Spark for Beginners: Tutorials – Apache Spark Streaming Twitter java example

Français Français

In this chapter, we will walk you through using Spark Streaming to process live tweet streams. Remember, Spark Streaming is a component of Spark that provides highly scalable, fault-tolerant streaming processing. These exercises are designed as standalone Java programs which will receive and process Twitter’s real sample tweet streams. You will find it the Gist Github links in the tutorial.

Create a Twitter developer account

This video Demonstrate how to create a twitter application. First go to https://apps.twitter.com/.




Adding dependencies Spark Streaming & Twitter

First we’ll create a new Maven project with Eclipse. I choose tn.esprit as Group Id and twitter  as Artifact Id.

We’re gonna change the pom.xml by adding dependencies.

Create TweetStream(java class) that will connect to Twitter API, and we’re going to steam live tweets.

Execution & test

To run our application, we must first generate it with Manven by running maven install and  right click on the TweetStream.java file and Run As Java Application .