Apache Spark :- Configure Spark on Windows with Eclipse

Hi Guys , in this post i am going to write about integration of spark with eclipse using Maven in Windows


Open eclipse and install scala IDE using eclipse marketplace

Help ->eclipse marketplace

Now let’s create a scala program with spark by following below steps:


  • Create new project (scala project)
  • Change this project to maven project :- rightclick -> configure :- convert to Maven project


  • Go to dependences , add following values
  • Save all
  • At bottom watch ;; workspace will be building
  • Once done add a scala file as below
  • Write code for word count simple in a file

import org.apache.spark._


object samplesparktest {

def main(args: Array[String]) {

System.setProperty(“hadoop.home.dir”, “c://winutil//”)

val conf = new SparkConf().setAppName(“Simple Application”).setMaster(“local[2]”)

val sc = new SparkContext(conf)

val    input=sc.textFile(“file:///C://Users//HA848869//Desktop//sparkdata//textfile.txt”)

val counts =input.flatMap(line => line.split(” “))

.map(word => (word, 1))

.reduceByKey(_ + _)



for (i – counts)









Run the above program as Run As Scala application and watch the console:-


Also you can see the spark web ui also:-

Web UI :-


2 thoughts on “Apache Spark :- Configure Spark on Windows with Eclipse

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s