Apache spark:- Maximum temperature in a file using scala

HI, in this blog I will explain how to find the maximum temperature in a text file     We need to configure eclipse with spark first which is already described in earlier blog     Sample input:- 91    67    Agra  Uttar Pradesh     33    19 90    67    Allahabad   Uttar Pradesh     32    19 87    60    Amritsar   … Read More Apache spark:- Maximum temperature in a file using scala

Spark Streaming :- Word Counts in a folder for text files

Hi,   In this blog i will write a program for spark streaming which will read textfiles in a folder on regular basis and print the word counts program as follows imp :- i use maven in this( u can find mavan dependency in my previous post) import org.apache.spark.SparkConf import org.apache.spark.storage.StorageLevel import org.apache.spark.streaming._ import org.apache.spark.streaming.StreamingContext._… Read More Spark Streaming :- Word Counts in a folder for text files

Apache Spark :- Configure Spark on Windows with Eclipse

Hi Guys , in this post i am going to write about integration of spark with eclipse using Maven in Windows   Open eclipse and install scala IDE using eclipse marketplace Help ->eclipse marketplace Now let’s create a scala program with spark by following below steps:   Create new project (scala project) Change this project… Read More Apache Spark :- Configure Spark on Windows with Eclipse

Running-hadoop-on-ubuntu-linux-single-node-cluster :version 2.6

    Check Java version:- root@ubuntuvm:/home/user# java -version java version “1.7.0_95” OpenJDK Runtime Environment (IcedTea 2.6.4) (7u95-2.6.4-0ubuntu0.14.04.1) OpenJDK Client VM (build 24.95-b01, mixed mode, sharing) root@ubuntuvm:/home/user#     untar downloaded Hadoop root@ubuntuvm:/home/user/Desktop# tar xzf hadoop-2.6.1.tar.gz root@ubuntuvm:/home/user/Desktop# ls -l total 192692 drwxr-xr-x 9 10011 10011      4096 Sep 16 17:18 hadoop-2.6.1 -rwxrw-rw- 1 user  user  197113367 Feb… Read More Running-hadoop-on-ubuntu-linux-single-node-cluster :version 2.6

HIve Tables and insert data

HI,   In this we are simply going to create a table with fields separated by tab “\t” character This a simple one  ,   let’s start , here i will provide only code in hive :- Sample data :- 2016-01-08T12:27:05.565Z 1452256025 836871 2016-01-08T12:27:12.634Z 1452256033 800798 2016-01-08T12:27:12.632Z 1452256033 795799 2016-01-08T12:27:13.694Z 1452256033 820359 2016-01-08T12:27:15.821Z 1452256036 294141… Read More HIve Tables and insert data