Running-hadoop-on-ubuntu-linux-single-node-cluster :version 2.6

 

 

Check Java version:-

root@ubuntuvm:/home/user# java -version

java version “1.7.0_95”

OpenJDK Runtime Environment (IcedTea 2.6.4) (7u95-2.6.4-0ubuntu0.14.04.1)

OpenJDK Client VM (build 24.95-b01, mixed mode, sharing)

root@ubuntuvm:/home/user#

 

 

untar downloaded Hadoop

root@ubuntuvm:/home/user/Desktop# tar xzf hadoop-2.6.1.tar.gz

root@ubuntuvm:/home/user/Desktop# ls -l

total 192692

drwxr-xr-x 9 10011 10011      4096 Sep 16 17:18 hadoop-2.6.1

-rwxrw-rw- 1 user  user  197113367 Feb 18 02:21 hadoop-2.6.1.tar.gz

 

Important: – where you untar the Hadoop , please remember it perfectly as we need it later

root@ubuntuvm:/home/user/Desktop# pwd

/home/user/Desktop

 

Check SSH with root :-

Steps as follows :-

  1. Create key gen

 

root@ubuntuvm:/home/user/Desktop# ssh-keygen -t rsa -P “”

Generating public/private rsa key pair.

Enter file in which to save the key (/root/.ssh/id_rsa):

/root/.ssh/id_rsa already exists.

Overwrite (y/n)? y

Your identification has been saved in /root/.ssh/id_rsa.

Your public key has been saved in /root/.ssh/id_rsa.pub.

The key fingerprint is:

6f:1c:85:e9:3c:4a:de:c7:60:bf:bb:7f:95:9c:c2:2e root@ubuntuvm

The key’s randomart image is:

+–[ RSA 2048]—-+

|                 |

|           o     |

|          o .    |

|         o .     |

|        S B . . o|

|       o * * o +.|

|        o = = . .|

|         . E o  .|

|            =+.. |

 

  1. Check ssh connection :-
  • cat $HOME/.ssh/id_rsa.pub >> $HOME/.ssh/authorized_keys
  • root@ubuntuvm:/home/user/Desktop# ssh localhost

Welcome to Ubuntu 14.04 LTS (GNU/Linux 3.13.0-24-generic i686)

 

Documentation:  https://help.ubuntu.com/

 

Last login: Tue Feb 23 02:57:38 2016 from localhost

root@ubuntuvm:~#

 

 

 

 

 

Now we need to edit bashrc file

Type nano ~/.bashrc

Copy below at the end of file

export JAVA_HOME=/usr/lib/jvm/java-7-openjdk-i386

export HADOOP_INSTALL=/home/user/Desktop/hadoop-2.6.1

export PATH=$PATH:$HADOOP_INSTALL/bin

export PATH=$PATH:$HADOOP_INSTALL/sbin

export HADOOP_MAPRED_HOME=$HADOOP_INSTALL

export HADOOP_COMMON_HOME=$HADOOP_INSTALL

export HADOOP_HDFS_HOME=$HADOOP_INSTALL

export YARN_HOME=$HADOOP_INSTALL

 

ctrl + x to save

 

 

to get value of JAVA_HOME

root@ubuntuvm:~# update-alternatives –config java

There is only one alternative in link group java (providing /usr/bin/java): /usr/lib/jvm/java-7-openjdk-i386/jre/bin/java

Nothing to configure.

 

Just take value till: –  /usr/lib/jvm/java-7-openjdk-i386

 

This is all we done at system level

 

 

 

 

 

 

 

 

 

 

Now we will done configuration changes at Hadoop level  

 

In Hadoop-env.sh just change the value as below :-

 

export JAVA_HOME=/usr/lib/jvm/java-7-openjdk-i386

 

you will get this file at  /home/user/Desktop/hadoop-2.6.1/etc/Hadoop

 

in hdfs-site.xml add following  between configuration

<property>

  <name>dfs.replication</name>

  <value>1</value>

  <description>Default block replication.

  The actual number of replications can be specified when the file is created.

  The default is used if replication is not specified in create time.

  </description>

</property>

 

Edit core-site.xml

Put following values in it :-

<configuration>

 

<property>

  <name>hadoop.tmp.dir</name>

  <value>/app/hadoop/tmp</value>

  <description>A base for other temporary directories.</description>

</property>

 

<property>

  <name>fs.default.name</name>

  <value>hdfs://localhost:50070</value>

  <description>The name of the default file system.  A URI whose

  scheme and authority determine the FileSystem implementation.  The

  uri’s scheme determines the config property (fs.SCHEME.impl) naming

  the FileSystem implementation class.  The uri’s authority is used to

  determine the host, port, etc. for a filesystem.</description>

</property>

</configuration>

 

 

Now we are ready to start our cluster

  • we need to format our namenode as follows :-

root@ubuntuvm:/home/user/Desktop/hadoop-2.6.1/etc/hadoop# hadoop namenode -format

DEPRECATED: Use of this script to execute hdfs command is deprecated.

Instead use the hdfs command for it.

 

16/02/23 03:46:26 INFO namenode.NameNode: STARTUP_MSG:

/************************************************************

STARTUP_MSG: Starting NameNode

STARTUP_MSG:   host = ubuntuvm/127.0.1.1

STARTUP_MSG:   args = [-format]

STARTUP_MSG:   version = 2.6.1

 

 

 

 

 

Now Start the daemons as follows:-

Go to Hadoop/sbin

Run following command:-

./start-all.sh

This script is Deprecated. Instead use start-dfs.sh and start-yarn.sh

16/02/23 03:48:27 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform… using builtin-java classes where applicable

 

After execution of command:-

 

root@ubuntuvm:/home/user/Desktop/hadoop-2.6.1/sbin# jps

12943 NodeManager

6318 NameNode

12818 ResourceManager

6479 DataNode

13804 Jps

 

Congratulations: – we have successfully installed single node cluster

Important url ‘s:-

http://localhost:50070

http://localhost:8088

 

root@ubuntuvm:/home/user/Desktop/hadoop-2.6.1/sbin# hadoop version
Hadoop 2.6.1
Subversion https://git-wip-us.apache.org/repos/asf/hadoop.git -r b4d876d837b830405ccdb6af94742f99d49f9c04
Compiled by jenkins on 2015-09-16T21:07Z
Compiled with protoc 2.5.0
From source with checksum ba9a9397365e3ec2f1b3691b52627f
This command was run using /home/user/Desktop/hadoop-2.6.1/share/hadoop/common/hadoop-common-2.6.1.jar

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s