Check Java version:-
root@ubuntuvm:/home/user# java -version
java version “1.7.0_95”
OpenJDK Runtime Environment (IcedTea 2.6.4) (7u95-2.6.4-0ubuntu0.14.04.1)
OpenJDK Client VM (build 24.95-b01, mixed mode, sharing)
root@ubuntuvm:/home/user#
untar downloaded Hadoop
root@ubuntuvm:/home/user/Desktop# tar xzf hadoop-2.6.1.tar.gz
root@ubuntuvm:/home/user/Desktop# ls -l
total 192692
drwxr-xr-x 9 10011 10011 4096 Sep 16 17:18 hadoop-2.6.1
-rwxrw-rw- 1 user user 197113367 Feb 18 02:21 hadoop-2.6.1.tar.gz
Important: – where you untar the Hadoop , please remember it perfectly as we need it later
root@ubuntuvm:/home/user/Desktop# pwd
/home/user/Desktop
Check SSH with root :-
Steps as follows :-
- Create key gen
root@ubuntuvm:/home/user/Desktop# ssh-keygen -t rsa -P “”
Generating public/private rsa key pair.
Enter file in which to save the key (/root/.ssh/id_rsa):
/root/.ssh/id_rsa already exists.
Overwrite (y/n)? y
Your identification has been saved in /root/.ssh/id_rsa.
Your public key has been saved in /root/.ssh/id_rsa.pub.
The key fingerprint is:
6f:1c:85:e9:3c:4a:de:c7:60:bf:bb:7f:95:9c:c2:2e root@ubuntuvm
The key’s randomart image is:
+–[ RSA 2048]—-+
| |
| o |
| o . |
| o . |
| S B . . o|
| o * * o +.|
| o = = . .|
| . E o .|
| =+.. |
- Check ssh connection :-
- cat $HOME/.ssh/id_rsa.pub >> $HOME/.ssh/authorized_keys
- root@ubuntuvm:/home/user/Desktop# ssh localhost
Welcome to Ubuntu 14.04 LTS (GNU/Linux 3.13.0-24-generic i686)
Documentation: https://help.ubuntu.com/
Last login: Tue Feb 23 02:57:38 2016 from localhost
root@ubuntuvm:~#
Now we need to edit bashrc file
Type nano ~/.bashrc
Copy below at the end of file
export JAVA_HOME=/usr/lib/jvm/java-7-openjdk-i386
export HADOOP_INSTALL=/home/user/Desktop/hadoop-2.6.1
export PATH=$PATH:$HADOOP_INSTALL/bin
export PATH=$PATH:$HADOOP_INSTALL/sbin
export HADOOP_MAPRED_HOME=$HADOOP_INSTALL
export HADOOP_COMMON_HOME=$HADOOP_INSTALL
export HADOOP_HDFS_HOME=$HADOOP_INSTALL
export YARN_HOME=$HADOOP_INSTALL
ctrl + x to save
to get value of JAVA_HOME
root@ubuntuvm:~# update-alternatives –config java
There is only one alternative in link group java (providing /usr/bin/java): /usr/lib/jvm/java-7-openjdk-i386/jre/bin/java
Nothing to configure.
Just take value till: – /usr/lib/jvm/java-7-openjdk-i386
This is all we done at system level
Now we will done configuration changes at Hadoop level
In Hadoop-env.sh just change the value as below :-
export JAVA_HOME=/usr/lib/jvm/java-7-openjdk-i386
you will get this file at /home/user/Desktop/hadoop-2.6.1/etc/Hadoop
in hdfs-site.xml add following between configuration
<property>
<name>dfs.replication</name>
<value>1</value>
<description>Default block replication.
The actual number of replications can be specified when the file is created.
The default is used if replication is not specified in create time.
</description>
</property>
Edit core-site.xml
Put following values in it :-
<configuration>
<property>
<name>hadoop.tmp.dir</name>
<value>/app/hadoop/tmp</value>
<description>A base for other temporary directories.</description>
</property>
<property>
<name>fs.default.name</name>
<value>hdfs://localhost:50070</value>
<description>The name of the default file system. A URI whose
scheme and authority determine the FileSystem implementation. The
uri’s scheme determines the config property (fs.SCHEME.impl) naming
the FileSystem implementation class. The uri’s authority is used to
determine the host, port, etc. for a filesystem.</description>
</property>
</configuration>
Now we are ready to start our cluster
- we need to format our namenode as follows :-
root@ubuntuvm:/home/user/Desktop/hadoop-2.6.1/etc/hadoop# hadoop namenode -format
DEPRECATED: Use of this script to execute hdfs command is deprecated.
Instead use the hdfs command for it.
16/02/23 03:46:26 INFO namenode.NameNode: STARTUP_MSG:
/************************************************************
STARTUP_MSG: Starting NameNode
STARTUP_MSG: host = ubuntuvm/127.0.1.1
STARTUP_MSG: args = [-format]
STARTUP_MSG: version = 2.6.1
Now Start the daemons as follows:-
Go to Hadoop/sbin
Run following command:-
./start-all.sh
This script is Deprecated. Instead use start-dfs.sh and start-yarn.sh
16/02/23 03:48:27 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform… using builtin-java classes where applicable
After execution of command:-
root@ubuntuvm:/home/user/Desktop/hadoop-2.6.1/sbin# jps
12943 NodeManager
6318 NameNode
12818 ResourceManager
6479 DataNode
13804 Jps
Congratulations: – we have successfully installed single node cluster
Important url ‘s:-
root@ubuntuvm:/home/user/Desktop/hadoop-2.6.1/sbin# hadoop version
Hadoop 2.6.1
Subversion https://git-wip-us.apache.org/repos/asf/hadoop.git -r b4d876d837b830405ccdb6af94742f99d49f9c04
Compiled by jenkins on 2015-09-16T21:07Z
Compiled with protoc 2.5.0
From source with checksum ba9a9397365e3ec2f1b3691b52627f
This command was run using /home/user/Desktop/hadoop-2.6.1/share/hadoop/common/hadoop-common-2.6.1.jar
