Install Hadoop on Ubuntu with two computers

This post is about the installation of Hadoop on Ubuntu with two computers. The system environment is Ubuntu 12.04, JDK 6, Hadoop 1.0.3. The information of two computers detail as follow:

hostname	IP address	Role
master	192.168.0.10	Namenode, Jobtracker, SecondaryNameNode
slave01	192.168.0.11	Datanode, Tasktracker

Supposing that the hadoop is installed, detail information : http://blog.csdn.net/hero_hegang/article/details/8910438.

Copying the virtual machine with hadoop to other physical machine. You had better configure the static IP address.

Step 1: modify the hosts mapping

$ sudo vi /etc/hosts

#type and add the contents as follows:

192.168.0.10 master
192.168.0.11 slave01

 

Step 2: modify the hostname

$ sudo vi /etc/hostname

#type and add the contents respectively as follows:

# the hostname of 192.168.0.10 is master.

# the hostname of 192.168.0.11 is slave01.

 

Step 3: test the ssh in master shell

$ ssh slave01

#If you don't enter the password, configure successfully.

 

Step 4: modify the hadoop configure file in master and slave with same confiurations

# modify core-site.xml, mapred-site.xml,masters, slaves as follows:

# modify the property fs.default.name tohdfs://master:9000 in core-site.xml.

# modify the property mapred.job.tracker to master:9001 in core-site.xml.

# modify the content of masters file to master.

# modify the content of slaves file to slave01.

 

Step 5: delete the contents of directory of dfs.data.dir in hdfs-site.xml in master and slave machines

$sudo rm -r /home/hadoop/data1/*

$sudo rm -r /home/hadoop/data2/*

 

Step 6: close the firewall in master and slave machines

$ sudo ufw disable

 

Step 7: reformat the namenode in master

$ bin/hadoop namenode -format

 

Step 8: see the process in masters and slave machines

$ jps

$ ssh slave01

$ jps

 

Step 9: see the report of hadoop

$ bin/hadoop dfsadmin -report