1.下载hadoop version 0.20.205.0
2.下载jdk1.6 version 1.6.0_29
3.准备4台机器
[table]
|name|ip|type|
|oplog.master|10.125.14.205|master|
|oplog.slave1|10.125.14.195|slave|
|oplog.slave2|10.125.14.19|slave|
|oplog.slave3|10.125.14.176|slave|
[/table]
4.修改/etc/profile,增加如下配置(四台机器均加入)
5.修改/etc/hosts,增加如下配置(四台机器均加入)
6.修改../hadoop20205/conf/hadoop-env.sh
7.修改../hadoop20205/conf/core-site.xml
8.修改hdfs-site.xml
9.修改mapred-site.xml
10.配置修改../hadoop20205/conf/masters
11.配置修改../hadoop20205/conf/slaves
12.拷贝master机器的目录到slave1 slave2 slave3
13.配置master本地免登
14.配置master和slave1 slave2 slave3 免登
15.启动集群
16.验证集群启动情况
1)验证master,有namenode jobtracker和secondaryNameNode进程
2)验证slave,有dataNode 和 tasktracker进程
3)命令校验
2.下载jdk1.6 version 1.6.0_29
3.准备4台机器
[table]
|name|ip|type|
|oplog.master|10.125.14.205|master|
|oplog.slave1|10.125.14.195|slave|
|oplog.slave2|10.125.14.19|slave|
|oplog.slave3|10.125.14.176|slave|
[/table]
4.修改/etc/profile,增加如下配置(四台机器均加入)
..
export JAVA_HOME=/home/jincheng.sunjc/softs/jdk1.6.0_29/
export PATH=$JAVA_HOME/bin:$PATH
export CLASSPATH=.:$JAVA_HOME/lib:
..
[jincheng.sunjc@v014205-bja hadoop20205]$ source /etc/profile
5.修改/etc/hosts,增加如下配置(四台机器均加入)
10.125.14.205 oplog.master
10.125.14.195 oplog.slave1
10.125.14.19 oplog.slave2
10.125.14.176 oplog.slave3
[jincheng.sunjc@v014205-bja hadoop20205]$ source /etc/profile
6.修改../hadoop20205/conf/hadoop-env.sh
export JAVA_HOME=/home/jincheng.sunjc/softs/jdk1.6.0_29/
expor tHADOOP_HOME_WARN_SUPPRESS=TRUE
7.修改../hadoop20205/conf/core-site.xml
<configuration>
<property>
<name></name>
<value>/home/jincheng.sunjc/softs/hadoop/data</value>
</property>
<property>
<name>fs.default.name</name>
<value>hdfs://10.125.14.205:9000</value>
</property>
</configuration>
8.修改hdfs-site.xml
<configuration>
<property>
<name>dfs.replication</name>
<value>3</value>
</property>
</configuration>
9.修改mapred-site.xml
<configuration>
<property>
<name>mapred.job.tracker</name>
<value>http://10.125.14.205:9001</value>
</property>
</configuration>
10.配置修改../hadoop20205/conf/masters
oplog.master
11.配置修改../hadoop20205/conf/slaves
oplog.slave1
oplog.slave2
oplog.slave3
12.拷贝master机器的目录到slave1 slave2 slave3
scp -r softs/ slave1:/home/jincheng.sunjc/.
scp -r softs/ slave2:/home/jincheng.sunjc/.
scp -r softs/ slave3:/home/jincheng.sunjc/.
13.配置master本地免登
ssh-keygen -t dsa -P '' -f ~/.ssh/id_dsa
cat ~/.ssh/id_dsa.pub >> ~/.ssh/authorized_keys
14.配置master和slave1 slave2 slave3 免登
master:scp ~/.ssh/id_dsa.pub slave1:/home/jincheng.sunjc/.
slave1:cat /home/jincheng.sunjc/id_dsa.pub >> ~/.ssh/authorized_keys
slave2和slave3同样操作
15.启动集群
[jincheng.sunjc@v014205-bja bin]$ ./hadoop namenode -format
13/04/22 16:50:39 INFO namenode.NameNode: STARTUP_MSG:
/************************************************************
STARTUP_MSG: Starting NameNode
STARTUP_MSG: host = v014205-bja/10.125.14.205
STARTUP_MSG: args = [-format]
STARTUP_MSG: version = 0.20.205.0
STARTUP_MSG: build = https://svn.apache.org/repos/asf/hadoop/common/branches/branch-0.20-security-205 -r 1179940; compiled by 'hortonfo' on Fri Oct 7 06:20:32 UTC 2011
************************************************************/
Re-format filesystem in /tmp/hadoop-jincheng.sunjc/dfs/name ? (Y or N) Y
13/04/22 16:50:41 INFO util.GSet: VM type = 64-bit
13/04/22 16:50:41 INFO util.GSet: 2% max memory = 19.33375 MB
13/04/22 16:50:41 INFO util.GSet: capacity = 2^21 = 2097152 entries
13/04/22 16:50:41 INFO util.GSet: recommended=2097152, actual=2097152
13/04/22 16:50:41 INFO namenode.FSNamesystem: fsOwner=jincheng.sunjc
13/04/22 16:50:41 INFO namenode.FSNamesystem: supergroup=supergroup
13/04/22 16:50:41 INFO namenode.FSNamesystem: isPermissionEnabled=true
13/04/22 16:50:41 INFO namenode.FSNamesystem: dfs.block.invalidate.limit=100
13/04/22 16:50:41 INFO namenode.FSNamesystem: isAccessTokenEnabled=false accessKeyUpdateInterval=0 min(s), accessTokenLifetime=0 min(s)
13/04/22 16:50:41 INFO namenode.NameNode: Caching file names occuring more than 10 times
13/04/22 16:50:42 INFO common.Storage: Image file of size 120 saved in 0 seconds.
13/04/22 16:50:42 INFO common.Storage: Storage directory /tmp/hadoop-jincheng.sunjc/dfs/name has been successfully formatted.
13/04/22 16:50:42 INFO namenode.NameNode: SHUTDOWN_MSG:
/************************************************************
SHUTDOWN_MSG: Shutting down NameNode at v014205-bja/10.125.14.205
************************************************************/
[jincheng.sunjc@v014205-bja bin]$ ./start-all.sh
starting namenode, logging to /home/jincheng.sunjc/softs/hadoop/hadoop20205/libexec/../logs/hadoop-jincheng.sunjc-namenode-v014205-bja.out
oplog.slave3: starting datanode, logging to /home/jincheng.sunjc/softs/hadoop/hadoop20205/libexec/../logs/hadoop-jincheng.sunjc-datanode-v014176-bja.out
oplog.slave1: starting datanode, logging to /home/jincheng.sunjc/softs/hadoop/hadoop20205/libexec/../logs/hadoop-jincheng.sunjc-datanode-v014195-bja.out
oplog.slave2: starting datanode, logging to /home/jincheng.sunjc/softs/hadoop/hadoop20205/libexec/../logs/hadoop-jincheng.sunjc-datanode-v014019-bja.out
oplog.master: starting secondarynamenode, logging to /home/jincheng.sunjc/softs/hadoop/hadoop20205/libexec/../logs/hadoop-jincheng.sunjc-secondarynamenode-v014205-bja.out
starting jobtracker, logging to /home/jincheng.sunjc/softs/hadoop/hadoop20205/libexec/../logs/hadoop-jincheng.sunjc-jobtracker-v014205-bja.out
oplog.slave1: starting tasktracker, logging to /home/jincheng.sunjc/softs/hadoop/hadoop20205/libexec/../logs/hadoop-jincheng.sunjc-tasktracker-v014195-bja.out
oplog.slave2: starting tasktracker, logging to /home/jincheng.sunjc/softs/hadoop/hadoop20205/libexec/../logs/hadoop-jincheng.sunjc-tasktracker-v014019-bja.out
oplog.slave3: starting tasktracker, logging to /home/jincheng.sunjc/softs/hadoop/hadoop20205/libexec/../logs/hadoop-jincheng.sunjc-tasktracker-v014176-bja.out
[jincheng.sunjc@v014205-bja bin]$
16.验证集群启动情况
1)验证master,有namenode jobtracker和secondaryNameNode进程
[jincheng.sunjc@v014205-bja bin]$ jps
4559 Jps
4345 SecondaryNameNode
4443 JobTracker
4170 NameNode
[jincheng.sunjc@v014205-bja bin]$
2)验证slave,有dataNode 和 tasktracker进程
[jincheng.sunjc@v014195-bja hadoop]$ jps
32618 DataNode
32718 TaskTracker
330 Jps
[jincheng.sunjc@v014195-bja hadoop]$
3)命令校验