配置三台hadoop集群（转）（有改动）(续)

6)配置文件：core-site.xml

[html] view plain copy

1. <configuration>  
2.     <property>  
3.         <name>fs.defaultFS</name>  
4.         <value>hdfs://master77:9000</value>  
5.     </property>  
6.     <property>  
7.         <name>io.file.buffer.size</name>  
8.         <value>131072</value>  
9.     </property>  
10.     <property>  
11.         <name>hadoop.tmp.dir</name>  
12.         <value>file:/usr/local/hadoop/tmp</value>  
13.         <description>Abase for other temporary   directories.</description>  
14.     </property>  
15. </configuration>  

 

7)配置文件：hdfs-site.xml

[html] view plain copy

1. <configuration>  
2.        <property>  
3.                 <name>dfs.namenode.secondary.http-address</name>  
4.                <value>master77:9001</value>  
5.        </property>  
6.      <property>  
7.              <name>dfs.namenode.name.dir</name>  
8.              <value>file:/usr/local/hadoop/dfs/name</value>  
9.        </property>  
10.       <property>  
11.               <name>dfs.datanode.data.dir</name>  
12.               <value>file:/usr/local/hadoop/dfs/data</value>  
13.        </property>  
14.        <property>  
15.                <name>dfs.replication</name>  
16.                <value>2</value>  
17.         </property>  
18.         <property>  
19.                  <name>dfs.webhdfs.enabled</name>  
20.                   <value>true</value>  
21.          </property>  
22. </configuration>  

 

8)配置文件：mapred-site.xml

先创建然后编辑

 cp etc/hadoop/mapred-site.xml.template etc/hadoop/mapred-site.xml

gedit etc/hadoop/mapred-site.xml

[html] view plain copy

1. <configuration>  
2.           <property>                                                                    
3. 　　　　　　　　<name>mapreduce.framework.name</name>  
4.                 <value>yarn</value>  
5.            </property>  
6.           <property>  
7.                   <name>mapreduce.jobhistory.address</name>  
8.                   <value>master77:10020</value>  
9.           </property>  
10.           <property>  
11.                 <name>mapreduce.jobhistory.webapp.address</name>  
12.                 <value>master77:19888</value>  
13.        </property>  
14. </configuration>  

 

9)配置文件：yarn-site.xml

[html] view plain copy

1. <configuration>  
2.         <property>  
3.                <name>yarn.nodemanager.aux-services</name>  
4.                <value>mapreduce_shuffle</value>  
5.         </property>  
6.         <property>                                                                  
7. <name>yarn.nodemanager.aux-services.mapreduce.shuffle.class</name>  
8.                <value>org.apache.hadoop.mapred.ShuffleHandler</value>  
9.         </property>  
10.         <property>  
11.                <name>yarn.resourcemanager.address</name>  
12.                <value>master77:8032</value>  
13.        </property>  
14.        <property>  
15.                <name>yarn.resourcemanager.scheduler.address</name>  
16.                <value>master77:8030</value>  
17.        </property>  
18.        <property>  
19.             <name>yarn.resourcemanager.resource-tracker.address</name>  
20.              <value>master77:8031</value>  
21.       </property>  
22.       <property>  
23.               <name>yarn.resourcemanager.admin.address</name>  
24.                <value>master77:8033</value>  
25.        </property>  
26.        <property>  
27.                <name>yarn.resourcemanager.webapp.address</name>  
28.                <value>master77:8088</value>  
29.        </property>  
30. </configuration>  

 

10)将hadoop传输到slave78和slave79 /usr/local/hadoop目录,（如果传输时报错说 ：权限拒绝，先把文件传送到非/usr目录下，然后在node上把这个文件再移动到/usr/hadoop）

 scp -r /usr/local/hadoop hadoop@192.168.8.78:/usr/local/hadoop

7、配置环境变量，并启动hadoop，检查是否安装成功

1）配置环境变量

#编辑/etc/profile
sudo gedit /etc/profile
#以上已经添加过java的环境变量,在后边添加就可以

1. #hadoop  
2. export HADOOP_HOME=/usr/local/hadoop
3. export PATH=$PATH:$HADOOP_HOME/sbin  
4. export PATH=$PATH:$HADOOP_HOME/bin  

执行 

source /etc/profile

 

使文件生效。

2）启动hadoop,进入hadoop安装目录

bin/hdfs namenode -format
sbin/start-all.sh

3)启动后分别在master, slave下输入jps查看进程

看到下面的结果，则表示成功。

Master77:

hadoop@master77:/usr/local/hadoop$ jps

4706 Jps

3653 ResourceManager

4357 SecondaryNameNode

4121 NameNode

slave79:

hadoop@slave79:~$ jps

3643 Jps

3149 DataNode

3294 NodeManager

 

hadoop@slave79:~$ 

 

 

 

 

 

补充：

【 分开启动】的问题

 

start-all 等于 先start dfs，后start yarn

遇到 ERROR: Can't get master address from ZooKeeper; znode data == null  ，启动时分开启动，就解决了