Hadoop2.0 RM计算框架的搭建

Hadoop2.0 RM计算框架的搭建

节点：
		   	NN-1	NN-2	DN		ZK		ZKFC	JNN		RS		NM	
node08   		*							*		*					
node09				*		*		*		*		*				*	
node10						*		*				*		*		*	
node11						*		*						*		*	

在hadoop 2.0 hdfs 完全分布式安装(HA)基础之上安装Hadoop2.0 on yarn模式。
系统环境：node10和node11之间免密钥

安装步骤：
【1】进入/opt/sxt/hadoop-2.6.5/etc/hadoop目录下
	vi mapred-site.xml
<property>
        <name>mapreduce.framework.name</name>
        <value>yarn</value>
  </property>

【2】编辑yarn-site.xml文件
 	<property>
        		<name>yarn.nodemanager.aux-services</name>
        		<value>mapreduce_shuffle</value>
  	</property>
	<property>
   		<name>yarn.resourcemanager.ha.enabled</name>
   		<value>true</value>
 	</property>
 	<property>
   		<name>yarn.resourcemanager.cluster-id</name>
   		<value>cluster1</value>
 	</property>
 	<property>
   		<name>yarn.resourcemanager.ha.rm-ids</name>
   		<value>rm1,rm2</value>
 	</property>
 	<property>
   		<name>yarn.resourcemanager.hostname.rm1</name>
   		<value>node10</value>
 	</property>
 	<property>
   		<name>yarn.resourcemanager.hostname.rm2</name>
   		<value>node11</value>
 	</property>
 	<property>
   		<name>yarn.resourcemanager.zk-address</name>
   		<value>node09:2181,node10:2181,node11:2181</value>
 	</property>
【3】分发
	scp yarn-site.xml mapred-site.xml node09:`pwd` 
	scp yarn-site.xml mapred-site.xml node10:`pwd` 
	scp yarn-site.xml mapred-site.xml node11:`pwd` 
【4】启动
	node08上：start-yarn.sh 
	node10上：yarn-daemon.sh  start resourcemanager
	node11上：yarn-daemon.sh  start resourcemanager
yarn 启动时，nodemanager 成功，但是resourcemanager失败解决
Namenode和ResourceManger如果不是同一台机器，不能在NameNode上启动 yarn，应该在ResouceManager所在的机器上启动yarn。现在问题解决了。
【5】web访问：
	http://node11:8088/cluster	http://192.168.88.21:8088/cluster
            http://node10:8088/cluster	http://192.168.88.20:8088/cluster

【6】测试：
	cd  /opt/sxt/hadoop-2.6.5/share/hadoop/mapreduce
	 hadoop jar hadoop-mapreduce-examples-2.6.5.jar wordcount  /user/root/test4.txt /data/wc/output