yarn集群搭建步骤
规划
NN1 | NN2 | DN | ZK | ZKFC | JNN | RS | NM | |
---|---|---|---|---|---|---|---|---|
node01 | √ | √ | √ | √ | ||||
node02 | √ | √ | √ | √ | √ | √ | ||
node03 | √ | √ | √ | √ | ||||
node04 | √ | √ | √ |
1.修改yarn-site.xml步骤
<property>
<name>yarn.nodemanager.aux-services</name>
<value>mapreduce_shuffle</value>//定义yarn的机制
</property>
<property>
<name>yarn.resourcemanager.ha.enabled</name>//是否使用HA
<value>true</value>
</property>
<property>
<name>yarn.resourcemanager.cluster-id</name//集群Id
<value>cluster1</value>
</property>
<property>
<name>yarn.resourcemanager.ha.rm-ids</name>//集群逻辑节点
<value>rm1,rm2</value>
</property>
<property>
<name>yarn.resourcemanager.hostname.rm1</name>//集群物理节点
<value>node01</value>
</property>
<property>
<name>yarn.resourcemanager.hostname.rm2</name>//集群物理节点
<value>node02</value>
</property>
<property>
<name>yarn.resourcemanager.zk-address</name>//与zookeeper通信
<value>node02:2181,node03:2181,node04:2181</value>
</property>
2.修改mapred-site.xml
<property>
<name>mapreduce.framework.name</name>
<value>local</value>
</property>
3.启动hdfs集群
node01上
start-dfs.sh
4.启动yarn集群
node01上
start-yarn.sh
5.启动node01和node02上的ResourceManager
yarn-daemon.sh start resourcemanager
6.jps检查节点情况和浏览器检查
node01:8080