Hadoop学习六:YARN的RM做HA

Hadoop学习六:YARN的RM做HA

标签（空格分隔）： hadoop

---

一，概述

1，快速故障自动转移
2，任务状态和资源的分配

二，配置

1,角色分配：
hadoop001.com.cn—-rm1
hadoop002.com.cn—-rm2

2,修改yarn-site.xml文件：

<configuration>
    <property>
        <name>yarn.nodemanager.aux-services</name>
        <value>mapreduce_shuffle</value>
    </property>

    <property>
        <name>yarn.log-aggregation-enable</name>
        <value>true</value>
    </property>
    <property>
        <name>yarn.log-aggregation.retain-seconds</name>
        <value>604800</value>
    </property>

     <property>
       <name>yarn.resourcemanager.ha.enabled</name>
       <value>true</value>
     </property>
     <property>
       <name>yarn.resourcemanager.cluster-id</name>
       <value>clusterrm</value>
     </property>
     <property>
       <name>yarn.resourcemanager.ha.rm-ids</name>
       <value>rm1,rm2</value>
     </property>
     <property>
       <name>yarn.resourcemanager.hostname.rm1</name>
       <value>hadoop001.com.cn</value>
     </property>
     <property>
       <name>yarn.resourcemanager.hostname.rm2</name>
       <value>hadoop002.com.cn</value>
     </property>
     <property>
       <name>yarn.resourcemanager.zk-address</name>
       <value>hadoop001.com.cn:2181,hadoop002.com.cn:2181,hadoop003.com.cn:2181</value>
     </property>

    <property>
       <name>yarn.resourcemanager.recovery.enabled</name>
       <value>true</value>
     </property>

    <property>
       <name>yarn.resourcemanager.store.class</name>
       <value>org.apache.hadoop.yarn.server.resourcemanager.recovery.ZKRMStateStore</value>
     </property> 

</configuration>

3,拷贝到其它服务器上

[hadoop001@hadoop001 hadoop]$ scp -r yarn-site.xml hadoop002.com.cn:/opt/app/hadoop-2.5.0/etc/hadoop/

[hadoop001@hadoop001 hadoop]$ scp -r yarn-site.xml hadoop003.com.cn:/opt/app/hadoop-2.5.0/etc/hadoop/

4,在rm节点上启动resourcemanager服务和nodemanager

$ sbin/yarn-daemon.sh start resourcemanager

$ sbin/yarn-daemon.sh start nodemanager

5,查看rm的状态

$ bin/yarn rmadmin -getServiceState rm1
$ bin/yarn rmadmin -getServiceState rm2

6,测试

在hadoop003.com.cn上执行一个mapreduce任务：

[hadoop001@hadoop003 hadoop-2.5.0]$ bin/yarn jar /opt/app/hadoop-2.5.0/share/hadoop/mapreduce/hadoop-mapreduce-examples-2.5.0.jar wordcount /core-site.xml /output001

在hadoop003.com.cn上执行一个mapreduce任务时关闭hadoop001.com.cn上的rm服务:

[hadoop001@hadoop003 hadoop-2.5.0]$ bin/yarn jar /opt/app/hadoop-2.5.0/share/hadoop/mapreduce/hadoop-mapreduce-examples-2.5.0.jar wordcount /core-site.xml /output002

[hadoop001@hadoop001 hadoop-2.5.0]$ kill -9 5357