hadoop高可用HA集群配置

一 、 搭建集群

Hadoop HA模式搭建前需要完成hadoop集群配置、时区同步设置和zookeeper安装配置

二 、修改配置文件

1. 修改core-site.xml文件

<configuration>
  <property>
     <name>fs.defaultFS</name>
     <value>hdfs://ns</value>
  </property>
  <property>
     <name>hadoop.tmp.dir</name>
     <value>/opt/hadoop/tmp</value>
  </property>
<!--引入zookeeper地址-->
  <property>
     <name>ha.zookeeper.quorum</name>
     <value>hadoop01:2181,hadoop02:2181,hadoop03:2181</value>
  </property>
  <property>
      <name>hadoop.proxyuser.root.hosts</name>
      <value>*</value>
  </property>
  <property>
      <name>hadoop.proxyuser.root.groups</name>
      <value>*</value>
  </property>
  <property>
      <name>hadoop.native.lib</name>
      <value>false</value>
      <description>Should native hadoop libraries, if present, be used.
      </description>
  </property>
</configuration>

2. 修改hdfs-site.xml文件

<configuration>
<!--节点数，ha中可不配置 -->
   <property>
        <name>dfs.replication</name>
        <value>3</value>
   </property>
<!--指定名称服务，与core-site.xml中的一致 -->
    <property>
        <name>dfs.nameservices</name>
        <value>ns</value>
   </property>
<!--指定名称服务下的namenode节点-->
    <property>
        <name>dfs.ha.namenodes.ns</name>
        <value>hadoop01,hadoop02</value>
   </property>
<!--rpc地址，9000端口-->
    <property>
        <name>dfs.namenode.rpc-address.ns.hadoop01</name>
        <value>hadoop01:9000</value>
   </property>
    <property>
        <name>dfs.namenode.rpc-address.ns.hadoop02</name>
        <value>hadoop02:9000</value>
   </property>
<!--http地址，50070端口,即页面地址-->
   <property>
        <name>dfs.namenode.http-address.ns.hadoop01</name>
        <value>hadoop01:50070</value>
   </property>
    <property>
        <name>dfs.namenode.http-address.ns.hadoop02</name>
        <value>hadoop02:50070</value>
   </property>
<!--namenode在jornalnode(datanode)上存放数据的端口8485-->
   <property>
        <name>dfs.namenode.shared.edits.dir</name>
        <value>qjournal://hadoop01:8485;hadoop02:8485;hadoop03:8485/ns</value>
   </property>
<!--开启namenode失败自动切换-->
   <property>
        <name>dfs.ha.automatic-failover.enabled</name>
        <value>true</value>
    </property>
<!--设置namenode自动切换方式-->
   <property>
        <name>dfs.client.failover.proxy.provider.ns</name>
        <value>org.apache.hadoop.hdfs.server.namenode.ha.ConfiguredFailoverProxyProvider</value>
   </property>
<!--设置隔离机制，多个机制换行分隔，每个机制占用一行-->
   <property>
        <name>dfs.ha.fencing.methods</name>
        <value>
           sshfence
           shell(/bin/true)
        </value>
   </property>
<!--超时设置-->
    <property>
       <name>dfs.ha.fencing.ssh.connect-timeout</name>
       <value>30000</value>
    </property>
    <property>
       <name>dfs.webhdfs.enabled</name>
       <value>true</value>
    </property>

<!-- 当使用HA配置集群，则不需要secondarynamenode
   </property>
        <name>dfs.namenode.secondary.http-address</name>
        <value>192.168.206.35:50090</value>
   </property>
-->
</configuration>

3. 修改yarn-site.xml文件

<configuration>
<!--HA配置-->
<!--开启HA-->
  <property>
      <name>yarn.resourcemanager.ha.enabled</name>
      <value>true</value>
  </property>
<!--指定resourcemanager集群id-->
   <property>
      <name>yarn.resourcemanager.cluster-id</name>
      <value>cluster_id</value>
   </property>
<!--rm集群的名称服务-->
   <property>
      <name>yarn.resourcemanager.ha.rm-ids</name>
      <value>rm1,rm2</value>
   </property>
<!--指定rm映射关系-->
   <property>
      <name>yarn.resourcemanager.hostname.rm1</name>
      <value>hadoop01</value>
   </property>
   <property>
      <name>yarn.resourcemanager.hostname.rm2</name>
      <value>hadoop02</value>
   </property>
   <property>
      <name>yarn.resourcemanager.webapp.address.rm1</name>
      <value>hadoop01:8088</value>
   </property>
   <property>
      <name>yarn.resourcemanager.webapp.address.rm2</name>
      <value>hadoop02:8088</value>
   </property>
<!--配置zookeeper集群-->
   <property>
      <name>yarn.resourcemanager.zk-address</name>
      <value>hadoop01:2181,hadoop02:2181,hadoop03:2181</value>
   </property>
   <!--非HA模式下指定resourcemanager
  <property>
      <name>yarn.resourcemanager.hostname</name>
      <value>hadoop01</value>
  </property>
-->
  <property>
      <name>yarn.nodemanager.aux-services</name>
      <value>mapreduce_shuffle</value>
  </property>
  <property>
      <name>yarn.nodemanager.aux-services.mapreduce.shuffle.class</name>
      <value>org.apache.hadoop.mapred.ShuffleHandler</value>
  </property>
<!-- 日志聚集功能使用 -->
  <property>
      <name>yarn.log-aggregation-enable</name>
      <value>true</value>
  </property>
<!-- 日志保留时间设置7天 -->
  <property>
      <name>yarn.log-aggregation.retain-seconds</name>
      <value>604800</value>
  </property>
</configuration>

4. 分发配置文件，要保证三台机器的配置文件一致

scp core-site.xml hadoop02:$PWD
scp core-site.xml hadoop03:$PWD
	
scp hdfs-site.xml hadoop02:$PWD
scp hdfs-site.xml hadoop02:$PWD
	
scp yarn-site.xml hadoop02:$PWD
scp yarn-site.xml hadoop02:$PWD

三 、启动集群

1. 确保集群处于关闭状态

2. 删除tmp文件夹，选择主节点重新format（可能需要提前启动journalnode）

#删除tmp和logs文件夹(三台机器都需要执行)
rm -rf tmp
rm -rf logs
# 可能需要提前启动 journalnode(三台机器都需要执行)
hadoop-daemon.sh start journalnode
#格式化hadoop集群(主节点上执行)
hadoop namenode -format

3. 把主节点新生成的tmp文件夹复制到备用节点(hadoop02)

scp -r tmp hadoop02:$PWD

4. 三台机器都需要启动zookeeper，查看zookeeper状态

#启动zookeeper
zkServer.sh start
#查看zookeeper状态
zkServer.sh status

5. 启动整个hadoop集群

cd /opt/hadoop
start-all.sh

6. 在备用节点(hadoop02)上启动备用resourcemanager

yarn-daemon.sh satrt resourcemanager

7. 主节点(hadoop01)格式化zkfc

hdfs zkfc -formatZK

8. 主、备用节点上启动zkfc

hadoop-daemon.sh start zkfc

9. 在负责jobhistory服务的机器上启动JobHistoryServer

mr-jobhistory-daemon.sh start historyserver

四 、测试集群是否部署成功

• 分别查看三台机器进程