ZooKeeper+Hadoop的ResourceManager HA集群高可用

一、centos配置 及克隆准备

1、安装VM及centos7 

 部署网络

2、安装JDK

3、修改网络名  hostnamectl --static set-hostname n1 

  n1为主机名

  reboot重启

4、etc/hosts配置  

 

127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4
::1         localhost localhost.localdomain localhost6 localhost6.localdomain6
192.168.146.128 n1
192.168.146.129 n2
192.168.146.130 n3

5、创建hadoop账号

 

 创建一个组 groupadd hadoop 

组下面创建一个账号 useradd -g hadoop hadoop

创建一个密码passwd hadoop

6、克隆虚拟机n2 n3

关闭虚拟机n1

打开VMware 选中n1即可-克隆源-虚拟机中的当前状态-创建链接克隆-名称改为n2 位置选跟n1一样的

变更 cd /etc/sysconfig/network-scripts/ifcfg-ens33  IPADDR=改成129  130

重新启动服务 service network restart

然后更改主机名 参考第3步

二、安装zookeper

1、上传到zookzookeeper-3.4.7.tar.gz

2、进入该目录解压 tar -zxvf /home/zookeeper-3.4.7.tar.gz -C /usr/local/

3、删除rm -rf ./zookeeper-3.4.7/docs/

4、配置环境变量 并变更n2 n3

 

JAVA_HOME=/usr/java/jdk1.7.0_80/
HADOOP_HOME=/usr/local/hadoop/
ZK_HOME=/usr/local/zookeeper-3.4.7/
CLASS_PATH=.:$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jar
PATH=$JAVA_HOME/bin:$PATH:$HADOOP_HOME/bin:$HADOOP_HOME/sbin:$ZK_HOME/bin

export JAVA_HOME CLASS_PATH  PATH 

生效  source /etc/profile 
测试zk  然后tab看有没有相应的命令

 

5、修改文件名mv /usr/local/zookeeper-3.4.7/conf/zoo_sample.cfg /usr/local/zookeeper-3.4.7/conf/zoo.cfg

 修改vi /usr/local/zookeeper-3.4.7/conf/zoo.cfg

 然后传给n2 n3    scp -r ../zookeeper-3.4.7 n2:/use/local/

6、创建目录 mkdir /home/zkdata/ 

  并在n1 n2 n3分别创建vi /home/zkdata/myid/  内容1、2、3

7、启动 各个节点都要启动

zkServer.sh start

进入客户端 zkCli.sh

三、安装hdfs HA

下载hadoop-2.6.5.tar.gz

1、安装tar  -zxvf /tmp/hadoop-2.6.5.tar.gz  -C /usr/local/  

hadoop-2.6.5改成hadoop

2、修改配置文件4个：

/usr/local/hadoop/etc/hadoop/hadoop-env.sh

export JAVA_HOME=${JAVA_HOME}

改成export JAVA_HOME=/usr/java/jdk1.7.0_80/

/usr/local/hadoop/etc/hadoop/core-site.xml

 

<?xml version="1.0" encoding="UTF-8"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<!--
  Licensed under the Apache License, Version 2.0 (the "License");
  you may not use this file except in compliance with the License.
  You may obtain a copy of the License at

    http://www.apache.org/licenses/LICENSE-2.0

  Unless required by applicable law or agreed to in writing, software
  distributed under the License is distributed on an "AS IS" BASIS,
  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
  See the License for the specific language governing permissions and
  limitations under the License. See accompanying LICENSE file.
-->

<!-- Put site-specific property overrides in this file. -->

<configuration>
  <!-- 指定hdfs的命名空间-->
     <property>
             <name>fs.defaultFS</name>
             <value>hdfs://qf</value>
     </property>
     <property>
             <name>hadoop.tmp.dir</name>
             <value>/home/hadoopdata/tmp</value> 
    </property>
     <property>    
        <name>io.file.buffer.size</name> 
        <value>4096</value>
      </property>
<!-- 指定zk的集群地址，用来协调namenode的服务 -->
   <property>
        <name>ha.zookeeper.quorum</name>
        <value>n1:2181,n2:2181,n3:2181</value>
</property>
</configuration>

/usr/local/hadoop/etc/hadoop/hdfs-site.xml

 

<?xml version="1.0" encoding="UTF-8"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<!--
  Licensed under the Apache License, Version 2.0 (the "License");
  you may not use this file except in compliance with the License.
  You may obtain a copy of the License at

    http://www.apache.org/licenses/LICENSE-2.0

  Unless required by applicable law or agreed to in writing, software
  distributed under the License is distributed on an "AS IS" BASIS,
  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
  See the License for the specific language governing permissions and
  limitations under the License. See accompanying LICENSE file.
-->

<!-- Put site-specific property overrides in this file. -->

<configuration>
       <!-- 指定hdfs的nameservice虚拟服务名 -->
            <property>
              	<name>dfs.nameservices</name>
	        <value>qf</value>
            </property>
      <!—指定hdfs的两个NameNode都是什么名字（等会儿下面会配置他们所对应的机器的信息）-->
           <property>
                <name>dfs.ha.namenodes.qf</name>
                 <value>nn1,nn2</value>
           </property>
         <!—指定Namenode内部通信地址 -->
           <property>
            <name>dfs.namenode.rpc-address.qf.nn1</name>
            <value>n1:9000</value>
            </property>
          <property>
             <name>dfs.namenode.rpc-address.qf.nn2</name>
             <value>n2:9000</value>
          </property>
         <!—指定Namenode的web ui通信地址 -->
          <property>
             <name>dfs.namenode.http-address.qf.nn1</name>
             <value>n1:50070</value>
          </property>
          <property>
               <name>dfs.namenode.http-address.qf.nn2</name>
               <value>n2:50070</value>
           </property>
     <!—指定jouranlnode数据共享目录-->
           <property>
               <name>dfs.namenode.shared.edits.dir</name>
               <value>qjournal://n1:8485;n2:8485;n3:8485/qf</value>
           </property>
       <!—指定jouranlnode本地共享目录-->
           <property>
              <name>dfs.journalnode.edits.dir</name>
              <value>/home/hahadoopdata/jouranl/data</value>
           </property>
    <!—开启hdfs的namenode死亡后自动切换-->
          <property>
               <name>dfs.ha.automatic-failover.enabled</name>
               <value>true</value>
         </property>
  
     <!—指定namenode失败进行自动切换的主类-->
           <property>
	        <name>dfs.client.failover.proxy.provider.qf </name>
	        <value>org.apache.hadoop.hdfs.server.namenode.ha.ConfiguredFailoverProxyProvider</value>
           </property>
        <!—防止多个namenode同时active（脑裂）的方式-->
            <property>
                  <name>dfs.ha.fencing.methods</name>
                  <value>sshfence</value>
             </property>

        <!—指定本机的私钥所在目录-->
             <property>
                    <name>dfs.ha.fencing.ssh.private-key-files</name>
                   <value>/root/.ssh/id_rsa</value>
            </property>
          <!—指定ssh通讯超时时间-->
            <property>
                  <name>dfs.ha.fencing.ssh.connect-timeout</name>
                  <value>30000</value>
           </property>
</configuration>

 

/usr/local/hadoop/etc/hadoop/slaves

 

 

 

 

n1
n2
n3

3、分发到n2 n3

scp -r  /usr/local/hadoop/ n2:/usr/local/

4、ssh免登陆配置

n1、n2、n3配置

 

ssh-keygen -t rsa一直回车
ssh-copy-id n1/n2/n3   选择yes就可以就不需要密码了

 

5、顺序启动执行

 

1、启动zk 
 每个服务器 zkServer.sh start  启动   jps查看状态
            zkServer.sh status 查看状态
            zkCli.sh 进入客户端  
2、启动journalnode服务(单个启动、多个进程启动)
./sbin/hadoop-daemon.sh start jouranlnode 单个启动
hadoop-daemons.sh start journalnode  多个启动 首选


3、挑选两个namenode之中的一台来格式化
hdfs namenode -format
5、单个namenode
n1  hadoop-daemons.sh start namenode
n2拉取源数据
hdfs namenode -bootstrapStandby
6、格式化zkfc
  hdfs zkfc -formatZK
6、启动
start-dfs.sh
7、测试 
先看jps对应的进程是否启动 n1 n2

DFSZKFailoverController
DataNode
NameNode
JournalNode
QuorumPeerMain
再看web ui是否正常
在hdfs中读写文件然后测试关闭一个namenode失败 查看是否能自动切换 
再看web ui是否正常
在hdfs中读写文件然后测试关闭一个namenode失败 查看是否能自动切换 

以后启动 直接n1、n2、n3zkServer.sh start

然后n1 start-dfs.sh

四、yarn-HA配置与测试

1、/usr/loca/hadoop/etc/hadoop/mapred-site.xml（把mapred-site.xml.template改成mapred-site.xml）

 

<configuration>
   <!--指定mapreduce运行框架 -->
   <property>
 <name>mapreduce.framework.name</name>
  <value>yarn</value>
  <final>true</final>
  </property>
</configuration>

 

     /usr/loca/hadoop/etc/hadoop/yarn-site.xml

 

<configuration>  
<!--指定的mr的shuffle -->  
<property>  
   <name>yarn.nodemanager.aux-services</name>  
   <value>mapreduce_shuffle</value>  
 </property>  
<property>  
            <name>yarn.nodemanager.aux-services.mapreduce_shuffle.class</name>  
            <value>org.apache.hadoop.mapred.ShuffleHandler</value>  
        </property>  
<!-- 是否启动yarn的HA -->  
<property>  
   <name>yarn.resourcemanager.ha.enabled</name>  
   <value>true</value>  
 </property>  
<!-- yarn的虚拟服务器名 -->  
 <property>  
   <name>yarn.resourcemanager.cluster-id</name>  
   <value>qfyarn</value>  
 </property>  
<!-- yarn的虚拟服务器名下的具体rm -->  
 <property>  
   <name>yarn.resourcemanager.ha.rm-ids</name>  
   <value>rm1,rm2</value>  
 </property>  
<!-- 指定rm1所启动的主机 -->  
 <property>  
   <name>yarn.resourcemanager.hostname.rm1</name>  
   <value>n1</value>  
 </property>  
 <property>  
   <name>yarn.resourcemanager.hostname.rm2</name>  
   <value>n2</value>  
 </property>  
<!-- 指定rm1web ui的通信地址-->  
 <property>  
   <name>yarn.resourcemanager.webapp.address.rm1</name>  
   <value>n1:8088</value>  
 </property>  
 <property>  
   <name>yarn.resourcemanager.webapp.address.rm2</name>  
   <value>n2:8088</value>  
 </property>  
<!-- 指定rm1web ui的通信地址-->  
 <property>  
   <name>yarn.resourcemanager.zk-address</name>  
   <value>n1:2181,n2:2181,n3:2181</value>  
 </property>  
</configuration>  

2、分发n2 n3

 

scp -r /usr/local/hadoop/etc/hadoop/mapred-site.xml /usr/local/hadoop/etc/hadoop/yarn-site.xml n2:/usr/local/hadoop/etc/hadoop/

scp -r /usr/local/hadoop/etc/hadoop/mapred-site.xml /usr/local/hadoop/etc/hadoop/yarn-site.xml n3:/usr/local/hadoop/etc/hadoop/

3、启动
      n1 

start-yarn.sh

   n2 

 

yarn-daemon.sh start resourcemanager

4、测试
创建于一个文件

 

vi /home/words
hadoop is nice
hadoop good 
hadoop better

上传hdfs dfs -put /home/words /
然后分析

 yarn jar /usr/local/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-examples-2.7.1.jar wordcount /words /out/01

查看数据

 

hdfs dfs -cat /out/00/part-00000

beter   1
good    1
hadoop  3
is      2
nice    1

5、看可以自动切换actvity不

kaill -9   ResourceManager对应的进程 进入n2的8088看变了没有