hadoop2.6.5 HDFS的高可用集群搭建

1.先来看一下要搭建hadoop集群的HDFS HA结构图

2.要配置的HDFS节点分布图

从分布图上可以看到，节点1和节点2作为namenode，节点2、3、4作为datanode，节点2、3、4作为Zookeeper，节点1和2作为ZKFC，节点1、2、3作为JournalNode。

3.准备4台Server

4.配置Zookeeper

1）下载Zookerper的jar包，下载链接如下：http://download.csdn.net/detail/aizhenshi/9857485

2）下载完成后，放到node2相应目录下，，一般放在和hadoop同级的目录里，然后解压。解压完成后，进入conf文件，执行命令：

cp zoo_sample.cfg zoo.cfg

复制该文件。

vi zoo.cfg

# The number of milliseconds of each tick
tickTime=2000
# The number of ticks that the initial
# synchronization phase can take
initLimit=10
# The number of ticks that can pass between
# sending a request and getting an acknowledgement
syncLimit=5
# the directory where the snapshot is stored.
# do not use /tmp for storage, /tmp here is just
# example sakes.
dataDir=/var/zk
# the port at which the clients will connect
clientPort=2181
# the maximum number of client connections.
# increase this if you need to handle more clients
#maxClientCnxns=60
#
# Be sure to read the maintenance section of the
# administrator guide before turning on autopurge.
#
# http://zookeeper.apache.org/doc/current/zookeeperAdmin.html#sc_maintenance
#
# The number of snapshots to retain in dataDir
#autopurge.snapRetainCount=3
# Purge task interval in hours
# Set to "0" to disable auto purge feature
#autopurge.purgeInterval=1
server.1=node2:2888:3888
server.2=node3:2888:3888
server.3=node4:2888:3888

其中dataDir=/var/zk，是Zookeeper的工作目录，server.1=node2:2888:3888，server.2=node3:2888:3888，server.3=node4:2888:3888是Zookeeper要部署在那些服务器上。zk这个目录是要手动创建的。执行以下命令：

mkdir zk
cd zk
echo 1 > myid

把Zookeeper文件目录拷贝到其他两个节点上

cp -r zookeeper-3.4/ node3:`pwd`/zookeeper-3.4
cp -r zookeeper-3.4/ node4:`pwd`/zookeeper-3.4

在node3执行：

mkdir zk
cd zk
echo 2 > myid

在node4执行：

mkdir zk
cd zk
echo 3 > myid

进入到zookeeper-3.4目录下的bin目录，并执行：

[root@node2 zookeeper-3.4]# cd bin
[root@node2 bin]# ./zkServer.sh start

所有部署了Zookeeper的节点（这里包括node2、node3、node4）都要执行./zkServer.sh start命令。
可使用./zkServer.sh status查看状态。

5.配置hdfs-site.xml

<?xml version="1.0" encoding="UTF-8"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<!--
  Licensed under the Apache License, Version 2.0 (the "License");
  you may not use this file except in compliance with the License.
  You may obtain a copy of the License at

    http://www.apache.org/licenses/LICENSE-2.0

  Unless required by applicable law or agreed to in writing, software
  distributed under the License is distributed on an "AS IS" BASIS,
  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
  See the License for the specific language governing permissions and
  limitations under the License. See accompanying LICENSE file.
-->

<!-- Put site-specific property overrides in this file. -->

<configuration>
    <property>
        <name>dfs.replication</name>
        <value>3</value>
    </property>
    <property>
      <name>dfs.nameservices</name>
      <value>mycluster</value>
    </property>
    <property>
      <name>dfs.ha.namenodes.mycluster</name>
      <value>nn1,nn2</value>
    </property>
    <property>
      <name>dfs.namenode.rpc-address.mycluster.nn1</name>
      <value>node1:8020</value>
    </property>
    <property>
      <name>dfs.namenode.rpc-address.mycluster.nn2</name>
      <value>node2:8020</value>
    </property>
    <property>
      <name>dfs.namenode.http-address.mycluster.nn1</name>
      <value>node1:50070</value>
    </property> 
    <property>
      <name>dfs.namenode.http-address.mycluster.nn2</name>
      <value>node2:50070</value>
    </property>
    <property>
      <name>dfs.namenode.shared.edits.dir</name>
      <value>qjournal://node1:8485;node2:8485;node3:8485/mycluster</value>
    </property>
    <property>
      <name>dfs.client.failover.proxy.provider.mycluster</name>
<value>org.apache.hadoop.hdfs.server.namenode.ha.ConfiguredFailoverProxyProvider</value>
    </property>
    <property>
      <name>dfs.ha.fencing.methods</name>
      <value>sshfence</value>
    </property>
    <property>
      <name>dfs.ha.fencing.ssh.private-key-files</name>
      <value>/root/.ssh/id_dsa</value>
    </property>
    <property>
       <name>dfs.ha.automatic-failover.enabled</name>
       <value>true</value>
     </property>
</configuration>

6.配置core-site.xml文件

<?xml version="1.0" encoding="UTF-8"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<!--
  Licensed under the Apache License, Version 2.0 (the "License");
  you may not use this file except in compliance with the License.
  You may obtain a copy of the License at

    http://www.apache.org/licenses/LICENSE-2.0

  Unless required by applicable law or agreed to in writing, software
  distributed under the License is distributed on an "AS IS" BASIS,
  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
  See the License for the specific language governing permissions and
  limitations under the License. See accompanying LICENSE file.
-->

<!-- Put site-specific property overrides in this file. -->

<configuration>
    <property>
        <name>fs.defaultFS</name>
        <value>hdfs://mycluster</value>
    </property>
    <property>
        <name>hadoop.tmp.dir</name>
        <value>/var/hadoop-2.6/ha</value>
    </property>
<property>
   <name>ha.zookeeper.quorum</name>
   <value>node2:2181,node3:2181,node4:2181</value>
 </property>
</configuration>

7.启动zookeeper集群

避免出现未知的问题，先查看一下部署zookeeper的节点上是否有zookeeper的进程，如果有的话，kill掉zookeeper进程。
在节点2、3、4上执行命令：

zkServer.sh start

8.启动JournalNode进程

在节点1、2、3上执行命令：

hadoop-daemon.sh start journalnode

9.格式化namenode，并启动hdfs集群

在节点1上执行命令：

hdfs namenode –format
hadoop-deamon.sh start namenode

通知节点2作为Standby的namenode，执行命令：

hdfs namenode –bootstrapStandby

启动hdfs集群，在节点1上执行命令：

start-dfs.sh

10.启动zkfc

先格式化zkfc，在节点1上执行命令：

hdfs zkfc -formatZK

启动zkfc，需要在所有namenode节点启动zkfc，执行以下命令：

hadoop-daemon.sh start zkfc

至此，一个高可用的hdfs集群就搭建完毕了！