在《Hadoop-2.7.3集群(HDFS)搭建》中,整理了如何搭建分布式的hdfs,但存在单点问题,如果NameNode挂了,整个集群就处于不可用的状态。这里记录了在之前基础上如何搭建HA HDFS。
1、增加了一台机器,并修改了host。
127.0.0.1 localhost
127.0.1.1 ubuntu
192.168.42.132 chan-takchi-03
192.168.42.131 chan-takchi-02
192.168.42.130 chan-takchi-01
192.168.42.129 chan-takchi
2、机器角色的划分
192.168.42.129 namenode
192.168.42.130 datanode journalnode
192.168.42.131 datanode journalnode
192.168.42.132 namenode(standby) journalnode
3、修改hdfs-site.xml
<configuration>
<property>
<name>dfs.name.dir</name>
<value>/usr/local/hdfs/name</value>
</property>
<property>
<name>dfs.data.dir</name>
<value>/usr/local/hdfs/data</value>
</property>
<property>
<name>dfs.nameservices</name>
<value>takchi-cluster</value>
</property>
<property>
<name>dfs.ha.namenodes.takchi-cluster</name>
<value>nn1,nn2</value>
</property>
<property>
<name>dfs.namenode.rpc-address.takchi-cluster.nn1</name>
<value>chan-takchi:9000</value>
</property>
<property>
<name>dfs.namenode.rpc-address.takchi-cluster.nn2</name>
<value>chan-takchi-03:9000</value>
</property>
<property>
<name>dfs.namenode.http-address.takchi-cluster.nn1</name>
<value>chan-takchi:50070</value>
</property>
<property>
<name>dfs.namenode.http-address.takchi-cluster.nn2</name>
<value>chan-takchi-03:50070</value>
</property>
<property>
<name>dfs.namenode.shared.edits.dir</name>
<value>qjournal://chan-takchi-03:8485;chan-takchi-01:8485;chan-takchi-02:8485/takchi-cluster</value>
</property>
<property>
<name>dfs.journalnode.edits.dir</name>
<value>/usr/local/hdfs/journal</value>
</property>
<property>
<name>dfs.client.failover.proxy.provider.takchi-cluster</name>
<value>org.apache.hadoop.hdfs.server.namenode.ha.ConfiguredFailoverProxyProvider</value>
</property>
<property>
<name>dfs.ha.fencing.methods</name>
<value>sshfence</value>
</property>
<property>
<name>dfs.ha.fencing.ssh.private-key-files</name>
<value>/home/takchi/.ssh/id_rsa</value>
</property>
</configuration>
4、修改core-site.xml
<configuration>
<property>
<name>fs.default.name</name>
<value>hdfs://takchi-cluster</value>
</property>
<property>
<name>dfs.replication</name>
<value>1</value>
</property>
<property>
<name>hadoop.tmp.dir</name>
<value>/tmp/hadoop</value>
</property>
</configuration>
5、先启动journalnode,格式化namenode时需要连接到journalnode。
sbin/hadoop-daemon.sh start journalnode
6、在设想成为active的namenode上,格式化并启动namenode。
bin/hdfs namenode -format
sbin/hadoop-daemon.sh start namenode
7、格式化standby的namenode并启动,这里的格式化是通过journalnode来同步上一个namenode的格式化数据。
bin/hdfs namenode -bootstrapStandby
sbin/hadoop-daemon.sh start namenode
8、启动datanode。
sbin/hadoop-daemons.sh start datanode
9、至此,可以在两个namenode的50070端口看到HDFS集群已经跑起来了,但都处于standby的状态,无法进行读写。
10、指定active的namenode。
bin/hdfs haadmin -transitionToActive nn1
bin/hdfs haadmin -transitionToActive --forceactive nn1
这样,HA HDFS集群处于了可用状态,当发生故障时便可参考步骤7、10,快速恢复节点至正常状态。美中不足的是,这需要手动来完成,但通过配合zookeeper集群可做到自动Failover。
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
在以上基础上,添加自动Failover功能。
11、先关闭dfs,搭建zookeeper集群,并启动server.2、server.3、server.4
# The number of milliseconds of each tick
tickTime=2000
# The number of ticks that the initial
# synchronization phase can take
initLimit=10
# The number of ticks that can pass between
# sending a request and getting an acknowledgement
syncLimit=5
# the directory where the snapshot is stored.
dataDir=/usr/apps/data/zookeeper
# the port at which the clients will connect
clientPort=2181
server.1=192.168.42.129:2888:3888
server.2=192.168.42.130:2888:3888
server.3=192.168.42.131:2888:3888
server.4=192.168.42.132:2888:3888
12、在hdfs-site.xml里添加一下内容。
<property>
<name>dfs.ha.automatic-failover.enabled</name>
<value>true</value>
</property>
<property>
<name>ha.zookeeper.quorum</name>
<value>chan-takchi-01:2181,chan-takchi-02:2181,chan-takchi-03:2181</value>
</property>
13、格式化zookeeper集群,目的是在ZooKeeper集群上建立HA的相应节点。
bin/hdfs zkfc -formatZK
14、重新启动dfs。
sbin/start-dfs.sh
一般情况下会自行启动相应的程序,命令跑完之后jps看一下是否有跑起来,没有则手工启动。
Starting namenodes on [chan-takchi chan-takchi-03]
chan-takchi-03: starting namenode, logging to /home/takchi/Bigdata/hadoop-2.7.3/logs/hadoop-takchi-namenode-ubuntu.out
chan-takchi: starting namenode, logging to /home/takchi/Bigdata/hadoop-2.7.3/logs/hadoop-takchi-namenode-ubuntu.out
chan-takchi-01: starting datanode, logging to /home/takchi/Bigdata/hadoop-2.7.3/logs/hadoop-takchi-datanode-ubuntu.out
chan-takchi-02: starting datanode, logging to /home/takchi/Bigdata/hadoop-2.7.3/logs/hadoop-takchi-datanode-ubuntu.out
Starting journal nodes [chan-takchi-03 chan-takchi-01 chan-takchi-02]
chan-takchi-03: starting journalnode, logging to /home/takchi/Bigdata/hadoop-2.7.3/logs/hadoop-takchi-journalnode-ubuntu.out
chan-takchi-02: starting journalnode, logging to /home/takchi/Bigdata/hadoop-2.7.3/logs/hadoop-takchi-journalnode-ubuntu.out
chan-takchi-01: starting journalnode, logging to /home/takchi/Bigdata/hadoop-2.7.3/logs/hadoop-takchi-journalnode-ubuntu.out
Starting ZK Failover Controllers on NN hosts [chan-takchi chan-takchi-03]
chan-takchi: starting zkfc, logging to /home/takchi/Bigdata/hadoop-2.7.3/logs/hadoop-takchi-zkfc-ubuntu.out
chan-takchi-03: starting zkfc, logging to /home/takchi/Bigdata/hadoop-2.7.3/logs/hadoop-takchi-zkfc-ubuntu.out
15、至此,自动Failover的HA HDFS集群已经搭建完成,把active的namenode节点关掉,看看standby的namenode有没有变active吧?!
sbin/hadoop-daemon.sh stop namenode