文章目录
概述
集群规划
主机名 | 安装的软件 | 运行的进程 |
---|---|---|
node1 | jdk、hadoop、zookeeper | QuorumPeerMain、NameNode(主)、DFSZKFailoverController |
node2 | jdk、hadoop、zookeeper | QuorumPeerMain、DataNode、NodeManager、JournalNode、NameNode、DFSZKFailoverController |
node3 | jdk、hadoop、zookeeper | QuorumPeerMain、DataNode、NodeManager、JournalNode、ResourceManager |
node4 | jdk、hadoop | DataNode、NodeManager、JournalNode、ResourceManager |
在hadoop2.0中通常由两个NameNode组成,一个处于active状态,另一个处于standby状态。Active NameNode对外提供服务,而Standby NameNode则不对外提供服务,仅同步active namenode的状态,以便能够在它失败时快速进行切换。
hadoop2.0官方提供了两种HDFS HA的解决方案,一种是NFS,另一种是QJM。这里我们使用简单的QJM。在该方案中,主备NameNode之间通过一组JournalNode同步元数据信息,一条数据只要成功写入多数JournalNode即认为写入成功。通常配置奇数个JournalNode
这里还配置了一个zookeeper集群,用于ZKFC(DFSZKFailoverController)故障转移,当Active NameNode挂掉了,会自动切换Standby NameNode为standby状态
安装ZK
安装
按照规划,我们再node1,node2,node3三台机器上安装ZK。
下面以node1的配置为例:
下载zookeeper-3.4.10.tar.gz
,上传至node1的/usr/java
目录下,并且解压。
然后进入/usr/java/zookeeper-3.4.10/conf
,执行cp zoo_sample.cfg zoo.cfg
命令创建配置文件。
编辑zoo.cfg
,设置数据及日志存放路径:
//设置数据及日志存放位置
dataDir=/opt/zookeeper/data
dataLogDir=/opt/zookeeper/logs
//省略其他内容。在zoo.cfg最末尾添加ZK集群信息
server.1=node1:2888:3888
server.2=node2:2888:3888
server.3=node3:2888:3888
server.1=node1:2888:3888
格式说明:server.myid=host:port1:port2
。
其中myid是zk服务器id,port1端口号是zookeeper服务之间通信的端口 ,port2 是zookeeper 与其他应用程序通信的端口 。
在/opt目录下创建相应的文件夹。
然后再/opt/zookeeper/data
路径下,创建myid
文件,文件内容为服务器id。
node1的myid文件内容为:
1
至此,node1的ZK部署完成。使用scp将/usr/java
目录下的ZK完整拷贝到node2和node3上去。
[root@localhost java]# scp -r zookeeper-3.4.10 root@node2:/usr/java
[root@localhost java]# scp -r zookeeper-3.4.10 root@node3:/usr/java
然后再node2和node3的opt
目录下也同样创建zookeeper目录,并且里面创建data和logs文件夹。同样需要在data目录下创建myid文件。这里注意,node2的myid文件的内容是2。node3的myid的文件内容是3。
验证
- 三台服务器均启动ZK服务:
[root@node1 zookeeper-3.4.10]# ./bin/zkServer.sh start conf/zoo.cfg
ZooKeeper JMX enabled by default
Using config: conf/zoo.cfg
Starting zookeeper ... STARTED
[root@node1 zookeeper-3.4.10]# jps
21589 QuorumPeerMain
21614 Jps
使用jps查看java进程,可以看到ZK的java进程名称是QuorumPeerMain
。
查看三台服务器的ZK状态
[root@node1 zookeeper-3.4.10]# ./bin/zkServer.sh status
ZooKeeper JMX enabled by default
Using config: /usr/java/zookeeper-3.4.10/bin/../conf/zoo.cfg
Mode: follower
[root@localhost zookeeper-3.4.10]# ./bin/zkServer.sh status
ZooKeeper JMX enabled by default
Using config: /usr/java/zookeeper-3.4.10/bin/../conf/zoo.cfg
Mode: leader
[root@localhost zookeeper-3.4.10]# ./bin/zkServer.sh status
ZooKeeper JMX enabled by default
Using config: /usr/java/zookeeper-3.4.10/bin/../conf/zoo.cfg
Mode: follower
可以看到node2是leader。
我们把node2的zk停掉,
[root@localhost zookeeper-3.4.10]# ./bin/zkServer.sh stop
ZooKeeper JMX enabled by default
Using config: /usr/java/zookeeper-3.4.10/bin/../conf/zoo.cfg
Stopping zookeeper ... STOPPED
再次检查服务器状态
[root@node1 zookeeper-3.4.10]# ./bin/zkServer.sh status
ZooKeeper JMX enabled by default
Using config: /usr/java/zookeeper-3.4.10/bin/../conf/zoo.cfg
Mode: follower
[root@localhost zookeeper-3.4.10]# ./bin/zkServer.sh status
ZooKeeper JMX enabled by default
Using config: /usr/java/zookeeper-3.4.10/bin/../conf/zoo.cfg
Mode: leader
可以看到node3成为了leader
把node2的zk再次开启服务,会发现他会成为follower。
[root@localhost zookeeper-3.4.10]# ./bin/zkServer.sh start conf/zoo.cfg
ZooKeeper JMX enabled by default
Using config: conf/zoo.cfg
Starting zookeeper ... STARTED
[root@localhost zookeeper-3.4.10]# ./bin/zkServer.sh status
ZooKeeper JMX enabled by default
Using config: /usr/java/zookeeper-3.4.10/bin/../conf/zoo.cfg
Mode: follower
验证zk安装成功。
安装配置Hadoop集群
这里首先安装部署在node1上,然后,scp拷贝到其他机器上去。
配置Hadoop环境变量
上传hadoop-2.8.5.tar.gz
到/usr/java
目录下,并且解压。
配置环境变量vim /etc/profile
export JAVA_HOME=/usr/java/jdk1.6.0_45
export HADOOP_HOME=/usr/java/hadoop-2.8.5
export PATH=$PATH:$JAVA_HOME/bin:$HADOOP_HOME/bin
注意:环境变量需要在每台机器都进行配置。
下面几个配置文件都在/usr/java/hadoop-2.8.5/etc/hadoop/
目录下,进入该目录。
hadoop-env.sh
hadoo-env.sh文件中配置java_home
export JAVA_HOME=/usr/java/jdk1.8.0_131
core-site.xml
<configuration>
<!-- 指定hdfs的nameservice为ns1 -->
<property>
<name>fs.defaultFS</name>
<value>hdfs://ns1</value>
</property>
<!-- 指定hadoop临时目录 -->
<property>
<name>hadoop.tmp.dir</name>
<value>/opt/hadoop-2.8.5/tmp</value>
</property>
<!-- 指定zookeeper地址 -->
<property>
<name>ha.zookeeper.quorum</name>
<value>node1:2181,node2:2181,node3:2181</value>
</property>
</configuration>
hdfs-site.xml
<configuration>
<!--指定hdfs的nameservice为ns1,需要和core-site.xml中的保持一致 -->
<property>
<name>dfs.nameservices</name>
<value>ns1</value>
</property>
<!-- ns1下面有两个NameNode,分别是nn1,nn2 -->
<property>
<name>dfs.ha.namenodes.ns1</name>
<value>nn1,nn2</value>
</property>
<!-- nn1的RPC通信地址 -->
<property>
<name>dfs.namenode.rpc-address.ns1.nn1</name>
<value>node1:9000</value>
</property>
<!-- nn1的http通信地址 -->
<property>
<name>dfs.namenode.http-address.ns1.nn1</name>
<value>node1:50070</value>
</property>
<!-- nn2的RPC通信地址 -->
<property>
<name>dfs.namenode.rpc-address.ns1.nn2</name>
<value>node2:9000</value>
</property>
<!-- nn2的http通信地址 -->
<property>
<name>dfs.namenode.http-address.ns1.nn2</name>
<value>node2:50070</value>
</property>
<!-- 指定NameNode的元数据在JournalNode上的存放位置 -->
<property>
<name>dfs.namenode.shared.edits.dir</name>
<value>qjournal://node2:8485;node3:8485;node4:8485/ns1</value>
</property>
<!-- 指定JournalNode在本地磁盘存放数据的位置 -->
<property>
<name>dfs.journalnode.edits.dir</name>
<value>/opt/hadoop-2.8.5/journal</value>
</property>
<!-- 开启NameNode失败自动切换 -->
<property>
<name>dfs.ha.automatic-failover.enabled</name>
<value>true</value>
</property>
<!-- 配置失败自动切换实现方式 -->
<property>
<name>dfs.client.failover.proxy.provider.ns1</name>
<value>org.apache.hadoop.hdfs.server.namenode.ha.ConfiguredFailoverProxyProvider</value>
</property>
<!-- 配置隔离机制 -->
<property>
<name>dfs.ha.fencing.methods</name>
<value>sshfence</value>
</property>
<!-- 使用隔离机制时需要ssh免登陆 -->
<property>
<name>dfs.ha.fencing.ssh.private-key-files</name>
<value>/root/.ssh/id_dsa</value>
</property>
</configuration>
上面dfs.ha.fencing.ssh.private-key-files
第一次配置值的时候写错了,写成了/root/.ssh/id_rsa
,导致出错,具体错误文章最后分析。
yarn-site.xml
<configuration>
<!-- Site specific YARN configuration properties -->
<!-- 开启RM高可靠 -->
<property>
<name>yarn.resourcemanager.ha.enabled</name>
<value>true</value>
</property>
<!-- 指定RM的cluster id -->
<property>
<name>yarn.resourcemanager.cluster-id</name>
<value>yrc</value>
</property>
<!-- 指定RM的名字 -->
<property>
<name>yarn.resourcemanager.ha.rm-ids</name>
<value>rm1,rm2</value>
</property>
<!-- 分别指定RM的地址 -->
<property>
<name>yarn.resourcemanager.hostname.rm1</name>
<value>node3</value>
</property>
<property>
<name>yarn.resourcemanager.hostname.rm2</name>
<value>node4</value>
</property>
<!-- 指定zk集群地址 -->
<property>
<name>yarn.resourcemanager.zk-address</name>
<value>node1:2181,node2:2181,node3:2181</value>
</property>
<property>
<name>yarn.nodemanager.aux-services</name>
<value>mapreduce_shuffle</value>
</property>
</configuration>
mapred-site.xml
<configuration>
<!-- 指定mr框架为yarn方式 -->
<property>
<name>mapreduce.framework.name</name>
<value>yarn</value>
</property>
</configuration>
slaves
按照规划,数据我们放在2,3,4这几台机器上,所以修改slaves配置文件的内容。
[root@localhost hadoop]# cat slaves
node2
node3
node4
opt文件下创建对应文件夹
在opt
目录下创建hadoop-2.8.5文件夹,并且在hadoop-2.8.5文件夹下创建两个文件夹:journal
和tmp
。对应上面配置文件中指定的路径。
将Hadoop拷贝到其他几台机器
scp -r hadoop-2.8.5/ root@node2:/usr/java/
scp -r hadoop-2.8.5/ root@node3:/usr/java/
scp -r hadoop-2.8.5/ root@node4:/usr/java/
启动Hadoop
启动ZK
参考上面安装ZK的步骤
启动journalnode
根据规划,在node2,3,4上启动journalnode。
[root@localhost data]# cd /usr/java/hadoop-2.8.5/sbin
[root@localhost sbin]# ./hadoop-daemon.sh start journalnode
The authenticity of host 'localhost (127.0.0.1)' can't be established.
ECDSA key fingerprint is 46:f8:c5:85:40:b6:0d:0a:66:3f:2e:25:65:5d:f4:7b.
Are you sure you want to continue connecting (yes/no)? yes
localhost: Warning: Permanently added 'localhost' (ECDSA) to the list of known hosts.
localhost: starting journalnode, logging to /usr/java/hadoop-2.8.5/logs/hadoop-root-journalnode-node4.out
[root@localhost sbin]# jps
32848 Jps
32713 JournalNode
格式化namenode
node1和node2计划启动namenode,node1作为主,在node1上格式化:
cd /usr/java/hadoop-2.8.5/bin/
./hdfs namenode -format
格式化zk
在node1上执行命令./hdfs zkfc -formatZK
。
执行完成后,会在zookeeper 上创建一个目录,查看是否创建成功:
进入cd /usr/java/zookeeper-3.4.10/bin/
目录,执行./zkCli.sh
客户端连接ZK。在ZK客户端的shell命令行查看:
[zk: localhost:2181(CONNECTED) 1] ls /
[zookeeper, hadoop-ha]
出现hadoop-ha即表示成功。
启动主namenode
node1是主namenode.
[root@node1 sbin]# ./hadoop-daemon.sh start namenode
starting namenode, logging to /usr/java/hadoop-2.8.5/logs/hadoop-root-namenode-node1.out
[root@node1 sbin]# jps
22196 QuorumPeerMain
22876 Jps
22799 NameNode
备用NN 同步主NN信息
node2是从node1的namenode同步数据。
在node2上执行./hdfs namenode -bootstrapStandby
:
[root@localhost bin]# ./hdfs namenode -bootstrapStandby
19/07/11 02:35:16 INFO namenode.NameNode: STARTUP_MSG:
/************************************************************
STARTUP_MSG: Starting NameNode
STARTUP_MSG: user = root
STARTUP_MSG: host = node2/192.168.46.130
STARTUP_MSG: args = [-bootstrapStandby]
STARTUP_MSG: version = 2.8.5
//省略部分日志
19/07/11 02:35:18 INFO common.Storage: Storage directory /opt/hadoop-2.8.5/name has been successfully formatted.
//省略部分日志
启动hadoop集群
由于上面我们启动了namenode,这里先将其关闭:在node1上执行./stop-dfs.sh
即可。然后重新启动整个集群。
在主节点NameNode(node1)启动集群./start-dfs.sh
:
改命令会在主和备NN上启动ZKFC守护进程
[root@node1 sbin]# ./start-dfs.sh
Starting namenodes on [node1 node2]
node1: namenode running as process 25857. Stop it first.
node2: starting namenode, logging to /usr/java/hadoop-2.8.5/logs/hadoop-root-namenode-node2.out
node3: starting datanode, logging to /usr/java/hadoop-2.8.5/logs/hadoop-root-datanode-node3.out
node2: starting datanode, logging to /usr/java/hadoop-2.8.5/logs/hadoop-root-datanode-node2.out
node4: starting datanode, logging to /usr/java/hadoop-2.8.5/logs/hadoop-root-datanode-node4.out
Starting journal nodes [node2 node3 node4]
node3: starting journalnode, logging to /usr/java/hadoop-2.8.5/logs/hadoop-root-journalnode-node3.out
node2: starting journalnode, logging to /usr/java/hadoop-2.8.5/logs/hadoop-root-journalnode-node2.out
node4: starting journalnode, logging to /usr/java/hadoop-2.8.5/logs/hadoop-root-journalnode-node4.out
Starting ZK Failover Controllers on NN hosts [node1 node2]
node1: starting zkfc, logging to /usr/java/hadoop-2.8.5/logs/hadoop-root-zkfc-node1.out
node2: starting zkfc, logging to /usr/java/hadoop-2.8.5/logs/hadoop-root-zkfc-node2.out
[root@node1 sbin]# jps
25857 NameNode
26258 DFSZKFailoverController
22196 QuorumPeerMain
26326 Jps
此时node2机器上也有DFSZKFailoverController服务:
[root@localhost hadoop]# jps
34369 NameNode
34548 JournalNode
31765 QuorumPeerMain
34661 DFSZKFailoverController
34439 DataNode
34718 Jps
查看两个namenode的状态
一个是active,一个是standby:
启动yarn
我们再node3和ndoe4上分别启动yarn:
[root@localhost sbin]# ./start-yarn.sh
starting yarn daemons
starting resourcemanager, logging to /usr/java/hadoop-2.8.5/logs/yarn-root-resourcemanager-node3.out
localhost: starting nodemanager, logging to /usr/java/hadoop-2.8.5/logs/yarn-root-nodemanager-node3.out
[root@localhost sbin]# jps
28128 Jps
27537 JournalNode
27766 NodeManager
27431 DataNode
27658 ResourceManager
25150 QuorumPeerMai
可以看到启动成功后在node3和node4上多了ResourceManager
和NodeManager
进程。
至此,所有服务启动完成。与当初规划的完全一致
[root@node1 sbin]# jps
25857 NameNode
26258 DFSZKFailoverController
22196 QuorumPeerMain
26363 Jps
node2:
[root@localhost hadoop]# jps
34960 Jps
34369 NameNode
34548 JournalNode
31765 QuorumPeerMain
34661 DFSZKFailoverController
34439 DataNode
34778 NodeManager
node3:
[root@localhost sbin]# jps
27537 JournalNode
27766 NodeManager
27431 DataNode
27658 ResourceManager
28140 Jps
25150 QuorumPeerMain
node4:
[root@localhost sbin]# jps
37139 Jps
36469 JournalNode
36249 DataNode
36681 NodeManager
36924 ResourceManager
验证HDFS HA
进入node1的/usr/java/hadoop-2.8.5/bin
目录,执行:
[root@node1 java]# cd /usr/java/hadoop-2.8.5/bin/
[root@node1 bin]# ./hadoop fs -mkdir /test1
[root@node1 bin]# ./hadoop fs -ls /
Found 1 items
drwxr-xr-x - root supergroup 0 2019-07-11 03:15 /test1
[root@node1 bin]# ./hadoop fs -put /usr/java/hadoop-2.8.5.tar.gz /test1
[root@node1 bin]# ./hadoop fs -ls /test1
Found 1 items
-rw-r--r-- 3 root supergroup 246543928 2019-07-11 03:18 /test1/hadoop-2.8.5.tar.gz
上面上传了一个240M左右的文件。下面看看他在Hadoop的实际存储情况。
可以看到这个文件被分成了两个块,hadoop-2.8.5.tar.gz大小为140M左右,所以会被分为2个块(默认一个快大小为128M)。每个块的副本分布在ndoe2,node3,node4机器上。
查看yarn控制台
我们的yarn部署在ndoe3上,直接访问:
namendoe高可用测试
上面配置了namenode的主从模式,我们将active的node1的nemenode停掉,看看结果如何。
[root@node1 bin]# jps
25857 NameNode
26258 DFSZKFailoverController
22196 QuorumPeerMain
26805 Jps
[root@node1 bin]# kill -9 25857
[root@node1 bin]# jps
26258 DFSZKFailoverController
22196 QuorumPeerMain
26815 Jps
node2还是standby状态,没有变化,怎么回事?
查看node2机器上的/usr/java/hadoop-2.8.5/logs/hadoop-root-zkfc-node2.log
文件:
2019-07-11 03:35:15,423 WARN org.apache.hadoop.ha.SshFenceByTcpPort: Unable to create SSH session
com.jcraft.jsch.JSchException: java.io.FileNotFoundException: /root/.ssh/id_rsa (No such file or directory)
at com.jcraft.jsch.KeyPair.load(KeyPair.java:543)
at com.jcraft.jsch.IdentityFile.newInstance(IdentityFile.java:40)
at com.jcraft.jsch.JSch.addIdentity(JSch.java:407)
at com.jcraft.jsch.JSch.addIdentity(JSch.java:367)
at org.apache.hadoop.ha.SshFenceByTcpPort.createSession(SshFenceByTcpPort.java:122)
at org.apache.hadoop.ha.SshFenceByTcpPort.tryFence(SshFenceByTcpPort.java:91)
at org.apache.hadoop.ha.NodeFencer.fence(NodeFencer.java:98)
at org.apache.hadoop.ha.ZKFailoverController.doFence(ZKFailoverController.java:536)
at org.apache.hadoop.ha.ZKFailoverController.fenceOldActive(ZKFailoverController.java:509)
at org.apache.hadoop.ha.ZKFailoverController.access$1100(ZKFailoverController.java:61)
at org.apache.hadoop.ha.ZKFailoverController$ElectorCallbacks.fenceOldActive(ZKFailoverController.java:895)
at org.apache.hadoop.ha.ActiveStandbyElector.fenceOldActive(ActiveStandbyElector.java:985)
at org.apache.hadoop.ha.ActiveStandbyElector.becomeActive(ActiveStandbyElector.java:882)
at org.apache.hadoop.ha.ActiveStandbyElector.processResult(ActiveStandbyElector.java:467)
at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:599)
at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:498)
Caused by: java.io.FileNotFoundException: /root/.ssh/id_rsa (No such file or directory)
at java.io.FileInputStream.open0(Native Method)
at java.io.FileInputStream.open(FileInputStream.java:195)
at java.io.FileInputStream.<init>(FileInputStream.java:138)
at java.io.FileInputStream.<init>(FileInputStream.java:93)
at com.jcraft.jsch.Util.fromFile(Util.java:508)
at com.jcraft.jsch.KeyPair.load(KeyPair.java:540)
... 15 more
2019-07-11 03:35:15,423 WARN org.apache.hadoop.ha.NodeFencer: Fencing method org.apache.hadoop.ha.SshFenceByTcpPort(null) was unsuccessful.
2019-07-11 03:35:15,423 ERROR org.apache.hadoop.ha.NodeFencer: Unable to fence service by any configured method.
2019-07-11 03:35:15,423 WARN org.apache.hadoop.ha.ActiveStandbyElector: Exception handling the winning of election
java.lang.RuntimeException: Unable to fence NameNode at node1/192.168.46.128:9000
at org.apache.hadoop.ha.ZKFailoverController.doFence(ZKFailoverController.java:537)
at org.apache.hadoop.ha.ZKFailoverController.fenceOldActive(ZKFailoverController.java:509)
at org.apache.hadoop.ha.ZKFailoverController.access$1100(ZKFailoverController.java:61)
at org.apache.hadoop.ha.ZKFailoverController$ElectorCallbacks.fenceOldActive(ZKFailoverController.java:895)
at org.apache.hadoop.ha.ActiveStandbyElector.fenceOldActive(ActiveStandbyElector.java:985)
at org.apache.hadoop.ha.ActiveStandbyElector.becomeActive(ActiveStandbyElector.java:882)
at org.apache.hadoop.ha.ActiveStandbyElector.processResult(ActiveStandbyElector.java:467)
at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:599)
at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:498)
2019-07-11 03:35:15,425 INFO org.apache.hadoop.ha.ActiveStandbyElector: Trying to re-establish ZK session
2019-07-11 03:35:15,429 INFO org.apache.zookeeper.ZooKeeper: Session: 0x26be03e0c220035 closed
很明显,找不到/root/.ssh/id_rsa
文件。
[root@localhost logs]# ls ~/.ssh/
authorized_keys id_dsa id_dsa.pub known_hosts
很明显,文件名应该是id_dsa 。配置错了,修改一下hdfs-site.xml
文件的配置:
<property>
<name>dfs.ha.fencing.ssh.private-key-files</name>
<value>/root/.ssh/id_dsa</value>
</property>
注意:所有机器均需要修改。
然后我们重新启动Hadoop集群
先停止所有服务,在node1上执行
[root@node1 sbin]# ./stop-all.sh
This script is Deprecated. Instead use stop-dfs.sh and stop-yarn.sh
Stopping namenodes on [node1 node2]
node1: no namenode to stop
node2: stopping namenode
node2: stopping datanode
node4: stopping datanode
node3: stopping datanode
Stopping journal nodes [node2 node3 node4]
node2: stopping journalnode
node3: stopping journalnode
node4: stopping journalnode
Stopping ZK Failover Controllers on NN hosts [node1 node2]
node2: stopping zkfc
node1: stopping zkfc
stopping yarn daemons
no resourcemanager to stop
node3: stopping nodemanager
node2: stopping nodemanager
node4: stopping nodemanager
no proxyserver to stop
使用jps检查四台机器的进程是否全部杀掉,。
接下来我们使用start-all.sh
来快速启动所有服务。他会一次启动 namenode、datanode、journalnode、zkfc、resourcemanager、nodemanager服务。
[root@node1 sbin]# ./start-all.sh
This script is Deprecated. Instead use start-dfs.sh and start-yarn.sh
Starting namenodes on [node1 node2]
node1: starting namenode, logging to /usr/java/hadoop-2.8.5/logs/hadoop-root-namenode-node1.out
node2: starting namenode, logging to /usr/java/hadoop-2.8.5/logs/hadoop-root-namenode-node2.out
node3: starting datanode, logging to /usr/java/hadoop-2.8.5/logs/hadoop-root-datanode-node3.out
node2: starting datanode, logging to /usr/java/hadoop-2.8.5/logs/hadoop-root-datanode-node2.out
node4: starting datanode, logging to /usr/java/hadoop-2.8.5/logs/hadoop-root-datanode-node4.out
Starting journal nodes [node2 node3 node4]
node2: starting journalnode, logging to /usr/java/hadoop-2.8.5/logs/hadoop-root-journalnode-node2.out
node3: starting journalnode, logging to /usr/java/hadoop-2.8.5/logs/hadoop-root-journalnode-node3.out
node4: starting journalnode, logging to /usr/java/hadoop-2.8.5/logs/hadoop-root-journalnode-node4.out
Starting ZK Failover Controllers on NN hosts [node1 node2]
node2: starting zkfc, logging to /usr/java/hadoop-2.8.5/logs/hadoop-root-zkfc-node2.out
node1: starting zkfc, logging to /usr/java/hadoop-2.8.5/logs/hadoop-root-zkfc-node1.out
starting yarn daemons
starting resourcemanager, logging to /usr/java/hadoop-2.8.5/logs/yarn-root-resourcemanager-node1.out
node2: starting nodemanager, logging to /usr/java/hadoop-2.8.5/logs/yarn-root-nodemanager-node2.out
node3: starting nodemanager, logging to /usr/java/hadoop-2.8.5/logs/yarn-root-nodemanager-node3.out
node4: starting nodemanager, logging to /usr/java/hadoop-2.8.5/logs/yarn-root-nodemanager-node4.out
如此一来,再次kill掉node1机器的namenode进程,可以看到node2的namenode自动成为active状态了。
再次启动node1的namenode服务,可以看到node1的namenode服务变成了standby了。
namenode高可用验证OK。