Hadoop 2.8.5 完全分布式HA高可用安装（二）--环境搭建

最新推荐文章于 2022-08-21 11:16:53 发布

快乐崇拜234

最新推荐文章于 2022-08-21 11:16:53 发布

阅读量2k

点赞数 2

分类专栏： hadoop 文章标签： Hadoop

本文链接：https://blog.csdn.net/liubenlong007/article/details/95483598

版权

hadoop 专栏收录该内容

4 篇文章 0 订阅

订阅专栏

概述

集群规划

主机名	安装的软件	运行的进程
node1	jdk、hadoop、zookeeper	QuorumPeerMain、NameNode（主）、DFSZKFailoverController
node2	jdk、hadoop、zookeeper	QuorumPeerMain、DataNode、NodeManager、JournalNode、NameNode、DFSZKFailoverController
node3	jdk、hadoop、zookeeper	QuorumPeerMain、DataNode、NodeManager、JournalNode、ResourceManager
node4	jdk、hadoop	DataNode、NodeManager、JournalNode、ResourceManager

在hadoop2.0中通常由两个NameNode组成，一个处于active状态，另一个处于standby状态。Active NameNode对外提供服务，而Standby NameNode则不对外提供服务，仅同步active namenode的状态，以便能够在它失败时快速进行切换。

hadoop2.0官方提供了两种HDFS HA的解决方案，一种是NFS，另一种是QJM。这里我们使用简单的QJM。在该方案中，主备NameNode之间通过一组JournalNode同步元数据信息，一条数据只要成功写入多数JournalNode即认为写入成功。通常配置奇数个JournalNode

这里还配置了一个zookeeper集群，用于ZKFC（DFSZKFailoverController）故障转移，当Active NameNode挂掉了，会自动切换Standby NameNode为standby状态

安装ZK

安装

按照规划，我们再node1,node2,node3三台机器上安装ZK。

下面以node1的配置为例：

下载zookeeper-3.4.10.tar.gz，上传至node1的/usr/java目录下，并且解压。
然后进入/usr/java/zookeeper-3.4.10/conf，执行cp zoo_sample.cfg zoo.cfg命令创建配置文件。
编辑zoo.cfg，设置数据及日志存放路径：

//设置数据及日志存放位置
dataDir=/opt/zookeeper/data
dataLogDir=/opt/zookeeper/logs

//省略其他内容。在zoo.cfg最末尾添加ZK集群信息
server.1=node1:2888:3888
server.2=node2:2888:3888
server.3=node3:2888:3888

server.1=node1:2888:3888格式说明：server.myid=host:port1:port2。
其中myid是zk服务器id，port1端口号是zookeeper服务之间通信的端口 ,port2 是zookeeper 与其他应用程序通信的端口。

在/opt目录下创建相应的文件夹。
然后再/opt/zookeeper/data路径下，创建myid文件，文件内容为服务器id。

node1的myid文件内容为：

至此，node1的ZK部署完成。使用scp将/usr/java目录下的ZK完整拷贝到node2和node3上去。

[root@localhost java]# scp -r zookeeper-3.4.10 root@node2:/usr/java
[root@localhost java]# scp -r zookeeper-3.4.10 root@node3:/usr/java

然后再node2和node3的opt目录下也同样创建zookeeper目录，并且里面创建data和logs文件夹。同样需要在data目录下创建myid文件。这里注意，node2的myid文件的内容是2。node3的myid的文件内容是3。

验证

三台服务器均启动ZK服务：

[root@node1 zookeeper-3.4.10]# ./bin/zkServer.sh start conf/zoo.cfg 
ZooKeeper JMX enabled by default
Using config: conf/zoo.cfg
Starting zookeeper ... STARTED
[root@node1 zookeeper-3.4.10]# jps
21589 QuorumPeerMain
21614 Jps

使用jps查看java进程，可以看到ZK的java进程名称是QuorumPeerMain。

查看三台服务器的ZK状态

[root@node1 zookeeper-3.4.10]# ./bin/zkServer.sh status
ZooKeeper JMX enabled by default
Using config: /usr/java/zookeeper-3.4.10/bin/../conf/zoo.cfg
Mode: follower

[root@localhost zookeeper-3.4.10]# ./bin/zkServer.sh status
ZooKeeper JMX enabled by default
Using config: /usr/java/zookeeper-3.4.10/bin/../conf/zoo.cfg
Mode: leader

[root@localhost zookeeper-3.4.10]# ./bin/zkServer.sh status
ZooKeeper JMX enabled by default
Using config: /usr/java/zookeeper-3.4.10/bin/../conf/zoo.cfg
Mode: follower

可以看到node2是leader。
我们把node2的zk停掉，

[root@localhost zookeeper-3.4.10]# ./bin/zkServer.sh stop
ZooKeeper JMX enabled by default
Using config: /usr/java/zookeeper-3.4.10/bin/../conf/zoo.cfg
Stopping zookeeper ... STOPPED

再次检查服务器状态

[root@node1 zookeeper-3.4.10]# ./bin/zkServer.sh status
ZooKeeper JMX enabled by default
Using config: /usr/java/zookeeper-3.4.10/bin/../conf/zoo.cfg
Mode: follower


[root@localhost zookeeper-3.4.10]# ./bin/zkServer.sh status
ZooKeeper JMX enabled by default
Using config: /usr/java/zookeeper-3.4.10/bin/../conf/zoo.cfg
Mode: leader

可以看到node3成为了leader

把node2的zk再次开启服务，会发现他会成为follower。

[root@localhost zookeeper-3.4.10]# ./bin/zkServer.sh start conf/zoo.cfg
ZooKeeper JMX enabled by default
Using config: conf/zoo.cfg
Starting zookeeper ... STARTED

[root@localhost zookeeper-3.4.10]# ./bin/zkServer.sh status
ZooKeeper JMX enabled by default
Using config: /usr/java/zookeeper-3.4.10/bin/../conf/zoo.cfg
Mode: follower

验证zk安装成功。

安装配置Hadoop集群

这里首先安装部署在node1上，然后，scp拷贝到其他机器上去。

配置Hadoop环境变量

上传hadoop-2.8.5.tar.gz到/usr/java目录下，并且解压。
配置环境变量vim /etc/profile

export JAVA_HOME=/usr/java/jdk1.6.0_45
export HADOOP_HOME=/usr/java/hadoop-2.8.5
export PATH=$PATH:$JAVA_HOME/bin:$HADOOP_HOME/bin

注意：环境变量需要在每台机器都进行配置。

下面几个配置文件都在/usr/java/hadoop-2.8.5/etc/hadoop/目录下，进入该目录。

hadoop-env.sh

hadoo-env.sh文件中配置java_home

export JAVA_HOME=/usr/java/jdk1.8.0_131

core-site.xml

<configuration>
        <!-- 指定hdfs的nameservice为ns1 -->
    <property>
        <name>fs.defaultFS</name>
        <value>hdfs://ns1</value>
    </property>
        <!-- 指定hadoop临时目录 -->
    <property>
        <name>hadoop.tmp.dir</name>
        <value>/opt/hadoop-2.8.5/tmp</value>
    </property>
        <!-- 指定zookeeper地址 -->
    <property>
        <name>ha.zookeeper.quorum</name>
        <value>node1:2181,node2:2181,node3:2181</value>
    </property>
</configuration>

hdfs-site.xml

<configuration>
	<!--指定hdfs的nameservice为ns1，需要和core-site.xml中的保持一致 -->
	<property>
	    <name>dfs.nameservices</name>
	    <value>ns1</value>
	</property>
	<!-- ns1下面有两个NameNode，分别是nn1，nn2 -->
	<property>
	    <name>dfs.ha.namenodes.ns1</name>
	    <value>nn1,nn2</value>
	</property>
	<!-- nn1的RPC通信地址 -->
	<property>
	    <name>dfs.namenode.rpc-address.ns1.nn1</name>
	    <value>node1:9000</value>
	</property>
	<!-- nn1的http通信地址 -->
	<property>
	    <name>dfs.namenode.http-address.ns1.nn1</name>
	    <value>node1:50070</value>
	</property>
	<!-- nn2的RPC通信地址 -->
	<property>
	    <name>dfs.namenode.rpc-address.ns1.nn2</name>
	    <value>node2:9000</value>
	</property>
	<!-- nn2的http通信地址 -->
	<property>
	    <name>dfs.namenode.http-address.ns1.nn2</name>
	    <value>node2:50070</value>
	</property>
	<!-- 指定NameNode的元数据在JournalNode上的存放位置 -->
	<property>
	    <name>dfs.namenode.shared.edits.dir</name>
	    <value>qjournal://node2:8485;node3:8485;node4:8485/ns1</value>
	</property>
	<!-- 指定JournalNode在本地磁盘存放数据的位置 -->
	<property>
	    <name>dfs.journalnode.edits.dir</name>
	    <value>/opt/hadoop-2.8.5/journal</value>
	</property>
	<!-- 开启NameNode失败自动切换 -->
	<property>
	    <name>dfs.ha.automatic-failover.enabled</name>
	    <value>true</value>
	</property>
	<!-- 配置失败自动切换实现方式 -->
	<property>
	    <name>dfs.client.failover.proxy.provider.ns1</name>
	    <value>org.apache.hadoop.hdfs.server.namenode.ha.ConfiguredFailoverProxyProvider</value>
	</property>
	<!-- 配置隔离机制 -->
	<property>
	    <name>dfs.ha.fencing.methods</name>
	    <value>sshfence</value>
	</property>
	<!-- 使用隔离机制时需要ssh免登陆 -->
	<property>
	    <name>dfs.ha.fencing.ssh.private-key-files</name>
	    <value>/root/.ssh/id_dsa</value>
	</property>
</configuration>

上面dfs.ha.fencing.ssh.private-key-files第一次配置值的时候写错了，写成了/root/.ssh/id_rsa，导致出错，具体错误文章最后分析。

yarn-site.xml

<configuration>

<!-- Site specific YARN configuration properties -->
<!-- 开启RM高可靠 -->
        <property>
            <name>yarn.resourcemanager.ha.enabled</name>
            <value>true</value>
        </property>
        <!-- 指定RM的cluster id -->
        <property>
            <name>yarn.resourcemanager.cluster-id</name>
            <value>yrc</value>
        </property>
        <!-- 指定RM的名字 -->
        <property>
            <name>yarn.resourcemanager.ha.rm-ids</name>
            <value>rm1,rm2</value>
        </property>
        <!-- 分别指定RM的地址 -->
        <property>
            <name>yarn.resourcemanager.hostname.rm1</name>
            <value>node3</value>
        </property>
        <property>
            <name>yarn.resourcemanager.hostname.rm2</name>
            <value>node4</value>
        </property>
        <!-- 指定zk集群地址 -->
        <property>
            <name>yarn.resourcemanager.zk-address</name>
            <value>node1:2181,node2:2181,node3:2181</value>
        </property>
        <property>
            <name>yarn.nodemanager.aux-services</name>
            <value>mapreduce_shuffle</value>
        </property>
</configuration>

mapred-site.xml

<configuration>
<!-- 指定mr框架为yarn方式 -->
    <property>
        <name>mapreduce.framework.name</name>
        <value>yarn</value>
    </property>
</configuration>

slaves

按照规划，数据我们放在2,3，4这几台机器上，所以修改slaves配置文件的内容。

[root@localhost hadoop]# cat slaves 
node2
node3
node4

opt文件下创建对应文件夹

在opt目录下创建hadoop-2.8.5文件夹，并且在hadoop-2.8.5文件夹下创建两个文件夹：journal和tmp。对应上面配置文件中指定的路径。

将Hadoop拷贝到其他几台机器

scp -r hadoop-2.8.5/ root@node2:/usr/java/
scp -r hadoop-2.8.5/ root@node3:/usr/java/
scp -r hadoop-2.8.5/ root@node4:/usr/java/

启动Hadoop

启动ZK

参考上面安装ZK的步骤

启动journalnode

根据规划，在node2,3,4上启动journalnode。

[root@localhost data]# cd /usr/java/hadoop-2.8.5/sbin

[root@localhost sbin]# ./hadoop-daemon.sh start journalnode
The authenticity of host 'localhost (127.0.0.1)' can't be established.
ECDSA key fingerprint is 46:f8:c5:85:40:b6:0d:0a:66:3f:2e:25:65:5d:f4:7b.
Are you sure you want to continue connecting (yes/no)? yes
localhost: Warning: Permanently added 'localhost' (ECDSA) to the list of known hosts.
localhost: starting journalnode, logging to /usr/java/hadoop-2.8.5/logs/hadoop-root-journalnode-node4.out

[root@localhost sbin]# jps
32848 Jps
32713 JournalNode

格式化namenode

node1和node2计划启动namenode，node1作为主，在node1上格式化：

cd /usr/java/hadoop-2.8.5/bin/

./hdfs namenode -format

格式化zk

在node1上执行命令./hdfs zkfc -formatZK。
执行完成后，会在zookeeper 上创建一个目录，查看是否创建成功：
进入cd /usr/java/zookeeper-3.4.10/bin/目录，执行./zkCli.sh客户端连接ZK。在ZK客户端的shell命令行查看：

[zk: localhost:2181(CONNECTED) 1] ls /
[zookeeper, hadoop-ha]

出现hadoop-ha即表示成功。

启动主namenode

node1是主namenode.

[root@node1 sbin]# ./hadoop-daemon.sh start namenode
starting namenode, logging to /usr/java/hadoop-2.8.5/logs/hadoop-root-namenode-node1.out

[root@node1 sbin]# jps
22196 QuorumPeerMain
22876 Jps
22799 NameNode

备用NN 同步主NN信息

node2是从node1的namenode同步数据。

在node2上执行./hdfs namenode -bootstrapStandby：

[root@localhost bin]# ./hdfs namenode -bootstrapStandby
19/07/11 02:35:16 INFO namenode.NameNode: STARTUP_MSG: 
/************************************************************
STARTUP_MSG: Starting NameNode
STARTUP_MSG:   user = root
STARTUP_MSG:   host = node2/192.168.46.130
STARTUP_MSG:   args = [-bootstrapStandby]
STARTUP_MSG:   version = 2.8.5

//省略部分日志

19/07/11 02:35:18 INFO common.Storage: Storage directory /opt/hadoop-2.8.5/name has been successfully formatted.

//省略部分日志

启动hadoop集群

由于上面我们启动了namenode,这里先将其关闭：在node1上执行./stop-dfs.sh即可。然后重新启动整个集群。

在主节点NameNode(node1)启动集群./start-dfs.sh：
改命令会在主和备NN上启动ZKFC守护进程

[root@node1 sbin]# ./start-dfs.sh
Starting namenodes on [node1 node2]
node1: namenode running as process 25857. Stop it first.
node2: starting namenode, logging to /usr/java/hadoop-2.8.5/logs/hadoop-root-namenode-node2.out
node3: starting datanode, logging to /usr/java/hadoop-2.8.5/logs/hadoop-root-datanode-node3.out
node2: starting datanode, logging to /usr/java/hadoop-2.8.5/logs/hadoop-root-datanode-node2.out
node4: starting datanode, logging to /usr/java/hadoop-2.8.5/logs/hadoop-root-datanode-node4.out
Starting journal nodes [node2 node3 node4]
node3: starting journalnode, logging to /usr/java/hadoop-2.8.5/logs/hadoop-root-journalnode-node3.out
node2: starting journalnode, logging to /usr/java/hadoop-2.8.5/logs/hadoop-root-journalnode-node2.out
node4: starting journalnode, logging to /usr/java/hadoop-2.8.5/logs/hadoop-root-journalnode-node4.out
Starting ZK Failover Controllers on NN hosts [node1 node2]
node1: starting zkfc, logging to /usr/java/hadoop-2.8.5/logs/hadoop-root-zkfc-node1.out
node2: starting zkfc, logging to /usr/java/hadoop-2.8.5/logs/hadoop-root-zkfc-node2.out

[root@node1 sbin]# jps
25857 NameNode
26258 DFSZKFailoverController
22196 QuorumPeerMain
26326 Jps

此时node2机器上也有DFSZKFailoverController服务：

[root@localhost hadoop]# jps
34369 NameNode
34548 JournalNode
31765 QuorumPeerMain
34661 DFSZKFailoverController
34439 DataNode
34718 Jps

查看两个namenode的状态

一个是active，一个是standby：
在这里插入图片描述

启动yarn

我们再node3和ndoe4上分别启动yarn：

[root@localhost sbin]# ./start-yarn.sh 
starting yarn daemons
starting resourcemanager, logging to /usr/java/hadoop-2.8.5/logs/yarn-root-resourcemanager-node3.out
localhost: starting nodemanager, logging to /usr/java/hadoop-2.8.5/logs/yarn-root-nodemanager-node3.out

[root@localhost sbin]# jps
28128 Jps
27537 JournalNode
27766 NodeManager
27431 DataNode
27658 ResourceManager
25150 QuorumPeerMai

可以看到启动成功后在node3和node4上多了ResourceManager和NodeManager进程。

至此，所有服务启动完成。与当初规划的完全一致

[root@node1 sbin]# jps
25857 NameNode
26258 DFSZKFailoverController
22196 QuorumPeerMain
26363 Jps

node2:
[root@localhost hadoop]# jps
34960 Jps
34369 NameNode
34548 JournalNode
31765 QuorumPeerMain
34661 DFSZKFailoverController
34439 DataNode
34778 NodeManager

node3:
[root@localhost sbin]# jps
27537 JournalNode
27766 NodeManager
27431 DataNode
27658 ResourceManager
28140 Jps
25150 QuorumPeerMain

node4:
[root@localhost sbin]# jps
37139 Jps
36469 JournalNode
36249 DataNode
36681 NodeManager
36924 ResourceManager

验证HDFS HA

进入node1的/usr/java/hadoop-2.8.5/bin目录，执行:

[root@node1 java]# cd /usr/java/hadoop-2.8.5/bin/
[root@node1 bin]# ./hadoop  fs -mkdir /test1
[root@node1 bin]# ./hadoop  fs -ls /
Found 1 items
drwxr-xr-x   - root supergroup          0 2019-07-11 03:15 /test1

[root@node1 bin]# ./hadoop  fs -put /usr/java/hadoop-2.8.5.tar.gz /test1
[root@node1 bin]# ./hadoop  fs -ls /test1
Found 1 items
-rw-r--r--   3 root supergroup  246543928 2019-07-11 03:18 /test1/hadoop-2.8.5.tar.gz

上面上传了一个240M左右的文件。下面看看他在Hadoop的实际存储情况。

在这里插入图片描述

可以看到这个文件被分成了两个块，hadoop-2.8.5.tar.gz大小为140M左右，所以会被分为2个块（默认一个快大小为128M）。每个块的副本分布在ndoe2,node3,node4机器上。

在这里插入图片描述

查看yarn控制台

我们的yarn部署在ndoe3上，直接访问：
在这里插入图片描述

namendoe高可用测试

上面配置了namenode的主从模式，我们将active的node1的nemenode停掉，看看结果如何。

[root@node1 bin]# jps
25857 NameNode
26258 DFSZKFailoverController
22196 QuorumPeerMain
26805 Jps
[root@node1 bin]# kill -9 25857
[root@node1 bin]# jps
26258 DFSZKFailoverController
22196 QuorumPeerMain
26815 Jps

在这里插入图片描述

node2还是standby状态，没有变化，怎么回事？

查看node2机器上的/usr/java/hadoop-2.8.5/logs/hadoop-root-zkfc-node2.log文件：

2019-07-11 03:35:15,423 WARN org.apache.hadoop.ha.SshFenceByTcpPort: Unable to create SSH session
com.jcraft.jsch.JSchException: java.io.FileNotFoundException: /root/.ssh/id_rsa (No such file or directory)
	at com.jcraft.jsch.KeyPair.load(KeyPair.java:543)
	at com.jcraft.jsch.IdentityFile.newInstance(IdentityFile.java:40)
	at com.jcraft.jsch.JSch.addIdentity(JSch.java:407)
	at com.jcraft.jsch.JSch.addIdentity(JSch.java:367)
	at org.apache.hadoop.ha.SshFenceByTcpPort.createSession(SshFenceByTcpPort.java:122)
	at org.apache.hadoop.ha.SshFenceByTcpPort.tryFence(SshFenceByTcpPort.java:91)
	at org.apache.hadoop.ha.NodeFencer.fence(NodeFencer.java:98)
	at org.apache.hadoop.ha.ZKFailoverController.doFence(ZKFailoverController.java:536)
	at org.apache.hadoop.ha.ZKFailoverController.fenceOldActive(ZKFailoverController.java:509)
	at org.apache.hadoop.ha.ZKFailoverController.access$1100(ZKFailoverController.java:61)
	at org.apache.hadoop.ha.ZKFailoverController$ElectorCallbacks.fenceOldActive(ZKFailoverController.java:895)
	at org.apache.hadoop.ha.ActiveStandbyElector.fenceOldActive(ActiveStandbyElector.java:985)
	at org.apache.hadoop.ha.ActiveStandbyElector.becomeActive(ActiveStandbyElector.java:882)
	at org.apache.hadoop.ha.ActiveStandbyElector.processResult(ActiveStandbyElector.java:467)
	at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:599)
	at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:498)
Caused by: java.io.FileNotFoundException: /root/.ssh/id_rsa (No such file or directory)
	at java.io.FileInputStream.open0(Native Method)
	at java.io.FileInputStream.open(FileInputStream.java:195)
	at java.io.FileInputStream.<init>(FileInputStream.java:138)
	at java.io.FileInputStream.<init>(FileInputStream.java:93)
	at com.jcraft.jsch.Util.fromFile(Util.java:508)
	at com.jcraft.jsch.KeyPair.load(KeyPair.java:540)
	... 15 more
2019-07-11 03:35:15,423 WARN org.apache.hadoop.ha.NodeFencer: Fencing method org.apache.hadoop.ha.SshFenceByTcpPort(null) was unsuccessful.
2019-07-11 03:35:15,423 ERROR org.apache.hadoop.ha.NodeFencer: Unable to fence service by any configured method.
2019-07-11 03:35:15,423 WARN org.apache.hadoop.ha.ActiveStandbyElector: Exception handling the winning of election
java.lang.RuntimeException: Unable to fence NameNode at node1/192.168.46.128:9000
	at org.apache.hadoop.ha.ZKFailoverController.doFence(ZKFailoverController.java:537)
	at org.apache.hadoop.ha.ZKFailoverController.fenceOldActive(ZKFailoverController.java:509)
	at org.apache.hadoop.ha.ZKFailoverController.access$1100(ZKFailoverController.java:61)
	at org.apache.hadoop.ha.ZKFailoverController$ElectorCallbacks.fenceOldActive(ZKFailoverController.java:895)
	at org.apache.hadoop.ha.ActiveStandbyElector.fenceOldActive(ActiveStandbyElector.java:985)
	at org.apache.hadoop.ha.ActiveStandbyElector.becomeActive(ActiveStandbyElector.java:882)
	at org.apache.hadoop.ha.ActiveStandbyElector.processResult(ActiveStandbyElector.java:467)
	at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:599)
	at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:498)
2019-07-11 03:35:15,425 INFO org.apache.hadoop.ha.ActiveStandbyElector: Trying to re-establish ZK session
2019-07-11 03:35:15,429 INFO org.apache.zookeeper.ZooKeeper: Session: 0x26be03e0c220035 closed

很明显，找不到/root/.ssh/id_rsa文件。

[root@localhost logs]# ls ~/.ssh/
authorized_keys  id_dsa  id_dsa.pub  known_hosts

很明显，文件名应该是id_dsa 。配置错了，修改一下hdfs-site.xml文件的配置：

<property>
    <name>dfs.ha.fencing.ssh.private-key-files</name>
    <value>/root/.ssh/id_dsa</value>
</property>

注意：所有机器均需要修改。

然后我们重新启动Hadoop集群

先停止所有服务，在node1上执行

[root@node1 sbin]# ./stop-all.sh 
This script is Deprecated. Instead use stop-dfs.sh and stop-yarn.sh
Stopping namenodes on [node1 node2]
node1: no namenode to stop
node2: stopping namenode
node2: stopping datanode
node4: stopping datanode
node3: stopping datanode
Stopping journal nodes [node2 node3 node4]
node2: stopping journalnode
node3: stopping journalnode
node4: stopping journalnode
Stopping ZK Failover Controllers on NN hosts [node1 node2]
node2: stopping zkfc
node1: stopping zkfc
stopping yarn daemons
no resourcemanager to stop
node3: stopping nodemanager
node2: stopping nodemanager
node4: stopping nodemanager
no proxyserver to stop

使用jps检查四台机器的进程是否全部杀掉，。

接下来我们使用start-all.sh来快速启动所有服务。他会一次启动 namenode、datanode、journalnode、zkfc、resourcemanager、nodemanager服务。

[root@node1 sbin]# ./start-all.sh 
This script is Deprecated. Instead use start-dfs.sh and start-yarn.sh
Starting namenodes on [node1 node2]
node1: starting namenode, logging to /usr/java/hadoop-2.8.5/logs/hadoop-root-namenode-node1.out
node2: starting namenode, logging to /usr/java/hadoop-2.8.5/logs/hadoop-root-namenode-node2.out
node3: starting datanode, logging to /usr/java/hadoop-2.8.5/logs/hadoop-root-datanode-node3.out
node2: starting datanode, logging to /usr/java/hadoop-2.8.5/logs/hadoop-root-datanode-node2.out
node4: starting datanode, logging to /usr/java/hadoop-2.8.5/logs/hadoop-root-datanode-node4.out
Starting journal nodes [node2 node3 node4]
node2: starting journalnode, logging to /usr/java/hadoop-2.8.5/logs/hadoop-root-journalnode-node2.out
node3: starting journalnode, logging to /usr/java/hadoop-2.8.5/logs/hadoop-root-journalnode-node3.out
node4: starting journalnode, logging to /usr/java/hadoop-2.8.5/logs/hadoop-root-journalnode-node4.out
Starting ZK Failover Controllers on NN hosts [node1 node2]
node2: starting zkfc, logging to /usr/java/hadoop-2.8.5/logs/hadoop-root-zkfc-node2.out
node1: starting zkfc, logging to /usr/java/hadoop-2.8.5/logs/hadoop-root-zkfc-node1.out
starting yarn daemons
starting resourcemanager, logging to /usr/java/hadoop-2.8.5/logs/yarn-root-resourcemanager-node1.out
node2: starting nodemanager, logging to /usr/java/hadoop-2.8.5/logs/yarn-root-nodemanager-node2.out
node3: starting nodemanager, logging to /usr/java/hadoop-2.8.5/logs/yarn-root-nodemanager-node3.out
node4: starting nodemanager, logging to /usr/java/hadoop-2.8.5/logs/yarn-root-nodemanager-node4.out

如此一来，再次kill掉node1机器的namenode进程，可以看到node2的namenode自动成为active状态了。

再次启动node1的namenode服务，可以看到node1的namenode服务变成了standby了。
在这里插入图片描述

namenode高可用验证OK。

快乐崇拜234

关注

2
点赞
踩
8

收藏

觉得还不错? 一键收藏
打赏
1
评论
Hadoop 2.8.5 完全分布式HA高可用安装（二）--环境搭建

文章目录概述集群规划安装ZK安装验证安装配置Hadoop集群配置Hadoop环境变量hadoo-env.shcore-site.xmlhdfs-site.xmlyarn-site.xmlmapred-site.xmlopt文件下创建对应文件夹将Hadoop拷贝到其他几台机器启动Hadoop启动ZK启动journalnode格式化namenode格式化zk启动主namenode备用NN 同步主NN信...
复制链接

扫一扫