Windows环境Docker环境搭建Hadoop3.2+zookeeper3.5.5+HBase2.2高可用集群（一）

参考1：https://blog.csdn.net/ccren/article/details/93485200

参考2：https://blog.csdn.net/qq_40235064/article/details/89074917

参考3：https://blog.csdn.net/qq_19524879/article/details/83659747

四个节点

docker run -itd --name hadoop1 --add-host hadoop1:172.17.0.2 --add-host hadoop2:172.17.0.3 --add-host hadoop3:172.17.0.4 --add-host hadoop4:172.17.0.5 -p 5002:22 -p 9870:9870 -p 8088:8088 -p 19888:19888 hadoop-ha-yarn /usr/sbin/sshd -D
docker run -itd --name hadoop2 --add-host hadoop1:172.17.0.2 --add-host hadoop2:172.17.0.3 --add-host hadoop3:172.17.0.4 --add-host hadoop4:172.17.0.5 -p 5003:22 -p 9871:9870 hadoop-ha-yarn /usr/sbin/sshd -D
docker run -itd --name hadoop3 --add-host hadoop1:172.17.0.2 --add-host hadoop2:172.17.0.3 --add-host hadoop3:172.17.0.4 --add-host hadoop4:172.17.0.5 -p 5004:22 -p 8087:8088 hadoop-ha-yarn /usr/sbin/sshd -D
docker run -itd --name hadoop4 --add-host hadoop1:172.17.0.2 --add-host hadoop2:172.17.0.3 --add-host hadoop3:172.17.0.4 --add-host hadoop4:172.17.0.5 -p 5005:22 -p 8086:8088 hadoop-ha-yarn /usr/sbin/sshd -D 

集群规划

hostname	namenode	datanode	zookeeper	zkfc	journalnode	resourcemanager	nodemanager
hadoop1	1	1	1	1	1
hadoop2	1	1	1	1	1		1
hadoop3		1	1		1	1	1
hadoop4			1			1	1

镜像制作好后启动容器：

docker run -itd --name hadoop1 --add-host hadoop1:172.17.0.2 --add-host hadoop2:172.17.0.3 --add-host hadoop3:172.17.0.4 --add-host hadoop4:172.17.0.5 -p 5002:22 -p 9870:9870 -p 19888:19888 hadoop-ha-yarn /usr/sbin/sshd -D
docker run -itd --name hadoop2 --add-host hadoop1:172.17.0.2 --add-host hadoop2:172.17.0.3 --add-host hadoop3:172.17.0.4 --add-host hadoop4:172.17.0.5 -p 5003:22 -p 9871:9870 hadoop-ha-yarn /usr/sbin/sshd -D
docker run -itd --name hadoop3 --add-host hadoop1:172.17.0.2 --add-host hadoop2:172.17.0.3 --add-host hadoop3:172.17.0.4 --add-host hadoop4:172.17.0.5 -p 5004:22 -p 8088:8088 hadoop-ha-yarn /usr/sbin/sshd -D
docker run -itd --name hadoop4 --add-host hadoop1:172.17.0.2 --add-host hadoop2:172.17.0.3 --add-host hadoop3:172.17.0.4 --add-host hadoop4:172.17.0.5 -p 5005:22 -p 8087:8088 hadoop-ha-yarn /usr/sbin/sshd -D 

使用CRT之类的客户端ssh登上去操作

1.启动zookeeper 4台(出现一个leader，三个follower，启动成功)
# bin/zkServer.sh start

1.1分别在3台虚拟机上查看zookeeper的状态。

#bin/zkServer.sh status

ZooKeeper JMX enabled by default
Using config: /opt/zookeeper/bin/../conf/zoo.cfg
Mode: follower

1.2连接其中一台的zookeeper

#bin/zkCli.sh -server hadoop1:2181

1.3写入数据

WatchedEvent state:SyncConnected type:None path:null

[zk: hadoop1:2181(CONNECTED) 0]

create /test data

Created /test

1.4再连接另外一台的zookeeper

#bin/zkCli.sh -server hadoop2:2181

1.5如果能获取到刚才写入的数据，则Zookeeper集群安装成功。

WatchedEvent state:SyncConnected type:None path:null

[zk: hadoop2:2181(CONNECTED) 0] get /test

data
cZxid = 0x100000002
ctime = Wed Jun 26 16:43:13 UTC 2019
mZxid = 0x100000002
mtime = Wed Jun 26 16:43:13 UTC 2019
pZxid = 0x100000002
cversion = 0
dataVersion = 0
aclVersion = 0
ephemeralOwner = 0x0
dataLength = 4
numChildren = 0

2.启动journalnode（分别在hadoop1、hadoop2、hadoop3上执行）
# cd /opt/modules/ha-hadoop/hadoop3.2/
# bin/hdfs --daemon start journalnode
或者以下命令也是开启 journalnode
# sbin/hadoop-deamon.sh start journalnode
# jps
1553 Jps
993 QuorumPeerMain
1514 JournalNode
# jps
993 QuorumPeerMain
1514 JournalNode
1563 Jps
出现JournalNode则表示journalnode启动成功。
3.格式化namenode（只要格式化一台，另一台同步，两台都格式化，你就做错了！！如：在hadoop2节点上）
# bin/hdfs namenode -format
如果在倒数8行左右的地方，出现这一句就表示成功
INFO common.Storage: Storage directory /home/hadoop/tmp/dfs/name has been successfully formatted.
启动namenode
# bin/hdfs --daemon start namenode
# jps
993 QuorumPeerMain
1681 NameNode
1747 Jps
1514 JournalNode
#cat /home/hadoop/tmp/dfs/name/current/VERSION
#Tue Jun 25 14:59:40 UTC 2019
namespaceID=1463663733
clusterID=CID-32938cd0-ed33-40f6-90c5-2326198e31bd
cTime=1561474780005
storageType=NAME_NODE
blockpoolID=BP-789586919-172.17.0.2-1561474780005
layoutVersion=-65
4.同步hadoop1元数据到hadoop2中（必须先启动hadoop1节点上的namenode）
在hadoop2主机上执行：
# bin/hdfs namenode -bootstrapStandby
如果出现：INFO common.Storage: Storage directory /home/hadoop/tmp/dfs/name has been successfully formatted.
说明同步成功
# cat /home/hadoop/tmp/dfs/name/current/VERSION
#Tue Jun 25 15:07:27 UTC 2019
namespaceID=1463663733
clusterID=CID-32938cd0-ed33-40f6-90c5-2326198e31bd
cTime=1561474780005
storageType=NAME_NODE
blockpoolID=BP-789586919-172.17.0.2-1561474780005
layoutVersion=-65
hadoop1和hadoop2显示的信息一致，则namenode数据同步成功
5.格式化ZKFC(在hadoop1上执行一次即可)
# bin/hdfs zkfc -formatZK
若在倒数第4行显示：INFO ha.ActiveStandbyElector: Successfully created /hadoop-ha/mycluster in ZK.
则表示ZKFC格式化成功
6.启动HDFS、yarn(在hadoop1上执行)
# sbin/start-dfs.sh
# sbin/start-yarn.sh

7、访问节点信息：

访问http://192.168.99.100:9870

yarn rmadmin -getServiceState rm1

yarn rmadmin -getServiceState rm2

拿active状态的节点映射端口比如8086访问
http://192.168.99.100:8086