ZooKeeper-3.4.6分布式集群安装

从Apache网站上(zookeeper.apache.org)下载ZooKeeper软件包,我选择了3.4.6版本的(zookeeper-3.4.6.tar.gz),下面说明Linux下分布式模式的安装配置,过程如下所示:

这里使用三台服务器来搭建,分别为node1、node2、node3;
对应的ip为:192.168.1.11、192.168.1.12、192.168.1.13

在搭建之前先配置三台机器之间的SSH免密登录。参考:集群内部的SSH密钥认证登陆机制配置(免密登陆)

一、主机名称与IP地址的映射配置
ZooKeeper集群中具有两个关键的角色:Leader和Follower。集群中所有的结点作为一个整体对分布式应用提供服务,集群中每个结点之间都互相连接,所以,在配置的ZooKeeper集群的时候,每一个结点的host到IP地址的映射都要配置上集群中其它结点的映射信息,方便结点之间的相互通信。
以node1为例:

vi /etc/hosts

进入配置文件hosts,配置对应映射

192.168.1.11 node1
192.168.1.12 node2
192.168.1.13 node3

ZooKeeper采用一种称为Leader election的选举算法。在整个集群运行过程中,只有一个Leader,其他的都是Follower,如果ZooKeeper集群在运行过程中Leader出了问题,系统会采用该算法重新选出一个Leader。因此,各个结点之间要能够保证互相连接,必须配置上述映射。
ZooKeeper集群启动的时候,会首先选出一个Leader,在Leader election过程中,某一个满足选举算的结点就能成为Leader。

二、修改Zookeeper配置文件

在node1上,解压zookeeper-3.4.6.tar.gz(因为zookeeper分布式集群搭建比较简单,不同机器上的不同之处很少,所以可以先在一台机器上配好,再分发到其他机器上,再进行局部配置的小修改)。
进入conf目录下,复制zoo_sample.cfg文件,改名为zoo.cfg;

cp zoo_sample.cfg zoo.cfg

修改zoo.cfg,内容如下:

# The number of milliseconds of each tick
tickTime=2000
# The number of ticks that the initial 
# synchronization phase can take
initLimit=10
# The number of ticks that can pass between 
# sending a request and getting an acknowledgement
syncLimit=5
# the directory where the snapshot is stored.
# do not use /tmp for storage, /tmp here is just 
# example sakes.
dataDir=/home/hadoop/app/zookeeper-3.4.6/data
# the port at which the clients will connect
clientPort=2181
# the maximum number of client connections.
# increase this if you need to handle more clients
#maxClientCnxns=60
#
# Be sure to read the maintenance section of the 
# administrator guide before turning on autopurge.
#
# http://zookeeper.apache.org/doc/current/zookeeperAdmin.html#sc_maintenance
#
# The number of snapshots to retain in dataDir
#autopurge.snapRetainCount=3
# Purge task interval in hours
# Set to "0" to disable auto purge feature
#autopurge.purgeInterval=1

server.1=node1:2888:3888
server.2=node2:2888:3888
server.3=node3:2888:3888

三、远程复制分发zookeeper到其他机器

这里分发到node2和node3上

scp -r zookeeper-3.4.6/ node2:/home/hadoop/app/
scp -r zookeeper-3.4.6/ node3:/home/hadoop/app/

四、设置myid

在我们配置的dataDir指定的目录下面,创建一个myid文件,里面内容为一个数字,用来标识当前主机,conf/zoo.cfg文件中配置的server.X中X为什么数字,则myid文件中就输入这个数字,例如:

dataDir=/home/hadoop/app/zookeeper-3.4.6/data

在/zookeeper-3.4.6/data目录下创建myid文件

server.1=node1:2888:3888
server.2=node2:2888:3888
server.3=node3:2888:3888

在node1下的myid文件中输入1,在node2的myid文件中输入2,node3 中的输入3,保存。

//示例
echo "1">data/myid

五、启动zookeeper集群

在zookeeper集群每个结点上,启动bin/目录下的zkServer.sh命令

bin/zkServer.sh start 
2017-12-08 09:58:21,819 [myid:] - INFO  [main:QuorumPeerConfig@103] - Reading configuration from: /home/hadoop/app/zookeeper-3.4.6/bin/../conf/zoo.cfg
2017-12-08 09:58:21,838 [myid:] - INFO  [main:QuorumPeerConfig@340] - Defaulting to majority quorums
2017-12-08 09:58:21,850 [myid:1] - INFO  [main:DatadirCleanupManager@78] - autopurge.snapRetainCount set to 3
2017-12-08 09:58:21,850 [myid:1] - INFO  [main:DatadirCleanupManager@79] - autopurge.purgeInterval set to 0
2017-12-08 09:58:21,852 [myid:1] - INFO  [main:DatadirCleanupManager@101] - Purge task is not scheduled.
2017-12-08 09:58:21,880 [myid:1] - INFO  [main:QuorumPeerMain@127] - Starting quorum peer
2017-12-08 09:58:21,965 [myid:1] - INFO  [main:NIOServerCnxnFactory@94] - binding to port 0.0.0.0/0.0.0.0:2181
2017-12-08 09:58:22,017 [myid:1] - INFO  [main:QuorumPeer@959] - tickTime set to 2000
2017-12-08 09:58:22,017 [myid:1] - INFO  [main:QuorumPeer@979] - minSessionTimeout set to -1
2017-12-08 09:58:22,017 [myid:1] - INFO  [main:QuorumPeer@990] - maxSessionTimeout set to -1
2017-12-08 09:58:22,018 [myid:1] - INFO  [main:QuorumPeer@1005] - initLimit set to 10
2017-12-08 09:58:22,092 [myid:1] - INFO  [main:FileSnap@83] - Reading snapshot /home/hadoop/app/zookeeper-3.4.6/data/version-2/snapshot.300000115
2017-12-08 09:58:22,205 [myid:1] - INFO  [Thread-1:QuorumCnxManager$Listener@504] - My election bind port: node1/192.168.1.11:3888
2017-12-08 09:58:22,232 [myid:1] - INFO  [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:2181:QuorumPeer@714] - LOOKING
2017-12-08 09:58:22,234 [myid:1] - INFO  [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:2181:FastLeaderElection@815] - New election. My id =  1, proposed zxid=0x4000000e7
2017-12-08 09:58:22,238 [myid:1] - INFO  [WorkerReceiver[myid=1]:FastLeaderElection@597] - Notification: 1 (message format version), 1 (n.leader), 0x4000000e7 (n.zxid), 0x1 (n.round), LOOKING (n.state), 1 (n.sid), 0x4 (n.peerEpoch) LOOKING (my state)
2017-12-08 09:58:22,255 [myid:1] - WARN  [WorkerSender[myid=1]:QuorumCnxManager@382] - Cannot open channel to 2 at election address node2/192.168.1.12:3888
java.net.ConnectException: Connection refused (Connection refused)
    at java.net.PlainSocketImpl.socketConnect(Native Method)
    at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
    at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
    at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
    at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
    at java.net.Socket.connect(Socket.java:589)
    at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectOne(QuorumCnxManager.java:368)
    at org.apache.zookeeper.server.quorum.QuorumCnxManager.toSend(QuorumCnxManager.java:341)
    at org.apache.zookeeper.server.quorum.FastLeaderElection$Messenger$WorkerSender.process(FastLeaderElection.java:449)
    at org.apache.zookeeper.server.quorum.FastLeaderElection$Messenger$WorkerSender.run(FastLeaderElection.java:430)
    at java.lang.Thread.run(Thread.java:748)
2017-12-08 09:58:22,271 [myid:1] - WARN  [WorkerSender[myid=1]:QuorumCnxManager@382] - Cannot open channel to 3 at election address node3/192.168.1.13:3888
java.net.ConnectException: Connection refused (Connection refused)
    at java.net.PlainSocketImpl.socketConnect(Native Method)
    at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
    at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
    at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
    at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
    at java.net.Socket.connect(Socket.java:589)
    at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectOne(QuorumCnxManager.java:368)
    at org.apache.zookeeper.server.quorum.QuorumCnxManager.toSend(QuorumCnxManager.java:341)
    at org.apache.zookeeper.server.quorum.FastLeaderElection$Messenger$WorkerSender.process(FastLeaderElection.java:449)
    at org.apache.zookeeper.server.quorum.FastLeaderElection$Messenger$WorkerSender.run(FastLeaderElection.java:430)
    at java.lang.Thread.run(Thread.java:748)
2017-12-08 09:58:22,461 [myid:1] - WARN  [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:2181:QuorumCnxManager@382] - Cannot open channel to 2 at election address node2/192.168.1.12:3888
java.net.ConnectException: Connection refused (Connection refused)
    at java.net.PlainSocketImpl.socketConnect(Native Method)
    at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
    at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
    at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
    at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
    at java.net.Socket.connect(Socket.java:589)
    at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectOne(QuorumCnxManager.java:368)
    at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectAll(QuorumCnxManager.java:402)
    at org.apache.zookeeper.server.quorum.FastLeaderElection.lookForLeader(FastLeaderElection.java:840)
    at org.apache.zookeeper.server.quorum.QuorumPeer.run(QuorumPeer.java:762)
2017-12-08 09:58:22,462 [myid:1] - WARN  [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:2181:QuorumCnxManager@382] - Cannot open channel to 3 at election address node3/192.168.1.13:3888
java.net.ConnectException: Connection refused (Connection refused)
    at java.net.PlainSocketImpl.socketConnect(Native Method)
    at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
    at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
    at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
    at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
    at java.net.Socket.connect(Socket.java:589)
    at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectOne(QuorumCnxManager.java:368)
    at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectAll(QuorumCnxManager.java:402)
    at org.apache.zookeeper.server.quorum.FastLeaderElection.lookForLeader(FastLeaderElection.java:840)
    at org.apache.zookeeper.server.quorum.QuorumPeer.run(QuorumPeer.java:762)
2017-12-08 09:58:22,463 [myid:1] - INFO  [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:2181:FastLeaderElection@849] - Notification time out: 400
2017-12-08 09:58:22,864 [myid:1] - WARN  [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:2181:QuorumCnxManager@382] - Cannot open channel to 2 at election address node2/192.168.1.12:3888
java.net.ConnectException: Connection refused (Connection refused)
    at java.net.PlainSocketImpl.socketConnect(Native Method)
    at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
    at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
    at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
    at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
    at java.net.Socket.connect(Socket.java:589)
    at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectOne(QuorumCnxManager.java:368)
    at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectAll(QuorumCnxManager.java:402)
    at org.apache.zookeeper.server.quorum.FastLeaderElection.lookForLeader(FastLeaderElection.java:840)
    at org.apache.zookeeper.server.quorum.QuorumPeer.run(QuorumPeer.java:762)
2017-12-08 09:58:22,865 [myid:1] - WARN  [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:2181:QuorumCnxManager@382] - Cannot open channel to 3 at election address node3/192.168.1.13:3888
java.net.ConnectException: Connection refused (Connection refused)
    at java.net.PlainSocketImpl.socketConnect(Native Method)
    at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
    at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
    at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
    at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
    at java.net.Socket.connect(Socket.java:589)
    at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectOne(QuorumCnxManager.java:368)
    at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectAll(QuorumCnxManager.java:402)
    at org.apache.zookeeper.server.quorum.FastLeaderElection.lookForLeader(FastLeaderElection.java:840)
    at org.apache.zookeeper.server.quorum.QuorumPeer.run(QuorumPeer.java:762)
2017-12-08 09:58:22,866 [myid:1] - INFO  [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:2181:FastLeaderElection@849] - Notification time out: 800
2017-12-08 09:58:22,985 [myid:1] - INFO  [node1/192.168.1.11:3888:QuorumCnxManager$Listener@511] - Received connection request /192.168.1.12:46579
2017-12-08 09:58:22,991 [myid:1] - INFO  [WorkerReceiver[myid=1]:FastLeaderElection@597] - Notification: 1 (message format version), 2 (n.leader), 0x4000000e7 (n.zxid), 0x1 (n.round), LOOKING (n.state), 2 (n.sid), 0x4 (n.peerEpoch) LOOKING (my state)
2017-12-08 09:58:22,992 [myid:1] - INFO  [WorkerReceiver[myid=1]:FastLeaderElection@597] - Notification: 1 (message format version), 2 (n.leader), 0x4000000e7 (n.zxid), 0x1 (n.round), LOOKING (n.state), 1 (n.sid), 0x4 (n.peerEpoch) LOOKING (my state)
2017-12-08 09:58:22,993 [myid:1] - WARN  [WorkerSender[myid=1]:QuorumCnxManager@382] - Cannot open channel to 3 at election address node3/192.168.1.13:3888
java.net.ConnectException: Connection refused (Connection refused)
    at java.net.PlainSocketImpl.socketConnect(Native Method)
    at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
    at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
    at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
    at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
    at java.net.Socket.connect(Socket.java:589)
    at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectOne(QuorumCnxManager.java:368)
    at org.apache.zookeeper.server.quorum.QuorumCnxManager.toSend(QuorumCnxManager.java:341)
    at org.apache.zookeeper.server.quorum.FastLeaderElection$Messenger$WorkerSender.process(FastLeaderElection.java:449)
    at org.apache.zookeeper.server.quorum.FastLeaderElection$Messenger$WorkerSender.run(FastLeaderElection.java:430)
    at java.lang.Thread.run(Thread.java:748)
2017-12-08 09:58:23,041 [myid:1] - INFO  [node1/192.168.1.11:3888:QuorumCnxManager$Listener@511] - Received connection request /192.168.1.13:43979
2017-12-08 09:58:23,045 [myid:1] - INFO  [WorkerReceiver[myid=1]:FastLeaderElection@597] - Notification: 1 (message format version), 3 (n.leader), 0x4000000e7 (n.zxid), 0x1 (n.round), LOOKING (n.state), 3 (n.sid), 0x4 (n.peerEpoch) LOOKING (my state)
2017-12-08 09:58:23,046 [myid:1] - INFO  [WorkerReceiver[myid=1]:FastLeaderElection@597] - Notification: 1 (message format version), 3 (n.leader), 0x4000000e7 (n.zxid), 0x1 (n.round), LOOKING (n.state), 1 (n.sid), 0x4 (n.peerEpoch) LOOKING (my state)
2017-12-08 09:58:23,048 [myid:1] - INFO  [WorkerReceiver[myid=1]:FastLeaderElection@597] - Notification: 1 (message format version), 3 (n.leader), 0x4000000e7 (n.zxid), 0x1 (n.round), LOOKING (n.state), 2 (n.sid), 0x4 (n.peerEpoch) LOOKING (my state)
2017-12-08 09:58:23,249 [myid:1] - INFO  [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:2181:QuorumPeer@784] - FOLLOWING
2017-12-08 09:58:23,257 [myid:1] - INFO  [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:2181:Learner@86] - TCP NoDelay set to: true
2017-12-08 09:58:23,267 [myid:1] - INFO  [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:2181:Environment@100] - Server environment:zookeeper.version=3.4.6-1569965, built on 02/20/2014 09:09 GMT
2017-12-08 09:58:23,267 [myid:1] - INFO  [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:2181:Environment@100] - Server environment:host.name=node1
2017-12-08 09:58:23,267 [myid:1] - INFO  [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:2181:Environment@100] - Server environment:java.version=1.8.0_131
2017-12-08 09:58:23,267 [myid:1] - INFO  [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:2181:Environment@100] - Server environment:java.vendor=Oracle Corporation
2017-12-08 09:58:23,268 [myid:1] - INFO  [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:2181:Environment@100] - Server environment:java.home=/home/hadoop/app/jdk1.8.0_131/jre
2017-12-08 09:58:23,268 [myid:1] - INFO  [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:2181:Environment@100] - Server environment:java.class.path=/home/hadoop/app/zookeeper-3.4.6/bin/../build/classes:/home/hadoop/app/zookeeper-3.4.6/bin/../build/lib/*.jar:/home/hadoop/app/zookeeper-3.4.6/bin/../lib/slf4j-log4j12-1.6.1.jar:/home/hadoop/app/zookeeper-3.4.6/bin/../lib/slf4j-api-1.6.1.jar:/home/hadoop/app/zookeeper-3.4.6/bin/../lib/netty-3.7.0.Final.jar:/home/hadoop/app/zookeeper-3.4.6/bin/../lib/log4j-1.2.16.jar:/home/hadoop/app/zookeeper-3.4.6/bin/../lib/jline-0.9.94.jar:/home/hadoop/app/zookeeper-3.4.6/bin/../zookeeper-3.4.6.jar:/home/hadoop/app/zookeeper-3.4.6/bin/../src/java/lib/*.jar:/home/hadoop/app/zookeeper-3.4.6/bin/../conf:.:/lib/dt.jar:/lib/tools.jar:/jre/lib
2017-12-08 09:58:23,268 [myid:1] - INFO  [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:2181:Environment@100] - Server environment:java.library.path=/usr/local/apr/lib:/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib
2017-12-08 09:58:23,268 [myid:1] - INFO  [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:2181:Environment@100] - Server environment:java.io.tmpdir=/tmp
2017-12-08 09:58:23,268 [myid:1] - INFO  [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:2181:Environment@100] - Server environment:java.compiler=<NA>
2017-12-08 09:58:23,268 [myid:1] - INFO  [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:2181:Environment@100] - Server environment:os.name=Linux
2017-12-08 09:58:23,269 [myid:1] - INFO  [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:2181:Environment@100] - Server environment:os.arch=amd64
2017-12-08 09:58:23,269 [myid:1] - INFO  [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:2181:Environment@100] - Server environment:os.version=2.6.32-431.el6.x86_64
2017-12-08 09:58:23,269 [myid:1] - INFO  [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:2181:Environment@100] - Server environment:user.name=root
2017-12-08 09:58:23,269 [myid:1] - INFO  [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:2181:Environment@100] - Server environment:user.home=/root
2017-12-08 09:58:23,269 [myid:1] - INFO  [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:2181:Environment@100] - Server environment:user.dir=/home/hadoop/app/zookeeper-3.4.6/bin
2017-12-08 09:58:23,272 [myid:1] - INFO  [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:2181:ZooKeeperServer@162] - Created server with tickTime 2000 minSessionTimeout 4000 maxSessionTimeout 40000 datadir /home/hadoop/app/zookeeper-3.4.6/data/version-2 snapdir /home/hadoop/app/zookeeper-3.4.6/data/version-2
2017-12-08 09:58:23,273 [myid:1] - INFO  [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:2181:Follower@63] - FOLLOWING - LEADER ELECTION TOOK - 1039
2017-12-08 09:58:23,277 [myid:1] - WARN  [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:2181:Learner@233] - Unexpected exception, tries=0, connecting to node3/192.168.1.13:2888
java.net.ConnectException: Connection refused (Connection refused)
    at java.net.PlainSocketImpl.socketConnect(Native Method)
    at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
    at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
    at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
    at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
    at java.net.Socket.connect(Socket.java:589)
    at org.apache.zookeeper.server.quorum.Learner.connectToLeader(Learner.java:225)
    at org.apache.zookeeper.server.quorum.Follower.followLeader(Follower.java:71)
    at org.apache.zookeeper.server.quorum.QuorumPeer.run(QuorumPeer.java:786)
2017-12-08 09:58:24,278 [myid:1] - WARN  [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:2181:Learner@233] - Unexpected exception, tries=1, connecting to node3/192.168.1.13:2888
java.net.ConnectException: Connection refused (Connection refused)
    at java.net.PlainSocketImpl.socketConnect(Native Method)
    at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
    at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
    at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
    at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
    at java.net.Socket.connect(Socket.java:589)
    at org.apache.zookeeper.server.quorum.Learner.connectToLeader(Learner.java:225)
    at org.apache.zookeeper.server.quorum.Follower.followLeader(Follower.java:71)
    at org.apache.zookeeper.server.quorum.QuorumPeer.run(QuorumPeer.java:786)
2017-12-08 09:58:25,279 [myid:1] - WARN  [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:2181:Learner@233] - Unexpected exception, tries=2, connecting to node3/192.168.1.13:2888
java.net.ConnectException: Connection refused (Connection refused)
    at java.net.PlainSocketImpl.socketConnect(Native Method)
    at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
    at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
    at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
    at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
    at java.net.Socket.connect(Socket.java:589)
    at org.apache.zookeeper.server.quorum.Learner.connectToLeader(Learner.java:225)
    at org.apache.zookeeper.server.quorum.Follower.followLeader(Follower.java:71)
    at org.apache.zookeeper.server.quorum.QuorumPeer.run(QuorumPeer.java:786)
2017-12-08 09:58:26,280 [myid:1] - WARN  [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:2181:Learner@233] - Unexpected exception, tries=3, connecting to node3/192.168.1.13:2888
java.net.ConnectException: Connection refused (Connection refused)
    at java.net.PlainSocketImpl.socketConnect(Native Method)
    at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
    at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
    at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
    at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
    at java.net.Socket.connect(Socket.java:589)
    at org.apache.zookeeper.server.quorum.Learner.connectToLeader(Learner.java:225)
    at org.apache.zookeeper.server.quorum.Follower.followLeader(Follower.java:71)
    at org.apache.zookeeper.server.quorum.QuorumPeer.run(QuorumPeer.java:786)
2017-12-08 09:58:27,282 [myid:1] - ERROR [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:2181:Learner@230] - Unexpected exception
java.net.ConnectException: Connection refused (Connection refused)
    at java.net.PlainSocketImpl.socketConnect(Native Method)
    at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
    at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
    at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
    at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
    at java.net.Socket.connect(Socket.java:589)
    at org.apache.zookeeper.server.quorum.Learner.connectToLeader(Learner.java:225)
    at org.apache.zookeeper.server.quorum.Follower.followLeader(Follower.java:71)
    at org.apache.zookeeper.server.quorum.QuorumPeer.run(QuorumPeer.java:786)
2017-12-08 09:58:27,282 [myid:1] - WARN  [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:2181:Follower@89] - Exception when following the leader
java.net.ConnectException: Connection refused (Connection refused)
    at java.net.PlainSocketImpl.socketConnect(Native Method)
    at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
    at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
    at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
    at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
    at java.net.Socket.connect(Socket.java:589)
    at org.apache.zookeeper.server.quorum.Learner.connectToLeader(Learner.java:225)
    at org.apache.zookeeper.server.quorum.Follower.followLeader(Follower.java:71)
    at org.apache.zookeeper.server.quorum.QuorumPeer.run(QuorumPeer.java:786)
2017-12-08 09:58:27,283 [myid:1] - INFO  [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:2181:Follower@166] - shutdown called
java.lang.Exception: shutdown Follower
    at org.apache.zookeeper.server.quorum.Follower.shutdown(Follower.java:166)
    at org.apache.zookeeper.server.quorum.QuorumPeer.run(QuorumPeer.java:790)
2017-12-08 09:58:27,284 [myid:1] - INFO  [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:2181:FollowerZooKeeperServer@139] - Shutting down
2017-12-08 09:58:27,284 [myid:1] - INFO  [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:2181:ZooKeeperServer@441] - shutting down
2017-12-08 09:58:27,284 [myid:1] - INFO  [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:2181:QuorumPeer@714] - LOOKING
2017-12-08 09:58:27,286 [myid:1] - INFO  [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:2181:FileSnap@83] - Reading snapshot /home/hadoop/app/zookeeper-3.4.6/data/version-2/snapshot.300000115
2017-12-08 09:58:27,310 [myid:1] - INFO  [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:2181:FastLeaderElection@815] - New election. My id =  1, proposed zxid=0x4000000e7
2017-12-08 09:58:27,311 [myid:1] - INFO  [WorkerReceiver[myid=1]:FastLeaderElection@597] - Notification: 1 (message format version), 1 (n.leader), 0x4000000e7 (n.zxid), 0x2 (n.round), LOOKING (n.state), 1 (n.sid), 0x4 (n.peerEpoch) LOOKING (my state)
2017-12-08 09:58:27,312 [myid:1] - INFO  [WorkerReceiver[myid=1]:FastLeaderElection@597] - Notification: 1 (message format version), 3 (n.leader), 0x4000000e7 (n.zxid), 0x1 (n.round), LEADING (n.state), 3 (n.sid), 0x4 (n.peerEpoch) LOOKING (my state)
2017-12-08 09:58:27,325 [myid:1] - INFO  [WorkerReceiver[myid=1]:FastLeaderElection@597] - Notification: 1 (message format version), 2 (n.leader), 0x4000000e7 (n.zxid), 0x2 (n.round), LOOKING (n.state), 2 (n.sid), 0x4 (n.peerEpoch) LOOKING (my state)
2017-12-08 09:58:27,325 [myid:1] - INFO  [WorkerReceiver[myid=1]:FastLeaderElection@597] - Notification: 1 (message format version), 2 (n.leader), 0x4000000e7 (n.zxid), 0x2 (n.round), LOOKING (n.state), 1 (n.sid), 0x4 (n.peerEpoch) LOOKING (my state)
2017-12-08 09:58:27,326 [myid:1] - INFO  [WorkerReceiver[myid=1]:FastLeaderElection@597] - Notification: 1 (message format version), 3 (n.leader), 0x4000000e7 (n.zxid), 0x1 (n.round), LEADING (n.state), 3 (n.sid), 0x4 (n.peerEpoch) LOOKING (my state)
2017-12-08 09:58:27,527 [myid:1] - INFO  [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:2181:FastLeaderElection@849] - Notification time out: 400
2017-12-08 09:58:27,527 [myid:1] - INFO  [WorkerReceiver[myid=1]:FastLeaderElection@597] - Notification: 1 (message format version), 2 (n.leader), 0x4000000e7 (n.zxid), 0x2 (n.round), LOOKING (n.state), 1 (n.sid), 0x4 (n.peerEpoch) LOOKING (my state)
2017-12-08 09:58:27,528 [myid:1] - INFO  [WorkerReceiver[myid=1]:FastLeaderElection@597] - Notification: 1 (message format version), 3 (n.leader), 0x4000000e7 (n.zxid), 0x1 (n.round), LEADING (n.state), 3 (n.sid), 0x4 (n.peerEpoch) LOOKING (my state)
2017-12-08 09:58:27,528 [myid:1] - INFO  [WorkerReceiver[myid=1]:FastLeaderElection@597] - Notification: 1 (message format version), 2 (n.leader), 0x4000000e7 (n.zxid), 0x2 (n.round), LEADING (n.state), 2 (n.sid), 0x4 (n.peerEpoch) LOOKING (my state)
2017-12-08 09:58:27,929 [myid:1] - INFO  [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:2181:FastLeaderElection@849] - Notification time out: 800
2017-12-08 09:58:27,929 [myid:1] - INFO  [WorkerReceiver[myid=1]:FastLeaderElection@597] - Notification: 1 (message format version), 2 (n.leader), 0x4000000e7 (n.zxid), 0x2 (n.round), LOOKING (n.state), 1 (n.sid), 0x4 (n.peerEpoch) LOOKING (my state)
2017-12-08 09:58:27,930 [myid:1] - INFO  [WorkerReceiver[myid=1]:FastLeaderElection@597] - Notification: 1 (message format version), 3 (n.leader), 0x4000000e7 (n.zxid), 0x1 (n.round), LEADING (n.state), 3 (n.sid), 0x4 (n.peerEpoch) LOOKING (my state)
2017-12-08 09:58:27,930 [myid:1] - INFO  [WorkerReceiver[myid=1]:FastLeaderElection@597] - Notification: 1 (message format version), 2 (n.leader), 0x4000000e7 (n.zxid), 0x2 (n.round), LEADING (n.state), 2 (n.sid), 0x4 (n.peerEpoch) LOOKING (my state)
2017-12-08 09:58:27,931 [myid:1] - INFO  [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:2181:QuorumPeer@784] - FOLLOWING
2017-12-08 09:58:27,931 [myid:1] - INFO  [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:2181:ZooKeeperServer@162] - Created server with tickTime 2000 minSessionTimeout 4000 maxSessionTimeout 40000 datadir /home/hadoop/app/zookeeper-3.4.6/data/version-2 snapdir /home/hadoop/app/zookeeper-3.4.6/data/version-2
2017-12-08 09:58:27,932 [myid:1] - INFO  [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:2181:Follower@63] - FOLLOWING - LEADER ELECTION TOOK - 647
2017-12-08 09:58:28,092 [myid:1] - INFO  [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:2181:Learner@323] - Getting a diff from the leader 0x4000000e7
2017-12-08 09:58:28,098 [myid:1] - INFO  [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:2181:FileTxnSnapLog@240] - Snapshotting: 0x4000000e7 to /home/hadoop/app/zookeeper-3.4.6/data/version-2/snapshot.4000000e7
2017-12-08 09:58:34,274 [myid:1] - WARN  [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:2181:Follower@118] - Got zxid 0x500000001 expected 0x1
2017-12-08 09:58:34,275 [myid:1] - INFO  [SyncThread:1:FileTxnLog@199] - Creating new log file: log.500000001
2017-12-08 09:58:35,381 [myid:1] - INFO  [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxnFactory@197] - Accepted socket connection from /127.0.0.1:34582
2017-12-08 09:58:35,391 [myid:1] - INFO  [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxn@827] - Processing srvr command from /127.0.0.1:34582
2017-12-08 09:58:35,397 [myid:1] - INFO  [Thread-2:NIOServerCnxn@1007] - Closed socket connection for client /127.0.0.1:34582 (no session established for client)
2017-12-08 09:58:43,331 [myid:1] - INFO  [WorkerReceiver[myid=1]:FastLeaderElection@597] - Notification: 1 (message format version), 3 (n.leader), 0x4000000e7 (n.zxid), 0x2 (n.round), LOOKING (n.state), 3 (n.sid), 0x4 (n.peerEpoch) FOLLOWING (my state)
2017-12-08 10:02:36,789 [myid:1] - INFO  [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxnFactory@197] - Accepted socket connection from /192.168.1.13:52885
2017-12-08 10:02:36,796 [myid:1] - INFO  [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:ZooKeeperServer@868] - Client attempting to establish new session at /192.168.1.13:52885
2017-12-08 10:02:36,818 [myid:1] - INFO  [CommitProcessor:1:ZooKeeperServer@617] - Established session 0x16033d866000000 with negotiated timeout 30000 for client /192.168.1.13:52885

由于ZooKeeper集群启动的时候,每个结点都试图去连接集群中的其它结点,先启动的肯定连不上后面还没启动的,所以上面日志前面部分的异常是可以忽略的。通过后面部分可以看到,集群在选出一个Leader后,最后稳定了。
其他结点可能也出现类似问题,属于正常。

六、检查启动情况

1)使用bin/目录下的zkServer.sh命令查看集群各结点的角色情况

bin/zkServer.sh status

显示结果如下:

[root@node1 bin]# ./zkServer.sh status
JMX enabled by default
Using config: /home/hadoop/app/zookeeper-3.4.6/bin/../conf/zoo.cfg
Mode: follower

[root@node2 bin]# ./zkServer.sh status
JMX enabled by default
Using config: /home/hadoop/app/zookeeper-3.4.6/bin/../conf/zoo.cfg
Mode: leader

[root@node3 bin]# ./zkServer.sh status
JMX enabled by default
Using config: /home/hadoop/app/zookeeper-3.4.6/bin/../conf/zoo.cfg
Mode: follower

2)可以通过bin/目录下的zkCli.sh客户端脚本,连接到ZooKeeper集群上。对于客户端来说,ZooKeeper是一个整体,连接到ZooKeeper集群实际上感觉在独享整个集群的服务,所以,你可以在任何一个结点上建立到服务集群的连接,例如:

[root@node3 bin]# ./zkCli.sh -server node1:2181
Connecting to node1:2181
2017-12-08 10:02:24,918 [myid:] - INFO  [main:Environment@100] - Client environment:zookeeper.version=3.4.6-1569965, built on 02/20/2014 09:09 GMT
2017-12-08 10:02:24,923 [myid:] - INFO  [main:Environment@100] - Client environment:host.name=node3
2017-12-08 10:02:24,923 [myid:] - INFO  [main:Environment@100] - Client environment:java.version=1.8.0_131
2017-12-08 10:02:24,926 [myid:] - INFO  [main:Environment@100] - Client environment:java.vendor=Oracle Corporation
2017-12-08 10:02:24,926 [myid:] - INFO  [main:Environment@100] - Client environment:java.home=/home/hadoop/app/jdk1.8.0_131/jre
2017-12-08 10:02:24,926 [myid:] - INFO  [main:Environment@100] - Client environment:java.class.path=/home/hadoop/app/zookeeper-3.4.6/bin/../build/classes:/home/hadoop/app/zookeeper-3.4.6/bin/../build/lib/*.jar:/home/hadoop/app/zookeeper-3.4.6/bin/../lib/slf4j-log4j12-1.6.1.jar:/home/hadoop/app/zookeeper-3.4.6/bin/../lib/slf4j-api-1.6.1.jar:/home/hadoop/app/zookeeper-3.4.6/bin/../lib/netty-3.7.0.Final.jar:/home/hadoop/app/zookeeper-3.4.6/bin/../lib/log4j-1.2.16.jar:/home/hadoop/app/zookeeper-3.4.6/bin/../lib/jline-0.9.94.jar:/home/hadoop/app/zookeeper-3.4.6/bin/../zookeeper-3.4.6.jar:/home/hadoop/app/zookeeper-3.4.6/bin/../src/java/lib/*.jar:/home/hadoop/app/zookeeper-3.4.6/bin/../conf:.:/home/hadoop/app/jdk1.8.0_131/lib/dt.jar:/home/hadoop/app/jdk1.8.0_131/lib/tools.jar:/home/hadoop/app/jdk1.8.0_131/jre/lib
2017-12-08 10:02:24,927 [myid:] - INFO  [main:Environment@100] - Client environment:java.library.path=/usr/local/apr/lib:/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib
2017-12-08 10:02:24,927 [myid:] - INFO  [main:Environment@100] - Client environment:java.io.tmpdir=/tmp
2017-12-08 10:02:24,927 [myid:] - INFO  [main:Environment@100] - Client environment:java.compiler=<NA>
2017-12-08 10:02:24,927 [myid:] - INFO  [main:Environment@100] - Client environment:os.name=Linux
2017-12-08 10:02:24,927 [myid:] - INFO  [main:Environment@100] - Client environment:os.arch=amd64
2017-12-08 10:02:24,927 [myid:] - INFO  [main:Environment@100] - Client environment:os.version=2.6.32-431.el6.x86_64
2017-12-08 10:02:24,927 [myid:] - INFO  [main:Environment@100] - Client environment:user.name=root
2017-12-08 10:02:24,928 [myid:] - INFO  [main:Environment@100] - Client environment:user.home=/root
2017-12-08 10:02:24,928 [myid:] - INFO  [main:Environment@100] - Client environment:user.dir=/home/hadoop/app/zookeeper-3.4.6/bin
2017-12-08 10:02:24,929 [myid:] - INFO  [main:ZooKeeper@438] - Initiating client connection, connectString=node1:2181 sessionTimeout=30000 watcher=org.apache.zookeeper.ZooKeeperMain$MyWatcher@277050dc
Welcome to ZooKeeper!
2017-12-08 10:02:24,988 [myid:] - INFO  [main-SendThread(node1:2181):ClientCnxn$SendThread@975] - Opening socket connection to server node1/192.168.1.11:2181. Will not attempt to authenticate using SASL (unknown error)
JLine support is enabled
2017-12-08 10:02:25,107 [myid:] - INFO  [main-SendThread(node1:2181):ClientCnxn$SendThread@852] - Socket connection established to node1/192.168.1.11:2181, initiating session
2017-12-08 10:02:25,139 [myid:] - INFO  [main-SendThread(node1:2181):ClientCnxn$SendThread@1235] - Session establishment complete on server node1/192.168.1.11:2181, sessionid = 0x16033d866000000, negotiated timeout = 30000

WATCHER::

WatchedEvent state:SyncConnected type:None path:null
[zk: node1:2181(CONNECTED) 0] ls /
[cluster, controller_epoch, brokers, zookeeper, yarn-leader-election, hadoop-ha, admin, isr_change_notification, consumers, config, hbase]
[zk: node1:2181(CONNECTED) 1] 

以上是我在node3结点上使用客户端从node1结点登录集群的日志信息,当前更路径下已存有

[cluster, controller_epoch, brokers, zookeeper, yarn-leader-election, hadoop-ha, admin, isr_change_notification, consumers, config, hbase]

等数据信息,关于zookeeper的数据结构,后面有时间再做详细的整理。

  • 1
    点赞
  • 1
    收藏
    觉得还不错? 一键收藏
  • 0
    评论

“相关推荐”对你有帮助么?

  • 非常没帮助
  • 没帮助
  • 一般
  • 有帮助
  • 非常有帮助
提交
评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值