相对于Hadoop、Spark、Flink、Kafka等大数据集群的搭建过程,Zookeeper集群的搭建相对来讲比较简单,现将其搭建过程记录下,方便日后查看。
说明:Zookeeper集群中机器的数量最好是奇数(n),这样可以保证在(n-1)/2台机器失败的情况下,依然可以保证高可用性。假如有4台集群,那也只能允许1台失败,而5台集群则可以允许2台坏掉。
Zookeeper集群步骤:
1、安装Java JDK并设置环境变量
export JAVA_HOME=/usr/local/jdk/jdk1.8.0_60
export JRE_HOME=${JAVA_HOME}/jre
export PATH=.:${JAVA_HOME}/bin:$PATH
export CLASS_PATH=.:${JAVA_HOME}/lib:${JRE_HOME}/lib
2、设置Java Heap(Swap空间)
网上资料很多,参考网上设置ubuntu swap空间。
3、安装、配置Zookeeper
版本:zookeeper-3.4.6
3.1、下载并解压:
tar -zxvf zookeeper-3.4.6.tar.gz
3.2、配置Zookeeper
(1)将{ZOOKEEPER_HOME}/conf下的zoo_sample.cfg文件改为zoo.cfg
(2)修改zoo.cfg:
tickTime=2000
initLimit=10
syncLimit=5
dataDir=/usr/local/zookeeper/zookeeper-3.4.6/data
dataLogdir=/usr/local/zookeeper/zookeeper-3.4.6/logs
server.0=master:2888:3888
server.1=worker1:2888:3888
server.2=worker2:2888:3888
clientPort=2181
注意:需要先创建dataDir和dataLogdir指定的目录;server.0、1、2分别指定zookeeper集群中的机器。
(3)创建myid
上一步创建了dataDir,在dataDir指定的目录中(/usr/local/zookeeper/zookeeper-3.4.6/data)创建文件myid:
注意:这里的myid中的内容只有“0”,目的是为了标示集群中的节点。但是,这个内容“0”必须与上一步zoo.cfg里server.0中的数字对应,即0->0,1->1,2->2.
4、设置环境变量,添加Zookeeper
export ZOOKEEPER_HOME=/usr/local/zookeeper/zookeeper-3.4.6
export PATH=.:${JAVA_HOME}/bin:${ZOOKEEPER_HOME}/bin:$PATH
生效:source ~/.bashrc
5、复制zookeeper目录以及环境变量到其他节点(worker1、worker2)
scp -r zookeeper-3.4.6/ root@worker1:/usr/local/zookeeper/
scp -r zookeeper-3.4.6/ root@worker2:/usr/local/zookeeper/
scp ~/.bashrc root@worker1:~/.bashrc
scp ~/.bashrc root@worker2:~/.bashrc
6、修改worker1、worker2上myid
worker1:
worker2:
7、启动zookeeper集群并测试
(1)在3台机器上都启动zookeeper:
root@master:/usr/local/zookeeper/zookeeper-3.4.6/bin# zkServer.sh start
root@worker1:/usr/local/zookeeper/zookeeper-3.4.6/bin# zkServer.sh start
root@worker2:/usr/local/zookeeper/zookeeper-3.4.6/bin# zkServer.sh start
(2)查看进程是否启动(3台都要看):
root@master:/usr/local/zookeeper/zookeeper-3.4.6/bin# jps
7734 Jps
7706 QuorumPeerMain
root@master:/usr/local/zookeeper/zookeeper-3.4.6/bin#
当看到QuorumPeerMain进程时,说明zookeeper的服务已启动。
(3)查看每台机器在zookeeper集群的状态:
root@master:/usr/local/zookeeper/zookeeper-3.4.6/bin# zkServer.sh status
JMX enabled by default
Using config: /usr/local/zookeeper/zookeeper-3.4.6/bin/../conf/zoo.cfg
Mode: follower
root@master:/usr/local/zookeeper/zookeeper-3.4.6/bin#
root@worker1:/usr/local/zookeeper/zookeeper-3.4.6/bin# zkServer.sh status
JMX enabled by default
Using config: /usr/local/zookeeper/zookeeper-3.4.6/bin/../conf/zoo.cfg
Mode: leader
root@worker1:/usr/local/zookeeper/zookeeper-3.4.6/bin#
root@worker2:/usr/local/zookeeper/zookeeper-3.4.6/bin# zkServer.sh status
JMX enabled by default
Using config: /usr/local/zookeeper/zookeeper-3.4.6/bin/../conf/zoo.cfg
Mode: follower
root@worker2:/usr/local/zookeeper/zookeeper-3.4.6/bin#
参考:
http://zookeeper.apache.org/doc/trunk/zookeeperAdmin.html#sc_zkMulitServerSetup