大数据之Zookeeper(一)：zookeeper集群搭建

下载安装包

进入zookeeper官网https://zookeeper.apache.org/，

 

Getting Started

Start by installing ZooKeeper on a single machine or a very small cluster.

1. Learn about ZooKeeper by reading the documentation.
2. Download ZooKeeper from the release page

点击Download,进入download页面，下面是官网download部分内容

 

Download

Apache ZooKeeper 3.6.0 is our latest stable release.

Apache ZooKeeper 3.6.0

Apache ZooKeeper 3.6.0(asc, sha512)

Apache ZooKeeper 3.6.0 Source Release(asc, sha512)

You can verify the integrity of a downloaded release using the PGP signatures and hashes (MD5 or SHA1) hosted at the main Apache distro site. For additional information, refer to the Apache documentation for verifying the integrity of Apache project releases.

---

Older releases are available.

Apache ZooKeeper 3.5.7

Apache ZooKeeper 3.5.7(asc, sha512)

Apache ZooKeeper 3.5.7 Source Release(asc, sha512)

 

可以看到，到目前为止，最新的zookeeper版本有3.6.0，3.5.7。3.6.0是3.6版本分支的第一个版本，按经验大版本分支第一个版本是不推荐使用的，因为可能存在不少需要修复的bug。所以我这里选择使用3.5.7版本。

 

zookeeper集群配置

1 解压

下载好zookeeper安装包后，上传到服务器，我存放的目录为/usr/bigdata_sotfs/ ,解压

 tar -zxvf apache-zookeeper-3.5.7-bin.tar.gz

查看解压后的目录

[root@hdp01 bigdata_sotfs]# ls -l
total 9096
drwxr-xr-x. 6 hadoop hadoop     134 Mar 10 22:03 apache-zookeeper-3.5.7-bin
-rw-r--r--. 1 hadoop hadoop 9311744 Apr  4  2020 apache-zookeeper-3.5.7-bin.tar.gz
 

解压成功！

 

2 创建软连接

因为这个目录名有点长，进入该目录后，名称所占位置有点长

[root@hdp01 bigdata_sotfs]# cd apache-zookeeper-3.5.7-bin
[root@hdp01 apache-zookeeper-3.5.7-bin]# pwd
/usr/bigdata_sotfs/apache-zookeeper-3.5.7-bin
[root@hdp01 apache-zookeeper-3.5.7-bin]# 

我们创建一个软连接

ln -s /usr/bigdata_sotfs/apache-zookeeper-3.5.7-bin /usr/bigdata_sotfs/zookeeper

[root@hdp01 bigdata_sotfs]# ls -l
total 9096
drwxr-xr-x. 6 hadoop hadoop     134 Mar 10 22:03 apache-zookeeper-3.5.7-bin
-rw-r--r--. 1 hadoop hadoop 9311744 Apr  4  2020 apache-zookeeper-3.5.7-bin.tar.gz
lrwxrwxrwx. 1 root   root        45 Mar 10 23:49 zookeeper -> /usr/bigdata_sotfs/apache-zookeeper-3.5.7-bin

通过查看文件，发现软连接已经创建成功

 

3 编辑配置文件

进入zookeeper配置文件目录 conf

[root@hdp01 bigdata_sotfs]# cd zookeeper/conf/
[root@hdp01 conf]# ls -l
total 16
-rw-r--r--. 1 hadoop hadoop  535 May  4  2018 configuration.xsl
-rw-r--r--. 1 hadoop hadoop 2712 Feb  7 21:54 log4j.properties
-rw-r--r--. 1 hadoop hadoop  922 Feb  7 21:54 zoo_sample.cfg

将zookeeper的配置文件zoo_sample.cfg备份一个

cp zoo_sample.cfg zoo.cfg

然后编辑

vim zoo.cfg

# The number of milliseconds of each tick
tickTime=2000
# The number of ticks that the initial 
# synchronization phase can take
initLimit=10
# The number of ticks that can pass between 
# sending a request and getting an acknowledgement
syncLimit=5
# the directory where the snapshot is stored.
# do not use /tmp for storage, /tmp here is just 
# example sakes.
dataDir=/var/lib/zookeeper
#transction logs to improve performance
dataLogDir=/var/lib/zookeeper/logs
# the port at which the clients will connect
clientPort=2181
# the maximum number of client connections.
# increase this if you need to handle more clients
maxClientCnxns=200
#
# Be sure to read the maintenance section of the 
# administrator guide before turning on autopurge.
#
# http://zookeeper.apache.org/doc/current/zookeeperAdmin.html#sc_maintenance
#
# The number of snapshots to retain in dataDir
#autopurge.snapRetainCount=3
# Purge task interval in hours
# Set to "0" to disable auto purge feature
#autopurge.purgeInterval=1
server.1=hdp01:2888:3888
server.2=hdp02:2888:3888
server.3=hdp03:2888:3888

 

这些配置，都有注释，我们重点说3个地方，

第一个dataDir 这个是设置 zookeeper数据目录，默认是/tmp/zookeeper 这个是会被系统定期清理的，所以需要更改为我们自己的目录 /var/lib/zookeeper   

第二个dataLogDir原始配置文件中没有这个配置，而官网推荐配上，它是存放zookeeper事务日志的地方，默认是和dataDir一个地方，官网说该配置可以提高zookeeper性能，我们应当另放一个地方。这里为 /var/lib/zookeeper/logs 

第三个就是 server.x=host:port1:port2 这个模式的配置了，

        那个x数字，代表zookeeper节点的编号，可配置的编号为1，2，3，4等数字，我们需要在dataDir对应的目录下创建一个myid 文件，并向其中写入zookeeper节点编号

            echo 1 > /var/lib/zookeeper/myid

       port1 代表zookeeper集群各节点的通信端口，比如leader和各follow节点的通信

       port2 代表zookeeper集群选取leader的端口，第一次启动zookeeper集群，或者当leader节点挂了之后，都需要选取新的leader节点，就要用到改端口

 

4 将配置复制到其他节点

一台zookeeper节点配置好后，将zookeeper目录及其文件复制到其他节点，并按照配置创建好相应的数据目录，和对应的myid文件，注意myid文件里面的编号需要依照zoo.cfg文件里面的

server.1=hdp01:2888:3888
server.2=hdp02:2888:3888
server.3=hdp03:2888:3888

这个依次编好，主机名也要一一对应上