zookeeper安装教程（伪分布式和分布式）

最新推荐文章于 2024-06-27 15:05:26 发布

大数据技术之路---花火

最新推荐文章于 2024-06-27 15:05:26 发布

阅读量1k

点赞数

分类专栏：分布式 zookeeper 大数据文章标签：分布式 zookeeper

本文链接：https://blog.csdn.net/qq_37095882/article/details/77658152

版权

大数据同时被 3 个专栏收录

24 篇文章 0 订阅

订阅专栏

分布式

12 篇文章 0 订阅

订阅专栏

zookeeper

1 篇文章 0 订阅

订阅专栏

伪分布式

创建环境目录
~ mkdir /home/conan/zoo/zk1
~ mkdir /home/conan/zoo/zk2
~ mkdir /home/conan/zoo/zk3

新建myid文件
~ echo “1” > /home/conan/zoo/zk1/myid
~ echo “2” > /home/conan/zoo/zk2/myid
~ echo “3” > /home/conan/zoo/zk3/myid

分别修改配置文件
修改：dataDir,clientPort
增加：集群的实例，server.X，”X”表示每个目录中的myid的值

~ vi /home/conan/toolkit/zookeeper345/conf/zk1.cfg
ckTime=2000
initLimit=10
syncLimit=5
dataDir=/home/conan/zoo/zk1
clientPort=2181
server.1=192.168.1.201:2888:3888
server.2=192.168.1.201:2889:3889
server.3=192.168.1.201:2890:3890

~ vi /home/conan/toolkit/zookeeper345/conf/zk2.cfg
ckTime=2000
initLimit=10
syncLimit=5
dataDir=/home/conan/zoo/zk2
clientPort=2182
server.1=192.168.1.201:2888:3888
server.2=192.168.1.201:2889:3889
server.3=192.168.1.201:2890:3890

~ vi /home/conan/toolkit/zookeeper345/conf/zk3.cfg
tickTime=2000
initLimit=10
syncLimit=5
dataDir=/home/conan/zoo/zk3
clientPort=2183
server.1=192.168.1.201:2888:3888
server.2=192.168.1.201:2889:3889
server.3=192.168.1.201:2890:3890
3个节点的ZooKeeper集群配置完成，接下来我们的启动服务。

启动集群

~ /home/conan/toolkit/zookeeper345/bin/zkServer.sh start zk1.cfg
~ /home/conan/toolkit/zookeeper345/bin/zkServer.sh start zk2.cfg
~ /home/conan/toolkit/zookeeper345/bin/zkServer.sh start zk3.cfg

~ jps
5422 QuorumPeerMain
5395 QuorumPeerMain
5463 QuorumPeerMain
5494 Jps

查看节点状态
~ /home/conan/toolkit/zookeeper345/bin/zkServer.sh status zk1.cfg
JMX enabled by default
Using config: /home/conan/toolkit/zookeeper345/bin/../conf/zk1.cfg
Mode: follower

~ /home/conan/toolkit/zookeeper345/bin/zkServer.sh status zk2.cfg
JMX enabled by default
Using config: /home/conan/toolkit/zookeeper345/bin/../conf/zk2.cfg
Mode: leader

~ /home/conan/toolkit/zookeeper345/bin/zkServer.sh status zk3.cfg
JMX enabled by default
Using config: /home/conan/toolkit/zookeeper345/bin/../conf/zk3.cfg
Mode: follower
我们可以看到zk2是leader，zk1和zk3是follower

查看ZooKeeper物理文件目录结构
~ tree -L 3 /home/conan/zoo
/home/conan/zoo
├── zk0
├── zk1
│ ├── myid
│ ├── version-2
│ │ ├── acceptedEpoch
│ │ ├── currentEpoch
│ │ ├── log.100000001
│ │ └── snapshot.0
│ └── zookeeper_server.pid
├── zk2
│ ├── myid
│ ├── version-2
│ │ ├── acceptedEpoch
│ │ ├── currentEpoch
│ │ ├── log.100000001
│ │ └── snapshot.0
│ └── zookeeper_server.pid
└── zk3
├── myid
├── version-2
│ ├── acceptedEpoch
│ ├── currentEpoch
│ ├── log.100000001
│ └── snapshot.100000000
└── zookeeper_server.pid

7 directories, 18 files

zookeeper命令行操作
我们通过客户端连接ZooKeeper的集群，我们可以任意的zookeeper是进行连接。
~ /home/conan/toolkit/zookeeper345/bin/zkCli.sh -server 192.168.1.201:2181

分布式

1.上传zk安装包

2.解压
3.在配置zookeeper之前，在集群中每一个节点上都配好主机名和所有节点的IP与主机名的映射，关闭防火墙
4.配置（先在一台节点上配置）
4.1添加一个zoo.cfg配置文件
$ZOOKEEPER/conf
mv zoo_sample.cfg zoo.cfg

4.2修改配置文件（zoo.cfg）
    dataDir=/itcast/zookeeper-3.4.5/data

    server.1=itcast05:2888:3888
    server.2=itcast06:2888:3888
    server.3=itcast07:2888:3888

4.3在（dataDir=/itcast/zookeeper-3.4.5/data）创建一个myid文件，里面内容是server.N中的N（server.2里面内容为2）
    echo "1" > myid

4.4将配置好的zk拷贝到其他节点
    scp -r /itcast/zookeeper-3.4.5/ itcast06:/itcast/
    scp -r /itcast/zookeeper-3.4.5/ itcast07:/itcast/

4.5注意：在其他节点上一定要修改myid的内容
    在itcast06应该讲myid的内容改为2 （echo "6" > myid）
    在itcast07应该讲myid的内容改为3 （echo "7" > myid）

4.启动集群
分别启动zk
./zkServer.sh start
查看集群节点状态
./zkServer.sh status

启动命令行客户端：
[root@itcast03 bin]# ./zkCli.sh -server itcast03:2181

zookeeper常见命令：

[zk: 192.168.1.201(CONNECTED) 1] help
ZooKeeper -server host:port cmd args
connect host:port
get path [watch]
ls path [watch]
set path data [version]
rmr path
delquota [-n|-b] path
quit
printwatches on|off
create [-s] [-e] path data acl
stat path [watch]
close
ls2 path [watch]
history
listquota path
setAcl path acl
getAcl path
sync path
redo cmdno
addauth scheme auth
delete path [version]
setquota -n|-b val path
ZooKeeper的结构，很像是目录结构，我们看到了像ls这样熟悉的命令。

ls,查看/目录内容
[zk: 192.168.1.201(CONNECTED) 1] ls /
[zookeeper]

create,创建一个znode节点
[zk: 192.168.1.201(CONNECTED) 2] create /node conan
Created /node

get,查看/node的数据信息
[zk: 192.168.1.201(CONNECTED) 4] get /node
conan
cZxid = 0x100000006
ctime = Mon Aug 12 05:32:49 CST 2013
mZxid = 0x100000006
mtime = Mon Aug 12 05:32:49 CST 2013
pZxid = 0x100000006
cversion = 0
dataVersion = 0
aclVersion = 0
ephemeralOwner = 0x0
dataLength = 5
numChildren = 0

set,修改数据
[zk: 192.168.1.201(CONNECTED) 5] set /node fens.me
cZxid = 0x100000006
ctime = Mon Aug 12 05:32:49 CST 2013
mZxid = 0x100000007
mtime = Mon Aug 12 05:34:32 CST 2013
pZxid = 0x100000006
cversion = 0
dataVersion = 1
aclVersion = 0
ephemeralOwner = 0x0
dataLength = 7
numChildren = 0

get,再查看/node的数据信息，已改为fens.me
[zk: 192.168.1.201(CONNECTED) 6] get /node
fens.me
cZxid = 0x100000006
ctime = Mon Aug 12 05:32:49 CST 2013
mZxid = 0x100000007
mtime = Mon Aug 12 05:34:32 CST 2013
pZxid = 0x100000006
cversion = 0
dataVersion = 1
aclVersion = 0
ephemeralOwner = 0x0
dataLength = 7
numChildren = 0

delete,删除/node
[zk: 192.168.1.201(CONNECTED) 7] delete /node
[zk: 192.168.1.201(CONNECTED) 8] ls /
[zookeeper]

quit,退出客户端连接
[zk: 192.168.1.201(CONNECTED) 19] quit
Quitting…
2013-08-12 05:40:29,304 [myid:] - INFO [main:ZooKeeper@684] - Session: 0x1406f3c1ef90002 closed
2013-08-12 05:40:29,305 [myid:] - INFO [main-EventThread:ClientCnxn$EventThread@509] - EventThread shut down

用Java开发zookeeper：

package org.conan.zookeeper.demo;

import java.io.IOException;

import org.apache.zookeeper.CreateMode;
import org.apache.zookeeper.KeeperException;
import org.apache.zookeeper.WatchedEvent;
import org.apache.zookeeper.Watcher;
import org.apache.zookeeper.ZooDefs.Ids;
import org.apache.zookeeper.ZooKeeper;

public class BasicDemo1 {

    public static void main(String[] args) throws IOException, KeeperException, InterruptedException {
        // 创建一个与服务器的连接
        ZooKeeper zk = new ZooKeeper("192.168.1.201:2181", 60000, new Watcher() {
            // 监控所有被触发的事件
            public void process(WatchedEvent event) {
                System.out.println("EVENT:" + event.getType());
            }
        });

        // 查看根节点
        System.out.println("ls / => " + zk.getChildren("/", true));

        // 创建一个目录节点
        if (zk.exists("/node", true) == null) {
            zk.create("/node", "conan".getBytes(), Ids.OPEN_ACL_UNSAFE, CreateMode.PERSISTENT);
            System.out.println("create /node conan");
            // 查看/node节点数据
            System.out.println("get /node => " + new String(zk.getData("/node", false, null)));
            // 查看根节点
            System.out.println("ls / => " + zk.getChildren("/", true));
        }

        // 创建一个子目录节点
        if (zk.exists("/node/sub1", true) == null) {
            zk.create("/node/sub1", "sub1".getBytes(), Ids.OPEN_ACL_UNSAFE, CreateMode.PERSISTENT);
            System.out.println("create /node/sub1 sub1");
            // 查看node节点
            System.out.println("ls /node => " + zk.getChildren("/node", true));
        }

        // 修改节点数据
        if (zk.exists("/node", true) != null) {
            zk.setData("/node", "changed".getBytes(), -1);
            // 查看/node节点数据
            System.out.println("get /node => " + new String(zk.getData("/node", false, null)));
        }

        // 删除节点
        if (zk.exists("/node/sub1", true) != null) {
            zk.delete("/node/sub1", -1);
            zk.delete("/node", -1);
            // 查看根节点
            System.out.println("ls / => " + zk.getChildren("/", true));
        }

        // 关闭连接
        zk.close();
    }
}

运行结果：

2013-08-12 15:33:29,699 [myid:] - INFO [main:Environment@97] - Client environment:zookeeper.version=3.3.1-942149, built on 05/07/2010 17:14 GMT
2013-08-12 15:33:29,702 [myid:] - INFO [main:Environment@97] - Client environment:host.name=PC201304202140
2013-08-12 15:33:29,702 [myid:] - INFO [main:Environment@97] - Client environment:java.version=1.6.0_45
2013-08-12 15:33:29,702 [myid:] - INFO [main:Environment@97] - Client environment:java.vendor=Sun Microsystems Inc.
2013-08-12 15:33:29,702 [myid:] - INFO [main:Environment@97] - Client environment:java.home=D:\toolkit\java\jdk6\jre
2013-08-12 15:33:29,703 [myid:] - INFO [main:Environment@97] - Client environment:java.class.path=D:\workspace\java\zkdemo\target\classes;C:\Users\Administrator.m2\repository\org\apache\hadoop\zookeeper\3.3.1\zookeeper-3.3.1.jar;C:\Users\Administrator.m2\repository\log4j\log4j\1.2.15\log4j-1.2.15.jar;C:\Users\Administrator.m2\repository\javax\mail\mail\1.4\mail-1.4.jar;C:\Users\Administrator.m2\repository\javax\activation\activation\1.1\activation-1.1.jar;C:\Users\Administrator.m2\repository\jline\jline\0.9.94\jline-0.9.94.jar;C:\Users\Administrator.m2\repository\junit\junit\3.8.1\junit-3.8.1.jar
2013-08-12 15:33:29,703 [myid:] - INFO [main:Environment@97] - Client environment:java.library.path=D:\toolkit\java\jdk6\bin;C:\Windows\Sun\Java\bin;C:\Windows\system32;C:\Windows;D:\toolkit\Rtools\bin;D:\toolkit\Rtools\gcc-4.6.3\bin;C:\Program Files (x86)\Common Files\NetSarang;C:\Windows\system32;C:\Windows;C:\Windows\System32\Wbem;C:\Windows\System32\WindowsPowerShell\v1.0\;D:\toolkit\Git\cmd;D:\toolkit\Git\bin;C:\Program Files (x86)\Microsoft SQL Server\100\Tools\Binn\;C:\Program Files\Microsoft SQL Server\100\Tools\Binn\;C:\Program Files\Microsoft SQL Server\100\DTS\Binn\;c:\Program Files (x86)\Common Files\Ulead Systems\MPEG;C:\Program Files (x86)\QuickTime\QTSystem\;D:\toolkit\MiKTex\miktex\bin\x64\;D:\toolkit\sshclient;D:\toolkit\ant19\bin;D:\toolkit\eclipse;D:\toolkit\gradle15\bin;D:\toolkit\java\jdk6\bin;D:\toolkit\maven3\bin;D:\toolkit\mysql56\bin;D:\toolkit\python27;D:\toolkit\putty;C:\Program Files\R\R-3.0.1\bin\x64;D:\toolkit\mongodb243\bin;D:\toolkit\php54;D:\toolkit\nginx140;D:\toolkit\nodejs;D:\toolkit\npm12\bin;D:\toolkit\java\jdk6\jre\bin\server;.
2013-08-12 15:33:29,703 [myid:] - INFO [main:Environment@97] - Client environment:java.io.tmpdir=C:\Users\ADMINI~1\AppData\Local\Temp\
2013-08-12 15:33:29,704 [myid:] - INFO [main:Environment@97] - Client environment:java.compiler=
2013-08-12 15:33:29,704 [myid:] - INFO [main:Environment@97] - Client environment:os.name=Windows 7
2013-08-12 15:33:29,704 [myid:] - INFO [main:Environment@97] - Client environment:os.arch=amd64
2013-08-12 15:33:29,705 [myid:] - INFO [main:Environment@97] - Client environment:os.version=6.1
2013-08-12 15:33:29,705 [myid:] - INFO [main:Environment@97] - Client environment:user.name=Administrator
2013-08-12 15:33:29,705 [myid:] - INFO [main:Environment@97] - Client environment:user.home=C:\Users\Administrator
2013-08-12 15:33:29,706 [myid:] - INFO [main:Environment@97] - Client environment:user.dir=D:\workspace\java\zkdemo
2013-08-12 15:33:29,707 [myid:] - INFO [main:ZooKeeper@373] - Initiating client connection, connectString=192.168.1.201:2181 sessionTimeout=60000 watcher=org.conan.zookeeper.demo.BasicDemo1 $1@3dfeca64 2013-08-12 15:33:29,731 [myid:] - INFO [main-SendThread():ClientCnxn$ SendThread@1000] - Opening socket connection to server /192.168.1.201:2181
2013-08-12 15:33:38,736 [myid:] - INFO [main-SendThread(192.168.1.201:2181):ClientCnxn $SendThread@908] - Socket connection established to 192.168.1.201/192.168.1.201:2181, initiating session 2013-08-12 15:33:38,804 [myid:] - INFO [main-SendThread(192.168.1.201:2181):ClientCnxn$ SendThread@701] - Session establishment complete on server 192.168.1.201/192.168.1.201:2181, sessionid = 0x1406f3c1ef9000d, negotiated timeout = 60000
EVENT:None
ls / => [zookeeper]
EVENT:NodeCreated
EVENT:NodeChildrenChanged
create /node conan
get /node => conan
ls / => [node, zookeeper]
EVENT:NodeCreated
create /node/sub1 sub1
ls /node => [sub1]
EVENT:NodeDataChanged
get /node => changed
EVENT:NodeDeleted
EVENT:NodeChildrenChanged
EVENT:NodeChildrenChanged
ls / => [zookeeper]
2013-08-12 15:33:38,877 [myid:] - INFO [main:ZooKeeper@538] - Session: 0x1406f3c1ef9000d closed

pom.xml，maven配置文件

<?xml version="1.0" encoding="UTF-8"?>
<project
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd"
xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
<modelVersion>4.0.0</modelVersion>
<groupId>org.conan.zookeeper</groupId>
<artifactId>zkdemo</artifactId>
<version>0.0.1</version>
<name>zkdemo</name>
<description>zkdemo</description>

<dependencies>
    <dependency>
    <groupId>org.apache.hadoop</groupId>
    <artifactId>zookeeper</artifactId>
    <version>3.3.1</version>
    <exclusions>
        <exclusion>
            <groupId>javax.jms</groupId>
            <artifactId>jms</artifactId>
        </exclusion>

        <exclusion>
            <groupId>com.sun.jdmk</groupId>
            <artifactId>jmxtools</artifactId>
        </exclusion>

        <exclusion>
            <groupId>com.sun.jmx</groupId>
            <artifactId>jmxri</artifactId>
        </exclusion>
    </exclusions>
    </dependency>
</dependencies>
</project>

Zoo.cfg中配置参数含义：

zookeeper的默认配置文件为zookeeper/conf/zoo_sample.cfg，需要将其修改为zoo.cfg。其中各配置项的含义，解释如下：

1.tickTime：CS通信心跳时间
Zookeeper 服务器之间或客户端与服务器之间维持心跳的时间间隔，也就是每个 tickTime 时间就会发送一个心跳。tickTime以毫秒为单位。
tickTime=2000

2.initLimit：LF初始通信时限
集群中的follower服务器(F)与leader服务器(L)之间初始连接时能容忍的最多心跳数（tickTime的数量）。
initLimit=5

3.syncLimit：LF同步通信时限
集群中的follower服务器与leader服务器之间请求和应答之间能容忍的最多心跳数（tickTime的数量）。
syncLimit=2

4.dataDir：数据文件目录
Zookeeper保存数据的目录，默认情况下，Zookeeper将写数据的日志文件也保存在这个目录里。
dataDir=/home/michael/opt/zookeeper/data

5.clientPort：客户端连接端口
客户端连接 Zookeeper 服务器的端口，Zookeeper 会监听这个端口，接受客户端的访问请求。
clientPort=2181

6.服务器名称与地址：集群信息（服务器编号，服务器地址，LF通信端口，选举端口）
这个配置项的书写格式比较特殊，规则如下：
server.N=YYY:A:B

server.1=itcast05:2888:3888
server.2=itcast06:2888:3888
server.3=itcast07:2888:3888

zookeeper客户端启动报错：
EndOfStreamException: Unable to read additional data from client sessionid 0x0, likely client has closed socket
at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:220)
at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
at java.lang.Thread.run(Thread.java:745)

问题：
配置的超时时间太短，Zookeeper没有读完Consumer的数据，连接就被Consumer断开了
解决：
在config/server.properties文件中将超时连接属性的值调大一点，如下：
1.zookeeper.session.timeout.ms=400000
注：也可以在java代码中设置！

大数据技术之路---花火

关注

0
点赞
踩
0

收藏

觉得还不错? 一键收藏
0
评论
zookeeper安装教程（伪分布式和分布式）

伪分布式创建环境目录 ~ mkdir /home/conan/zoo/zk1 ~ mkdir /home/conan/zoo/zk2 ~ mkdir /home/conan/zoo/zk3新建myid文件 ~ echo “1” > /home/conan/zoo/zk1/myid ~ echo “2” > /home/conan/zoo/zk2/myid ~ echo “3” > /ho
复制链接

扫一扫

专栏目录