Zookeeper学习

最新推荐文章于 2023-08-29 11:36:53 发布

QingyunChen

最新推荐文章于 2023-08-29 11:36:53 发布

阅读量208

点赞数

分类专栏： zookeeper 大数据文章标签： zookeeper

本文链接：https://blog.csdn.net/piaoyunlive/article/details/107945666

版权

zookeeper 同时被 2 个专栏收录

1 篇文章 0 订阅

订阅专栏

大数据

1 篇文章 0 订阅

订阅专栏

Zookeeper

1. 下载安装Zookeeper

下载地址: https://www.apache.org/dyn/closer.cgi/zookeeper/

解压文件至目录: /opt/module/zookeeper-3.4.14

2. 配置Zookeeper

vim /opt/module/zookeeper-3.4.14/conf/zoo.cfg

dataDir=/opt/module/zookeeper-3.4.14/zkData

在这里插入图片描述

3. 启动Zookeeper

#Usage: ./zkServer.sh {start|start-foreground|stop|restart|status|upgrade|print-cmd}
#启动服务器
[root@MiWiFi-R3-srv bin]# ./zkServer.sh start
ZooKeeper JMX enabled by default
Using config: /opt/module/zookeeper-3.4.14/bin/../conf/zoo.cfg
Starting zookeeper ... STARTED
[root@MiWiFi-R3-srv bin]# jps
8636 QuorumPeerMain
8653 Jps
#查看启动状态
[root@MiWiFi-R3-srv bin]# ./zkServer.sh status
ZooKeeper JMX enabled by default
Using config: /opt/module/zookeeper-3.4.14/bin/../conf/zoo.cfg
Mode: standalone
#启动客户端(没有start)
[root@MiWiFi-R3-srv bin]# ./zkCli.sh
#退出客户端 quit
[zk: localhost:2181(CONNECTED) 3] quit
Quitting...
2019-03-24 20:54:52,000 [myid:] - INFO  [main:ZooKeeper@693] - Session: 0x10002be78c50000 closed
2019-03-24 20:54:52,002 [myid:] - INFO  [main-EventThread:ClientCnxn$EventThread@522] - EventThread shut down for session: 0x10002be78c50000
#退出服务
[root@MiWiFi-R3-srv bin]# ./zkServer.sh stop
ZooKeeper JMX enabled by default
Using config: /opt/module/zookeeper-3.4.14/bin/../conf/zoo.cfg
Stopping zookeeper ... STOPPED

3.1 Zookeeper启动报错

Zookeeper启动报错 already running as process xxx/...Error contacting service. It is probably not running.

rm -rf zookeeper_server.pid

在这里插入图片描述

4. Zookeeper内部原理

4.1 选举机制

半数机制:集群中半数以上机器存活,集群可用.所以Zookeeper适合安装奇数台服务器.

4.2 节点类型

持久(Persistent):客户端和服务器断开连接后,创建的节点不删除;
短暂(Ephemeral):客户端和服务器端断开连接后,创建的节点自己删除.

4.3 Stat结构体

[zk: localhost:2181(CONNECTED) 6] stat /
cZxid = 0x0  #创建节点的事务zxid
ctime = Wed Dec 31 19:00:00 EST 1969	#被创建的毫秒数
mZxid = 0x0	 #最后更新的事务zxid
mtime = Wed Dec 31 19:00:00 EST 1969	#最后修改的毫秒数
pZxid = 0x20000001b	#最后更新的子节点
cversion = 1		#子节点变化号,znode子节点修改次数
dataVersion = 0		#数据变化号
aclVersion = 0		#访问控制列表的变化号
ephemeralOwner = 0x0	#如果是临时节点,该znode拥有者的session id .如果不是临时节点则是0
dataLength = 0		#znode的数据长度
numChildren = 1		#znode子节点数据

4.4 监听器原理

4.4.1 监听原理详解

首先要有一个main()线程;
在main()创建Zookeeper客户端,这时就会创建两个线程,一个负责网络连接通信(connet),一个负责监听(Listener);
通过Connect线程将注册的监听事件发送给Zookeeper;
在Zookeeper的注册监听器列表中将注册的监听事件添加到列表中;
Zookeeper监听到所有数据或路径变化,就会将这个消息发送给Listener线程.s

在这里插入图片描述

4.4.2 常见的监听

监听节点数据变化: get path [watch]
监听子节点增减变化: ls path [watch]

4.5 写数据流程

Client向Zookeeper的Server1上写数据,发送一个写请求;
如果Server1不是Leader,那么Server1会把接受到的进一步转发给Leader,因为每个Zookeeper的Server里面有一个是Leader.这个Leader会将写请求广播给各个Server,比如Server1和Server2,各个Server写成功后就会通知Leader;
当Leader收到大多数Server数据写成功了,那么就说明数据写成功了.如果这里三个节点的话,只要有两个数据写成功了,那么就认为数据写成功了.写成功后,Leader会告诉Server1数据写成功了.

5. 分布式部署

在 /opt/module/zookeeper-3.4.14/目录下创建zkData
在 /opt/module/zookeeper-3.4.14/zkData目录下创建一个myid文件;
编辑myid文件,在文件中添加与server对应的编号(001,002,003服务器对应myid内容为1,2,3);
配置zoo.cfg文件(/opt/module/zookeeper-3.4.14/conf),目录下的zoo_sample.cfg重命名为zoo.cfg;
打开zoo.cfg文件,修改数据存储路径配置:
```
dataDir=/opt/module/zookeeper-3.4.14/zkData
```
增加如下配置
```
############cluster##############
server.1=0.0.0.0:2888:3888
server.2=192.168.31.75:2888:3888
server.3=192.168.31.188:2888:3888
```
配置参数解读

server.A=B:C:D
- A 是一个数字,表示这个是第几号服务器;
集群模式下配置一个文件myid,这个文件在dataDir目录下,该文件中有一个数据就是A的值,Zookeeper启动时读取此文件,拿到里面的数据与zoo.cfg里面的配置信息比较从而判断到底是哪个server;
- B 是这个服务的IP地址;
- C 是这个服务器与集群中的Leader服务器交换信息的端口;
- D 是万一集群中的Leader服务器挂了,需要一个端口来重新进行选举选出一个新的;

6. 客户端命令操作

6.1 启动客户端

#启动客户端
[root@server2 bin]# ./zkCli.sh

6.2 帮助

[zk: localhost:2181(CONNECTED) 2] help
ZooKeeper -server host:port cmd args
	stat path [watch]
	set path data [version]
	ls path [watch]
	delquota [-n|-b] path
	ls2 path [watch]
	setAcl path acl
	setquota -n|-b val path
	history 
	redo cmdno
	printwatches on|off
	delete path [version]
	sync path
	listquota path
	rmr path
	get path [watch]
	create [-s] [-e] path data acl
	addauth scheme auth
	quit 
	getAcl path
	close 
	connect host:port

6.3 查看当前znode中所包含的内容:ls path [watch]

[zk: localhost:2181(CONNECTED) 3] ls /
[zookeeper]

6.4 查看当前节点详细数据

[zk: localhost:2181(CONNECTED) 4] ls2 /
[zookeeper]
cZxid = 0x0
ctime = Wed Dec 31 19:00:00 EST 1969
mZxid = 0x0
mtime = Wed Dec 31 19:00:00 EST 1969
pZxid = 0x0
cversion = -1
dataVersion = 0
aclVersion = 0
ephemeralOwner = 0x0
dataLength = 0
numChildren = 1

6.5 创建2个普通节点

[zk: localhost:2181(CONNECTED) 5] create /sanguo "jinlian"
Created /sanguo
[zk: localhost:2181(CONNECTED) 6] ls / 
[zookeeper, sanguo]

[zk: localhost:2181(CONNECTED) 7] create /sanguo/shuguo "liubei"
Created /sanguo/shuguo
[zk: localhost:2181(CONNECTED) 8] ls /sanguo
[shuguo]

6.6 获取节点数据

[zk: localhost:2181(CONNECTED) 10] get /sanguo/shuguo
liubei
cZxid = 0x200000003
ctime = Sat Apr 27 19:27:52 EDT 2019
mZxid = 0x200000003
mtime = Sat Apr 27 19:27:52 EDT 2019
pZxid = 0x200000003
cversion = 0
dataVersion = 0
aclVersion = 0
ephemeralOwner = 0x0
dataLength = 6
numChildren = 0

6.7 创建短暂节点

[zk: localhost:2181(CONNECTED) 12] create -e /sanguo/wuguo "zouyu"
Created /sanguo/wuguo
[zk: localhost:2181(CONNECTED) 13] ls /sanguo
[wuguo, shuguo]

6.8 创建带序号的节点

如果原来没有序号节点,序号从0开始依次递增.如果原节点下已有2个节点,则再排序时从2开始,以此类推.

[zk: localhost:2181(CONNECTED) 30] create -s /sanguo/weiguo "caocao"
Created /sanguo/webguo0000000002
[zk: localhost:2181(CONNECTED) 32] create -s /sanguo/weiguo "caocao"
Created /sanguo/webguo0000000003
[zk: localhost:2181(CONNECTED) 33] create -s /sanguo/weiguo "caocao"
Created /sanguo/webguo0000000004
[zk: localhost:2181(CONNECTED) 34] ls /sanguo                       
[wuguo, weiguo0000000004, weiguo0000000003, shuguo, weiguo0000000002]

6.9 修改节点数据值

[zk: localhost:2181(CONNECTED) 35]  set /sanguo/shuguo "liucan"
cZxid = 0x200000003
ctime = Sat Apr 27 19:27:52 EDT 2019
mZxid = 0x200000008
mtime = Sat Apr 27 19:43:02 EDT 2019
pZxid = 0x200000003
cversion = 0
dataVersion = 1
aclVersion = 0
ephemeralOwner = 0x0
dataLength = 6
numChildren = 0
[zk: localhost:2181(CONNECTED) 36] get /sanguo/shuguo
liucan
cZxid = 0x200000003
ctime = Sat Apr 27 19:27:52 EDT 2019
mZxid = 0x200000008
mtime = Sat Apr 27 19:43:02 EDT 2019
pZxid = 0x200000003
cversion = 0
dataVersion = 1
aclVersion = 0
ephemeralOwner = 0x0
dataLength = 6
numChildren = 0

6.10 节点值变化监听

一次监听仅有效一次

#server2客户端修改值
[zk: localhost:2181(CONNECTED) 1] set /sanguo "jingjing"
cZxid = 0x200000002
ctime = Sat Apr 27 19:27:18 EDT 2019
mZxid = 0x20000000b
mtime = Sat Apr 27 19:47:42 EDT 2019
pZxid = 0x200000007
cversion = 5
dataVersion = 1
aclVersion = 0
ephemeralOwner = 0x0
dataLength = 8
numChildren = 5

#server3客户端监听值
[zk: localhost:2181(CONNECTED) 2] get /sanguo watch
jinlian
cZxid = 0x200000002
ctime = Sat Apr 27 19:27:18 EDT 2019
mZxid = 0x200000002
mtime = Sat Apr 27 19:27:18 EDT 2019
pZxid = 0x200000007
cversion = 5
dataVersion = 0
aclVersion = 0
ephemeralOwner = 0x0
dataLength = 7
numChildren = 5
[zk: localhost:2181(CONNECTED) 3] 
WATCHER::

WatchedEvent state:SyncConnected type:NodeDataChanged path:/sanguo

在这里插入图片描述

6.11 监听节点的子节点变化(路径变化)

一次监听仅有效一次

#Server2修改
[zk: localhost:2181(CONNECTED) 1] create /sanguo/bangzhang "banzhang"
Created /sanguo/bangzhang

#server3监听
[zk: localhost:2181(CONNECTED) 8] ls /sanguo watch
[wuguo, webguo0000000004, webguo0000000003, shuguo, webguo0000000002]
[zk: localhost:2181(CONNECTED) 9] 
WATCHER::

WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/sanguo

[zk: localhost:2181(CONNECTED) 0] ls /sanguo
[wuguo, webguo0000000004, webguo0000000003, bangzhang, shuguo, webguo0000000002]

在这里插入图片描述

6.12 删除节点

[zk: localhost:2181(CONNECTED) 0] ls /sanguo
[wuguo, webguo0000000004, webguo0000000003, bangzhang, shuguo, webguo0000000002]

[zk: localhost:2181(CONNECTED) 2] delete /sanguo/bangzhang

[zk: localhost:2181(CONNECTED) 3] ls /sanguo
[wuguo, webguo0000000004, webguo0000000003, shuguo, webguo0000000002]

6.12 递归删除节点

[zk: localhost:2181(CONNECTED) 3] ls /sanguo
[wuguo, webguo0000000004, webguo0000000003, shuguo, webguo0000000002]

[zk: localhost:2181(CONNECTED) 4] rmr /sanguo   

[zk: localhost:2181(CONNECTED) 5] ls /
[zookeeper]

6.13 查看节点状态

[zk: localhost:2181(CONNECTED) 6] stat /
cZxid = 0x0  #创建节点的事务zxid
ctime = Wed Dec 31 19:00:00 EST 1969	#被创建的毫秒数
mZxid = 0x0	 #最后更新的事务zxid
mtime = Wed Dec 31 19:00:00 EST 1969	#最后修改的毫秒数
pZxid = 0x20000001b	#最后更新的子节点
cversion = 1		#子节点变化号,znode子节点修改次数
dataVersion = 0		#数据变化号
aclVersion = 0		#访问控制列表的变化号
ephemeralOwner = 0x0	#如果是临时节点,该znode拥有者的session id .如果不是临时节点则是0
dataLength = 0		#znode的数据长度
numChildren = 1		#znode子节点数据

6.14 JAVA连接zookeeper

MANVAN引入依赖


<dependency>
    <groupId>org.apache.zookeeper</groupId>
    <artifactId>zookeeper</artifactId>
    <version>3.4.10</version>
</dependency>

java测试代码

package pub.qingyun;

import org.apache.zookeeper.*;
import org.junit.Before;
import org.junit.Test;

import java.io.IOException;

public class TestZookeeper {

    private String connectString = "192.168.31.100:2181,192.168.31.75:2181,192.168.31.188:2181";
    private int sessionTimeout = 2000;
    private ZooKeeper zkClient;

    @Before
    public void init() throws IOException {
        zkClient = new ZooKeeper(connectString, sessionTimeout, new Watcher() {
            public void process(WatchedEvent watchedEvent) {

            }
        });
    }

    //1. 创建节点
    @Test
    public void createNode() throws KeeperException, InterruptedException {
        String path = zkClient.create("/qingyun", "huihui".getBytes(), ZooDefs.Ids.OPEN_ACL_UNSAFE, CreateMode.PERSISTENT);
        System.out.println("path:" + path);
    }
}

QingyunChen

关注

0
点赞
踩
0

收藏

觉得还不错? 一键收藏
0
评论
Zookeeper学习

Zookeeper1. 下载安装Zookeeper下载地址: https://www.apache.org/dyn/closer.cgi/zookeeper/解压文件至目录: /opt/module/zookeeper-3.4.142. 配置Zookeepervim /opt/module/zookeeper-3.4.14/conf/zoo.cfgdataDir=/opt/module/zookeeper-3.4.14/zkData3. 启动Zookeeperbin目录文件#Usa
复制链接

扫一扫

专栏目录