JAVA分布式篇2——Zookeeper
1、简介
Zookeeper是一个开源的分布式(多台服务器干一件事)的,为分布式应用提供协调服务的 Apache项目
2、工作机制
-
Zookeeper从设计模式角度来理解:是一个基于观察者模式(一个人干活,有人盯着他)设计的分 布式服务管理框架
- 存储 和 管理 大家都关心的数据 然后接受观察者的注册
- 通知已经注册的那些观察者做出相应的反应
-
Zookeeper = 文件系统 + 通知机制
-
类似美团
- 商家营业并入驻
- 获取到当前营业的饭店列表
- 服务器节点下线
- 服务器节点上下线事件通知
- 重新再去获取服务器列表,并注册监听
3、特点
- 是一个leader和多个follower来组成的集群
- 集群中只要有半数以上的节点存活,Zookeeper就能正常工作(5台服务器挂2台,没问题;4台服 务器挂2台,就停止)
- 全局数据一致性,每台服务器都保存一份相同的数据副本,无论client连接哪台server,数据都是 一致的
- 数据更新原子性,一次数据要么成功,要么失败(不成功便成仁)
- 实时性,在一定时间范围内,client能读取到最新数据
- 更新的请求按照顺序执行,会按照发送过来的顺序,逐一执行(发来123,执行123,而不是321 或者别的)
4、应用场景
- 统一命名服务
- 统一配置管理
- 服务器节点动态上下线
- 软负载均衡
5、下载安装(linux)
zookeeper官网下载:https://downloads.apache.org/zookeeper/
zookeeper3.6.3 百度云:https://pan.baidu.com/s/1OM0Lq2rm-8qcd8nb8TGfEg提取码:izco
linux需要有jdk
移动zookeeper
mv /root/apache-zookeeper-3.6.3-bin.tar.gz /usr/local/apache-zookeeper-3.6.3-bin.tar.gz
解压zookeeper
tar -zxvf apache-zookeeper-3.6.3-bin.tar.gz
zookeeper目录下创建文件夹
mkdir zkData
mkdir zkLog
拷贝配置文件
cp zoo_sample.cfg zoo.cfg
vim zoo.cfg
添加配置
dataDir=/usr/local/apache-zookeeper-3.6.3-bin/zkData
dataLogDir=/usr/local/apache-zookeeper-3.6.3-bin/zkLog
启动
[root@VM-16-14-centos bin]# ./zkServer.sh start
/usr/bin/java
ZooKeeper JMX enabled by default
Using config: /usr/local/apache-zookeeper-3.6.3-bin/bin/../conf/zoo.cfg
Starting zookeeper ... STARTED
测试
[root@VM-16-14-centos bin]# jps
1898 QuorumPeerMain
1981 Jps
[root@VM-16-14-centos bin]# netstat -apn | grep 2181
tcp6 0 0 :::2181 :::* LISTEN 1898/java
[root@VM-16-14-centos bin]# ./zkServer.sh status
/usr/bin/java
ZooKeeper JMX enabled by default
Using config: /usr/local/apache-zookeeper-3.6.3-bin/bin/../conf/zoo.cfg
Client port found: 2181. Client address: localhost. Client SSL: false.
Mode: standalone
启动客户端
[root@VM-16-14-centos bin]# ./zkCli.sh
[zk: localhost:2181(CONNECTED) 0] ls
ls [-s] [-w] [-R] path
[zk: localhost:2181(CONNECTED) 1] ls /
[zookeeper]
[zk: localhost:2181(CONNECTED) 2] quit
6、配置参数详解
- tickTime =2000:通信心跳数,Zookeeper服务器与客户端心跳时间,单位毫秒,服务器之间或客户端与服务器之间维持心跳的时间间隔,也就 是每个tickTime时间就会发送一个心跳,时间单位为毫秒
- initLimit =10:LF初始通信时限,集群中的Follower跟随者服务器与Leader领导者服务器之间,启动时能容忍的最多心跳数,10*2000(10个心跳时间)如果领导和跟随者没有发出心跳通信,就视为失效的连接,领导 和跟随者彻底断开
- syncLimit =5:LF同步通信时限,集群启动后,Leader与Follower之间的最大响应时间单位,假如响应超过syncLimit * tickTime->10秒,Leader就认为Follwer已经死掉,会将Follwer从服务器列表中删除
- dataDir:数据文件目录+数据持久化路径 主要用于保存Zookeeper中的数据
- dataLogDir:日志文件目录
- clientPort =2181:客户端连接端口 监听客户端连接的端口
7、节点及其类型
7.1、节点
- ZooKeeper数据模型的结构与linux文件系统很类似,整体上可以看作是一棵树,每个节点称做一 个ZNode(ZookeeperNode)
- 每一个ZNode默认能够存储1MB的数据(元数据),每个ZNode的路径都是唯一的
- 元数据(Metadata),又称中介数据、中继数据,为描述数据的数据(data about data),主要是描述数据属性(property)的信息,用来支持如指示存储位置、历史数据、 资源查找、文件记录等功能
7.2、节点类型
7.2.1、持久型
- 持久化目录节点(persistent)客户端与zookeeper断开连接后,该节点依旧存在
- 持久化顺序编号目录节点(persistent_sequential)客户端与zookeeper断开连接后,该节 点依旧存在,创建znode时设置顺序标识,znode名称后会附加一个值,顺序号是一个单调 递增的计数器,由父节点维护,例如:Znode001,Znode002…
7.2.2、短暂型
- 临时目录节点(ephemeral)客户端和服务器端断开连接后,创建的节点自动删除
- 临时顺序编号目录节点(ephemeral_sequential)客户端与zookeeper断开连接后,该节点 被删除,创建znode时设置顺序标识,znode名称后会附加一个值,顺序号是一个单调递增 的计数器,由父节点维护,例如:Znode001,Znode002…
8、分布式部署
8.1、编号配置
在zkData目录下书写myid文件(依次写上编号1、2、3)
vim myid
8.2、zoo.cfg配置
# The number of milliseconds of each tick
tickTime=2000
# The number of ticks that the initial
# synchronization phase can take
initLimit=10
# The number of ticks that can pass between
# sending a request and getting an acknowledgement
syncLimit=5
# the directory where the snapshot is stored.
# do not use /tmp for storage, /tmp here is just
# example sakes.
dataDir=/usr/local/apache-zookeeper-3.6.3-bin/zkData
# the port at which the clients will connect
clientPort=2181
# the maximum number of client connections.
# increase this if you need to handle more clients
#maxClientCnxns=60
#
# Be sure to read the maintenance section of the
# administrator guide before turning on autopurge.
#
# http://zookeeper.apache.org/doc/current/zookeeperAdmin.html#sc_maintenance
#
# The number of snapshots to retain in dataDir
#autopurge.snapRetainCount=3
# Purge task interval in hours
# Set to "0" to disable auto purge feature
#autopurge.purgeInterval=1
## Metrics Providers
#
# https://prometheus.io Metrics Exporter
#metricsProvider.className=org.apache.zookeeper.metrics.prometheus.PrometheusMetricsProvider
#metricsProvider.httpPort=7000
#metricsProvider.exportJvmInfo=true
#cluster
quorumListenOnAllIPs=true
server.1=101.34.116.9:2888:3888
server.2=106.54.85.216:2888:3888
server.3=81.70.1.65:2888:3888
server.A=B:C:D
- A:一个数字,表示第几号服务器(集群模式下配置的/opt/zookeeper/zkData/myid文件里面的数据就是A的值)
- B:服务器的ip地址
- C:与集群中Leader服务器交换信息的端口
- D:选举时专用端口,万一集群中的Leader服务器挂了,需要一个端口来重新进行选举,选 出一个新的Leader,而这个端口就是用来执行选举时服务器相互通信的端口
8.3、启动
[root@VM-8-13-centos bin]# ./zkServer.sh start
/usr/bin/java
ZooKeeper JMX enabled by default
Using config: /usr/local/apache-zookeeper-3.6.3-bin/bin/../conf/zoo.cfg
Starting zookeeper ... STARTED
[root@VM-8-13-centos bin]# jps
29545 QuorumPeerMain
29597 Jps
[root@VM-8-13-centos bin]# ./zkServer.sh status
/usr/bin/java
ZooKeeper JMX enabled by default
Using config: /usr/local/apache-zookeeper-3.6.3-bin/bin/../conf/zoo.cfg
Client port found: 2181. Client address: localhost. Client SSL: false.
Mode: leader
[root@VM-0-3-centos bin]# ./zkServer.sh start
/usr/bin/java
ZooKeeper JMX enabled by default
Using config: /usr/local/apache-zookeeper-3.6.3-bin/bin/../conf/zoo.cfg
Starting zookeeper ... STARTED
[root@VM-0-3-centos bin]# ./zkServer.sh status
/usr/bin/java
ZooKeeper JMX enabled by default
Using config: /usr/local/apache-zookeeper-3.6.3-bin/bin/../conf/zoo.cfg
Client port found: 2181. Client address: localhost. Client SSL: false.
Mode: follower
[root@VM-16-14-centos bin]# ./zkServer.sh start
/usr/bin/java
ZooKeeper JMX enabled by default
Using config: /usr/local/apache-zookeeper-3.6.3-bin/bin/../conf/zoo.cfg
Starting zookeeper ... STARTED
[root@VM-16-14-centos bin]# jps
29545 QuorumPeerMain
29597 Jps
[root@VM-16-14-centos bin]# ./zkServer.sh status
/usr/bin/java
ZooKeeper JMX enabled by default
Using config: /usr/local/apache-zookeeper-3.6.3-bin/bin/../conf/zoo.cfg
Client port found: 2181. Client address: localhost. Client SSL: false.
Mode: follower
8.4、问题解决
- 检查配置文件
- ping一下服务器
- 查看一下防火墙是否在关闭状态
- 查看防火墙状态:firewall-cmd --state
- 启动防火墙:systemctl start firewalld
- 关闭防火墙:systemctl stop firewalld.service
- 禁止防火墙开机启动:systemctl disable firewalld.service
9、客户端操作
9.1、客户端启动
./zkCli.sh
9.2、帮助
[zk: localhost:2181(CONNECTED) 0] help
ZooKeeper -server host:port -client-configuration properties-file cmd args
addWatch [-m mode] path # optional mode is one of [PERSISTENT, PERSISTENT_RECURSIVE] - default is PERSISTENT_RECURSIVE
addauth scheme auth
close
config [-c] [-w] [-s]
connect host:port
create [-s] [-e] [-c] [-t ttl] path [data] [acl]
delete [-v version] path
deleteall path [-b batch size]
delquota [-n|-b] path
get [-s] [-w] path
getAcl [-s] path
getAllChildrenNumber path
getEphemerals path
history
listquota path
ls [-s] [-w] [-R] path
printwatches on|off
quit
reconfig [-s] [-v version] [[-file path] | [-members serverID=host:port1:port2;port3[,...]*]] | [-add serverId=host:port1:port2;port3[,...]]* [-remove serverId[,...]*]
redo cmdno
removewatches path [-c|-d|-a] [-l]
set [-s] [-v version] path data
setAcl [-s] [-v version] [-R] path acl
setquota -n|-b val path
stat [-w] path
sync path
version
Command not found: Command not found help
9.3、基本命令
#查看当前znode中所包含的内容
[zk: localhost:2181(CONNECTED) 1] ls /
[zookeeper]
#查看当前节点详细数据
[zk: localhost:2181(CONNECTED) 2] ls -s /
[zookeeper]
cZxid = 0x0
ctime = Thu Jan 01 08:00:00 CST 1970
mZxid = 0x0
mtime = Thu Jan 01 08:00:00 CST 1970
pZxid = 0x0
cversion = -1
dataVersion = 0
aclVersion = 0
ephemeralOwner = 0x0
dataLength = 0
numChildren = 1
- cZxid:创建节点的事务 每次修改ZooKeeper状态都会收到一个zxid形式的时间戳,也就是ZooKeeper事务ID。 事务ID是ZooKeeper中所有修改总的次序。 每个修改都有唯一的zxid,如果zxid1小于zxid2,那么zxid1在zxid2之前发生
- ctime:被创建的毫秒数(从1970年开始)
- mZxid:最后更新的事务zxid
- mtime:最后修改的毫秒数(从1970年开始)
- pZxid:最后更新的子节点zxid
- cversion:创建版本号,子节点修改次数
- dataVersion:数据变化版本号
- aclVersion:权限版本号
- ephemeralOwner:如果是临时节点,这个是znode拥有者的session id,如果不是临时节点 则是0
- dataLength:数据长度 numChildren:子节点数
#创建节点
[zk: localhost:2181(CONNECTED) 3] create /china
Created /china
[zk: localhost:2181(CONNECTED) 4] create /usa
Created /usa
[zk: localhost:2181(CONNECTED) 5] ls /
[china, usa, zookeeper]
#获得节点的值
[zk: localhost:2181(CONNECTED) 6] create /china/nanyang "home"
Created /china/nanyang
[zk: localhost:2181(CONNECTED) 7] get /china/nanyang
home
#创建短暂节点
[zk: localhost:2181(CONNECTED) 8] create -e /xxx
Created /xxx
#创建带序号的节点
[zk: localhost:2181(CONNECTED) 9] create -s /uk
Created /uk0000000003
[zk: localhost:2181(CONNECTED) 10] create -s /uk
Created /uk0000000004
#获取值,设置值,监听值的变化
[zk: localhost:2181(CONNECTED) 11] set /usa "short"
[zk: localhost:2181(CONNECTED) 12] get /usa
short
#(另一个机器)
[zk: localhost:2181(CONNECTED) 1] addWatch /usa
[zk: localhost:2181(CONNECTED) 2]
WATCHER::
WatchedEvent state:SyncConnected type:NodeDataChanged path:/usa
# 删除节点
[zk: localhost:2181(CONNECTED) 13] delete /usa
[zk: localhost:2181(CONNECTED) 14] ls /
[china, uk0000000003, uk0000000004, xxx, zookeeper]
[zk: localhost:2181(CONNECTED) 15] create /uk0000000003/uk
Created /uk0000000003/uk
[zk: localhost:2181(CONNECTED) 16] deleteall /uk0000000003
10、API
10.1、导入依赖
<dependencies>
<!--zookeeper-->
<dependency>
<groupId>org.apache.zookeeper</groupId>
<artifactId>zookeeper</artifactId>
<version>3.6.3</version>
</dependency>
<dependency>
<groupId>org.apache.logging.log4j</groupId>
<artifactId>log4j-core</artifactId>
<version>2.14.0</version>
</dependency>
<!--junit-->
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>4.13</version>
<scope>test</scope>
</dependency>
</dependencies>
10.2、创建节点
public class ZookeeperTest {
private String con="101.34.116.9:2181,106.54.85.216:2181,81.70.1.65:2181";
//单位毫秒
private int time=60*1000;
private ZooKeeper zooKeeper;
@Before
public void zookeeper() throws IOException {
Watcher watcher=(watchedEvent)->System.out.println("监听到了变化");
zooKeeper = new ZooKeeper(con, time, watcher);
}
@Test
public void createNode() throws KeeperException, InterruptedException {
String s = zooKeeper.create("/idea", "idea".getBytes(), ZooDefs.Ids.OPEN_ACL_UNSAFE, CreateMode.PERSISTENT);
System.out.println(s);
}
}
- 一个ACL对象就是一个Id和permission对
- 类似linux的文件权限,不同的是共有5种操作:CREATE、READ、WRITE、DELETE、ADMIN(对应更改ACL的权限)
- OPEN_ACL_UNSAFE:创建开放节点,允许任意操作 (用的最少,其余的权限用的很 少)
- READ_ACL_UNSAFE:创建只读节点
- CREATOR_ALL_ACL:创建者才有全部权限
- CreateMode.PERSISTENT:持久型节点
10.3、查询节点
@Test
public void selectNode() throws KeeperException, InterruptedException {
byte[] data = zooKeeper.getData("/idea", false, new Stat());
System.out.println(new String(data));
}
10.4、修改节点
@Test
public void updateNode() throws KeeperException, InterruptedException {
Stat stat = zooKeeper.setData("/idea", "idea1".getBytes(), 0);
System.out.println(stat);
}
[zk: localhost:2181(CONNECTED) 18] ls -s /idea
[]
cZxid = 0x100000013
ctime = Thu Jul 29 16:43:11 CST 2021
mZxid = 0x100000013
mtime = Thu Jul 29 16:43:11 CST 2021
pZxid = 0x100000013
cversion = 0
dataVersion = 0
aclVersion = 0
ephemeralOwner = 0x0
dataLength = 4
numChildren = 0
[zk: localhost:2181(CONNECTED) 19] ls -s /idea
[]
cZxid = 0x100000013
ctime = Thu Jul 29 16:43:11 CST 2021
mZxid = 0x10000001b
mtime = Thu Jul 29 17:09:11 CST 2021
pZxid = 0x100000013
cversion = 0
dataVersion = 1
aclVersion = 0
ephemeralOwner = 0x0
dataLength = 5
numChildren = 0
10.5、删除节点
@Test
public void deleteNode() throws KeeperException, InterruptedException {
zooKeeper.delete("/idea",1);
}
[zk: localhost:2181(CONNECTED) 21] create /athens
Created /athens
监听到了变化
xxx
uk0000000004
zookeeper
china
监听到了变化
10.6、判断节点
@Test
public void existsNode() throws KeeperException, InterruptedException {
Stat exists = zooKeeper.exists("/china", false);
System.out.println(exists);
}
11、监听器
也是美团等的工作模式
[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-Dj4aI8pH-1627572451887)(C:\Users\yoya\AppData\Roaming\Typora\typora-user-images\image-20210729223559847.png)]
12、分布式锁
- redis:高性能
- zookeeper:高可用(可靠性)
12.1、传统锁
12.2、分布式锁
12.3、实现秒杀(ssm)
待补