Zookeeper(一)

最新推荐文章于 2024-09-11 10:30:00 发布

无为丶

最新推荐文章于 2024-09-11 10:30:00 发布

阅读量247

点赞数

文章标签： zookeeper

本文链接：https://blog.csdn.net/weixin_43946848/article/details/106386789

版权

一、入门

1、概述

Zookeeper 是一个开源的分布式的，为分布式应用提供协调服务的 Apache 项目。
Zookeeper=文件系统+通知机制
Zookeeper工作机制:从设计模式角度来理解：是一个基于观察者模式设计的分布式服务管理框架,它负责存储和管理大家都关心的数据，然后接受观察者的注册，一旦这些数据发生了变化，Zookeeper就将负责通知已经在Zookeeper上注册的那些观察者做出相应的反应。

2、特点

一个领导者(Leader)和多个跟随着(Follower)组成的集群
集群中只要有半数以上的节点(server)存活，集群就能正常工作
全局数据一致：每个Server保存一份相同的数据副本，Client无论连接到哪一个Server，获取数据都是一致的。
更新请求顺序执行：来自同一个Client的更新请求按其发送顺序依次执行。
数据更新原子性：一次数据要么成功，要么失败。
实时性：在一定的时间范围内，Client能读到最新数据。

3、数据结构：树状结构

在这里插入图片描述
ZooKeeper数据模型的结构与Unix文件系统很类似，整体看做一棵树。每个节点称作一个ZNode，每一个ZNode默认存储1MB数据，每一个ZNode都可以通过其路径唯一标识。

4、应用场景

提供的服务包括：统一命名服务、统一配置管理、统一集群管理、服务器节点动态上下线、软负载均衡等。

二、常用命令

在zookeeper服务器中使用命令:./zkCli.sh

1、新增节点

创建持久化节点/持久化有序节点：create (-s)有序节点节点路径节点数据
创建临时节点：create (-s) -e 节点路径节点数据

2、更新节点

set 节点路径更新数据 (数据版本号dataVersion)

3、删除节点

没有子节点：delete 节点路径 (版本号)
存在子节点：rmr 节点路径

4、查看节点

get 节点路径：返回的节点数据和节点属性
stat 节点路径：返回节点属性
ls 节点路径：查看节点列表即所有子节点
ls2 节点路径：查看节点列表以及当前节点属性

5、监听

监听配置文件的变化
监听器触发一次后便失效
get/stat 节点路径 watch:监听当前节点
ls/ls2 节点路径 watch:监听当前节点子节点增加和删除操作

三、ACL权限控制

1、授权格式

setAcl 节点名称权限模式 :授权对象:权限

2、权限模式和权限

权限模式

world:只有一个用户，anyone，代表登录zookeeper的所有人。默认
ip：对客户端使用的ip地址认证
auth：使用已添加认证的用户认证
digest：使用“用户名”“密码”方式认证

权限：cdrwa : 创建/删除/读/写/管理

例如：

setAcl /node1 world:anyone:cdrwa
setAcl /node1 ip:192.168.0.1:cdrwq
addauth digest (username)ty:(password)123456 #添加认证用户
setAcl /node1 auth:ty:cdrwa
digest方式需要获得通过linux处理之后用户的加密后的密码

四、javaAPI

1、引入所需依赖

     <dependencies>
        <dependency>
            <groupId>org.apache.zookeeper</groupId>
            <artifactId>zookeeper</artifactId>
            <version>3.4.10</version>
        </dependency>
    </dependencies>

2、java简单连接代码

注意这里要关闭centos7的防火墙，否则可能连接超时

关闭防火墙指令
CentOS 7.0默认使用的是firewall作为防火墙
查看防火墙状态

firewall-cmd --state

停止firewall

systemctl stop firewalld.service

禁止firewall开机启动

systemctl disable firewalld.service

package com.ty.zookeeper;

import org.apache.zookeeper.WatchedEvent;
import org.apache.zookeeper.Watcher;
import org.apache.zookeeper.ZooKeeper;

import java.util.concurrent.CountDownLatch;

public class ZookeeperConnection {
    public static void main(String[] args) {
        try {
            //创建计数器对象
            final CountDownLatch countDownLatch = new CountDownLatch(1);
            //arg1:zookeeper服务器的ip和端口号
            //arg2：客户端与服务器之间的会话超时时间，ms
            //arg3：监视器对象
            ZooKeeper zooKeeper = new ZooKeeper("192.168.101.41:2181", 5000, new Watcher() {
                public void process(WatchedEvent watchedEvent) {
                    if(watchedEvent.getState() == Event.KeeperState.SyncConnected);
                    System.out.println("连接创建成功");
                    countDownLatch.countDown();//通知主线程不要继续阻塞
                }
            });
            //由于zookeeper连接对现创建过程为异步的，当执行完上面的代码并不代表连接已经创建成功，所以通过计数器对象让主线程阻塞
            countDownLatch.await();//主线程阻塞,等待连接对象的创建成功
            System.out.println(zooKeeper.getSessionId());
            zooKeeper.close();
        }catch (Exception e){
            e.printStackTrace();
        }
    }
}

在这里插入图片描述

3、新建节点

使用junit的@After和@Before已经@Test来实现
@before中放连接代码
@After中放关闭连接代码
@Test中放测定代码/连接代码

同步方式：

public class ZookeeperConnection {
    String IP = "192.168.101.41:2181";
    private ZooKeeper zooKeeper;

    @Before
    public void before() throws Exception{
            //创建计数器对象
            final CountDownLatch countDownLatch = new CountDownLatch(1);
            //arg1:zookeeper服务器的ip和端口号
            //arg2：客户端与服务器之间的会话超时时间，ms
            //arg3：监视器对象
            zooKeeper = new ZooKeeper(IP, 5000, new Watcher() {
                public void process(WatchedEvent watchedEvent) {
                    if(watchedEvent.getState() == Event.KeeperState.SyncConnected);
                    System.out.println("连接创建成功");
                    countDownLatch.countDown();//通知主线程不要继续阻塞
                }
            });
            //由于zookeeper连接对现创建过程为异步的，当执行完上面的代码并不代表连接已经创建成功，所以通过计数器对象让主线程阻塞
            countDownLatch.await();//主线程阻塞,等待连接对象的创建成功
            System.out.println(zooKeeper.getSessionId());
    }

    @After
    public void after() throws Exception{
        zooKeeper.close();
    }

    @Test
    //world权限方式
    public void create01() throws Exception{
        //arg1:节点路径
        //arg2：节点数据
        //arg3：权限列表，此时为world:anyone:cdrwa;READ_ACL_UNSAFE为world:anyone:r
        //arg4: 节点类型，持久化节点;临时节点：EPHEMERAL
        zooKeeper.create("/node01","node01".getBytes(), ZooDefs.Ids.OPEN_ACL_UNSAFE, CreateMode.PERSISTENT);
    }

    @Test
    //ip权限方式
    public void create02() throws Exception{
        //自定义权限列表
        List<ACL> acls = new ArrayList<ACL>();
//        Id id = new Id("world","anyone");
        Id id = new Id("ip","192.168.101.41");
        acls.add(new ACL(ZooDefs.Perms.CREATE,id));
        acls.add(new ACL(ZooDefs.Perms.READ,id));
        zooKeeper.create("/node01","node01".getBytes(), acls, CreateMode.PERSISTENT);
    }

    @Test
    //auth权限方式
    public void create03() throws Exception{
        zooKeeper.addAuthInfo("digest","ty:123456".getBytes());
        List<ACL> acls = new ArrayList<ACL>();
        Id id = new Id("auth","ty");
        acls.add(new ACL(ZooDefs.Perms.CREATE,id));
        zooKeeper.create("/node01","node01".getBytes(), acls, CreateMode.PERSISTENT);
    }

    @Test
    //持久化有序节点
    public void create04() throws Exception{
       String result = zooKeeper.create("/node01","node01".getBytes(), ZooDefs.Ids.OPEN_ACL_UNSAFE, CreateMode.PERSISTENT_SEQUENTIAL);
        System.out.println(result);
    }
}

还有一种异步回调接口方式创建节点。

4、更新节点

同步方式：

 @Test
    public void set01() throws Exception{
        //arg1:节点路径
        //arg2:更新数据
        //arg3:数据版本号，-1代表版本号不参与更新
        //Stat可以拿到节点的属性值
        Stat stat = zooKeeper.setData("/set/node1","node111".getBytes(),-1);
        
    }

异步方式：

    @Test
    public void delete2() throws Exception{
        //arg1:节点路径
        //arg2：节点数据版本号
        zooKeeper.setData("/set/node1","node111".getBytes(),-1, new AsyncCallback.VoidCallback() {
            public void processResult(int i, String s, Object o) {
                //0代表修改成功
                System.out.println(i);
                //节点的路径
                System.out.println(s);
                //上下文参数对象
                System.out.println(o);
            }
        },"I am Context");
        Thread.sleep(1000);
        System.out.println("结束");
    }

5、删除节点

同步方式：

    @Test
    public void delete() throws Exception{
        //arg1:节点路径
        //arg2：节点数据版本号
        zooKeeper.delete("/set/node2",-1);
    }

异步方式：

    @Test
    public void delete2() throws Exception{
        //arg1:节点路径
        //arg2：节点数据版本号
        zooKeeper.delete("/set/node2", -1, new AsyncCallback.VoidCallback() {
            public void processResult(int i, String s, Object o) {
                //0代表删除成功
                System.out.println(i);
                //节点的路径
                System.out.println(s);
                //上下文参数对象
                System.out.println(o);
            }
        },"I am Context");
        Thread.sleep(1000);
        System.out.println("结束");
    }

6、查看节点

同步方式：

    @Test
    public void get01() throws Exception{
        Stat stat = new Stat();
        byte[] bytes = zooKeeper.getData("/get/node1",false,stat);
        System.out.println(new String(bytes));
        System.out.println(stat.getVersion());
    }

异步方式：

 @Test
    public void get02() throws Exception{

        zooKeeper.getData("/get/node1", false, new AsyncCallback.DataCallback() {
            public void processResult(int i, String s, Object o, byte[] bytes, Stat stat) {
                //arg1:0代表删除成功
                System.out.println(i);
                //arg2:节点的路径
                System.out.println(s);
                //arg3:上下文参数对象
                System.out.println(o);
                //arg4:数据
                System.out.println(new String(bytes));
                //arg5:属性信息
                System.out.println(stat.getVersion());
            }
        },"I am Context");
        Thread.sleep(1000);
        System.out.println("结束");
    }

6、查看子节点、检查子节点是否存在

查看子节点

  @Test
    public void getChild01() throws Exception{
        List<String> list = new ArrayList<String>();
        list = zooKeeper.getChildren("/get",false);
        for(String s:list){
            System.out.println(s);
        }
    }

    @Test
    public void getChild02() throws Exception{

        zooKeeper.getChildren("/get", false, new AsyncCallback.ChildrenCallback() {
            public void processResult(int i, String s, Object o, List<String> list) {
                //arg1:0代表删除成功
                System.out.println(i);
                //arg2:节点的路径
                System.out.println(s);
                //arg3:上下文参数对象
                System.out.println(o);
                //arg4:数据
                for(String child:list){
                    System.out.println(child);
                }
            }
        },"I an Context");
        Thread.sleep(1000);
        System.out.println("结束");
    }

检查子节点是否存在：exists()

四、事件监听机制

1、watch概念

zookeeper采用Watch机制实现数据的发布/订阅功能，该机制在被订阅对象发生变化时会异步通知客户端，因此客户端不必在Watch注册后轮询阻塞，从而减轻客户端压力

2、watcher架构

watcher实现由三部分组成：

zookeeper服务端
zookeeper客户端
客户端的ZkWatchManaget对象

实现过程：
客户端首先将Watcher注册到服务端，同时将Watch对象保存到客户端的Watch管理器中，当zookeeper服务端监听到数据变化时，会主动通知客户端，接着客户端的Watcher管理器会触发相关Watch对象来回调处理逻辑，从而实现完整的发布/订阅流程

3、watcher特性

一次性：一旦触发就会失效，再次使用需再次注册。
客户端顺序回调
轻量级：WatchEvent是最小通信单元，结构上只包括通知状态、事件类型和节点路径，并不会告诉数据节点变化前后的具体内容。
时效性：watcher与sesseion绑定，只有session失效才会失效。

4、接口设计，连接状态

①、KeeperState

SyscConnected:客户端与服务端正常连接时
Disconnected:客户端与服务端断开连接时
Expired：会话session失效时,超时
AuthFailed：身份认证失败时

②、EventType

None
NodeCreated
NodeDeleted
NodeDataChanged
NodeChildrenChanged

public class ZookeeperWatcher implements Watcher {

    static final CountDownLatch count = new CountDownLatch(1);
    static ZooKeeper zooKeeper;
    private static final String IP = "192.168.101.41:2181";
    public void process(WatchedEvent watchedEvent) {
        if(watchedEvent.getType() == Event.EventType.None){
            if(watchedEvent.getState() == Event.KeeperState.SyncConnected){
                count.countDown();
                System.out.println("连接成功");
            }
            if(watchedEvent.getState() == Event.KeeperState.Disconnected){
                System.out.println("连接断开");
            }
            if(watchedEvent.getState() == Event.KeeperState.Expired){
                System.out.println("会话超时");
                try {
                    zooKeeper = new ZooKeeper(IP,5000,new ZookeeperWatcher());
                } catch (IOException e) {
                    e.printStackTrace();
                }
            }
            if(watchedEvent.getState() == Event.KeeperState.AuthFailed){
                System.out.println("用户认证失败");
            }
        }

    }

    public static void main(String[] args) throws Exception{
        zooKeeper = new ZooKeeper(IP,5000,new ZookeeperWatcher());
        count.await();
        System.out.println(zooKeeper.getSessionId());
        Thread.sleep(5000);
        zooKeeper.close();
        System.out.println("结束");

    }
}

5、注册监听

①、为某个node创建监听

注册方式	Created	ChildrenChanfged	Changed	Deleted
zk.exists(“节点名称”,watcher)	可监控		可监控	可监控
zk.getData(“节点名称”,watcher)			可监控	可监控
zk.getChildren(“节点名称”,watcher)		可监控		可监控

②、exists()
exists(String path,boolean b)
exists(String path,watcher w)

package com.ty.zookeeper;

import org.apache.zookeeper.*;
import org.apache.zookeeper.data.ACL;
import org.apache.zookeeper.data.Id;
import org.apache.zookeeper.data.Stat;
import org.junit.After;
import org.junit.Before;
import org.junit.Test;

import java.util.ArrayList;
import java.util.List;
import java.util.concurrent.CountDownLatch;

public class ZookeeperWatcherExists {
    String IP = "192.168.101.41:2181";
    private ZooKeeper zooKeeper;

    @Before
    public void before() throws Exception{
            //创建计数器对象
            final CountDownLatch countDownLatch = new CountDownLatch(1);
            //arg1:zookeeper服务器的ip和端口号
            //arg2：客户端与服务器之间的会话超时时间，ms
            //arg3：监视器对象
            zooKeeper = new ZooKeeper(IP, 5000, new Watcher() {
                public void process(WatchedEvent watchedEvent) {
                    if(watchedEvent.getState() == Event.KeeperState.SyncConnected);{
                        countDownLatch.countDown();//通知主线程不要继续阻塞
                        System.out.println("连接创建成功");
                    }
                    System.out.println(watchedEvent.getPath());
                    System.out.println(watchedEvent.getType());
                }
            });
            //由于zookeeper连接对现创建过程为异步的，当执行完上面的代码并不代表连接已经创建成功，所以通过计数器对象让主线程阻塞
            countDownLatch.await();//主线程阻塞,等待连接对象的创建成功
            System.out.println(zooKeeper.getSessionId());
    }

    @After
    public void after() throws Exception{
        zooKeeper.close();
    }

    @Test
    public void exists01() throws KeeperException,InterruptedException{
        zooKeeper.exists("/watcher",true);
        Thread.sleep(50000);
        System.out.println("================");
    }

    @Test
    public void exists02() throws Exception{
        Watcher watcher = new Watcher() {
            public void process(WatchedEvent watchedEvent) {
                System.out.println(watchedEvent.getPath());
                System.out.println(watchedEvent.getType());
            }
        };
        zooKeeper.exists("/test02",watcher);
        Thread.sleep(50000);
        System.out.println("================");
    }

    @Test
    public void exists03() throws InterruptedException,KeeperException{
    //实现多次监听    
        Watcher watcher = new Watcher() {
            public void process(WatchedEvent watchedEvent) {
                System.out.println(watchedEvent.getPath());
                System.out.println(watchedEvent.getType());
                try {
                    zooKeeper.exists("/test02",this);
                } catch (KeeperException e) {
                    e.printStackTrace();
                } catch (InterruptedException e) {
                    e.printStackTrace();
                }
            }
        };
        zooKeeper.exists("/test02",watcher);
        Thread.sleep(50000);
        System.out.println("================");
    }  
}

③、getData()与getChiledren()用法同exist()类似
getData(String path,boolean b,Stat s)
getData(String path,watcher w,Stat s)
getChiledren(String path,boolean b)
getChiledren(String path,watcher w)

6、监听实例：使用zookeeper作为配置中心

设计思路：
通过将连接数据库的相关配置信息存在zookeeper中，通过连接zookeeper服务器，读取相应信息并通过配置watcher监听，将这些配置存在本地属性变量，来实时获取更新。

package com.ty.zookeeper;

import org.apache.zookeeper.WatchedEvent;
import org.apache.zookeeper.Watcher;
import org.apache.zookeeper.ZooKeeper;

import java.io.IOException;
import java.util.concurrent.CountDownLatch;

public class ZookeeperConfigCenter implements Watcher {
    private String url;
    private String username;
    private String password;

    static final CountDownLatch count = new CountDownLatch(1);
    static ZooKeeper zooKeeper;
    private static final String IP = "192.168.101.41:2181";
    public void process(WatchedEvent watchedEvent) {
        if(watchedEvent.getType() == Event.EventType.None){
            if(watchedEvent.getState() == Event.KeeperState.SyncConnected){
                count.countDown();
                System.out.println("连接成功");
            }
            if(watchedEvent.getState() == Event.KeeperState.Disconnected){
                System.out.println("连接断开");
            }
            if(watchedEvent.getState() == Event.KeeperState.Expired){
                System.out.println("会话超时");
                try {
                    try {
                        zooKeeper = new ZooKeeper(IP,5000,new ZookeeperConfigCenter());
                    } catch (Exception e) {
                        e.printStackTrace();
                    }
                } catch (Exception e) {
                    e.printStackTrace();
                }
            }
            if(watchedEvent.getState() == Event.KeeperState.AuthFailed){
                System.out.println("用户认证失败");
            }
        }else if(watchedEvent.getType() == Event.EventType.NodeDataChanged){
            try {
                this.url = new String(zooKeeper.getData("/config/url",true,null));
                this.username = new String(zooKeeper.getData("/config/username",true,null));
                this.password = new String(zooKeeper.getData("/config/password",true,null));
            }catch (Exception e){
                e.printStackTrace();
            }
        }

    }

    public ZookeeperConfigCenter() throws Exception{
        initCondig();
    }

    public void initCondig() throws  Exception{
        zooKeeper = new ZooKeeper(IP,5000,this);
        count.await();
        this.url = new String(zooKeeper.getData("/config/url",true,null));
        this.username = new String(zooKeeper.getData("/config/username",true,null));
        this.password = new String(zooKeeper.getData("/config/password",true,null));
    }

    public String getUrl() {
        return url;
    }

    public void setUrl(String url) {
        this.url = url;
    }

    public String getUsername() {
        return username;
    }

    public void setUsername(String username) {
        this.username = username;
    }

    public String getPassword() {
        return password;
    }

    public void setPassword(String password) {
        this.password = password;
    }

    public static void main(String[] args) {
        for(int i=0;i<20;i++){
            try {
                ZookeeperConfigCenter zookeeperConfigCenter = new ZookeeperConfigCenter();
                Thread.sleep(5000);
                System.out.println(zookeeperConfigCenter.getUrl());
                System.out.println(zookeeperConfigCenter.getUsername());
                System.out.println(zookeeperConfigCenter.getPassword());
            } catch (Exception e) {
                e.printStackTrace();
            }
        }
    }
}

7、生成分布式唯一id

通过create方法生成有序节点，其自动生成的序列号极为分布式唯一id

五、分布式锁

1、zookeeper实现排他锁过程

每个客户端往/Locks下创建临时有序节点/Locks/Lock_,创建成功后zookeeper会为每个节点生成一个节点序号。
客户端获取/Locks下面的节点，并进行排序，判断排在最前面的节点是否为自己，若自己位于第一位，则获取锁。
若自己不是位于第一个节点，则监听自己的上一位的锁节点。
当上一位锁节点对应客户端执行完毕之后，释放锁。然后自己获取锁，并进行相关逻辑
重复第二步

package com.ty.zookeeper;

import org.apache.zookeeper.*;
import org.apache.zookeeper.data.Stat;

import java.util.Collections;
import java.util.List;
import java.util.concurrent.CountDownLatch;

public class ZookeeperLocks {
    private String IP = "192.168.101.41:2181";
    private static final String  LOCK_ROOT_PATH= "/Locks";
    private static final String LOCK_NODE_NAME = "Lock_";
    private String lockPath;

    private CountDownLatch countDownLatch = new CountDownLatch(1);
    private ZooKeeper zooKeeper;

    public ZookeeperLocks(){
        try {
            zooKeeper = new ZooKeeper(IP, 5000, new Watcher() {
                public void process(WatchedEvent watchedEvent) {
                    if(watchedEvent.getType() == Event.EventType.None){
                        if (watchedEvent.getState() == Event.KeeperState.SyncConnected){
                            countDownLatch.countDown();
                        }
                    }
                }
            });
            countDownLatch.await();
        }catch (Exception e){
            e.printStackTrace();
        }
    }

    //获取锁
    public void acquireLock() throws Exception{
        createlOCK();
        attemptLock();
    }

    //创建锁节点
    public void createlOCK() throws Exception{
        Stat stat = zooKeeper.exists(LOCK_ROOT_PATH,false);
        if(stat == null){
            zooKeeper.create(LOCK_ROOT_PATH,new byte[0], ZooDefs.Ids.OPEN_ACL_UNSAFE, CreateMode.PERSISTENT);
        }
        lockPath = zooKeeper.create(LOCK_ROOT_PATH +"/" + LOCK_NODE_NAME,new byte[0],ZooDefs.Ids.OPEN_ACL_UNSAFE,CreateMode.EPHEMERAL_SEQUENTIAL);
        System.out.println("节点获取成功"+lockPath);
    }

    Watcher watcher = new Watcher() {
        public void process(WatchedEvent watchedEvent) {
           if(watchedEvent.getType() == Event.EventType.NodeDeleted){
               synchronized (this){
                   notifyAll();
               }
           }
        }
    };

    //尝试获取锁
    public void attemptLock() throws Exception{
        List<String> lockLists = zooKeeper.getChildren(LOCK_ROOT_PATH,false);
        Collections.sort(lockLists);
        //获取当前锁节点在子节点列表中的位置
        int index = lockLists.indexOf(lockPath.substring(LOCK_ROOT_PATH.length()+1));
        //如果是第一个节点就创建锁
        if(index == 0){
            System.out.println("创建锁成功");
        }else { //如果不是第一个节点就监听上一个节点锁状态
           String path = lockLists.get(index - 1);//上一个节点名称
           String lastPath = LOCK_ROOT_PATH + "/" +path;//上一个节点路径
           Stat stat = zooKeeper.exists(lastPath,watcher);
           if(stat == null){ //可能在执行上面3个语句时节点已经删除了
               attemptLock();//此时当前节点位于子节点列表第一位，重新进行尝试获取锁方法。
           }else{
               synchronized (watcher){
                  watcher.wait();//程序执行到此阻塞直到watcher监听器中监听到节点删除并唤醒程序
               }
               attemptLock();
           }
        }
    }

    //释放锁
    public void releaseLock() throws Exception{
        zooKeeper.delete(lockPath,-1);
        zooKeeper.close();
        System.out.println("锁已经释放"+this.lockPath);
    }


    public static void main(String[] args) {
        ZookeeperLocks locks = new ZookeeperLocks();
        try {
            locks.createlOCK();
            locks.createlOCK();
        } catch (Exception e) {
            e.printStackTrace();
        }
    }
}