1、准备工作
(1)、关闭防火墙
systemctl stop firewalld
systemctl disable firewalld
(2)、三台机器关闭selinux
三台机器在root用户下执行以下命令关闭selinux 三台机器执行以下命令,关闭selinux
# vim /etc/selinux/config
SELINUX=disabled
(3)、同步时间
三台机器执行以下命令定时同步阿里云服务器时间(每一分钟去阿里云上效验时间)
yum -y install ntpdate
crontab -e
*/1 * * * * /usr/sbin/ntpdate time1.aliyun.com
(4)ssh免密登录
(5)、三台机器添加普通用户
三台linux服务器统一添加普通用户hadoop,并给以sudo权限,用于以后所有的大数据软件的安装 并统一设置普通用户的密码为 123456
useradd hadoop
passwd hadoop
三台机器为普通用户添加sudo权限
visudo
hadoop ALL=(ALL) ALL
(6)、三台定义统一目录
定义三台linux服务器软件压缩包存放目录,以及解压后安装目录,三台机器执行以下命令,创建两个文 件夹,一个用于存放软件压缩包目录,一个用于存放解压后目录
mkdir -p /kkb/soft # 软件压缩包存放目录
mkdir -p /kkb/install # 软件解压后存放目录
chown -R hadoop:hadoop /kkb # 将文件夹权限更改为hadoop用户
(7)、安装JDK
2、安装zookeeper
node01先安装zookeeper,配置好之后,再把文件夹复制给node02 ,node03
(1)、node01修改配置文件
cd /kkb/install/zookeeper-3.4.5-cdh5.14.2/conf
cp zoo_sample.cfg zoo.cfg
mkdir -p /kkb/install/zookeeper-3.4.5-cdh5.14.2/zkdatas
vim zoo.cfg
zoo.cfg的配置参数如下
dataDir=/kkb/install/zookeeper-3.4.5-cdh5.14.2/zkdatas
autopurge.snapRetainCount=3
autopurge.purgeInterval=1
server.1=node01:2888:3888
server.2=node02:2888:3888
server.3=node03:2888:3888
(2)、添加myid配置
在第一台机器的/kkb/install/zookeeper-3.4.5-cdh5.14.2/zkdatas/ 这个路径下创建一个文件,文件名为myid ,文件内容为1
echo 1 > /kkb/install/zookeeper-3.4.5-cdh5.14.2/zkdatas/myid
cat /kkb/install/zookeeper-3.4.5-cdh5.14.2/zkdatas/myid
(3)将配置好的zookeeper-3.4.5-cdh5.14.2环境拷贝到node02和node03
scp -r /kkb/install/zookeeper-3.4.5-cdh5.14.2/ node02:/kkb/install/
scp -r /kkb/install/zookeeper-3.4.5-cdh5.14.2/ node03:/kkb/install/
(4)修改node02和node03的my.in的值
node02
echo 2 > /kkb/install/zookeeper-3.4.5-cdh5.14.2/myid
node03
echo 3 > /kkb/install/zookeeper-3.4.5-cdh5.14.2/myid
cat /kkb/install/zookeeper-3.4.5-cdh5.14.2/zkdatas/myid
3、配置环境并启动
(1) 如果不配置环境,必须要在zookeeper-3.4.5-cdh5.14.2/bin/目录下才能启动
环境配置如下
vim etc/profile
export ZK_HOME=/kkb/install/zookeeper-3.4.5-cdh5.14.2
export PATH=$PATH:$ZK_HOME/bin
source /etc/profile
(2)、 启动 ,关闭,查看状态zookeeper
zkServer.sh start
zkServer.sh stop
zkServer.sh status
(3)遇见到问题:zookeeper正常启动,但是状态查询不到
[root@node01 soft]# zkServer.sh status
JMX enabled by default
Using config: /kkb/install/zookeeper-3.4.5-cdh5.14.2/bin/../conf/zoo.cfg
Error contacting service. It is probably not running.
解决方法
(1)、查看防火墙是否关闭
(2)、检查是否按照jdk(如果命令jps正常,则jdk是能识别的)
[root@node03 soft]# jps
40213 Jps
39207 QuorumPeerMain
(3)、查看节点是否重复,myid的值不能相同
cat /kkb/install/zookeeper-3.4.5-cdh5.14.2/zkdatas/myid
(4)、未全部启动完全
正常状态如下
root@node03 soft]# zkServer.sh status
JMX enabled by default
Using config: /kkb/install/zookeeper-3.4.5-cdh5.14.2/bin/../conf/zoo.cfg
Mode: follower
You have new mail in /var/spool/mail/root
[root@node02 soft]# zkServer.sh status
JMX enabled by default
Using config: /kkb/install/zookeeper-3.4.5-cdh5.14.2/bin/../conf/zoo.cfg
Mode: leader
[root@node01 kkb]# zkServer.sh status
JMX enabled by default
Using config: /kkb/install/zookeeper-3.4.5-cdh5.14.2/bin/../conf/zoo.cfg
Mode: follower
4、zookpeer的使用
-
ZooKeeper
-
一个主从架构的分布式框架、开源的
-
对其他的分布式框架的提供协调服务(service)
-
-
Zookeeper 作为一个分布式的服务框架
-
它提供类似于linux文件系统(有目录节点树)的简版文件系统来存储数据
-
Zookeeper 维护和监控存储的数据的状态变化,通过监控这些数据状态的变化,从而达到基于数据的集群管理
-
主要用来解决分布式集群中应用系统的一致性问题
-
使用ZooKeeper自带的脚本,连接ZooKeeper的服务器
# 使用ZooKeeper自带的脚本,连接ZooKeeper的服务器
zkCli.sh -server node01:2181,node02:2181,node03:2181
-server选项后指定参数node01:2181,node02:2181,node03:2181
客户端随机的连接三个服务器中的一个
-
客户端发出对ZooKeeper集群的读写请求
-
ZooKeeper集群中有类似于linux文件系统的一个简版的文件系统;
-
目录结构也是树状结构(目录树)
-
1、ZooKeeper常用命令
#查看ZooKeeper根目录/下的文件列表
ls /
#创建节点,并指定数据(create后面不指定,默认创建永久节点,-e创建临时节点)
create /kkb kkb
#获得某节点的数据
get /kkb
#修改节点的数据
set /kkb kkb01
#删除节点
delete /kkb
持久节点 | 临时节点 | |
---|---|---|
非有序节点 | create | create -e |
有序节点 | create -s | create -s -e |
有序节点
-
ZNode也可以设置为有序节点
-
为什么设计有序节点?
-
防止多个不同的客户端在同一目录下,创建同名ZNode,由于重名,导致创建失败
-
如何创建有序节点
-
命令行使用-s选项:create -s path data
-
一旦节点被标记上这个属性,那么在这个节点被创建时,ZooKeeper 就会自动在其节点后面追加上一个整型数字
-
这个整数是一个由父节点维护的自增数字。
-
提供了创建唯一名字的ZNode的方式
create -s /test01 test01-data
Created /test010000000009
2、javaAPI使用
<dependencies> <dependency> <groupId>org.apache.zookeeper</groupId> <artifactId>zookeeper</artifactId> <version>3.4.13</version> </dependency> <dependency> <groupId>org.apache.curator</groupId> <artifactId>curator-recipes</artifactId> <version>4.2.0</version> </dependency> <dependency> <groupId>org.apache.curator</groupId> <artifactId>curator-framework</artifactId> <version>4.2.0</version> </dependency> <dependency> <groupId>junit</groupId> <artifactId>junit</artifactId> <version>4.12</version> <scope>compile</scope> </dependency> <dependency> <groupId>org.testng</groupId> <artifactId>testng</artifactId> <version>RELEASE</version> </dependency> <dependency> <groupId>log4j</groupId> <artifactId>log4j</artifactId> <version>1.2.17</version> </dependency> </dependencies>
1、ZK连接与关闭
//定义会话的超时时间 private final static int SESSION_TIME = 30000; //定义zookeeper集群的ip地址;根据自己的实际情况,修改主机名 private final static String ZK_SERVERS = "node01:2181,node02:2181,node03:2181"; private static final Logger LOGGER = Logger.getLogger(ZKCurd.class); //定义zookeeper实例 static ZooKeeper zk = null; //创建监视器 Watcher watcher = new Watcher() { //回调函数 public void process(WatchedEvent watchedEvent) { LOGGER.info("event:" + watchedEvent.toString()); } }; /** * 初始化zookeeper连接 * * @return * @throws Exception */ @Before public void connect() throws Exception { zk = new ZooKeeper(ZK_SERVERS, SESSION_TIME, this.watcher); long sessionId = zk.getSessionId(); LOGGER.info("sessionId:"+sessionId); } /** * 关闭zookeeper连接 * * @return * @throws Exception */ @After public void closeZk() throws Exception { zk.close(); }
2、创建节点数据
@Test public void createPersistentNode() throws Exception { //1.创建持久化节点 String result = zk.create("/kkb", "kkbdata1".getBytes(), ZooDefs.Ids.OPEN_ACL_UNSAFE, CreateMode.PERSISTENT); //2.查看节点是否创建成功 LOGGER.info("create result: " + result); LOGGER.info("查看zk_test节点是否存在:" + zk.getData("/zk_test", false, null)); }
3、删除节点数据
@Test public void testDeleteZnode() throws Exception { zk.delete("/kkb", -1); if (null == zk.exists("/kkb", false)) { System.out.println("节点删除成功!"); } }
4、创建零时节点
@Test public void createEphemeralNode() { try { //创建临时节点 zk.create("/tmp", "tmpdata".getBytes(), ZooDefs.Ids.OPEN_ACL_UNSAFE, CreateMode.EPHEMERAL); //休眠20秒;根据是否休眠,命令行端观察现象 Thread.sleep(20000); //查看节点时候创建成功 LOGGER.info("/tmp是否存在:" + zk.exists("/tmp", null)); } catch (KeeperException e) { LOGGER.error(e.getMessage()); e.printStackTrace(); } catch (InterruptedException e) { LOGGER.error(e.getMessage()); e.printStackTrace(); } }
5、判断节点是否存在
@Test public void exist() throws Exception { Stat stat = zk.exists("/kkb", null); if (stat != null) { System.out.println(stat.getVersion()); LOGGER.info("node exists!"); } else { LOGGER.info("node not exists!"); } }
6、获取节点数据
@Test public void testGetData() throws KeeperException, InterruptedException { //zk.sync("kkb", null, null); byte[] data = zk.getData("/kkb", null, null); System.out.println("data is :" + new String(data)); }
7、监听节点的创建删除修改变化
public class CuratorWatcherDemo { /** * Zookeeper info */ // private static final String ZK_ADDRESS = "note01:2181,node02:2181,node03:2181"; private final static String ZK_ADDRESS = "node01:2181,node02:2181,node03:2181"; private static final String ZK_PATH = "/zk_test"; public static void main(String[] args) throws Exception { // 1.Connect to zk CuratorFramework client = CuratorFrameworkFactory.newClient( ZK_ADDRESS, new RetryNTimes(10, 5000) ); client.start(); System.out.println("zk client start successfully!"); //path cache ///zktest/b/a PathChildrenCache pathCache = new PathChildrenCache(client, ZK_PATH, true); //Listener for PathChildrenCache changes PathChildrenCacheListener listener = new PathChildrenCacheListener() { @Override public void childEvent(CuratorFramework client, PathChildrenCacheEvent event) throws Exception { switch (event.getType()) { case CHILD_ADDED: { System.out.println("Node added: " + ZKPaths.getNodeFromPath(event.getData().getPath())); break; } case CHILD_UPDATED: { System.out.println("Node changed: " + ZKPaths.getNodeFromPath(event.getData().getPath())); break; } case CHILD_REMOVED: { System.out.println("Node removed: " + ZKPaths.getNodeFromPath(event.getData().getPath())); break; } default: break; } } }; //添加监听器 pathCache.getListenable().addListener(listener); pathCache.start(PathChildrenCache.StartMode.BUILD_INITIAL_CACHE); System.out.println("Register zk pathCache successfully!"); Thread.sleep(60000); pathCache.close(); //关闭zk连接 client.close(); } }
7、 监听Znode节点数据的变化
public class CuratorClientDemo { //根据自己集群的实际情况,Zookeeper info 替换 private static final String ZK_ADDRESS = "node01:2181,node02:2181,node03:2181"; private static final String ZK_PATH = "/zk_test"; static CuratorFramework client = null; //初始化,建立连接 public static void init() { //重试连接策略,失败重试次数;每次休眠5000毫秒 //RetryPolicy policy = new ExponentialBackoffRetry(3000, 3); RetryNTimes retryPolicy = new RetryNTimes(10, 5000); // 1.设置客户端参数,参数1:指定连接的服务器集端口列表;参数2:重试策略 client = CuratorFrameworkFactory.newClient(ZK_ADDRESS, retryPolicy); //启动客户端,连接到zk集群 client.start(); System.out.println("zk client start successfully!"); } //关闭连接 public static void clean() { System.out.println("close session"); client.close(); } // 创建永久节点 public static void createPersistentZNode() throws Exception { String zNodeData = "火辣的"; ///a/b/c client.create(). creatingParentsIfNeeded(). //如果父目录不存在,则创建 withMode(CreateMode.PERSISTENT). //创建永久节点 forPath("/beijing/goddess/anzhulababy", zNodeData.getBytes());//指定路径及节点数据 } // 创建临时节点 public static void createEphemeralZNode() throws Exception { String zNodeData2 = "流星雨"; client.create(). creatingParentsIfNeeded(). withMode(CreateMode.EPHEMERAL). forPath("/beijing/star", zNodeData2.getBytes()); Thread.sleep(10000); } //查询znode数据 public static void queryZNodeData() throws Exception { // 查询列表 print("ls", "/"); print(client.getChildren().forPath("/")); //查询节点数据 print("get", ZK_PATH); if(client.checkExists().forPath(ZK_PATH) != null) { print(client.getData().forPath(ZK_PATH)); } else { System.out.println("节点不存在"); } } public static void modifyZNodeData() throws Exception { // 修改节点数据 String data2 = "world"; print("set", ZK_PATH, data2); client.setData().forPath(ZK_PATH, data2.getBytes()); print("get", ZK_PATH); print(client.getData().forPath(ZK_PATH)); } public static void deleteZNode() throws Exception { // 删除节点 print("delete", ZK_PATH); client.delete().forPath(ZK_PATH); print("ls", "/"); print(client.getChildren().forPath("/")); } //监听ZNode public static void watchZNode() throws Exception { //cache: TreeCache\PathChildrenCache\DataCache //设置节点的cache TreeCache treeCache = new TreeCache(client, "/zk_test"); //设置监听器和处理过程 treeCache.getListenable().addListener(new TreeCacheListener() { @Override public void childEvent(CuratorFramework client, TreeCacheEvent event) throws Exception { ChildData data = event.getData(); if (data != null) { switch (event.getType()) { case NODE_ADDED: System.out.println("NODE_ADDED : " + data.getPath() + " 数据:" + new String(data.getData())); break; case NODE_REMOVED: System.out.println("NODE_REMOVED : " + data.getPath() + " 数据:" + new String(data.getData())); break; case NODE_UPDATED: System.out.println("NODE_UPDATED : " + data.getPath() + " 数据:" + new String(data.getData())); break; default: break; } } else { System.out.println("data is null : " + event.getType()); } } }); //开始监听 treeCache.start(); Thread.sleep(30000); //关闭cache System.out.println("关闭treeCache"); treeCache.close(); } public static void main(String[] args) throws Exception { init(); // createPersistentZNode(); // createEphemeralZNode(); // queryZNodeData(); // modifyZNodeData(); // deleteZNode(); watchZNode(); clean(); } private static void print(String... cmds) { StringBuilder text = new StringBuilder("$ "); for (String cmd : cmds) { text.append(cmd).append(" "); } System.out.println(text.toString()); } private static void print(Object result) { System.out.println( result instanceof byte[] ? new String((byte[]) result) : result); } }
8、Zk创建分布式锁
public class DistributedLock implements Watcher { //定义会话的超时时间 private final static int SESSION_TIME = 30000; private int threadId; private ZooKeeper zk = null; private String selfPath; private String waitPath; private String LOG_PREFIX_OF_THREAD; private static final int SESSION_TIMEOUT = 10000; private static final String GROUP_PATH = "/disLocks"; private static final String SUB_PATH = "/disLocks/sub"; private static final String CONNECTION_STRING = "node01:2181,node02:2181,node03:2181"; private static final int THREAD_NUM = 10; //确保连接zk成功; private CountDownLatch connectedSemaphore = new CountDownLatch(1); //确保所有线程运行结束;semaphore信号 private static final CountDownLatch threadSemaphore = new CountDownLatch(THREAD_NUM); private static final Logger LOG = Logger.getLogger(DistributedLock.class); //构造方法 public DistributedLock(int id) { this.threadId = id; LOG_PREFIX_OF_THREAD = "【第" + threadId + "个线程】"; } //程序入口 public static void main(String[] args) { //多个线程中zk是否是同一个对象? BasicConfigurator.configure(); for (int i = 0; i < THREAD_NUM; i++) { final int threadId = i + 1; new Thread() { @Override public void run() { try { DistributedLock dc = new DistributedLock(threadId); //连接zookeeper集群 dc.createConnection(CONNECTION_STRING, SESSION_TIMEOUT); System.out.println("3、在createConnection中,线程等待结束,向下执行"); //GROUP_PATH不存在的话,由一个线程创建即可; synchronized (threadSemaphore) { dc.createPath(GROUP_PATH, "该节点由线程" + threadId + "创建", true); } //获得锁 dc.getLock(); } catch (Exception e) { LOG.error("【第" + threadId + "个线程】 抛出的异常:"); e.printStackTrace(); } } }.start(); } try { threadSemaphore.await(); LOG.info("所有线程运行结束!"); } catch (InterruptedException e) { e.printStackTrace(); } } /** * 获取锁 * * @return */ private void getLock() throws KeeperException, InterruptedException { //创建临时有序节点 selfPath = zk.create(SUB_PATH, null, ZooDefs.Ids.OPEN_ACL_UNSAFE, CreateMode.EPHEMERAL_SEQUENTIAL); LOG.info(LOG_PREFIX_OF_THREAD + "创建锁路径:" + selfPath); if (checkMinPath()) {//判断当前线程,创建的临时节点,时候是编号最小的;如果是,表示此线程可获得锁 getLockSuccess(); } } /** * 创建节点 * * @param path 节点path * @param data 初始数据内容 * @return */ public boolean createPath(String path, String data, boolean needWatch) throws KeeperException, InterruptedException { if (zk.exists(path, needWatch) == null) { LOG.warn(LOG_PREFIX_OF_THREAD + "节点创建成功, Path: " + this.zk.create(path, data.getBytes(), ZooDefs.Ids.OPEN_ACL_UNSAFE, CreateMode.PERSISTENT) + ", content: " + data); } return true; } /** * 创建ZK连接 * * @param connectString ZK服务器地址列表 * @param sessionTimeout Session超时时间 */ public void createConnection(String connectString, int sessionTimeout) throws IOException, InterruptedException { zk = new ZooKeeper(connectString, sessionTimeout, this); System.out.println("1、创建连接,并等待"); //CountDownLatch connectedSemaphore.await(); System.out.println("2、创建连接后,等待结束;理应执行3、"); } /** * 获取锁成功 */ public void getLockSuccess() throws KeeperException, InterruptedException { if (zk.exists(this.selfPath, false) == null) { LOG.error(LOG_PREFIX_OF_THREAD + "本节点已不在了..."); return; } LOG.info(LOG_PREFIX_OF_THREAD + "获取锁成功,赶紧干活!"); Thread.sleep(2000); LOG.info(LOG_PREFIX_OF_THREAD + "删除本节点:" + selfPath); //删除本节点 zk.delete(this.selfPath, -1); //释放zk连接 releaseConnection(); //threadSemaphore数字递减;达到零后,让等待的线程继续执行 threadSemaphore.countDown(); } /** * 关闭ZK连接 */ public void releaseConnection() { if (this.zk != null) { try { this.zk.close(); } catch (InterruptedException e) { } } LOG.info(LOG_PREFIX_OF_THREAD + "释放连接"); } /** * 检查自己创建的临时节点是不是最小的节点 * * @return */ public boolean checkMinPath() throws KeeperException, InterruptedException { //获得所有子节点的路径 List<String> subNodes = zk.getChildren(GROUP_PATH, false); //对子节点列表做排序 Collections.sort(subNodes); //获得当前线程创建的临时节点,在子节点列表中排第几? int index = subNodes.indexOf(selfPath.substring(GROUP_PATH.length() + 1)); switch (index) { case -1: { LOG.error(LOG_PREFIX_OF_THREAD + "本节点已不在了..." + selfPath); return false; } //当前线程创建的临时节点是最小的节点 case 0: { LOG.info(LOG_PREFIX_OF_THREAD + "子节点中,我果然是老大" + selfPath); //获得锁 return true; } default: { //找到比自己编号小,紧邻的临时节点 this.waitPath = GROUP_PATH + "/" + subNodes.get(index - 1); LOG.info(LOG_PREFIX_OF_THREAD + "获取子节点中,排在我前面的" + waitPath); try { //注册监听器 zk.getData(waitPath, true, new Stat()); //没有获得锁 return false; } catch (KeeperException e) { if (zk.exists(waitPath, false) == null) { LOG.info(LOG_PREFIX_OF_THREAD + "子节点中,排在我前面的" + waitPath + "已失踪,幸福来得太突然?"); //回调自己 return checkMinPath(); } else { throw e; } } } } } //回调函数 public void process(WatchedEvent event) { if (event == null) { return; } Event.KeeperState keeperState = event.getState(); Event.EventType eventType = event.getType(); //Event.KeeperState.SyncConnected =>> The client is in the connected state 客户端处于连接状态 if (Event.KeeperState.SyncConnected == keeperState) { if (Event.EventType.None == eventType) {//客户端连接上zkServer后,执行此分支 LOG.info(LOG_PREFIX_OF_THREAD + "成功连接上ZK服务器"); //connectedSemaphore数字递减;达到零后,让等待的线程继续执行 System.out.println("4、createConnection后,客户端成功连接zkServer"); connectedSemaphore.countDown(); System.out.println("5、CountDownLatch: connectedSemaphore 递减为0;理应执行 -> 2、创建连接后,等待结束"); } else if (event.getType() == Event.EventType.NodeDeleted && event.getPath().equals(waitPath)) { LOG.info(LOG_PREFIX_OF_THREAD + "收到情报,排我前面的家伙已挂,我是不是可以出山了?"); try { if (checkMinPath()) { getLockSuccess(); } } catch (KeeperException e) { e.printStackTrace(); } catch (InterruptedException e) { e.printStackTrace(); } } } else if (Event.KeeperState.Disconnected == keeperState) { LOG.info(LOG_PREFIX_OF_THREAD + "与ZK服务器断开连接"); } else if (Event.KeeperState.AuthFailed == keeperState) { LOG.info(LOG_PREFIX_OF_THREAD + "权限检查失败"); } else if (Event.KeeperState.Expired == keeperState) { LOG.info(LOG_PREFIX_OF_THREAD + "会话失效"); } } }
3、 会话(Session)
1、 什么是会话
-
客户端要对ZooKeeper集群进行读写操作,得先与某一ZooKeeper服务器建立TCP长连接;此TCP长连接称为建立一个会话Session。
-
每个会话有超时时间:SessionTimeout
-
当客户端与集群建立会话后,如果超过SessionTimeout时间,两者间没有通信,会话超时
-
2、 会话的特点
-
客户端打开一个Session中的请求以FIFO(先进先出)的顺序执行;
-
如客户端client01与集群建立会话后,先发出一个create请求,再发出一个get请求;
-
那么在执行时,会先执行create,再执行get
-
-
若打开两个Session,无法保证Session间,请求FIFO执行;只能保证一个session中请求的FIFO
3、zk写入过程
客户端写操作
-
①客户端向zk集群写入数据,如create /kkb;与一个follower建立Session连接,从节点follower01
-
②follower将写请求转发给leader
-
③leader收到消息后,发出proposal提案(创建/kkb),每个follower先记录下要创建/kkb
-
④超过半数quorum(包括leader自己)发回反馈,则leader提交commit提案,leader本地创建/kkb节点ZNode
-
⑤leader通知所有follower,也commit提案;follower各自在本地创建/kkb
-
⑥follower01响应client
4、Watcher监视器
-
客户端在服务器端,注册的事件监听器;
-
watcher用于监听znode上的某些事件
-
比如znode数据修改、节点增删等;
-
当监听到事件后,watcher会触发通知客户端
-
在node01,如下监听zk_test的节点下面的变化
ls /zk_test watch
在node03下面的zk_test节点下面创建新的znode
create /zk_test/dir01 dir01-data
当/zk_test节点下面的节点发生变化,node01就能监听到
[zk: node01:2181,node02:2181,node03:2181(CONNECTED) 6] ls /zk_test watch
[dir01, dir0, dir02]
[zk: node01:2181,node02:2181,node03:2181(CONNECTED) 7]
WATCHER::WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/zk_test
监听节点数据变化
node01上面监听
get /zk_test watch
node03上面改变节点数据
set /zk_test "junk01"
当数据发生改变,则能看到数据变化
5、 Zk工作原理
-
ZooKeeper使用原子广播协议叫做Zab(ZooKeeper Automic Broadcast)协议
-
Zab协议有两种模式
-
恢复模式(选主):因为ZooKeeper也是主从架构;当ZooKeeper集群没有主的角色leader时,从众多服务器中选举leader时,处于此模式
-
广播模式(同步):当集群有了leader后,客户端向ZooKeeper集群读写数据时,集群处于此模式
-
-
为了保证事务的顺序一致性,ZooKeeper采用了递增的事务id号(zxid)来标识事务,所有提议(proposal)都有zxid
应用场景
-
NameNode使用ZooKeeper实现高可用.
-
Yarn ResourceManager使用ZooKeeper实现高可用.
-
利用ZooKeeper对HBase集群做高可用配置
-
kafka使用ZooKeeper
-
保存消息消费信息比如offset.
-
用于检测崩溃
-
主题topic发现
-
保持主题的生产和消费状态
-