第1章 Zookeeper入门
1.1 特点:
- 1)Zookeeper:一个领导者(leader),多个跟随者(follower)组成的集群。
- 2)Leader负责进行投票的发起和决议,更新系统状态
- 3)Follower用于接收客户请求并向客户端返回结果,在选举Leader过程中参与投票
- 4)集群中只要有半数以上节点存活,Zookeeper集群就能正常服务
- 5)全局数据一致:每个server保存一份相同的数据副本,client无论连接到哪个server,数据都是一致的。
- 6)更新请求顺序进行,来自同一个client的更新请求按其发送顺序依次执行。
- 7)数据更新原子性,一次数据更新要么成功,要么失败。
- 8)实时性,在一定时间范围内,client能读到最新数据。
1.2 应用场景
-
统一命名服务
-
统一配管理
将配置信息写入ZooKeeper上的Znode,各个客户端服务器监听这个Znode,一旦Znode中数据被修改,ZooKeeper将通知各个客户端服务器
- 统一集群管理
实时监控节点状态变化
- 服务器节点动态上下线
客户端实时洞察到服务器上下线变化
- 软负载均衡
Zookeeper 中记录每台服务器的访问数,让访问数最少的服务器去处理最新客户端请求
第3章 Zookeeper内部原理
3.1 选举机制(面试重点)
- (1)半数机制:集群中半数以上机器存活,集群可用。所以Zookeeper适合安装奇数台服务器。
- (2)Zookeeper工作时,是有一个节点为Leader,其他则为Follower,Leader是通过内部的选举机制临时产生的。
- (3)选举流程:理解!!!!
3.2 节点类型(持久、非持久、有无顺序编号)
- 持久:(客户端与Zookeeper断开后,节点依旧存在)
- 短暂:(客户端与Zookeeper断开后,节点被删除)
节点的创建(持久、非持久、有无顺序)方式,下面会讲到
3.3 监听器原理(面试重点)
- main()线程:connect、listener
- connect将监听事件发送给Zookeeper
- Zookeeper将监听事件添加到列表
- Zookeeper监听到数据变化或路径变化,将消息发送给listener
- listener线程内部调用process方法
3.4 写数据流程
第4章 Zookeeper实战(开发重点)
4.1 分布式安装部署
1.集群规划
在hadoop102、hadoop103和hadoop104三个节点上部署Zookeeper
2.解压安装
(1)解压Zookeeper安装包到/opt/module/目录下
(2)同步/opt/module/zookeeper-3.4.10目录内容到hadoop103、hadoop104
3.配置zoo.cfg文件
- (1)将/opt/module/zookeeper-3.4.10/conf这个路径下的zoo_sample.cfg修改为zoo.cfg;
- (2)打开zoo.cfg文件,修改dataDir路径:
修改成:dataDir=/opt/module/zookeeper-3.4.10/zkData
- (3)打开zoo.cfg文件,增加如下配置
#######################cluster##########################
server.2=hadoop102:2888:3888
server.3=hadoop103:2888:3888
server.4=hadoop104:2888:3888
- (4)同步zoo.cfg配置文件
- (5)配置参数解读
4.配置服务器编号
(1)在/opt/module/zookeeper-3.4.10/这个目录下创建zkData
[atguigu@hadoop102 zookeeper-3.4.10]$ mkdir -p zkData
(2)在/opt/module/zookeeper-3.4.10/zkData目录下创建一个myid的文件
(3)编辑myid文件
(4)拷贝配置好的zookeeper到其他机器上(拷贝好后,需要去103,104x修改myid为3,4)
5.集群操作
(1)分别启动Zookeeper
[atguigu@hadoop102 zookeeper-3.4.10]$ bin/zkServer.sh start
[atguigu@hadoop103 zookeeper-3.4.10]$ bin/zkServer.sh start
[atguigu@hadoop104 zookeeper-3.4.10]$ bin/zkServer.sh start
(2)查看状态
[atguigu@hadoop102 zookeeper-3.4.10]# bin/zkServer.sh status
4.2 客户端命令行操作
1.启动客户端
[atguigu@hadoop103 zookeeper-3.4.10]$ bin/zkCli.sh
2.显示所有操作命令
[zk: localhost:2181(CONNECTED) 1] help
3.查看当前znode中所包含的内容
[zk: localhost:2181(CONNECTED) 0] ls /
[zookeeper]
4.查看当前节点详细数据
[zk: localhost:2181(CONNECTED) 1] ls2 /
5.分别创建2个普通节点:create
6.获得节点的值:get
[zk: localhost:2181(CONNECTED) 5] get /sanguo
7.创建短暂节点:-e
[zk: localhost:2181(CONNECTED) 7] create -e /sanguo/wuguo "zhouyu"
8.创建带序号的节点:-s
9.修改节点数据值:set
[zk: localhost:2181(CONNECTED) 6] set /sanguo/weiguo "simayi"
10.节点的值变化监听:watch
11.删除节点:delete
[zk: localhost:2181(CONNECTED) 4] delete /sanguo/jin
12.递归删除节点:rmr
[zk: localhost:2181(CONNECTED) 15] rmr /sanguo/shuguo
13.查看节点状态:stat
[zk: localhost:2181(CONNECTED) 17] stat /sanguo
4.3 API应用
4.3.1 Eclipse环境搭建
1.创建一个Maven工程
2.添加pom文件
<dependencies>
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>RELEASE</version>
</dependency>
<dependency>
<groupId>org.apache.logging.log4j</groupId>
<artifactId>log4j-core</artifactId>
<version>2.8.2</version>
</dependency>
<!-- https://mvnrepository.com/artifact/org.apache.zookeeper/zookeeper -->
<dependency>
<groupId>org.apache.zookeeper</groupId>
<artifactId>zookeeper</artifactId>
<version>3.4.10</version>
</dependency>
</dependencies>
3.拷贝log4j.properties文件到项目根目录
需要在项目的src/main/resources目录下,新建一个文件,命名为“log4j.properties”,在文件中填入。
log4j.rootLogger=INFO, stdout
log4j.appender.stdout=org.apache.log4j.ConsoleAppender
log4j.appender.stdout.layout=org.apache.log4j.PatternLayout
log4j.appender.stdout.layout.ConversionPattern=%d %p [%c] - %m%n
log4j.appender.logfile=org.apache.log4j.FileAppender
log4j.appender.logfile.File=target/spring.log
log4j.appender.logfile.layout=org.apache.log4j.PatternLayout
log4j.appender.logfile.layout.ConversionPattern=%d %p [%c] - %m%n
4.3.2 创建ZooKeeper客户端
package com.atguigu.com;
import java.io.IOException;
import java.util.List;
import org.apache.zookeeper.CreateMode;
import org.apache.zookeeper.KeeperException;
import org.apache.zookeeper.WatchedEvent;
import org.apache.zookeeper.Watcher;
import org.apache.zookeeper.ZooDefs.Ids;
import org.apache.zookeeper.ZooKeeper;
import org.apache.zookeeper.data.Stat;
import org.junit.Before;
import org.junit.Test;
public class TestZookeeper {
private String connectString="hadoop102:2181,hadoop103:2181,hadoop104:2181";
private int sessionTimeout = 2000;
private ZooKeeper zkClient;
@Before
public void init() throws IOException{
zkClient = new ZooKeeper(connectString, sessionTimeout , new Watcher() {
@Override
public void process(WatchedEvent event) {
System.out.println("---------start----------");
List<String> children;
try {
children = zkClient.getChildren("/", true);
for (String child : children) {
System.out.println(child);
}
System.out.println("---------end----------");
} catch (KeeperException e) {
// TODO Auto-generated catch block
e.printStackTrace();
} catch (InterruptedException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
});
}
4.3.3 创建子节点
package com.atguigu.com;
import java.io.IOException;
import java.util.List;
import org.apache.zookeeper.CreateMode;
import org.apache.zookeeper.KeeperException;
import org.apache.zookeeper.WatchedEvent;
import org.apache.zookeeper.Watcher;
import org.apache.zookeeper.ZooDefs.Ids;
import org.apache.zookeeper.ZooKeeper;
import org.apache.zookeeper.data.Stat;
import org.junit.Before;
import org.junit.Test;
public class TestZookeeper {
private String connectString="hadoop102:2181,hadoop103:2181,hadoop104:2181";
private int sessionTimeout = 2000;
private ZooKeeper zkClient;
@Before
public void init() throws IOException{
zkClient = new ZooKeeper(connectString, sessionTimeout , new Watcher() {
@Override
public void process(WatchedEvent event) {
System.out.println("---------start----------");
List<String> children;
try {
children = zkClient.getChildren("/", true);
for (String child : children) {
System.out.println(child);
}
System.out.println("---------end----------");
} catch (KeeperException e) {
// TODO Auto-generated catch block
e.printStackTrace();
} catch (InterruptedException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
});
}
// 1 创建节点
@Test
public void createNode() throws KeeperException, InterruptedException{
String path = zkClient.create("/sunyl", "dahaigezuishuai".getBytes(), Ids.OPEN_ACL_UNSAFE, CreateMode.PERSISTENT);
System.out.println(path);
}
}
4.3.4 获取子节点并监听节点变化
package com.atguigu.com;
import java.io.IOException;
import java.util.List;
import org.apache.zookeeper.CreateMode;
import org.apache.zookeeper.KeeperException;
import org.apache.zookeeper.WatchedEvent;
import org.apache.zookeeper.Watcher;
import org.apache.zookeeper.ZooDefs.Ids;
import org.apache.zookeeper.ZooKeeper;
import org.apache.zookeeper.data.Stat;
import org.junit.Before;
import org.junit.Test;
public class TestZookeeper {
private String connectString="hadoop102:2181,hadoop103:2181,hadoop104:2181";
private int sessionTimeout = 2000;
private ZooKeeper zkClient;
@Before
public void init() throws IOException{
zkClient = new ZooKeeper(connectString, sessionTimeout , new Watcher() {
@Override
public void process(WatchedEvent event) {
System.out.println("---------start----------");
List<String> children;
try {
children = zkClient.getChildren("/", true);
for (String child : children) {
System.out.println(child);
}
System.out.println("---------end----------");
} catch (KeeperException e) {
// TODO Auto-generated catch block
e.printStackTrace();
} catch (InterruptedException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
});
}
// 2 获取子节点 并监控节点的变化
@Test
public void getDataAndWatch() throws KeeperException, InterruptedException{
List<String> children = zkClient.getChildren("/", true);
for (String child : children) {
System.out.println(child);
}
Thread.sleep(Long.MAX_VALUE);
}
}
4.3.5 判断Znode是否存在
package com.atguigu.com;
import java.io.IOException;
import java.util.List;
import org.apache.zookeeper.CreateMode;
import org.apache.zookeeper.KeeperException;
import org.apache.zookeeper.WatchedEvent;
import org.apache.zookeeper.Watcher;
import org.apache.zookeeper.ZooDefs.Ids;
import org.apache.zookeeper.ZooKeeper;
import org.apache.zookeeper.data.Stat;
import org.junit.Before;
import org.junit.Test;
public class TestZookeeper {
private String connectString="hadoop102:2181,hadoop103:2181,hadoop104:2181";
private int sessionTimeout = 2000;
private ZooKeeper zkClient;
@Before
public void init() throws IOException{
zkClient = new ZooKeeper(connectString, sessionTimeout , new Watcher() {
@Override
public void process(WatchedEvent event) {
System.out.println("---------start----------");
List<String> children;
try {
children = zkClient.getChildren("/", true);
for (String child : children) {
System.out.println(child);
}
System.out.println("---------end----------");
} catch (KeeperException e) {
// TODO Auto-generated catch block
e.printStackTrace();
} catch (InterruptedException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
});
}
// 3 判断节点是否存在
@Test
public void exist() throws KeeperException, InterruptedException{
Stat stat = zkClient.exists("/sunyl", false);
System.out.println(stat==null? "not exist":"exist");
}
}
4.4 监听服务器节点动态上下线案例
1.需求
某分布式系统中,主节点可以有多台,可以动态上下线,任意一台客户端都能实时感知到主节点服务器的上下线。
2.具体实现
- (0)先在集群上创建/servers节点
[zk: localhost:2181(CONNECTED) 1] create /servers "servers"
Created /servers
- (1)服务器端向Zookeeper注册代码
package com.atguigu;
import java.io.IOException;
import org.apache.zookeeper.CreateMode;
import org.apache.zookeeper.KeeperException;
import org.apache.zookeeper.WatchedEvent;
import org.apache.zookeeper.Watcher;
import org.apache.zookeeper.ZooKeeper;
import org.apache.zookeeper.ZooDefs.Ids;
public class DistributeServer {
public static void main(String[] args) throws Exception {
DistributeServer server = new DistributeServer();
// 1 连接zookeeper集群
server.getConnect();
// 2 注册节点
server.regist(args[0]);
// 3 业务逻辑处理
server.business();
}
private void business() throws InterruptedException {
Thread.sleep(Long.MAX_VALUE);
}
private void regist(String hostname) throws KeeperException, InterruptedException {
String path = zkClient.create("/servers/server", hostname.getBytes(), Ids.OPEN_ACL_UNSAFE, CreateMode.EPHEMERAL_SEQUENTIAL);
System.out.println(hostname +" is online ");
}
private String connectString = "hadoop102:2181,hadoop103:2181,hadoop104:2181";
private int sessionTimeout = 2000;
private ZooKeeper zkClient;
private void getConnect() throws IOException {
zkClient = new ZooKeeper(connectString , sessionTimeout , new Watcher() {
@Override
public void process(WatchedEvent event) {
// TODO Auto-generated method stub
}
});
}
}
测试:注册hadoop104节点
此时客户端也会实时监控到
- (2)客户端代码
package com.atguigu;
import java.io.IOException;
import java.util.ArrayList;
import java.util.List;
import org.apache.zookeeper.KeeperException;
import org.apache.zookeeper.WatchedEvent;
import org.apache.zookeeper.Watcher;
import org.apache.zookeeper.ZooKeeper;
public class DistributeClient {
public static void main(String[] args) throws IOException, KeeperException, InterruptedException {
DistributeClient client = new DistributeClient();
// 1 获取zookeeper集群连接
client.getConnect();
// 2 注册监听
client.getChlidren();
// 3 业务逻辑处理
client.business();
}
private void business() throws InterruptedException {
Thread.sleep(Long.MAX_VALUE);
}
private void getChlidren() throws KeeperException, InterruptedException {
List<String> children = zkClient.getChildren("/servers", true);
// 存储服务器节点主机名称集合
ArrayList<String> hosts = new ArrayList<>();
for (String child : children) {
byte[] data = zkClient.getData("/servers/"+child, false, null);
hosts.add(new String(data));
}
// 将所有在线主机名称打印到控制台
System.out.println(hosts);
}
private String connectString = "hadoop102:2181,hadoop103:2181,hadoop104:2181";
private int sessionTimeout = 2000;
private ZooKeeper zkClient;
private void getConnect() throws IOException {
zkClient = new ZooKeeper(connectString , sessionTimeout , new Watcher() {
@Override
public void process(WatchedEvent event) {
try {
getChlidren();
} catch (KeeperException e) {
e.printStackTrace();
} catch (InterruptedException e) {
e.printStackTrace();
}
}
});
}
}
第5章 企业面试真题
5.1 请简述ZooKeeper的选举机制
详见3.1。
5.2 ZooKeeper的监听原理是什么?
详见3.4。
5.3 ZooKeeper的部署方式有哪几种?集群中的角色有哪些?集群最少需要几台机器?
(1)部署方式单机模式、集群模式
(2)角色:Leader和Follower
(3)集群最少需要机器数:3
5.4 ZooKeeper的常用命令
ls create get delete set…