zookeeper 集群和java实例

最新推荐文章于 2021-03-24 12:48:35 发布

weixin_33895016

最新推荐文章于 2021-03-24 12:48:35 发布

阅读量123

点赞数

文章标签：大数据 java python

原文链接：https://my.oschina.net/kaishui/blog/1590601

版权

2019独角兽企业重金招聘Python工程师标准>>>

zookeeper 从安装到java 实例

1. zookeeper 单节点启动

修改配置文件 conf/zoo.cfg


tickTime=2000 ##心跳时间
dataDir=/var/lib/zookeeper ##保存zookeeper内存中的快照和事物日志
clientPort=2181 ##client 连接端口

服务器端启动：


bin/zkServer.sh start

1.1 客户端连接服务器


bin/zkCli.sh -server 127.0.0.1:2181

成功连接显示如下：


Connecting to localhost:2181

2017-12-12 16:35:02,744 [myid:] - INFO  [main:Environment@100] - Client environment:zookeeper.version=3.4.10-39d3a4f269333c922ed3db283be479f9deacaa0f, built on 03/23/2017 10:13 GMT

2017-12-12 16:35:02,748 [myid:] - INFO  [main:Environment@100] - Client environment:host.name=localhost

2017-12-12 16:35:02,748 [myid:] - INFO  [main:Environment@100] - Client environment:java.version=1.8.0_131

2017-12-12 16:35:02,750 [myid:] - INFO  [main:Environment@100] - Client environment:java.vendor=Oracle Corporation

2017-12-12 16:35:02,750 [myid:] - INFO  [main:Environment@100] - Client environment:java.home=/softhware/path/jdk1.8.0_131/jre

2017-12-12 16:35:02,750 [myid:] - INFO  [main:Environment@100] - Client environment:java.class.path=/softhware/path/zookeeper-3.4.10/bin/../build/classes:/softhware/path/zookeeper-3.4.10/bin/../build/lib/*.jar:/softhware/path/zookeeper-3.4.10/bin/../lib/slf4j-log4j12-1.6.1.jar:/softhware/path/zookeeper-3.4.10/bin/../lib/slf4j-api-1.6.1.jar:/softhware/path/zookeeper-3.4.10/bin/../lib/netty-3.10.5.Final.jar:/softhware/path/zookeeper-3.4.10/bin/../lib/log4j-1.2.16.jar:/softhware/path/zookeeper-3.4.10/bin/../lib/jline-0.9.94.jar:/softhware/path/zookeeper-3.4.10/bin/../zookeeper-3.4.10.jar:/softhware/path/zookeeper-3.4.10/bin/../src/java/lib/*.jar:/softhware/path/zookeeper-3.4.10/bin/../conf:

2017-12-12 16:35:02,750 [myid:] - INFO  [main:Environment@100] - Client environment:java.library.path=/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib

2017-12-12 16:35:02,751 [myid:] - INFO  [main:Environment@100] - Client environment:java.io.tmpdir=/tmp

2017-12-12 16:35:02,751 [myid:] - INFO  [main:Environment@100] - Client environment:java.compiler=<NA>

2017-12-12 16:35:02,751 [myid:] - INFO  [main:Environment@100] - Client environment:os.name=Linux

2017-12-12 16:35:02,751 [myid:] - INFO  [main:Environment@100] - Client environment:os.arch=amd64

2017-12-12 16:35:02,751 [myid:] - INFO  [main:Environment@100] - Client environment:os.version=4.13.0-17-generic

2017-12-12 16:35:02,751 [myid:] - INFO  [main:Environment@100] - Client environment:user.name=kaishui

2017-12-12 16:35:02,751 [myid:] - INFO  [main:Environment@100] - Client environment:user.home=/home/kaishui

2017-12-12 16:35:02,751 [myid:] - INFO  [main:Environment@100] - Client environment:user.dir=/softhware/path/zookeeper-3.4.10

2017-12-12 16:35:02,753 [myid:] - INFO  [main:ZooKeeper@438] - Initiating client connection, connectString=localhost:2181 sessionTimeout=30000 watcher=org.apache.zookeeper.ZooKeeperMain$MyWatcher@506c589e

Welcome to ZooKeeper!

2017-12-12 16:35:02,816 [myid:] - INFO  [main-SendThread(localhost:2181):ClientCnxn$SendThread@1032] - Opening socket connection to server localhost/127.0.0.1:2181. Will not attempt to authenticate using SASL (unknown error)

JLine support is enabled

2017-12-12 16:35:02,904 [myid:] - INFO  [main-SendThread(localhost:2181):ClientCnxn$SendThread@876] - Socket connection established to localhost/127.0.0.1:2181, initiating session

[zk: localhost:2181(CONNECTING) 0] 2017-12-12 16:35:03,067 [myid:] - INFO  [main-SendThread(localhost:2181):ClientCnxn$SendThread@1299] - Session establishment complete on server localhost/127.0.0.1:2181, sessionid = 0x16049d862680000, negotiated timeout = 30000

WATCHER::



WatchedEvent state:SyncConnected type:None path:null



[zk: localhost:2181(CONNECTED) 0] help

可以尝试使用里面的命令：


help ##查看有哪些命令

create /zk_demo chiji  ##创建一个目录赋值 "chiji"

get /zk_demo ## 获取一个值

delete /zk_demo ##删除

使用java zookeeper client 连接服务器，启动集群时再做demo解释，这里就不一一贴代码了。

1.2 可以使用jvisualvm 查看服务器的内存，CPU使用情况

从图表上看，启用单节点时，需要的内存还是很小的，也方便开发者调试。

2. 集群模式

单节点情况很方便的让开发人员测试调试，对于生产环境官方还是建议使用奇数个的复制集(注意：使用两个复制集情况，稳定性比单节点还差，不够选举的法定数目)

2.1 在单台机器上启动三个节点的集群：

2.1.1 配置文件node1 conf/zoo.cfg:

详细配置可以查看zookeeper admin


# The number of milliseconds of each tick

# 心跳间隔时间

tickTime=2000

# The number of ticks that the initial 

# synchronization phase can take

# 最多心跳数

initLimit=10

# The number of ticks that can pass between 

# sending a request and getting an acknowledgement

# 心跳个数请求超过这个数字，就放弃这个节点

syncLimit=5

# the directory where the snapshot is stored.

# do not use /tmp for storage, /tmp here is just 

# example sakes.

dataDir=/data/zookeeper/1

dataLogDir=/data/zookeeper/1/logs

# the port at which the clients will connect

clientPort=2181

# the maximum number of client connections.

# increase this if you need to handle more clients

#maxClientCnxns=60

#

# Be sure to read the maintenance section of the 

# administrator guide before turning on autopurge.

#

# http://zookeeper.apache.org/doc/current/zookeeperAdmin.html#sc_maintenance

#

# The number of snapshots to retain in dataDir

#autopurge.snapRetainCount=3

# Purge task interval in hours

# Set to "0" to disable auto purge feature

#autopurge.purgeInterval=1



server.1=192.168.10.105:2888:3888

server.2=192.168.10.105:2889:3889

server.3=192.168.10.105:2890:3890

2.1.2 创建dataDir , dataLogDir 和 myid文件：


mkdir -p /data/zookeeper/1/logs

echo "1" > /data/zookeeper/1/myid

2.1.3 启动


 ./bin/zkServer.sh start

2.2 启动节点node2, node3

2.2.1 从node1 复制出 node2, node3


 cp -r zookeeper-3.4.10/ zookeeper-3.4.10-node2

 cp -r zookeeper-3.4.10/ zookeeper-3.4.10-node3

2.2.2 node2 修改 conf/zoo.cfg


dataDir=/data/zookeeper/2

dataLogDir=/data/zookeeper/2/logs

clientPort=2182

2.2.3 再新建node2相应的myid文件启动即可


mkdir -p /data/zookeeper/2/logs

echo "2" > /data/zookeeper/2/myid

node3重复node2,启动即集群搭建完毕。

2.2.4 可以通过netstat查看相应端口情况


 netstat -nap | grep 218

3. 为相应的目录加上权限


create /authdir authdirVal  ##创建相应的目录

addauth digest kaishui:123456 ##设置相应的权证，类似帐号密码

setAcl /authdir auth:kaishui:123456:cdrwa ## 为相应目录加上用户和权限， 格式： setAcl path schema:username:password:acl_permissions

参考：schema & acl permissions，新建其他目录，设置权限必须先addauth登录再进行设置

4. java zookeeper client

4.1 对目录文件增删改查

4.1.1 maven中引入zookeeper client 依赖


<dependency>
    <groupId>org.apache.zookeeper</groupId>
    <artifactId>zookeeper</artifactId>
    <version>3.4.10</version>
</dependency>

4.1.2 无权限的增删改查


public static void main(String[] args) throws Exception {

    ZooKeeper zk = new ZooKeeper("localhost:2182,localhost:2181,localhost:2183", 3000, event -> {
        System.out.println(event.getPath());

    });
    // 创建一个目录节点
    try {
        zk.create("/testRootPath", "testRootData".getBytes(), ZooDefs.Ids.OPEN_ACL_UNSAFE,
                CreateMode.PERSISTENT);
        // 创建一个子目录节点
        zk.create("/testRootPath/testChildPathOne", "testChildDataOne".getBytes(),
                ZooDefs.Ids.OPEN_ACL_UNSAFE, CreateMode.PERSISTENT);
        System.out.println(new String(zk.getData("/testRootPath", false, null)));
        // 取出子目录节点列表
        System.out.println(zk.getChildren("/testRootPath", true));
        // 修改子目录节点数据
        zk.setData("/testRootPath/testChildPathOne", "modifyChildDataOne".getBytes(), -1);
        System.out.println("目录节点状态：[" + zk.exists("/testRootPath", true) + "]");
        // 创建另外一个子目录节点
        zk.create("/testRootPath/testChildPathTwo", "testChildDataTwo".getBytes(),
                ZooDefs.Ids.OPEN_ACL_UNSAFE, CreateMode.PERSISTENT);
        System.out.println(new String(zk.getData("/testRootPath/testChildPathTwo", true, null)));
        // 删除子目录节点
        zk.delete("/testRootPath/testChildPathTwo", -1);
        zk.delete("/testRootPath/testChildPathOne", -1);
        // 删除父目录节点
        zk.delete("/testRootPath", -1);
        // 关闭连接
        zk.close();
    } catch (Exception e) {
        e.printStackTrace();
    }
}

4.1.3 有权限的创建查询


public static void main(String[] args) throws Exception {

    ZooKeeper zk = new ZooKeeper("localhost:2182,localhost:2181,localhost:2183", 3000, event -> {
        System.out.println(event.getPath());

    });
    // 创建一个目录节点
    try {
        //login
        zk.addAuthInfo("digest", "feng:123456".getBytes());
        //构造一个权限列表
        List<ACL> acls = new ArrayList<>();
        acls.add(new ACL(ZooDefs.Perms.ALL, new Id("digest", DigestAuthenticationProvider.generateDigest("feng:123456"))));
        //创建目录时并设置权限
        zk.create("/ab", "value".getBytes(), acls, CreateMode.PERSISTENT);

        System.out.println(new String(zk.getData("/ab", true, null)));
        zk.close();
    } catch (Exception e) {
        e.printStackTrace();
    }
}

转载于:https://my.oschina.net/kaishui/blog/1590601