zookeeper源码（四）——客户端源码

https://www.cnblogs.com/jing99/p/12723817.html

看zookeeper源码的server启动还是比较模糊，因此看了下zookeeper集群的，zab协议详解，能更好了解zookeeper的核心，也明确源码阅读的方向。

下面，还是先了解下zookeeper客户端源码，再Java或者其他语言中 ，经常要用到客户端连接进行api操作，因此客户端源码能更好的使用zookeeper。

我们先从一个zookeeper客户端测试类开始。这是一个创建节点然后删除节点的测试。

@Test
    public void testDeleteRecursive() throws IOException, InterruptedException, KeeperException {
        final ZooKeeper zk = createClient();
        setupDataTree(zk);

        assertTrue(ZKUtil.deleteRecursive(zk, "/a/c", 1000));
        List<String> children = zk.getChildren("/a", false);
        assertEquals("1 children - c should be deleted ", 1, children.size());
        assertTrue(children.contains("b"));

        assertTrue(ZKUtil.deleteRecursive(zk, "/a", 1000));
        assertNull(zk.exists("/a", null));
    }

测试类需要启动zkserver，启动之前需要赋予sever端口，有个给端口范围，检测端口是否可用的方式，可以参考下

public static synchronized int unique() {
        if (portRange == null) {
            Integer threadId = Integer.getInteger("zookeeper.junit.threadid");
            portRange = setupPortRange(
                System.getProperty("test.junit.threads"),
                threadId != null ? "threadid=" + threadId : System.getProperty("sun.java.command"));
            nextPort = portRange.getMinimum();
        }
        int candidatePort = nextPort;
        for (; ; ) {
            ++candidatePort;
            if (candidatePort > portRange.getMaximum()) {
                candidatePort = portRange.getMinimum();
            }
            if (candidatePort == nextPort) {
                throw new IllegalStateException(String.format(
                    "Could not assign port from range %s.  The entire range has been exhausted.",
                    portRange));
            }
            try {
                ServerSocket s = new ServerSocket(candidatePort); //起一个seversocket校验
                s.close();//然后关闭，表示校验成功
                nextPort = candidatePort;
                LOG.info("Assigned port {} from range {}.", nextPort, portRange);
                return nextPort;
            } catch (IOException e) {
                LOG.debug(
                    "Could not bind to port {} from range {}.  Attempting next port.",
                    candidatePort,
                    portRange,
                    e);
            }
        }
    }

然后看下如何创建一个zookeeper客户端。

/**
     * Creates a connection object. The actual network connect doesn't get
     * established until needed. The start() instance method must be called
     * subsequent to construction.
     *
     * @param chrootPath - the chroot of this client. Should be removed from this Class in ZOOKEEPER-838
     * @param hostProvider
     *                the list of ZooKeeper servers to connect to
     * @param sessionTimeout
     *                the timeout for connections.
     * @param zooKeeper
     *                the zookeeper object that this connection is related to.
     * @param watcher watcher for this connection
     * @param clientCnxnSocket
     *                the socket implementation used (e.g. NIO/Netty)
     * @param sessionId session id if re-establishing session
     * @param sessionPasswd session passwd if re-establishing session
     * @param canBeReadOnly
     *                whether the connection is allowed to go to read-only
     *                mode in case of partitioning
     * @throws IOException in cases of broken network
     */
    public ClientCnxn(
        String chrootPath,
        HostProvider hostProvider,
        int sessionTimeout,
        ZooKeeper zooKeeper,
        ClientWatchManager watcher,
        ClientCnxnSocket clientCnxnSocket,
        long sessionId,
        byte[] sessionPasswd,
        boolean canBeReadOnly) throws IOException {
        this.zooKeeper = zooKeeper;
        this.watcher = watcher;
        this.sessionId = sessionId;
        this.sessionPasswd = sessionPasswd;
        this.sessionTimeout = sessionTimeout;
        this.hostProvider = hostProvider;
        this.chrootPath = chrootPath;

        connectTimeout = sessionTimeout / hostProvider.size(); //根绝集群个数来分配连接时间
        readTimeout = sessionTimeout * 2 / 3;
        readOnly = canBeReadOnly;

        sendThread = new SendThread(clientCnxnSocket);//clinetCnxsocket nio连接
        eventThread = new EventThread();
        this.clientConfig = zooKeeper.getClientConfig();
        initRequestTimeout();
    }

创建客户端用的类与 服务端的类实现方式相似，都是采用nio或者netty两种方式实现，我们这里还是看Nio的通讯方式。上面是初始化类，核心代码在sendthread的线程run方法内。

run方法比较常，先看客户端第一次连接部分

 @Override
        public void run() {
            //clientCnxnSocket 为封装的与socke通信nio类，继承抽象类ClientCnxnSocket
            //outgoingQueue为发送队列，存储的发送包
            clientCnxnSocket.introduce(this, sessionId, outgoingQueue);
            clientCnxnSocket.updateNow(); //更新发送时间 毫秒计数
            clientCnxnSocket.updateLastSendAndHeard();
            int to;
            long lastPingRwServer = Time.currentElapsedTime();
            final int MAX_SEND_PING_INTERVAL = 10000; //10 seconds
            InetSocketAddress serverAddress = null;
            while (state.isAlive()) {//判断连接状态，非关闭或者非鉴权
                try {
                    if (!clientCnxnSocket.isConnected()) { //socketchanel还未selectkey与绑定
                        // don't re-establish connection if we are closing
                        if (closing) { //volatile变量
                            break;
                        }
                        if (rwServerAddress != null) {
                            serverAddress = rwServerAddress;
                            rwServerAddress = null;
                        } else {
                            serverAddress = hostProvider.next(1000); //遍历zk集群的连接，等待1秒再次连接
                        }
                        onConnecting(serverAddress); //集群测试方式，无需考虑
                        startConnect(serverAddress); //开始连接zk server
                        // Update now to start the connection timer right after we make a connection attempt
                        clientCnxnSocket.updateNow();
                        clientCnxnSocket.updateLastSendAndHeard(); //更新尝试连接后的时间
                    }
...
}

nio连接 startConnect方法

   private void startConnect(InetSocketAddress addr) throws IOException {
            // initializing it for new connection
            saslLoginFailed = false; //初始化判断sasl连接
            if (!isFirstConnect) { //客户端断连后重连，连接需要缓冲休眠
                try {
                    Thread.sleep(ThreadLocalRandom.current().nextLong(1000));
                } catch (InterruptedException e) {
                    LOG.warn("Unexpected exception", e);
                }
            }
            changeZkState(States.CONNECTING); //更改连接状态为连接中

            String hostPort = addr.getHostString() + ":" + addr.getPort();
            MDC.put("myid", hostPort);
            setName(getName().replaceAll("\\(.*\\)", "(" + hostPort + ")")); //更新send线程名字myid:127.0.0.1:11222
            if (clientConfig.isSaslClientEnabled()) { //默认客户端sasl是开启的
                try {
                    if (zooKeeperSaslClient != null) {
                        zooKeeperSaslClient.shutdown();
                    }
                    //sasl认证，和zkserver一样回去读取jass变量，判断是否开启了java security的jass保护，若开启了，会读取对应设置的用户和密码
                    zooKeeperSaslClient = new ZooKeeperSaslClient(SaslServerPrincipal.getServerPrincipal(addr, clientConfig), clientConfig);
                } catch (LoginException e) {
                    // An authentication error occurred when the SASL client tried to initialize:
                    // for Kerberos this means that the client failed to authenticate with the KDC.
                    // This is different from an authentication error that occurs during communication
                    // with the Zookeeper server, which is handled below.
                    LOG.warn(
                        "SASL configuration failed. "
                            + "Will continue connection to Zookeeper server without "
                            + "SASL authentication, if Zookeeper server allows it.", e);
                    eventThread.queueEvent(new WatchedEvent(Watcher.Event.EventType.None, Watcher.Event.KeeperState.AuthFailed, null));
                    saslLoginFailed = true;
                }
            }
            logStartConnect(addr);//打印连接的serverIp和Port，还有打印sall状态，是否使用sasl

            clientCnxnSocket.connect(addr); //这次socketChanel到clientCnxnSocket 的selectKey上，监听事件
        }

socketchannel注册selectkey，因为是非阻塞的，所以primeconnection方法后面判断连接上了，也会再次调用。

void registerAndConnect(SocketChannel sock, InetSocketAddress addr) throws IOException {
        sockKey = sock.register(selector, SelectionKey.OP_CONNECT);
        boolean immediateConnect = sock.connect(addr);//异步连接
        if (immediateConnect) { //连接成功
            sendThread.primeConnection();
        }
    }

    @Override
    void connect(InetSocketAddress addr) throws IOException {
        SocketChannel sock = createSock(); //初始化socketchanel 设置非阻塞模式，读写和连接不回阻塞，Buffer类费祖舍
        try {
            registerAndConnect(sock, addr); //注册socketchanel 到 selectkey 的connect 事件
        } catch (UnresolvedAddressException | UnsupportedAddressTypeException | SecurityException | IOException e) {
            LOG.error("Unable to open socket to {}", addr);
            sock.close();
            throw e;
        }
        initialized = false;

        /*
         * Reset incomingBuffer
         */
        lenBuffer.clear();
        incomingBuffer = lenBuffer;
    }

上述就是client首次连接server的代码，后面部分再看其他连接逻辑。