Curator源码分析之分布式锁

Curator介绍

关于什么是Curator，我们看一下官网是怎么说的。

What is Curator?

Apache Curator is a Java/JVM client library for Apache ZooKeeper, a distributed coordination service. It includes a highlevel API framework and utilities to make using Apache ZooKeeper much easier and more reliable. It also includes recipes for common use cases and extensions such as service discovery and a Java 8 asynchronous DSL.

Apache Curator是对ZooKeeper的Java客户端库的封装，使我们在操作ZK的时候更容易，更可靠； 并且它还包括了一些常见用例和扩展功能（如服务发现和Java 8异步DSL）。

下面这句话可以说很形象了：

Curator对于Zookeeper来说就像Guava for Java，Guava我们都使用过，它是谷歌开源的Java类库，该库经过高度优化，运用得当可极大提高我们的代码效率和质量。

Curator实现分布式锁

首先我们上一段Curator实现分布式锁的代码

    public static void main(String[] args) throws Exception {
        private  String ZK_ADDRESS = "10.2.1.1:2181";
        private  String ZK_LOCK_PATH = "/zktest/lock0";
        // 1.Connect to zk
        final CuratorFramework client = CuratorFrameworkFactory.newClient(ZK_ADDRESS, new RetryNTimes(10, 5000));
        client.start();
        System.out.println(client.getState());
        System.out.println("zk client start successfully!");
        // 2.创建分布式锁, 锁空间的根节点路径为/zktest/lock0
        final InterProcessMutex mutex = new InterProcessMutex(client, ZK_LOCK_PATH);
        // 3. 获得锁, 执行业务流程
        if (mutex.acquire(1, TimeUnit.SECONDS)) {
            try {
                // do something ...
            } catch (Exception e) {
                e.printStackTrace();
            } finally {
                // 4. 释放锁
                mutex.release();
            }
        }
        // 5. 关闭客户端
        client.close();
    }

获取锁的过程

看完上面代码你会发现一个很关键的方法：mutex.acquire()

    /**
     * Acquire the mutex - blocking until it's available. Note: the same thread
     * can call acquire re-entrantly. Each call to acquire must be balanced by a call
     * to {@link #release()}
     *
     * @throws Exception ZK errors, connection interruptions
     */
    @Override
    public void acquire() throws Exception
    {
        if ( !internalLock(-1, null) )
        {
            throw new IOException("Lost connection while trying to acquire lock: " + basePath);
        }
    }

这个方法没有入参，意思是没有设置超时时间，如果获取不到锁，会一直阻塞直到获取锁。the same thread
can call acquire re-entrantly，它还是一个重入锁。每一个获取锁操作必须对应一个释放锁操作。

我在实例中调用的是acquire(long time, TimeUnit unit)方法，该方法有两个入参分别是超时时间和时间单位。当到达超时时间时会抛出异常终止方法继续阻塞。

    @Override
    public boolean acquire(long time, TimeUnit unit) throws Exception
    {
        return internalLock(time, unit);
    }

然后我们继续往下跟进代码：internalLock()方法

    private boolean internalLock(long time, TimeUnit unit) throws Exception
    {
        /*
           Note on concurrency: a given lockData instance
           can be only acted on by a single thread so locking isn't necessary
        */
        // 获取当前的线程
        Thread currentThread = Thread.currentThread();
        // 从Map中取出与当前线程绑定的LockData对象
        LockData lockData = threadData.get(currentThread);
        // lockData不为空说明当前线程已经获得锁
        if ( lockData != null )
        {
            // re-entering 重入锁计数+1
            lockData.lockCount.incrementAndGet();
            return true;
        }
        // lockData为空时，进行获得锁的操作
        String lockPath = internals.attemptLock(time, unit, getLockNodeBytes());
        // 第一次成功获得锁后，将当前线程和锁信息放入map中
        if ( lockPath != null )
        {
            LockData newLockData = new LockData(currentThread, lockPath);
            threadData.put(currentThread, newLockData);
            return true;
        }

        return false;
    }

上面代码中有几个关键点

1. lockData，他是一个map，以线程为key存储了线程获取锁的信息。包括lockPath(锁路径)和lockCount(c重入次数)。

    private final ConcurrentMap<Thread, LockData> threadData = Maps.newConcurrentMap();

继续跟进获取锁的关键方法attemptLock()

 String attemptLock(long time, TimeUnit unit, byte[] lockNodeBytes) throws Exception
    {
        final long      startMillis = System.currentTimeMillis();
        final Long      millisToWait = (unit != null) ? unit.toMillis(time) : null;
        final byte[]    localLockNodeBytes = (revocable.get() != null) ? new byte[0] : lockNodeBytes;
        int             retryCount = 0;

        String          ourPath = null;
        boolean         hasTheLock = false;
        boolean         isDone = false;
        while ( !isDone )
        {
            isDone = true;

            try
            {
                // 在zk上创建一个临时节点
                ourPath = driver.createsTheLock(client, path, localLockNodeBytes);
                // 根据创建的节点，判断是否获取到锁
                hasTheLock = internalLockLoop(startMillis, millisToWait, ourPath);
            }
            catch ( KeeperException.NoNodeException e )
            {
                // gets thrown by StandardLockInternalsDriver when it can't find the lock node
                // this can happen when the session expires, etc. So, if the retry allows, just try it all again
                if ( client.getZookeeperClient().getRetryPolicy().allowRetry(retryCount++, System.currentTimeMillis() - startMillis, RetryLoop.getDefaultRetrySleeper()) )
                {
                    isDone = false;
                }
                else
                {
                    throw e;
                }
            }
        }

        if ( hasTheLock )
        {
            return ourPath;
        }

        return null;
    }

这段代码中最关键的只有两行，一个是创建节点的createsTheLock()方法;另一个是internalLockLoop()，该方法对创建的临时顺序节点进行判断，判断该节点是否为最小节点。

咱们直接看internalLockLoop()方法：

private boolean internalLockLoop(long startMillis, Long millisToWait, String ourPath) throws Exception
    {
        boolean     haveTheLock = false;
        boolean     doDelete = false;
        try
        {
            if ( revocable.get() != null )
            {
                client.getData().usingWatcher(revocableWatcher).forPath(ourPath);
            }
            // 自旋直至获得锁，或者到达超时时间
            while ( (client.getState() == CuratorFrameworkState.STARTED) && !haveTheLock )
            {
                // 获取所有子节点，并且按小到大排序
                List<String>        children = getSortedChildren();
                // 将当前节点名去掉父节点的前缀，用于比较是否最小
                String              sequenceNodeName = ourPath.substring(basePath.length() + 1); // +1 to include the slash
                // 这个方法就是比较当前节点是否最小
                PredicateResults    predicateResults = driver.getsTheLock(client, children, sequenceNodeName, maxLeases);
                if ( predicateResults.getsTheLock() ) // 获得锁 (ಡωಡ)hiahiahia
                {
                    haveTheLock = true;
                }
                else
                {
                    // 咪有获取到锁，那就监听比自己“小一点”的那个节点
                    String  previousSequencePath = basePath + "/" + predicateResults.getPathToWatch();

                    synchronized(this)
                    {
                        try 
                        {
                            // use getData() instead of exists() to avoid leaving unneeded watchers which is a type of resource leak
                            client.getData().usingWatcher(watcher).forPath(previousSequencePath);
                            if ( millisToWait != null )
                            {
                                millisToWait -= (System.currentTimeMillis() - startMillis);
                                startMillis = System.currentTimeMillis();
                                // 如果到达超时时间，则删除创建的临时节点
                                if ( millisToWait <= 0 )
                                {
                                    doDelete = true;    // timed out - delete our node
                                    break;
                                }

                                wait(millisToWait);
                            }
                            else
                            {
                                wait();
                            }
                        }
                        catch ( KeeperException.NoNodeException e ) 
                        {
                            // it has been deleted (i.e. lock released). Try to acquire again
                        }
                    }
                }
            }
        }
        ...
        return haveTheLock;
    }

上面方法的核心逻辑：1.获取所有子节点，并且按小到大排序 2.判断当前节点是否最小 3.是最小则成功获取锁，否则监听比自己小的那个节点。
我们继续跟进getsTheLock()，看一下如何进行节点比较。

    @Override
    public PredicateResults getsTheLock(CuratorFramework client, List<String> children, String sequenceNodeName, int maxLeases) throws Exception
    {
        // 在已排序的节点列表中，查找当前节点的下标
        int             ourIndex = children.indexOf(sequenceNodeName);
        validateOurIndex(sequenceNodeName, ourIndex);
	    // maxLeases在创建锁InterProcessMutex实例时被初始化为1，如果ourIndex比maxLeases小，则说明是最小的节点
        boolean         getsTheLock = ourIndex < maxLeases;
        // 如果节点不是最小的，则监听比自己小1的节点。也就是最近的那个节点
        String          pathToWatch = getsTheLock ? null : children.get(ourIndex - maxLeases);
        
        return new PredicateResults(pathToWatch, getsTheLock);
    }

释放锁的过程

获取锁的过程分析的差不多了，下面来简单看一下释放锁的过程：

    /**
     * Perform one release of the mutex if the calling thread is the same thread that acquired it. If the
     * thread had made multiple calls to acquire, the mutex will still be held when this method returns.
     *
     * @throws Exception ZK errors, interruptions, current thread does not own the lock
     */
    @Override
    public void release() throws Exception
    {
        /*
            Note on concurrency: a given lockData instance
            can be only acted on by a single thread so locking isn't necessary
         */

        Thread currentThread = Thread.currentThread();
        LockData lockData = threadData.get(currentThread);
        // 当前线程没有lockData信息绑定，则抛出异常
        if ( lockData == null )
        {
            throw new IllegalMonitorStateException("You do not own the lock: " + basePath);
        }
		//LockCount计数减一
        int newLockCount = lockData.lockCount.decrementAndGet();
        // 由于是重入锁如果newLockCount>0，则节点不能删除，直接返回。
        if ( newLockCount > 0 )
        {
            return;
        }
        // 计数小于0，抛出异常
        if ( newLockCount < 0 )
        {
            throw new IllegalMonitorStateException("Lock count has gone negative for lock: " + basePath);
        }
        try
        {
            // 删除临时节点，移除监听。将锁释放
            internals.releaseLock(lockData.lockPath);
        }
        finally
        {
            // 删除当前线程的信息
            threadData.remove(currentThread);
        }
    }

小结

经过漫长的跟踪分析，获取锁的过程基本就是这几步：

1. 判断线程是否已经获得锁，有则计数加1
2. 未获得则尝试创建节点，如果当前线程创建的节点值是最小的，则获取锁
3. 如果不是最小的就监听比自己小的那个节点。
4. 超时则删除创建的节点，抛出异常，此次获取锁失败

释放锁就更加简单了：

1. 判断重入锁计数如果大于0，直接返回，有始有终嘛
2. 小于0，抛出异常
3. 等于0正常，删除临时节点，移除监听，将锁释放
4. 删除map中当前线程的信息