zookeeper-curator分布式锁实现及源码分析（三）

zookeeper-curator分布式锁实现

1. curator分布式锁类型

1.1. InterProcessMutex

特性：分布式可重入排它锁，此锁可以重入，但是重入几次需要释放几次
原理：InterProcessMutex通过在zookeeper的某路径节点下创建临时序列节点来实现分布式锁，即每个线程（跨进程的线程）获取同一把锁前，都需要在同样的路径下创建一个节点，节点名字由uuid + 递增序列组成。而通过对比自身的序列数是否在所有子节点的第一位，来判断是否成功获取到了锁。当获取锁失败时，它会添加watcher来监听前一个节点的变动情况，然后进行等待状态。直到watcher的事件生效将自己唤醒，或者超时时间异常返回。

1.2. InterProcessSemaphoreMutex

InterProcessSemaphoreMutex是一种不可重入的互斥锁，也就意味着即使是同一个线程也无法在持有锁的情况下再次获得锁，所以需要注意，不可重入的锁很容易在一些情况导致死锁

1.3. InterProcessReadWriteLock

分布式读写锁

1.4. InterProcessMultiLock

将多个锁作为单个实体管理的容器,创建多重锁对象

例如
// 可重入锁
final InterProcessLock interProcessLock1 = new InterProcessMutex(client, lockPath);
// 不可重入锁
final InterProcessLock interProcessLock2 = new InterProcessSemaphoreMutex(client2, lockPath);
// 创建多重锁对象
final InterProcessLock lock = new InterProcessMultiLock(Arrays.asList(interProcessLock1, interProcessLock2));

1.5 InterProcessSemaphoreV2

共享信号量

2. InterProcessMutex实现分布式锁

public class DistributedLockDemo {
    public static void main(String[] args) {
        CuratorFramework curatorFramework = CuratorFrameworkFactory.builder().
                connectString("114.55.254.108:2181,114.55.254.108:2182,114.55.254.108:2183").
                sessionTimeoutMs(5000).
                // ExponentialBackoffRetry 重试指定次数
                // ReTryOneTime 仅重试一次
                        retryPolicy(new ExponentialBackoffRetry(1000,3)).
                        build();
        curatorFramework.start(); //启动

        InterProcessLock interProcessLock = new InterProcessMutex(curatorFramework,"/path");
        for (int i = 0; i < 5; i++) {
            new Thread(()->{
                System.out.println(Thread.currentThread().getName() + " 尝试获取锁");
                try {
                    interProcessLock.acquire();
                    System.out.println(Thread.currentThread().getName() + " 成功获取到锁");
                    Thread.sleep(1000);
                } catch (Exception e) {
                    e.printStackTrace();
                }finally {
                    try {
                        interProcessLock.release();
                        System.out.println(Thread.currentThread().getName() + " 成功释放锁");
                    } catch (Exception e) {
                        e.printStackTrace();
                    }
                }
            },"thread-" + i).start();
        }
    }
}

执行结果

thread-1 尝试获取锁
thread-0 尝试获取锁
thread-2 尝试获取锁
thread-3 尝试获取锁
thread-4 尝试获取锁
thread-4 成功获取到锁
thread-4 成功释放锁
thread-1 成功获取到锁
thread-1 成功释放锁
thread-3 成功获取到锁
thread-3 成功释放锁
thread-2 成功获取到锁
thread-2 成功释放锁
thread-0 成功获取到锁
thread-0 成功释放锁

3. InterProcessMutex源码分析

3.1 锁的初始化

InterProcessLock interProcessLock = new InterProcessMutex(curatorFramework,"/path");

    public InterProcessMutex(CuratorFramework client, String path)
    {
        this(client, path, new StandardLockInternalsDriver());
    }

LockInternalsDriver的实现类StandardLockInternalsDriver后面加锁的时候会用到

    public InterProcessMutex(CuratorFramework client, String path, LockInternalsDriver driver)
    {
        this(client, path, LOCK_NAME, 1, driver);
    }

maxLeases为1，在后面进行节点监听会用到

3.2 获取锁

   //获取锁
    public void acquire() throws Exception
    {
        if ( !internalLock(-1, null) )
        {
            throw new IOException("Lost connection while trying to acquire lock: " + basePath);
        }
    }

    private boolean internalLock(long time, TimeUnit unit) throws Exception
    {
        // 同一个线程可以多次获取锁
        Thread currentThread = Thread.currentThread();
        LockData lockData = threadData.get(currentThread);
        if ( lockData != null )
        {
            // re-entering
            lockData.lockCount.incrementAndGet();
            return true;
        }

		// 如果是不同线程尝试去获取锁
        String lockPath = internals.attemptLock(time, unit, getLockNodeBytes());
        // 如果lockPath 不为空代表获取锁成功
        if ( lockPath != null )
        {
            LockData newLockData = new LockData(currentThread, lockPath);
            threadData.put(currentThread, newLockData);
            return true;
        }

        return false;
    }

LockData lockData = threadData.get(currentThread);
上面的代码可以看出如果是同一线程可以多次直接返回获取锁成功，只是计数会增加，可见InterProcessMutex是支持可重入的

   String attemptLock(long time, TimeUnit unit, byte[] lockNodeBytes) throws Exception
    {
        final long      startMillis = System.currentTimeMillis();
        final Long      millisToWait = (unit != null) ? unit.toMillis(time) : null;
        final byte[]    localLockNodeBytes = (revocable.get() != null) ? new byte[0] : lockNodeBytes;
        int             retryCount = 0;

        String          ourPath = null;
        boolean         hasTheLock = false;
        boolean         isDone = false;
        while ( !isDone )
        {
            isDone = true;

            try
            {
                ourPath = driver.createsTheLock(client, path, localLockNodeBytes);
                hasTheLock = internalLockLoop(startMillis, millisToWait, ourPath);
            }
            catch ( KeeperException.NoNodeException e )
            {
                // gets thrown by StandardLockInternalsDriver when it can't find the lock node
                // this can happen when the session expires, etc. So, if the retry allows, just try it all again
                if ( client.getZookeeperClient().getRetryPolicy().allowRetry(retryCount++, System.currentTimeMillis() - startMillis, RetryLoop.getDefaultRetrySleeper()) )
                {
                    isDone = false;
                }
                else
                {
                    throw e;
                }
            }
        }
        if ( hasTheLock )
        {
            return ourPath;
        }
        return null;
    }

driver.createsTheLock(client, path, localLockNodeBytes);
去创建节点

	// 创建临时有序节点
    public String createsTheLock(CuratorFramework client, String path, byte[] lockNodeBytes) throws Exception
    {
        String ourPath;
        if ( lockNodeBytes != null )
        {
            ourPath = client.create().creatingParentContainersIfNeeded().withProtection().withMode(CreateMode.EPHEMERAL_SEQUENTIAL).forPath(path, lockNodeBytes);
        }
        else
        {
            ourPath = client.create().creatingParentContainersIfNeeded().withProtection().withMode(CreateMode.EPHEMERAL_SEQUENTIAL).forPath(path);
        }
        return ourPath;
    }

client.create().creatingParentContainersIfNeeded().withProtection().withMode(CreateMode.EPHEMERAL_SEQUENTIAL).forPath(path);
创建节点的类型是临时有序节点

    private boolean internalLockLoop(long startMillis, Long millisToWait, String ourPath) throws Exception
    {
        boolean     haveTheLock = false;
        boolean     doDelete = false;
        try
        {
            if ( revocable.get() != null )
            {
                client.getData().usingWatcher(revocableWatcher).forPath(ourPath);
            }

            while ( (client.getState() == CuratorFrameworkState.STARTED) && !haveTheLock )
            {
                List<String>        children = getSortedChildren();
                String              sequenceNodeName = ourPath.substring(basePath.length() + 1); // +1 to include the slash

                PredicateResults    predicateResults = driver.getsTheLock(client, children, sequenceNodeName, maxLeases);
                if ( predicateResults.getsTheLock() )
                {
                	//超线程获取到锁直接返回
                    haveTheLock = true;
                }
                else
                {
                	//该线程未获取到锁
                    String  previousSequencePath = basePath + "/" + predicateResults.getPathToWatch();

                    synchronized(this)
                    {
                        try 
                        {
                            // 把当前线程对应的子节点绑定到上一个子节点对应的监听器上，该线程被阻塞，当上一个子节点对应的线程结束后，对应的临时节点就会消失，此时会触发监听事件，在监听事件中，唤醒该线程，则获取锁成功
                            client.getData().usingWatcher(watcher).forPath(previousSequencePath);
                            // 超时等待
                            if ( millisToWait != null )
                            {
                                millisToWait -= (System.currentTimeMillis() - startMillis);
                                startMillis = System.currentTimeMillis();
                                if ( millisToWait <= 0 )
                                {
                                	// 超时删除节点标志位
                                    doDelete = true;    // timed out - delete our node
                                    break;
                                }
								// 线程未获取锁进行超时等待
                                wait(millisToWait);
                            }
                            else
                            {
                            	// 线程未获取锁进行阻塞
                                wait();
                            }
                        }
                        catch ( KeeperException.NoNodeException e ) 
                        {
                            // it has been deleted (i.e. lock released). Try to acquire again
                        }
                    }
                }
            }
        }
        catch ( Exception e )
        {
            ThreadUtils.checkInterrupted(e);
            doDelete = true;
            throw e;
        }
        finally
        {
        	// 如果线程等待时间超时则会删除该临时节点
            if ( doDelete )
            {
                deleteOurPath(ourPath);
            }
        }
        return haveTheLock;
    }

getSortedChildren()
对path下的所有的临时有序子节点进行排序

    public PredicateResults getsTheLock(CuratorFramework client, List<String> children, String sequenceNodeName, int maxLeases) throws Exception
    {
        int             ourIndex = children.indexOf(sequenceNodeName);
        validateOurIndex(sequenceNodeName, ourIndex);

        boolean         getsTheLock = ourIndex < maxLeases;
        String          pathToWatch = getsTheLock ? null : children.get(ourIndex - maxLeases);

        return new PredicateResults(pathToWatch, getsTheLock);
    }

上面的方法在类StandardLockInternalsDriver中从初始化中可以看出来，根据children.indexOf(sequenceNodeName)获取本节点的排序位置，
maxLeases在初始化时赋值为1
例如：如果该节点为0号节点，此时ourIndex=0 ， maxLeases=1，则getsTheLock=true；获取到锁，直接返回；
r如果此时再有1号节点进行争抢锁时，此时此时ourIndex=1，maxLeases=1，则getsTheLock=false；获取锁失败，此时
pathToWatch=children.get(0)；则pathToWatch为0号节点，也即是1号对应的线程的PredicateResults对象中的节点是0号节点的信息；
在后面进行绑定监听器，在就会出现1号节点绑定在0号监听器，以此类推，2号绑定1号上，3号绑定的2号号上，
为何如此设计？
原因是：防止出现惊群效应

	//初始化StandardLockInternalsDriver代码
    public InterProcessMutex(CuratorFramework client, String path)
    {
        this(client, path, new StandardLockInternalsDriver());
    }

3.3 释放锁

锁是通过release进行释放的

    public void release() throws Exception
    {
        /*
            Note on concurrency: a given lockData instance
            can be only acted on by a single thread so locking isn't necessary
         */

        Thread currentThread = Thread.currentThread();
        LockData lockData = threadData.get(currentThread);
        if ( lockData == null )
        {
            throw new IllegalMonitorStateException("You do not own the lock: " + basePath);
        }
		// 同一线程多次获取锁，计数进行减一
        int newLockCount = lockData.lockCount.decrementAndGet();
        if ( newLockCount > 0 )
        {
            return;
        }
        if ( newLockCount < 0 )
        {
            throw new IllegalMonitorStateException("Lock count has gone negative for lock: " + basePath);
        }
        try
        {
        	// 释放锁
            internals.releaseLock(lockData.lockPath);
        }
        finally
        {
            threadData.remove(currentThread);
        }
    }

释放锁也就是删除该线程对应的临时有序子节点

    void releaseLock(String lockPath) throws Exception
    {
        revocable.set(null);
        deleteOurPath(lockPath);
    }

    private void deleteOurPath(String ourPath) throws Exception
    {
        try
        {
            client.delete().guaranteed().forPath(ourPath);
        }
        catch ( KeeperException.NoNodeException e )
        {
            // ignore - already deleted (possibly expired session, etc.)
        }
    }