一、背景
在分布式环境中,多线程中的各种锁是不生效的(由于程序在多台机器上运行),即使是单线程,一旦访问临界资源,也会出错的。
因此我们需要分布式锁来对临界资源进行同步访问。
在此,介绍基于zookeeper,curator实现的分布式锁,包括InterProcessMutex锁、InterProcessReadWriteLock读写锁、信号量InterProcessSemaphoreV2、
分布式栅栏DistributedBarrier。
二、添加maven依赖
<dependency>
<groupId>org.apache.zookeeper</groupId>
<artifactId>zookeeper</artifactId>
<version>3.4.6</version>
</dependency>
<dependency>
<groupId>org.apache.curator</groupId>
<artifactId>curator-framework</artifactId>
<version>4.0.0</version>
</dependency>
<dependency>
<groupId>org.apache.curator</groupId>
<artifactId>curator-client</artifactId>
<version>4.0.0</version>
</dependency>
<dependency>
<groupId>org.apache.curator</groupId>
<artifactId>curator-recipes</artifactId>
<version>4.0.0</version>
</dependency>
三、InterProcessMutex锁
通过InterProcessMutex创建锁,然后使用acquire()获取锁,使用release()释放锁,
代码如下:
import org.apache.curator.framework.CuratorFramework;
import org.apache.curator.framework.CuratorFrameworkFactory;
import org.apache.curator.framework.recipes.locks.InterProcessMutex;
import org.apache.curator.retry.ExponentialBackoffRetry;
import java.util.ArrayList;
import java.util.List;
import java.util.concurrent.Callable;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.concurrent.Future;
import java.util.function.Consumer;
public class ZookeeperLockMain {
public static void main(String[] args) throws Exception {
//创建zookeeper客户端
CuratorFramework client = CuratorFrameworkFactory.newClient("127.0.0.1:2181", new ExponentialBackoffRetry(1000, 3));
client.start();
//指定锁路径
String lockPath = "/zkLockRoot/lock_1";
//创建锁,为可重入锁,即是获锁后,还可以再次获取,本例以此为例
InterProcessMutex lock = new InterProcessMutex(client, lockPath);
// //创建锁,为不可重入锁,即是获锁后,不可以再次获取,这里不作例子,使用和重入锁类似
// InterProcessSemaphoreMutex lock = new InterProcessSemaphoreMutex(client, lockPath);
ExecutorService executor = Executors.newCachedThreadPool();
Consumer<InterProcessMutex> consumer = (InterProcessMutex typeLock)->{
try{
List<Callable<String>> callList = new ArrayList<>();
Callable<String> call = () -> {
try{
//获取锁
typeLock.acquire();
System.out.println(Thread.currentThread() + " acquire read lock");
}catch (Exception e){
}finally {
//释放锁
typeLock.release();
System.out.println(Thread.currentThread() + " release read lock");
}
return "true";
};
//5个并发线程
for (int i = 0; i < 5; i++) {
callList.add(call);
}
List<Future<String>> futures = executor.invokeAll(callList);
}catch (Exception e){
}
};
//分布式锁测试
consumer.accept(lock);
executor.shutdown();
}
}
输出:
Thread[pool-3-thread-1,5,main] acquire read lock
Thread[pool-3-thread-1,5,main] release read lock
Thread[pool-3-thread-3,5,main] acquire read lock
Thread[pool-3-thread-3,5,main] release read lock
Thread[pool-3-thread-5,5,main] acquire read lock
Thread[pool-3-thread-5,5,main] release read lock
Thread[pool-3-thread-2,5,main] acquire read lock
Thread[pool-3-thread-2,5,main] release read lock
Thread[pool-3-thread-4,5,main] acquire read lock
Thread[pool-3-thread-4,5,main] release read lock
结果分析:
可以发现,只有等前一个获取锁的线程释放锁后,下一个线程才能获取锁。
四、InterProcessReadWriteLock读写锁
首先通过InterProcessReadWriteLock创建读写锁,然后再通过readLock()获取读锁,通过writeLock()获取写锁。
最后基于读锁或写锁,使用acquire()获取锁,使用release()释放锁。
可以同时有多个线程获取读锁,但同时只能有一个线程获取写锁。
代码如下:
import org.apache.curator.framework.CuratorFramework;
import org.apache.curator.framework.CuratorFrameworkFactory;
import org.apache.curator.framework.recipes.locks.InterProcessMutex;
import org.apache.curator.framework.recipes.locks.InterProcessReadWriteLock;
import org.apache.curator.retry.ExponentialBackoffRetry;
import java.util.ArrayList;
import java.util.List;
import java.util.concurrent.Callable;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.concurrent.Future;
import java.util.function.Consumer;
public class ZookeeperReadWriteLockMain {
public static void main(String[] args) throws Exception {
//创建zookeeper客户端
CuratorFramework client = CuratorFrameworkFactory.newClient("127.0.0.1:2181", new ExponentialBackoffRetry(1000, 3));
client.start();
//指定锁路径
String lockPath = "/zkLockRoot/lock_1";
//创建读写锁
InterProcessReadWriteLock lock = new InterProcessReadWriteLock(client, lockPath);
//生成线程池
ExecutorService executor = Executors.newCachedThreadPool();
Consumer<InterProcessMutex> consumer = (InterProcessMutex typeLock)->{
try{
List<Callable<String>> callList = new ArrayList<>();
Callable<String> call = () -> {
try{
typeLock.acquire();
System.out.println(Thread.currentThread() + " acquire read lock");
}catch (Exception e){
}finally {
typeLock.release();
System.out.println(Thread.currentThread() + " release read lock");
}
return "true";
};
//5个并发线程
for (int i = 0; i < 5; i++) {
callList.add(call);
}
List<Future<String>> futures = executor.invokeAll(callList);
}catch (Exception e){
}
};
//读锁测试(多个线程可同时获取读锁)
System.out.println("5个并发线程,读锁测试");
InterProcessMutex readLock = lock.readLock();
consumer.accept(readLock);
//写锁测试(同时只有一个线程获取写锁)
System.out.println("5个并发线程,写锁测试");
InterProcessMutex writeLock = lock.writeLock();
consumer.accept(writeLock);
executor.shutdown();
}
}
输出:
5个并发线程,读锁测试
Thread[pool-3-thread-5,5,main] acquire read lock
Thread[pool-3-thread-3,5,main] acquire read lock
Thread[pool-3-thread-1,5,main] acquire read lock
Thread[pool-3-thread-4,5,main] acquire read lock
Thread[pool-3-thread-2,5,main] acquire read lock
Thread[pool-3-thread-5,5,main] release read lock
Thread[pool-3-thread-3,5,main] release read lock
Thread[pool-3-thread-1,5,main] release read lock
Thread[pool-3-thread-4,5,main] release read lock
Thread[pool-3-thread-2,5,main] release read lock
5个并发线程,写锁测试
Thread[pool-3-thread-2,5,main] acquire read lock
Thread[pool-3-thread-2,5,main] release read lock
Thread[pool-3-thread-4,5,main] acquire read lock
Thread[pool-3-thread-4,5,main] release read lock
Thread[pool-3-thread-1,5,main] acquire read lock
Thread[pool-3-thread-1,5,main] release read lock
Thread[pool-3-thread-3,5,main] acquire read lock
Thread[pool-3-thread-3,5,main] release read lock
Thread[pool-3-thread-5,5,main] acquire read lock
Thread[pool-3-thread-5,5,main] release read lock
结果分析:
可以发现,同时多个线程可获取读锁,但同时只能有一个线程获取写锁。
五、InterProcessSemaphoreV2信息量
InterProcessSemaphoreV2信号量与多线程中的Semaphonre信息量含义是一致的,
即同时最多只能允许指定数量的线程访问临界资源。
通过InterProcessSemaphoreV2创建信息量,然后使用acquire()获取访问权限,
使用returnLease(lease)翻译访问权限。
代码如下:
import org.apache.curator.framework.CuratorFramework;
import org.apache.curator.framework.CuratorFrameworkFactory;
import org.apache.curator.framework.recipes.locks.InterProcessSemaphoreV2;
import org.apache.curator.framework.recipes.locks.Lease;
import org.apache.curator.retry.ExponentialBackoffRetry;
import java.util.ArrayList;
import java.util.List;
import java.util.concurrent.*;
import java.util.function.Consumer;
public class ZookeeperSemaphoreMain {
public static void main(String[] args) throws Exception {
//创建zookeeper客户端
CuratorFramework client = CuratorFrameworkFactory.newClient("127.0.0.1:2181", new ExponentialBackoffRetry(1000, 3));
client.start();
//指定锁路径
String lockPath = "/zkLockRoot/lock_1";
//创建信号量,指定同时最大访问数为3个
InterProcessSemaphoreV2 semaphoreLock = new InterProcessSemaphoreV2(client, lockPath, 3);
//生成线程池
ExecutorService executor = Executors.newCachedThreadPool();
Consumer<InterProcessSemaphoreV2> consumer = (InterProcessSemaphoreV2 semaphore)->{
try{
List<Callable<String>> callList = new ArrayList<>();
Callable<String> call = () -> {
Lease lease = semaphore.acquire();
try{
System.out.println(Thread.currentThread() + " acquire semaphore");
TimeUnit.MILLISECONDS.sleep(200);
}catch (Exception e){
}finally {
semaphore.returnLease(lease);
System.out.println(Thread.currentThread() + " release semaphore");
}
return "true";
};
//5个并发线程
for (int i = 0; i < 5; i++) {
callList.add(call);
}
List<Future<String>> futures = executor.invokeAll(callList);
}catch (Exception e){
}
};
//读锁测试(多个线程可同时获取读锁)
System.out.println("5个并发线程,信号量测试");
consumer.accept(semaphoreLock);
executor.shutdown();
}
}
输出:
5个并发线程,信号量测试
Thread[pool-3-thread-1,5,main] acquire semaphore
Thread[pool-3-thread-2,5,main] acquire semaphore
Thread[pool-3-thread-1,5,main] release semaphore
Thread[pool-3-thread-5,5,main] acquire semaphore
Thread[pool-3-thread-4,5,main] acquire semaphore
Thread[pool-3-thread-2,5,main] release semaphore
Thread[pool-3-thread-3,5,main] acquire semaphore
Thread[pool-3-thread-5,5,main] release semaphore
Thread[pool-3-thread-4,5,main] release semaphore
Thread[pool-3-thread-3,5,main] release semaphore
六、DistributedBarrier分布式栅栏
分布式栅栏DistributedBarrier用于在分布式环境下,阻塞指定数量线程(不一定在同一台机器),当这些线程达到某一点时,
再放开阻塞。和多线程编程中的CyclicBarrier类似。
barrier.waitOnBarrier()方法用于阻塞,当所有线程都调用了该方法后,阻塞解除。
示例代码:
import org.apache.curator.framework.CuratorFramework;
import org.apache.curator.framework.CuratorFrameworkFactory;
import org.apache.curator.framework.recipes.barriers.DistributedBarrier;
import org.apache.curator.retry.ExponentialBackoffRetry;
import java.util.concurrent.Callable;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.function.Consumer;
public class ZookeeperDistributedBarrierMain {
public static void main(String[] args) throws Exception {
//创建zookeeper客户端
CuratorFramework client = CuratorFrameworkFactory.newClient("127.0.0.1:2181", new ExponentialBackoffRetry(1000, 3));
client.start();
//指定锁路径
String lockPath = "/zkLockRoot/lock_1";
//创建分布式栅栏
DistributedBarrier distributedBarrier = new DistributedBarrier(client, lockPath);
distributedBarrier.setBarrier();
//生成线程池
ExecutorService executor = Executors.newCachedThreadPool();
Consumer<DistributedBarrier> consumer = (DistributedBarrier barrier)->{
try{
Callable<Boolean> call = () -> {
try{
System.out.println(Thread.currentThread() + " rearch barrier, waiting");
barrier.waitOnBarrier();
System.out.println(Thread.currentThread() + " do next");
}catch (Exception e){
}
return true;
};
//5个并发线程
for (int i = 0; i < 5; i++) {
executor.submit(call);
}
}catch (Exception e){
}
};
//栅栏测试(多个线程栅栏测试)
System.out.println("5个并发线程,栅栏测试");
consumer.accept(distributedBarrier);
distributedBarrier.removeBarrier();
executor.shutdown();
client.close();
}
}
输出:
5个并发线程,栅栏测试
Thread[pool-3-thread-2,5,main] rearch barrier, waiting
Thread[pool-3-thread-1,5,main] rearch barrier, waiting
Thread[pool-3-thread-4,5,main] rearch barrier, waiting
Thread[pool-3-thread-3,5,main] rearch barrier, waiting
Thread[pool-3-thread-5,5,main] rearch barrier, waiting
Thread[pool-3-thread-5,5,main] do next
Thread[pool-3-thread-3,5,main] do next
Thread[pool-3-thread-4,5,main] do next
Thread[pool-3-thread-1,5,main] do next
Thread[pool-3-thread-2,5,main] do next