参考:https://www.jianshu.com/p/6d1fd5d8fe0a
参考:https://www.cnblogs.com/harry335/p/5473480.html
参考:https://blog.csdn.net/qq_35349114/article/details/84027745
分布式锁:排他性、避免死锁、高可用
实现方式为:数据库实现、基于Redis实现、基于ZooKeeper实现。
一、数据库实现
1、基于数据库的乐观锁实现
乐观锁机制其实就是在数据库表中引入一个版本号(version)字段来实现的。读取和更新
2、基于数据库的悲观锁实现
悲观锁也叫作排它锁,在Mysql中是基于 for update 来实现加锁的。通过 for update 操作,数据库在查询的时候就会给正在查询的记录加上排它锁。
当某条记录加上排它锁之后,其它线程是无法操作这条记录的。
如此,我们就可以认为获得了排它锁的这个线程是拥有了分布式锁,然后就可以执行我们想要做的业务逻辑,当逻辑完成之后,再释放锁即可。
SQL语句加锁:
//使用数据库的悲观锁for update
String sql = "select value from t_table_id where table_name=? for update";
for update是在数据库中上锁用的,可以为数据库中的行上一个排它锁。当一个事务的操作未完成时候,其他事务可以读取但是不能写入或更新。
例子:
比如一张表三个字段 , id(商品id), name(商品名字) , count(数量)
当商品抢购时候会显示剩余商品件数,如果并发量大的时候,商品自减的值可能不准确。所以当我们在一个事务中对count字段进行修改的时候,其他事务应该只能读取指定id的count,而不能进行update等操作。这个时候就需要用到for update.
sql语句:
start transaction ;
select * from table_name where id =1 for update ;
update table_name set count = count - 1 where id= 1;
此时如果另一个事务也想执行类似的操作:
start transaction ;
select * from table_name where id =1 for update ;
//下面的这行sql会等待,直到上面的事务回滚或者commit才得到执行。
update table_name set count = count - 1 where id= 1;
*注:当选中某一个行的时候,如果是通过主键id选中的。那么这个时候是行级锁。
其他的行还是可以直接insert 或者update的。如果是通过其他的方式选中行,或者选中的条件不明确包含主键。这个时候会锁表。其他的事务对该表的任意一行记录都无法进行插入或者更新操作。只能读取。*
采用悲观锁来实现同步
在sql语句后加 for update就加上了锁,在查询的时候进行加锁,在加锁后不能进行查询。提交时候后其他人才能查询。
public static int generate(String tableName){
//使用数据库的悲观锁for update
String sql = "select value from t_table_id where table_name=? for update";
Connection conn = null;
PreparedStatement pstmt = null;
ResultSet rs = null;
int value = 0;
try{
conn = DbUtil.getConnection();
//设置自动提交为false
DbUtil.beginTransaction(conn);
pstmt = conn.prepareStatement(sql);
pstmt.setString(1, tableName);
rs = pstmt.executeQuery();
rs.next();
// if(!rs.next()){
// throw new RuntimeException();
// }
value = rs.getInt("value");
value++;
modifyValueField(conn,tableName,value);
//提交事务
DbUtil.commitTransaction(conn);
}catch(Exception e){
e.printStackTrace();
//回滚事务
DbUtil.rollbackTranscation(conn);
throw new RuntimeException();
}finally{
DbUtil.close(rs);
DbUtil.close(pstmt);
DbUtil.resetConnection(conn);
DbUtil.close(conn);
}
return value;
}
二、基于Redis实现
基于Redis实现的锁机制,主要是依赖Redis自身的原子操作:
SET user_key user_value MX PX 100
NX:只在在键不存在时,才对键进行设置操作,SET key value NX 效果等同于 SETNX key value。
PX millisecond:设置键的过期时间为millisecond毫秒,当超过这个时间后,设置的键会自动失效。
当redis中不存在user_key这个键的时候,才会去设置一个user_key键,并且给这个键的值设置为 user_value,且这个键的存活时间为100ms。
解锁很简单,只需要删除这个key就可以了,不过删除之前需要判断,这个key对应的value是当初自己设置的那个。
另外,针对Redis集群模式的分布式锁,可以采用Redis的Redlock机制。
/**
* @author http://blog.csdn.net/java2000_wl
* @version <b>1.0.0</b>
*/
public class RedisBillLockHandler implements IBatchBillLockHandler {
private static final Logger LOGGER = LoggerFactory.getLogger(RedisBillLockHandler.class);
private static final int DEFAULT_SINGLE_EXPIRE_TIME = 3;
private static final int DEFAULT_BATCH_EXPIRE_TIME = 6;
private final JedisPool jedisPool;
/**
* 构造
* @author http://blog.csdn.net/java2000_wl
*/
public RedisBillLockHandler(JedisPool jedisPool) {
this.jedisPool = jedisPool;
}
/**
* 获取锁 如果锁可用 立即返回true, 否则返回false
* @author http://blog.csdn.net/java2000_wl
* @see com.fx.platform.components.lock.IBillLockHandler#tryLock(com.fx.platform.components.lock.IBillIdentify)
* @param billIdentify
* @return
*/
public boolean tryLock(IBillIdentify billIdentify) {
return tryLock(billIdentify, 0L, null);
}
/**
* 锁在给定的等待时间内空闲,则获取锁成功 返回true, 否则返回false
* @author http://blog.csdn.net/java2000_wl
* @see com.fx.platform.components.lock.IBillLockHandler#tryLock(com.fx.platform.components.lock.IBillIdentify,
* long, java.util.concurrent.TimeUnit)
* @param billIdentify
* @param timeout
* @param unit
* @return
*/
public boolean tryLock(IBillIdentify billIdentify, long timeout, TimeUnit unit) {
String key = (String) billIdentify.uniqueIdentify();
Jedis jedis = null;
try {
jedis = getResource();
long nano = System.nanoTime();
do {
LOGGER.debug("try lock key: " + key);
Long i = jedis.setnx(key, key);
if (i == 1) {
jedis.expire(key, DEFAULT_SINGLE_EXPIRE_TIME);
LOGGER.debug("get lock, key: " + key + " , expire in " + DEFAULT_SINGLE_EXPIRE_TIME + " seconds.");
return Boolean.TRUE;
} else { // 存在锁
if (LOGGER.isDebugEnabled()) {
String desc = jedis.get(key);
LOGGER.debug("key: " + key + " locked by another business:" + desc);
}
}
if (timeout == 0) {
break;
}
Thread.sleep(300);
} while ((System.nanoTime() - nano) < unit.toNanos(timeout));
return Boolean.FALSE;
} catch (JedisConnectionException je) {
LOGGER.error(je.getMessage(), je);
returnBrokenResource(jedis);
} catch (Exception e) {
LOGGER.error(e.getMessage(), e);
} finally {
returnResource(jedis);
}
return Boolean.FALSE;
}
/**
* 如果锁空闲立即返回 获取失败 一直等待
* @author http://blog.csdn.net/java2000_wl
* @see com.fx.platform.components.lock.IBillLockHandler#lock(com.fx.platform.components.lock.IBillIdentify)
* @param billIdentify
*/
public void lock(IBillIdentify billIdentify) {
String key = (String) billIdentify.uniqueIdentify();
Jedis jedis = null;
try {
jedis = getResource();
do {
LOGGER.debug("lock key: " + key);
Long i = jedis.setnx(key, key);
if (i == 1) {
jedis.expire(key, DEFAULT_SINGLE_EXPIRE_TIME);
LOGGER.debug("get lock, key: " + key + " , expire in " + DEFAULT_SINGLE_EXPIRE_TIME + " seconds.");
return;
} else {
if (LOGGER.isDebugEnabled()) {
String desc = jedis.get(key);
LOGGER.debug("key: " + key + " locked by another business:" + desc);
}
}
Thread.sleep(300);
} while (true);
} catch (JedisConnectionException je) {
LOGGER.error(je.getMessage(), je);
returnBrokenResource(jedis);
} catch (Exception e) {
LOGGER.error(e.getMessage(), e);
} finally {
returnResource(jedis);
}
}
/**
* 释放锁
* @author http://blog.csdn.net/java2000_wl
* @see com.fx.platform.components.lock.IBillLockHandler#unLock(com.fx.platform.components.lock.IBillIdentify)
* @param billIdentify
*/
public void unLock(IBillIdentify billIdentify) {
List<IBillIdentify> list = new ArrayList<IBillIdentify>();
list.add(billIdentify);
unLock(list);
}
/**
* 批量获取锁 如果全部获取 立即返回true, 部分获取失败 返回false
* @author http://blog.csdn.net/java2000_wl
* @date 2013-7-22 下午10:27:44
* @see com.fx.platform.components.lock.IBatchBillLockHandler#tryLock(java.util.List)
* @param billIdentifyList
* @return
*/
public boolean tryLock(List<IBillIdentify> billIdentifyList) {
return tryLock(billIdentifyList, 0L, null);
}
/**
* 锁在给定的等待时间内空闲,则获取锁成功 返回true, 否则返回false
* @author http://blog.csdn.net/java2000_wl
* @param billIdentifyList
* @param timeout
* @param unit
* @return
*/
public boolean tryLock(List<IBillIdentify> billIdentifyList, long timeout, TimeUnit unit) {
Jedis jedis = null;
try {
List<String> needLocking = new CopyOnWriteArrayList<String>();
List<String> locked = new CopyOnWriteArrayList<String>();
jedis = getResource();
long nano = System.nanoTime();
do {
// 构建pipeline,批量提交
Pipeline pipeline = jedis.pipelined();
for (IBillIdentify identify : billIdentifyList) {
String key = (String) identify.uniqueIdentify();
needLocking.add(key);
pipeline.setnx(key, key);
}
LOGGER.debug("try lock keys: " + needLocking);
// 提交redis执行计数
List<Object> results = pipeline.syncAndReturnAll();
for (int i = 0; i < results.size(); ++i) {
Long result = (Long) results.get(i);
String key = needLocking.get(i);
if (result == 1) { // setnx成功,获得锁
jedis.expire(key, DEFAULT_BATCH_EXPIRE_TIME);
locked.add(key);
}
}
needLocking.removeAll(locked); // 已锁定资源去除
if (CollectionUtils.isEmpty(needLocking)) {
return true;
} else {
// 部分资源未能锁住
LOGGER.debug("keys: " + needLocking + " locked by another business:");
}
if (timeout == 0) {
break;
}
Thread.sleep(500);
} while ((System.nanoTime() - nano) < unit.toNanos(timeout));
// 得不到锁,释放锁定的部分对象,并返回失败
if (!CollectionUtils.isEmpty(locked)) {
jedis.del(locked.toArray(new String[0]));
}
return false;
} catch (JedisConnectionException je) {
LOGGER.error(je.getMessage(), je);
returnBrokenResource(jedis);
} catch (Exception e) {
LOGGER.error(e.getMessage(), e);
} finally {
returnResource(jedis);
}
return true;
}
/**
* 批量释放锁
* @author http://blog.csdn.net/java2000_wl
* @see com.fx.platform.components.lock.IBatchBillLockHandler#unLock(java.util.List)
* @param billIdentifyList
*/
public void unLock(List<IBillIdentify> billIdentifyList) {
List<String> keys = new CopyOnWriteArrayList<String>();
for (IBillIdentify identify : billIdentifyList) {
String key = (String) identify.uniqueIdentify();
keys.add(key);
}
Jedis jedis = null;
try {
jedis = getResource();
jedis.del(keys.toArray(new String[0]));
LOGGER.debug("release lock, keys :" + keys);
} catch (JedisConnectionException je) {
LOGGER.error(je.getMessage(), je);
returnBrokenResource(jedis);
} catch (Exception e) {
LOGGER.error(e.getMessage(), e);
} finally {
returnResource(jedis);
}
}
/**
* @author http://blog.csdn.net/java2000_wl
* @date 2013-7-22 下午9:33:45
* @return
*/
private Jedis getResource() {
return jedisPool.getResource();
}
/**
* 销毁连接
* @author http://blog.csdn.net/java2000_wl
* @param jedis
*/
private void returnBrokenResource(Jedis jedis) {
if (jedis == null) {
return;
}
try {
//容错
jedisPool.returnBrokenResource(jedis);
} catch (Exception e) {
LOGGER.error(e.getMessage(), e);
}
}
/**
* @author http://blog.csdn.net/java2000_wl
* @param jedis
*/
private void returnResource(Jedis jedis) {
if (jedis == null) {
return;
}
try {
jedisPool.returnResource(jedis);
} catch (Exception e) {
LOGGER.error(e.getMessage(), e);
}
}
三、基于Zookeeper实现
基于ZooKeeper,就是使用它的临时有序节点来实现的分布式锁。
其原理是当某客户端要进行逻辑的加锁时,就在zookeeper上的某个指定节点的目录下,去生成一个唯一的临时有序节点, 然后判断自己是否是这些有序节点中序号最小的一个,如果是,则算是获取了锁。如果不是,则说明没有获取到锁,那么就需要在序列中找到比自己小的那个节点,并对其调用exist()方法,对其注册事件监听,当监听到这个节点被删除了,那就再去判断一次自己当初创建的节点是否变成了序列中最小的。如果是,则获取锁,如果不是,则重复上述步骤,当释放锁的时候,只需将这个临时节点删除即可。
使用zookeeper实现的分布式锁
分布式锁,实现了Lock接口
复制代码 代码如下:
package com.concurrent;
import java.io.IOException;
import java.util.ArrayList;
import java.util.Collections;
import java.util.List;
import java.util.concurrent.CountDownLatch;
import java.util.concurrent.TimeUnit;
import java.util.concurrent.locks.Condition;
import java.util.concurrent.locks.Lock;
import org.apache.zookeeper.CreateMode;
import org.apache.zookeeper.KeeperException;
import org.apache.zookeeper.WatchedEvent;
import org.apache.zookeeper.Watcher;
import org.apache.zookeeper.ZooDefs;
import org.apache.zookeeper.ZooKeeper;
import org.apache.zookeeper.data.Stat;
/**
DistributedLock lock = null;
try {
lock = new DistributedLock("127.0.0.1:2182","test");
lock.lock();
//do something...
} catch (Exception e) {
e.printStackTrace();
}
finally {
if(lock != null)
lock.unlock();
}
* @author xueliang
*
*/
public class DistributedLock implements Lock, Watcher{
private ZooKeeper zk;
private String root = "/locks";//根
private String lockName;//竞争资源的标志
private String waitNode;//等待前一个锁
private String myZnode;//当前锁
private CountDownLatch latch;//计数器
private int sessionTimeout = 30000;
private List<Exception> exception = new ArrayList<Exception>();
/**
* 创建分布式锁,使用前请确认config配置的zookeeper服务可用
* @param config 127.0.0.1:2181
* @param lockName 竞争资源标志,lockName中不能包含单词lock
*/
public DistributedLock(String config, String lockName){
this.lockName = lockName;
// 创建一个与服务器的连接
try {
zk = new ZooKeeper(config, sessionTimeout, this);
Stat stat = zk.exists(root, false);
if(stat == null){
// 创建根节点
zk.create(root, new byte[0], ZooDefs.Ids.OPEN_ACL_UNSAFE,CreateMode.PERSISTENT);
}
} catch (IOException e) {
exception.add(e);
} catch (KeeperException e) {
exception.add(e);
} catch (InterruptedException e) {
exception.add(e);
}
}
/**
* zookeeper节点的监视器
*/
public void process(WatchedEvent event) {
if(this.latch != null) {
this.latch.countDown();
}
}
public void lock() {
if(exception.size() > 0){
throw new LockException(exception.get(0));
}
try {
if(this.tryLock()){
System.out.println("Thread " + Thread.currentThread().getId() + " " +myZnode + " get lock true");
return;
}
else{
waitForLock(waitNode, sessionTimeout);//等待锁
}
} catch (KeeperException e) {
throw new LockException(e);
} catch (InterruptedException e) {
throw new LockException(e);
}
}
public boolean tryLock() {
try {
String splitStr = "_lock_";
if(lockName.contains(splitStr))
throw new LockException("lockName can not contains \\u000B");
//创建临时子节点
myZnode = zk.create(root + "/" + lockName + splitStr, new byte[0], ZooDefs.Ids.OPEN_ACL_UNSAFE,CreateMode.EPHEMERAL_SEQUENTIAL);
System.out.println(myZnode + " is created ");
//取出所有子节点
List<String> subNodes = zk.getChildren(root, false);
//取出所有lockName的锁
List<String> lockObjNodes = new ArrayList<String>();
for (String node : subNodes) {
String _node = node.split(splitStr)[0];
if(_node.equals(lockName)){
lockObjNodes.add(node);
}
}
Collections.sort(lockObjNodes);
System.out.println(myZnode + "==" + lockObjNodes.get(0));
if(myZnode.equals(root+"/"+lockObjNodes.get(0))){
//如果是最小的节点,则表示取得锁
return true;
}
//如果不是最小的节点,找到比自己小1的节点
String subMyZnode = myZnode.substring(myZnode.lastIndexOf("/") + 1);
waitNode = lockObjNodes.get(Collections.binarySearch(lockObjNodes, subMyZnode) - 1);
} catch (KeeperException e) {
throw new LockException(e);
} catch (InterruptedException e) {
throw new LockException(e);
}
return false;
}
public boolean tryLock(long time, TimeUnit unit) {
try {
if(this.tryLock()){
return true;
}
return waitForLock(waitNode,time);
} catch (Exception e) {
e.printStackTrace();
}
return false;
}
private boolean waitForLock(String lower, long waitTime) throws InterruptedException, KeeperException {
Stat stat = zk.exists(root + "/" + lower,true);
//判断比自己小一个数的节点是否存在,如果不存在则无需等待锁,同时注册监听
if(stat != null){
System.out.println("Thread " + Thread.currentThread().getId() + " waiting for " + root + "/" + lower);
this.latch = new CountDownLatch(1);
this.latch.await(waitTime, TimeUnit.MILLISECONDS);
this.latch = null;
}
return true;
}
public void unlock() {
try {
System.out.println("unlock " + myZnode);
zk.delete(myZnode,-1);
myZnode = null;
zk.close();
} catch (InterruptedException e) {
e.printStackTrace();
} catch (KeeperException e) {
e.printStackTrace();
}
}
public void lockInterruptibly() throws InterruptedException {
this.lock();
}
public Condition newCondition() {
return null;
}
public class LockException extends RuntimeException {
private static final long serialVersionUID = 1L;
public LockException(String e){
super(e);
}
public LockException(Exception e){
super(e);
}
}
}
并发测试工具
复制代码 代码如下:
package com.concurrent;
import java.util.ArrayList;
import java.util.Collections;
import java.util.List;
import java.util.concurrent.CopyOnWriteArrayList;
import java.util.concurrent.CountDownLatch;
import java.util.concurrent.atomic.AtomicInteger;
/**
ConcurrentTask[] task = new ConcurrentTask[5];
for(int i=0;i<task.length;i++){
task[i] = new ConcurrentTask(){
public void run() {
System.out.println("==============");
}};
}
new ConcurrentTest(task);
* @author xueliang
*
*/
public class ConcurrentTest {
private CountDownLatch startSignal = new CountDownLatch(1);//开始阀门
private CountDownLatch doneSignal = null;//结束阀门
private CopyOnWriteArrayList<Long> list = new CopyOnWriteArrayList<Long>();
private AtomicInteger err = new AtomicInteger();//原子递增
private ConcurrentTask[] task = null;
public ConcurrentTest(ConcurrentTask... task){
this.task = task;
if(task == null){
System.out.println("task can not null");
System.exit(1);
}
doneSignal = new CountDownLatch(task.length);
start();
}
/**
* @param args
* @throws ClassNotFoundException
*/
private void start(){
//创建线程,并将所有线程等待在阀门处
createThread();
//打开阀门
startSignal.countDown();//递减锁存器的计数,如果计数到达零,则释放所有等待的线程
try {
doneSignal.await();//等待所有线程都执行完毕
} catch (InterruptedException e) {
e.printStackTrace();
}
//计算执行时间
getExeTime();
}
/**
* 初始化所有线程,并在阀门处等待
*/
private void createThread() {
long len = doneSignal.getCount();
for (int i = 0; i < len; i++) {
final int j = i;
new Thread(new Runnable(){
public void run() {
try {
startSignal.await();//使当前线程在锁存器倒计数至零之前一直等待
long start = System.currentTimeMillis();
task[j].run();
long end = (System.currentTimeMillis() - start);
list.add(end);
} catch (Exception e) {
err.getAndIncrement();//相当于err++
}
doneSignal.countDown();
}
}).start();
}
}
/**
* 计算平均响应时间
*/
private void getExeTime() {
int size = list.size();
List<Long> _list = new ArrayList<Long>(size);
_list.addAll(list);
Collections.sort(_list);
long min = _list.get(0);
long max = _list.get(size-1);
long sum = 0L;
for (Long t : _list) {
sum += t;
}
long avg = sum/size;
System.out.println("min: " + min);
System.out.println("max: " + max);
System.out.println("avg: " + avg);
System.out.println("err: " + err.get());
}
public interface ConcurrentTask {
void run();
}
}
测试
复制代码 代码如下:
package com.concurrent;
import com.concurrent.ConcurrentTest.ConcurrentTask;
public class ZkTest {
public static void main(String[] args) {
Runnable task1 = new Runnable(){
public void run() {
DistributedLock lock = null;
try {
lock = new DistributedLock("127.0.0.1:2182","test1");
//lock = new DistributedLock("127.0.0.1:2182","test2");
lock.lock();
Thread.sleep(3000);
System.out.println("===Thread " + Thread.currentThread().getId() + " running");
} catch (Exception e) {
e.printStackTrace();
}
finally {
if(lock != null)
lock.unlock();
}
}
};
new Thread(task1).start();
try {
Thread.sleep(1000);
} catch (InterruptedException e1) {
e1.printStackTrace();
}
ConcurrentTask[] tasks = new ConcurrentTask[60];
for(int i=0;i<tasks.length;i++){
ConcurrentTask task3 = new ConcurrentTask(){
public void run() {
DistributedLock lock = null;
try {
lock = new DistributedLock("127.0.0.1:2183","test2");
lock.lock();
System.out.println("Thread " + Thread.currentThread().getId() + " running");
} catch (Exception e) {
e.printStackTrace();
}
finally {
lock.unlock();
}
}
};
tasks[i] = task3;
}
new ConcurrentTest(tasks);
}
}