使用redis scan方法无法获取connection,导致线程锁死。
0、关键字
- redis
- springboot
- redistemplate
- scan
- try-with-resource
1、异常现象
应用部署后,功能正常使用,但约数小时左右,部分功能接口异常,接口请求无响应。
2、异常排查
查看堆栈信息,jstask pid。首先找到java进程pid;输出堆栈信息至log文件,jstask 30 > stask.log,看到与redis相关的日志,线程状态为waiting。
"pool-13-thread-6" prio=10 tid=0x00007f754800e800 nid=0x71b5 waiting on condition [0x00007f758f0ee000]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x0000000779b75f40> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
at org.apache.commons.pool2.impl.LinkedBlockingDeque.takeFirst(LinkedBlockingDeque.java:583)
at org.apache.commons.pool2.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:442)
at org.apache.commons.pool2.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:363)
at redis.clients.util.Pool.getResource(Pool.java:49)
at redis.clients.jedis.JedisPool.getResource(JedisPool.java:99)
at org.reborndb.reborn.RoundRobinJedisPool.getResource(RoundRobinJedisPool.java:300)
at com.le.smartconnect.adapter.spring.RebornConnectionFactory.getConnection(RebornConnectionFactory.java:43)
at org.springframework.data.redis.core.RedisConnectionUtils.doGetConnection(RedisConnectionUtils.java:128)
at org.springframework.data.redis.core.RedisConnectionUtils.getConnection(RedisConnectionUtils.java:91)
at org.springframework.data.redis.core.RedisConnectionUtils.getConnection(RedisConnectionUtils.java:78)
at xxx.run(xxx.java:80)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Locked ownable synchronizers:
- <0x000000074f529b08> (a java.util.concurrent.ThreadPoolExecutor$Worker)
也就是说,redis连接获取不到,线程一直在等待可用的redis连接。大概率是应用中有功能模块获取到连接,并没有释放。找到一个功能使用了scan,具体如下:
public void releaseCallbackMessage() throws Exception {
Cursor<Map.Entry<Object, Object>> cursor = RedisCacheUtils.scan(key)
if (cursor == null) {
logger.info("通过scan(H key, ScanOptions options)方法获取匹配键值对记录为空");
return;
}
while (cursor.hasNext()) {
// 遍历缓存
Map.Entry<Object, Object> entry