背景描述:
(1) 某项目,使用redis做存储,用redis的set性质来做实时统计,同时也存放其他统计数据;
(2) 用到的key不少,value集合量较多;
(3) 每天零点的时候,回清理当前redis中所有的数据;
(4) 异常都出现在 零点清理之后;
异常现场:
2018/03/13 00:00:20 OSS INFO [com.xxx.RedisDAO] - error
redis.clients.jedis.exceptions.JedisConnectionException: java.net.SocketTimeoutException: Read timed out
at redis.clients.util.RedisInputStream.ensureFill(RedisInputStream.java:202)
at redis.clients.util.RedisInputStream.readByte(RedisInputStream.java:40)
at redis.clients.jedis.Protocol.process(Protocol.java:151)
at redis.clients.jedis.Protocol.read(Protocol.java:215)
at redis.clients.jedis.Connection.readProtocolWithCheckingBroken(Connection.java:340)
at redis.clients.jedis.Connection.getIntegerReply(Connection.java:265)
at redis.clients.jedis.Jedis.sadd(Jedis.java:1109)
at com.xxx.RedisDAO.setXXXKPI(RedisDAO.java:66)
at com.xxx.xxxRunnable.run(xxxRunnable.java:69)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.net.SocketTimeoutException: Read timed out
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
at java.net.SocketInputStream.read(SocketInputStream.java:171)
at java.net.SocketInputStream.read(SocketInputStream.java:141)
at java.net.SocketInputStream.read(SocketInputStream.java:127)
at redis.clients.util.RedisInputStream.ensureFill(RedisInputStream.java:196)
... 9 more
2018/03/13 00:00:20 OSS INFO [com.xxx.RedisDAO] - error
java.lang.ClassCastException: java.lang.Long cannot be cast to [B
at redis.clients.jedis.Connection.getStatusCodeReply(Connection.java:239)
at redis.clients.jedis.Jedis.set(Jedis.java:121)
at com.xxx.RedisDAO.setXXXXXKPI(RedisDAO.java:100)
at com.xxx.xxxRunnable.run(xxxRunnable.java:70)
at java.lang.Thread.run(Thread.java:745)
2018/03/13 00:00:20 OSS INFO [com.xxx.RedisDAO] - error
java.lang.ClassCastException: [B cannot be cast to java.lang.Long
at redis.clients.jedis.Connection.getIntegerReply(Connection.java:265)
at redis.clients.jedis.Jedis.sadd(Jedis.java:1109)
at com.xxx.RedisDAO.setXXXXXKPI(RedisDAO.java:167)
at com.xxx.xxxRunnable.run(xxxRunnable.java:71)
at java.lang.Thread.run(Thread.java:745)
(1) 若当前redis中的数据很多,FLUSHDB 操作,需要花费一定的时间,阻塞了redis的响应,于是先出现了SocketTimeoutException;
(2) 连接出现异常后,从日志看,接着出现了ClassCastException,且后续一直报这个异常错误;
(3) 经搜索查询, 找到如下两个网页:
ClassCastException - [B cannot be cast to java.lang.Long #186:
https://github.com/xetorthio/jedis/issues/186
Jedis使用过程中踩过的那些坑:
http://bert82503.iteye.com/blog/2184225
综上, 查项目代码,发现几处问题:
a. redis连接操作出现异常后,应该关闭close, 归还资源,重新去一个redis连接使用。程序写的是:出了异常,记录日志,接着用这个redis连接,一直出异常。
b. 连接池设置过大,重启项目的时候,本次初始化的加上之前未释放的连接(直接kill的进程,关闭时未释放),瞬间链接过大;
c. 程序设计不合理, 由于数据每天是实时统计展示,可以每天的key前缀不一样,设置key的过期时间,这样就不用每天FLUSHDB。