记录生产环境SpringBoot Redis哨兵模式：JedisConnectionException: Could not get a resource from the pool

最新推荐文章于 2024-09-12 09:46:55 发布

丁乾文

最新推荐文章于 2024-09-12 09:46:55 发布

阅读量697

点赞数 1

分类专栏： spring boot 文章标签： redis spring java

本文链接：https://blog.csdn.net/weixin_42756198/article/details/119796861

版权

spring boot 专栏收录该内容

4 篇文章 0 订阅

订阅专栏

问题由来

应工作需要，进行修改一个老项目，修改完后进行投产，发现登陆时无法获取到Redis连接，无法进行登陆，然后项目进行了回滚。

错误提示：

2021-08-18 21:44:08.130 ERROR 24558 [http-nio-8089-exec-9] --- .a.c.l.AbstractPhoneClientRequestHandler : Cannot get Jedis connection; nested exception is redis.clients.jedis.exceptions.JedisConnectionException: Could not get a resource from the pool
org.springframework.data.redis.RedisConnectionFailureException: Cannot get Jedis connection; nested exception is redis.clients.jedis.exceptions.JedisConnectionException: Could not get a resource from the pool
	at org.springframework.data.redis.connection.jedis.JedisConnectionFactory.fetchJedisConnector(JedisConnectionFactory.java:204)
	at org.springframework.data.redis.connection.jedis.JedisConnectionFactory.getConnection(JedisConnectionFactory.java:348)
	at org.springframework.data.redis.core.RedisConnectionUtils.doGetConnection(RedisConnectionUtils.java:129)
	at org.springframework.data.redis.core.RedisConnectionUtils.getConnection(RedisConnectionUtils.java:92)
	at org.springframework.data.redis.core.RedisConnectionUtils.getConnection(RedisConnectionUtils.java:79)
	at org.springframework.data.redis.core.RedisTemplate.execute(RedisTemplate.java:194)
	at org.springframework.data.redis.core.RedisTemplate.execute(RedisTemplate.java:169)
	at org.springframework.data.redis.core.RedisTemplate.execute(RedisTemplate.java:157)
Caused by: redis.clients.jedis.exceptions.JedisConnectionException: Could not get a resource from the pool
	at redis.clients.util.Pool.getResource(Pool.java:53)
	at redis.clients.jedis.JedisSentinelPool.getResource(JedisSentinelPool.java:209)
	at redis.clients.jedis.JedisSentinelPool.getResource(JedisSentinelPool.java:17)
	at org.springframework.data.redis.connection.jedis.JedisConnectionFactory.fetchJedisConnector(JedisConnectionFactory.java:194)
Caused by: redis.clients.jedis.exceptions.JedisConnectionException: java.net.SocketTimeoutException: connect timed out
	at redis.clients.jedis.Connection.connect(Connection.java:207)
	at redis.clients.jedis.BinaryClient.connect(BinaryClient.java:93)
	at redis.clients.jedis.BinaryJedis.connect(BinaryJedis.java:1767)
	at redis.clients.jedis.JedisFactory.makeObject(JedisFactory.java:106)
	at redis.clients.util.Pool.getResource(Pool.java:49)
	at redis.clients.jedis.Connection.connect(Connection.java:184)

问题排查

初步怀疑代码可能存在问题，然后进行了排查，发现使用Redis的地方全局只有登录的时候一块代码用到（这次修改触发了这一块的业务执行导致暴露出问题），经过多次测试，发现本地环境以及Test环境均正常。
本地debug 在JedisConnectionFactory中fetchJedisConnector方法进行问题追踪，首先获取链接池中的连接数据进行查看，发现如下数据：
在这里插入图片描述

我配置的哨兵节点为：192.168.16.14:26379，192.168.16.11:26379，192.168.16.13:26379，debug发现当前选举的Master为192.168.16.13:6379，然后进一步排查发现：
在这里插入图片描述
通过哨兵配置的地址进行Socket连接获取到的Master确实为192.168.16.13:6379，然后去翻生产日志，试图找到以下日志内容，找到生产Redis初始化哨兵的日志

日志文件如下：

2021-08-18 21:43:22.817  INFO 24558 [main] --- redis.clients.jedis.JedisSentinelPool    : Trying to find master from available Sentinels...
2021-08-18 21:43:22.850  INFO 24558 [main] --- redis.clients.jedis.JedisSentinelPool    : Redis master running at 10.*.*.26:6379, starting Sentinel listeners...
2021-08-18 21:43:22.866  INFO 24558 [main] --- redis.clients.jedis.JedisSentinelPool    : Created JedisPool to master at 10.*.*.26:6379

则去找运维验证10...26:6379相关Ip以及端口访问权限确实没有打开，完美破案！