Redis Connection reset by peer 异常的懒人解法

最近在使用 Lettuce 集成 Redis 的过程中遇到了一些问题，主要的表现是在测试环境使用正常，但是生产环境中 redis 会出现 Connection reset by peer 的异常。

基础环境 SpringBoot + Lettuce

Redis exception; nested exception is io.lettuce.core.RedisException

org.springframework.data.redis.RedisSystemException: Redis exception; nested exception is io.lettuce.core.RedisException: java.io.IOException: Connection reset by peer
        at org.springframework.data.redis.connection.lettuce.LettuceExceptionConverter.convert(LettuceExceptionConverter.java:74)
        at org.springframework.data.redis.connection.lettuce.LettuceExceptionConverter.convert(LettuceExceptionConverter.java:41)
        at org.springframework.data.redis.PassThroughExceptionTranslationStrategy.translate(PassThroughExceptionTranslationStrategy.java:44)
        at org.springframework.data.redis.FallbackExceptionTranslationStrategy.translate(FallbackExceptionTranslationStrategy.java:42)
        at org.springframework.data.redis.connection.lettuce.LettuceConnection.convertLettuceAccessException(LettuceConnection.java:277)
        at org.springframework.data.redis.connection.lettuce.LettuceConnection.await(LettuceConnection.java:1085)
        at org.springframework.data.redis.connection.lettuce.LettuceConnection.lambda$doInvoke$4(LettuceConnection.java:938)
        at org.springframework.data.redis.connection.lettuce.LettuceInvoker$Synchronizer.invoke(LettuceInvoker.java:673)
        at org.springframework.data.redis.connection.lettuce.LettuceInvoker$DefaultSingleInvocationSpec.get(LettuceInvoker.java:589)
        at org.springframework.data.redis.connection.lettuce.LettuceStringCommands.setEx(LettuceStringCommands.java:167)
        at org.springframework.data.redis.connection.DefaultedRedisConnection.setEx(DefaultedRedisConnection.java:335)
        at org.springframework.data.redis.core.DefaultValueOperations$8.potentiallyUsePsetEx(DefaultValueOperations.java:337)
        at org.springframework.data.redis.core.DefaultValueOperations$8.doInRedis(DefaultValueOperations.java:330)
        at org.springframework.data.redis.core.RedisTemplate.execute(RedisTemplate.java:224)
        at org.springframework.data.redis.core.RedisTemplate.execute(RedisTemplate.java:191)
        at org.springframework.data.redis.core.AbstractOperations.execute(AbstractOperations.java:97)
        at org.springframework.data.redis.core.DefaultValueOperations.set(DefaultValueOperations.java:325)

java.io.IOException: Connection reset by peer

Caused by: io.lettuce.core.RedisException: java.io.IOException: Connection reset by peer
        at io.lettuce.core.internal.Exceptions.bubble(Exceptions.java:83)
        at io.lettuce.core.internal.Futures.awaitOrCancel(Futures.java:250)
        at io.lettuce.core.LettuceFutures.awaitOrCancel(LettuceFutures.java:74)
        at org.springframework.data.redis.connection.lettuce.LettuceConnection.await(LettuceConnection.java:1083)
        ... 75 common frames omitted
Caused by: java.io.IOException: Connection reset by peer
        at sun.nio.ch.FileDispatcherImpl.read0(Native Method)
        at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:39)
        at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:223)
        at sun.nio.ch.IOUtil.read(IOUtil.java:192)
        at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:380)
        at io.netty.buffer.PooledByteBuf.setBytes(PooledByteBuf.java:256)
        at io.netty.buffer.AbstractByteBuf.writeBytes(AbstractByteBuf.java:1132)
        at io.netty.channel.socket.nio.NioSocketChannel.doReadBytes(NioSocketChannel.java:357)
        at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:151)
        at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:788)
        at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:724)
        at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:650)
        at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:562)
        at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997)
        at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
        at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
        ... 1 common frames omitted

last destination | Reconnected to

Reconnecting, last destination was /10.1.2.3:26381
Reconnected to 10.1.2.4:26381

如果redis服务本身没有问题，存在java.io.IOException: Connection reset by peer 、Reconnecting, last destination，而且是很有规律的出现，基本上可以断定是 redis 连接双方的 keepalive 机制被某种原因破坏导致的。

根本原因需要通过网络抓包来深入的分析，需要的一定的网络知识，对整理链路的请求转发路径需要比较清楚。

这里介绍一直懒人解法：

import org.springframework.beans.factory.InitializingBean;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.data.redis.connection.RedisConnectionFactory;
import org.springframework.data.redis.connection.lettuce.LettuceConnectionFactory;
import org.springframework.stereotype.Component;


@Component
public class InitializinConfig  implements InitializingBean {
    @Autowired
    private RedisConnectionFactory redisConnectionFactory;
    
    /**
     * @see org.springframework.data.redis.connection.lettuce.LettuceConnectionFactory.SharedConnection#getConnection()
     */
    @Override
    public void afterPropertiesSet() throws Exception {
        if(redisConnectionFactory instanceof LettuceConnectionFactory){
            LettuceConnectionFactory lettuceConnectionFactory = (LettuceConnectionFactory)redisConnectionFactory;
            lettuceConnectionFactory.setValidateConnection(true);
        }
    }

}

通过上面的代码可以每次获取连接前都对连接进行检查，如果连接不可用就重新创建连接，从而避免 Redis Connection reset by peer 的问题。

感兴趣的同学可以去 org.springframework.data.redis.connection.lettuce.LettuceConnectionFactory.SharedConnection#getConnection() 这里去翻翻官方的源代码。

希望这个能帮助你解决问题。