在使用springdata操作缓存中,当访问量比较大时,有可能返回null导致数据不准确,发生几率在0.01%或以下,虽然已经低于压测标准,但是还是会影响部分用户,经过一番筛查,发现原因如下:
RedisCache 类中 有get方法,存在明显的逻辑错误 “先判断是否存在,再去get”,代码执行过程中总有时间差,如果这个时间过期,则 判定为存在,又取不到数据,所以发生了 本文所描述的情况
/**
* Return the value to which this cache maps the specified key.
*
* @param cacheKey the key whose associated value is to be returned via its binary representation.
* @return the {@link RedisCacheElement} stored at given key or {@literal null} if no value found for key.
* @since 1.5
*/
public RedisCacheElement get(final RedisCacheKey cacheKey) {
Assert.notNull(cacheKey, "CacheKey must not be null!");
Boolean exists = (Boolean) redisOperations.execute(new RedisCallback<Boolean>() {
@Override
public Boolean doInRedis(RedisConnection connection) throws DataAccessException {
return connection.exists(cacheKey.getKeyBytes());
}
});
if (!exists.booleanValue()) {
return null;
}
return new RedisCacheElement(cacheKey, fromStoreValue(lookup(cacheKey)));
}
改进方法如下(网上很多写法也有bug,所以自己稍微做了一点改动):
redis缓存类:
package com.jinhuhang.risk.plugins.redis;
import org.springframework.dao.DataAccessException;
import org.springframework.data.redis.cache.RedisCache;
import org.springframework.data.redis.cache.RedisCacheElement;
import org.springframework.data.redis.cache.RedisCacheKey;
import org.springframework.data.redis.connection.RedisConnection;
import org.springframework.data.redis.core.RedisCallback;
import org.springframework.data.redis.core.RedisOperations;
import org.springframework.util.Assert;
/**
* 自定义的redis缓存
*
* @author yuhao.wang
*/
public class CustomizedRedisCache extends RedisCache {
private final RedisOperations redisOperations;
private final byte[] prefix;
public CustomizedRedisCache(String name, byte[] prefix, RedisOperations<? extends Object, ? extends Object> redisOperations, long expiration) {
super(name, prefix, redisOperations, expiration);
this.redisOperations = redisOperations;
this.prefix = prefix;
}
public CustomizedRedisCache(String name, byte[] prefix, RedisOperations<? extends Object, ? extends Object> redisOperations, long expiration, boolean allowNullValues) {
super(name, prefix, redisOperations, expiration, allowNullValues);
this.redisOperations = redisOperations;
this.prefix = prefix;
}
/**
* 重写父类的get函数。
* 父类的get方法,是先使用exists判断key是否存在,不存在返回null,存在再到redis缓存中去取值。这样会导致并发问题,
* 假如有一个请求调用了exists函数判断key存在,但是在下一时刻这个缓存过期了,或者被删掉了。
* 这时候再去缓存中获取值的时候返回的就是null了。
* 可以先获取缓存的值,再去判断key是否存在。
*
* @param cacheKey
* @return
*/
@Override
public RedisCacheElement get(final RedisCacheKey cacheKey) {
Assert.notNull(cacheKey, "CacheKey must not be null!");
RedisCacheElement redisCacheElement = new RedisCacheElement(cacheKey, fromStoreValue(lookup(cacheKey)));
if(redisCacheElement.get()==null)//如果取出来的值为空 ,则直接返回null
return null;
Boolean exists = (Boolean) redisOperations.execute(new RedisCallback<Boolean>() {
@Override
public Boolean doInRedis(RedisConnection connection) throws DataAccessException {
return connection.exists(cacheKey.getKeyBytes());
}
});
if (!exists.booleanValue()) {
return null;
}
return redisCacheElement;
}
/**
* 获取RedisCacheKey
*
* @param key
* @return
*/
private RedisCacheKey getRedisCacheKey(Object key) {
return new RedisCacheKey(key).usePrefix(this.prefix)
.withKeySerializer(redisOperations.getKeySerializer());
}
}
cacheManager:
package com.jinhuhang.risk.plugins.redis;
import java.util.Collection;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.cache.Cache;
import org.springframework.data.redis.cache.RedisCacheManager;
import org.springframework.data.redis.core.RedisOperations;
/**
* 自定义的redis缓存管理器
* @author yuhao.wang
*/
public class CustomizedRedisCacheManager extends RedisCacheManager {
private static final Logger logger = LoggerFactory.getLogger(CustomizedRedisCacheManager.class);
public CustomizedRedisCacheManager(RedisOperations redisOperations) {
super(redisOperations);
}
public CustomizedRedisCacheManager(RedisOperations redisOperations, Collection<String> cacheNames) {
super(redisOperations, cacheNames);
}
@Override
protected Cache getMissingCache(String name) {
long expiration = computeExpiration(name);
return new CustomizedRedisCache(
name,
(this.isUsePrefix() ? this.getCachePrefix().prefix(name) : null),
this.getRedisOperations(),
expiration);
}
}
配置类:
package com.jinhuhang.risk.plugins;
import com.fasterxml.jackson.annotation.JsonAutoDetect;
import com.fasterxml.jackson.annotation.PropertyAccessor;
import com.fasterxml.jackson.databind.ObjectMapper;
import com.jinhuhang.risk.plugins.redis.CustomizedRedisCacheManager;
import com.jinhuhang.risk.util.JedisUtil;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.cache.CacheManager;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.data.redis.cache.RedisCacheManager;
import org.springframework.data.redis.connection.RedisConnectionFactory;
import org.springframework.data.redis.core.RedisOperations;
import org.springframework.data.redis.core.RedisTemplate;
import org.springframework.data.redis.repository.query.RedisOperationChain;
import org.springframework.data.redis.serializer.Jackson2JsonRedisSerializer;
import org.springframework.data.redis.serializer.StringRedisSerializer;
import redis.clients.jedis.HostAndPort;
import redis.clients.jedis.JedisCluster;
import redis.clients.jedis.JedisPoolConfig;
import java.net.UnknownHostException;
import java.util.HashSet;
import java.util.List;
import java.util.Set;
@Configuration
public class JedisConfiguration {
@Autowired
private JedisProperties jedisProperties;
@Bean
public JedisCluster jedisCluster() {
List<String> nodes = jedisProperties.getCluster().getNodes();
Set<HostAndPort> hps = new HashSet<>();
for (String node : nodes) {
String[] hostPort = node.split(":");
hps.add(new HostAndPort(hostPort[0].trim(), Integer.valueOf(hostPort[1].trim())));
}
JedisPoolConfig poolConfig = new JedisPoolConfig();
poolConfig.setMaxIdle(jedisProperties.getPool().getMaxIdle());
poolConfig.setMinIdle(jedisProperties.getPool().getMinIdle());
poolConfig.setMaxWaitMillis(jedisProperties.getPool().getMaxWait());
poolConfig.setMaxTotal(jedisProperties.getMaxTotal());
JedisCluster jedisCluster1;
if (1 == jedisProperties.getIsAuth()) {
jedisCluster1 = new JedisCluster(
hps,
jedisProperties.getTimeout(),
jedisProperties.getSoTimeout(),
jedisProperties.getMaxAttempts(),
jedisProperties.getPassword(),
poolConfig);
} else {
jedisCluster1 = new JedisCluster(
hps,
jedisProperties.getTimeout(),
jedisProperties.getSoTimeout(),
poolConfig);
}
JedisUtil.setJedisCluster(jedisCluster1);
return jedisCluster1;
}
/**
* 设置数据存入redis 的序列化方式
*</br>redisTemplate序列化默认使用的jdkSerializeable,存储二进制字节码,导致key会出现乱码,所以自定义
*序列化类
*
* @paramredisConnectionFactory
*/
@Bean
public RedisTemplate<Object,Object> redisTemplate(RedisConnectionFactory redisConnectionFactory)throws UnknownHostException {
RedisTemplate<Object,Object> redisTemplate = new RedisTemplate<>();
redisTemplate.setConnectionFactory(redisConnectionFactory);
Jackson2JsonRedisSerializer jackson2JsonRedisSerializer =new Jackson2JsonRedisSerializer(Object.class);
ObjectMapper objectMapper =new ObjectMapper();
objectMapper.setVisibility(PropertyAccessor.ALL,JsonAutoDetect.Visibility.ANY);
objectMapper.enableDefaultTyping(ObjectMapper.DefaultTyping.NON_FINAL);
jackson2JsonRedisSerializer.setObjectMapper(objectMapper);
redisTemplate.setValueSerializer(jackson2JsonRedisSerializer);
redisTemplate.setKeySerializer(new StringRedisSerializer());
redisTemplate.afterPropertiesSet();
return redisTemplate;
}
@Bean
public CacheManager cacheManager(RedisTemplate<Object, Object> redisTemplate) {
RedisCacheManager rcm = new CustomizedRedisCacheManager(redisTemplate);
// 设置缓存过期时间,单位:秒
rcm.setDefaultExpiration(60L);
return rcm;
}
}
嗯,完美解决,性能稍微下降了一点点,不过对业务系统来说稳定性最重要