多级缓存
应用场景:我们知道redis的tps读写能力在10w/s左右,在大促或者双11场景,很多商品的访问高达百万千万级别,如果只使用redis缓存,是不能满足业务需要。
缓存混合存在问题
基于以上场景,我们需要使用多级缓存实现,利用本地缓存与redis缓存来实现:
- 本地缓存 ,使用ehcache来实现,ehcache作为JVM级别的缓存,不能够保证分布式集群部署一致性,无法实现分布式场景下缓存共享;
- 本地缓存和分布式redis缓存如何混合使用;
方式1:sboot代码实现
(1) 配置文件配置, ehcache.xml网上有很多配置,可以根据实际需要配置
#encache 本地缓存
spring.cache.type=ehcache
spring.cache.ehcache.config=classpath:ehcache.xml
(2) Springboot开启Config配置
/
**
* @author libiao
* 开启本地缓存EnableCaching扫描spring.cache.ehcache.config
*/
@Configuration
@EnableCaching
public class CacheConfig {
@Resource
private CacheManager cacheManager;
/**
* ehcache缓存处理
* @return
* @throws Exception
*/
@Bean("ehcache")
public Cache initEhcache()throws Exception{
return cacheManager.getCache("userCache");
}
}
(3) 业务代码
- 查上商品信息
//1、 从本地缓存获取
Product product = ehcache.get(Constants.CACHE_PRODUCT_PREFIX + productId, Product.class);
if (product != null){
return product;
}
//2、从redis缓存获取
product = redis.get(Constants.CACHE_PRODUCT_PREFIX + productId, Product.class);
if (product != null){
return product;
}
//3、从db查数据,并放入redis和本地缓存
- 添加或修改商品信息zk
//1、淘汰商品redis缓存,此处省略
//2、商品的zk路径监控,如下所示;
String zkMonitorProductPath = Constants.getZkMonitorProductPath(productId);
if (zooKeeper.exists(zkMonitorProductPath,true) == null){
//路径不存在,则创建路径,状态为true
zooKeeper.create(zkMonitorProductPath, "true".getBytes(), ZooDefs.Ids.OPEN_ACL_UNSAFE, CreateMode.PERSISTENT);
}
//监听zk节点某个商品状态
zooKeeper.exists(zkMonitorProductPath, true);
方式2:guava实现
(1)商品查询
private static final String PRODUCT_PREFIX = "PRODUCT:";
private final Cache<String, Product> cache = CacheBuilder.newBuilder()
.initialCapacity(1000)
.maximumSize(10000)
.expireAfterWrite(5, TimeUnit.HOURS).build();
private final ReentrantLock updateLock = new ReentrantLock();
public Product queryProductById(Long productId){
String key = PRODUCT_PREFIX + productId;
Product cachePrd = cache.getIfPresent(key);
if (null == cachePrd){
updateLock.lock();
try{
cachePrd = cache.getIfPresent(key);
if (null == cachePrd){
//从db查数据,并放入redis和本地缓存
cachePrd = findProductInDB(productId);
cache.put(key, cachePrd);
log.info("商品写入本地缓存,cachePrd:{}", cachePrd);
}
}finally {
updateLock.unlock();
}
}
return cachePrd;
}
(2) 添加或修改商品信息zk
//1、淘汰商品redis缓存,此处省略
//2、商品的zk路径监控,如下所示;
String zkMonitorProductPath = Constants.getZkMonitorProductPath(productId);
if (zooKeeper.exists(zkMonitorProductPath,true) == null){
//路径不存在,则创建路径,状态为true
zooKeeper.create(zkMonitorProductPath, "true".getBytes(), ZooDefs.Ids.OPEN_ACL_UNSAFE, CreateMode.PERSISTENT);
}
//监听zk节点某个商品状态
Stat stat =zooKeeper.exists(zkMonitorProductPath, true);
zookeeper.setData(zkMonotorProductPath,"false".getBytes(),stat.getVersion());
本地缓存一致性保证
本地缓存使用zookeeper保证,针对当前商品添加zk的path,如果商品信息发生变更通过zk的watch机制进行淘汰本地缓存
@Bean
public ZooKeeper initZookeeper()throws Exception{
ZookeeperWatcher zkWatcher = new ZookeeperWatcher();
ZooKeeper zooKeeper = new ZooKeeper(zookeeperAddress, 30000, zkWatcher);
zkWatcher.setZooKeeper(zooKeeper);
zkWatcher.setCache(cache); //见上cache配置
return zooKeeper;
}
/**zk淘汰本地缓存*/
public class ZookeeperWatcher implements Watcher {
private ZooKeeper zooKeeper;
private Cache cache;
public void setZooKeeper(ZooKeeper zooKeeper, Cache cache){
this.zooKeeper = zooKeeper;
this.cache = cache;
}
@Override
public void process(WatchedEvent event) {
if (event.getType() == Event.EventType.None && event.getPath() == null){
log.info("zookeeper connected success!");
//创建zk的商品标记根节点
try{
//App启动时候创建标记root节点ZK_PRODUCT_MONITOR_FLAG
if (zooKeeper.exists(Constants.ZK_PRODUCT_MONITOR_FLAG, false) == null){
//创建根节点,zk的标记为无数据
zooKeeper.create(Constants.ZK_PRODUCT_MONITOR_FLAG, "".getBytes(), ZooDefs.Ids.OPEN_ACL_UNSAFE, CreateMode.PERSISTENT);
}
}catch (Exception e){
log.error("商品标记失败", e);
}
}else if (event.getType() == Event.EventType.NodeDataChanged){
//zk目录节点数据变化通知事件
try{
String path = event.getPath();
if (path.startsWith(Constants.ZK_PRODUCT_MONITOR_FLAG)) {
String monitorFlag = new String(zooKeeper.getData(path, true, new Stat()));
log.info("zookeeper 数据节点修改变动,path:{},value:{}", path, monitorFlag );
if (Constants.ZK_FALSE.equals(monitorFlag )) {
String productId = path.substring(path.lastIndexOf("/") + 1);
cache.evict(Constants.ZK_PRODUCT_MONITOR_FLAG + productId);
log.info("清除商品{}本地内存", productId);
}
}
}catch (Exception e){
log.error("zookeeper数据节点修改回调事件异常", e);
}
}
}
}