谷粒商城微服务分布式高级篇八——分布式缓存-Redis-Jedis-分布式锁

最新推荐文章于 2024-07-06 15:46:08 发布

hi163hello

最新推荐文章于 2024-07-06 15:46:08 发布

阅读量499

点赞数

分类专栏： Gulimall

原文链接：https://blog.csdn.net/D_A_I_H_A_O/article/details/106771416

版权

Gulimall 专栏收录该内容

20 篇文章 5 订阅

订阅专栏

文章目录

缓存使用
SpringBoot使用redis异常
Jedis
- 使用jedis

缓存使用

为了系统性能的提升,我们一般都会将部分数据放入缓存中,加速访问。而db承担数据落盘工作。

哪些数据适合放入缓存?

即时性、数据一致性要求不高的
访问量大且更新频率不高的数据(读多,写少)
举例:电商类应用,商品分类,商品列表等适合缓存并加一个失效时间(根据数据更新频率来定),后台如果发布一个商品,买家需要5分钟才能看到新的商品一般还是可以接受的。

本地缓存可以使用map来做缓存，分布式下则不行，则需要加入缓存中间件来统一缓存

这里我们使用redis，Java使用redis有很多方法。

SpringBoot使用redis异常

springBoot可以直接使用redis，但是可能会有异常

    @Override
    public Map<String, List<Catalog2Vo>> getCatalogJson() {
        String catalogJSON = stringRedisTemplate.opsForValue().get("catalogJSON");
        if(StringUtils.isBlank("catalogJSON") || catalogJSON == null){
            Map<String, List<Catalog2Vo>> catalogJsonFromDb = getCatalogJsonFromDb();
            String s = JSON.toJSONString(catalogJsonFromDb);
            stringRedisTemplate.opsForValue().set("catalogJSON",s,1,TimeUnit.SECONDS);
            return catalogJsonFromDb;
        }
        Map<String, List<Catalog2Vo>> map = JSON.parseObject(catalogJSON, new TypeReference<Map<String, List<Catalog2Vo>>>() {
        });
        return map;
    }

   
   
   
   1
2
3
4
5
6
7
8
9
10
11
12
13

堆外内存溢出：
在这里插入图片描述
原因：
springboot在2.0后默认使用lettuce操作redis客户端，它使用netty进行网络通信，lettuce的bug导致堆外内存溢出，底层实现是只要有操作，就会统计内存使用量，操作完会decrement减内存，可能是lettuce客户端在减内存的过程出错
在这里插入图片描述

解决：
对这个问题调节-Xmx大小不起作用，而且如果netty（netty如没有指定，默认是-Xmx300m）没指定堆外内存，默认使用-Xmx，所以一旦出现问题，就算调大-Xmx也总会到堆外内存溢出的地步。所以我们不能用io.netty.maxDirectMemory只修改堆外内存大小。
还有2种方案：

1、升级lettuce客户端
2、排除lettuce依赖，使用jedis

Jedis

jedis是什么
Jedis是Redis的Java实现的客户端，其API提供了比较全面的Redis命令的支持；

使用jedis

Jedis实例不是线程安全的,所以不可以多个线程共用一个Jedis实例，但是创建太多的实现也不好因为这意味着会建立很多sokcet连接。
JedisPool是一个线程安全的网络连接池。可以用JedisPool创建一些可靠Jedis实例，可以从池中获取Jedis实例，使用完后再把Jedis实例还回JedisPool。这种方式可以避免创建大量socket连接并且会实现高效的性能.

依赖

        <dependency>
            <groupId>redis.clients</groupId>
            <artifactId>jedis</artifactId>
            <version>2.9.3</version>
        </dependency>

   
   
   
   1
2
3
4
5

RedisUtil

public class RedisUtil {
<span class="token keyword">private</span>  JedisPool jedisPool<span class="token punctuation">;</span>

<span class="token keyword">public</span> <span class="token keyword">void</span> <span class="token function">initPool</span><span class="token punctuation">(</span>String host<span class="token punctuation">,</span><span class="token keyword">int</span> port <span class="token punctuation">,</span><span class="token keyword">int</span> database<span class="token punctuation">)</span><span class="token punctuation">{</span>
    JedisPoolConfig poolConfig <span class="token operator">=</span> <span class="token keyword">new</span> <span class="token class-name">JedisPoolConfig</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">;</span>
    poolConfig<span class="token punctuation">.</span><span class="token function">setMaxTotal</span><span class="token punctuation">(</span><span class="token number">200</span><span class="token punctuation">)</span><span class="token punctuation">;</span>
    poolConfig<span class="token punctuation">.</span><span class="token function">setMaxIdle</span><span class="token punctuation">(</span><span class="token number">30</span><span class="token punctuation">)</span><span class="token punctuation">;</span>
    poolConfig<span class="token punctuation">.</span><span class="token function">setBlockWhenExhausted</span><span class="token punctuation">(</span><span class="token boolean">true</span><span class="token punctuation">)</span><span class="token punctuation">;</span>
    poolConfig<span class="token punctuation">.</span><span class="token function">setMaxWaitMillis</span><span class="token punctuation">(</span><span class="token number">10</span><span class="token operator">*</span><span class="token number">1000</span><span class="token punctuation">)</span><span class="token punctuation">;</span>
    poolConfig<span class="token punctuation">.</span><span class="token function">setTestOnBorrow</span><span class="token punctuation">(</span><span class="token boolean">true</span><span class="token punctuation">)</span><span class="token punctuation">;</span>
    jedisPool<span class="token operator">=</span><span class="token keyword">new</span> <span class="token class-name">JedisPool</span><span class="token punctuation">(</span>poolConfig<span class="token punctuation">,</span>host<span class="token punctuation">,</span>port<span class="token punctuation">,</span><span class="token number">20</span><span class="token operator">*</span><span class="token number">1000</span><span class="token punctuation">)</span><span class="token punctuation">;</span>
<span class="token punctuation">}</span>

<span class="token keyword">public</span> Jedis <span class="token function">getJedis</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">{</span>
    Jedis jedis <span class="token operator">=</span> jedisPool<span class="token punctuation">.</span><span class="token function">getResource</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">;</span>
    <span class="token keyword">return</span> jedis<span class="token punctuation">;</span>
<span class="token punctuation">}</span>

}

RedisConfig

@Configuration
public class RedisConfig {
<span class="token comment">//读取配置文件中的redis的ip地址</span>
<span class="token annotation punctuation">@Value</span><span class="token punctuation">(</span><span class="token string">"${spring.redis.host:disabled}"</span><span class="token punctuation">)</span>
<span class="token keyword">private</span> String host<span class="token punctuation">;</span>

<span class="token annotation punctuation">@Value</span><span class="token punctuation">(</span><span class="token string">"${spring.redis.port:0}"</span><span class="token punctuation">)</span>
<span class="token keyword">private</span> <span class="token keyword">int</span> port<span class="token punctuation">;</span>

<span class="token annotation punctuation">@Value</span><span class="token punctuation">(</span><span class="token string">"${spring.redis.database:0}"</span><span class="token punctuation">)</span>
<span class="token keyword">private</span> <span class="token keyword">int</span> database<span class="token punctuation">;</span>

<span class="token annotation punctuation">@Bean</span>
<span class="token keyword">public</span> RedisUtil <span class="token function">getRedisUtil</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">{</span>
    <span class="token keyword">if</span><span class="token punctuation">(</span>host<span class="token punctuation">.</span><span class="token function">equals</span><span class="token punctuation">(</span><span class="token string">"disabled"</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">{</span>
        <span class="token keyword">return</span> null<span class="token punctuation">;</span>
    <span class="token punctuation">}</span>
    RedisUtil redisUtil<span class="token operator">=</span><span class="token keyword">new</span> <span class="token class-name">RedisUtil</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">;</span>
    redisUtil<span class="token punctuation">.</span><span class="token function">initPool</span><span class="token punctuation">(</span>host<span class="token punctuation">,</span>port<span class="token punctuation">,</span>database<span class="token punctuation">)</span><span class="token punctuation">;</span>
    <span class="token keyword">return</span> redisUtil<span class="token punctuation">;</span>
<span class="token punctuation">}</span>

}

application.properties

spring.redis.host=redis服务地址
spring.redis.port=6379
spring.redis.database=0

 
 
 
 1
2
3

使用测试

    @Autowired
    RedisUtil redisUtil;
<span class="token annotation punctuation">@Test</span>
<span class="token keyword">public</span> <span class="token keyword">void</span> <span class="token function">testRedis</span><span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token punctuation">{</span>

    Jedis jedis <span class="token operator">=</span> redisUtil<span class="token punctuation">.</span><span class="token function">getJedis</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">;</span>
    <span class="token comment">//jedis.set("test","test");</span>
    String s <span class="token operator">=</span> jedis<span class="token punctuation">.</span><span class="token function">get</span><span class="token punctuation">(</span><span class="token string">"test"</span><span class="token punctuation">)</span><span class="token punctuation">;</span>
    System<span class="token punctuation">.</span>out<span class="token punctuation">.</span><span class="token function">println</span><span class="token punctuation">(</span>s<span class="token punctuation">)</span><span class="token punctuation">;</span>
<span class="token punctuation">}</span>

业务使用

 /**
     * 查出所有分类 返回首页json
     */
    @Override
    public Map<String, List<Catalog2Vo>> getCatalogJson() {
        //加入缓存逻辑
        Jedis jedis = redisUtil.getJedis();
        String catalogJson = jedis.get("catalogJson");
    Map<span class="token operator">&lt;</span>String<span class="token punctuation">,</span> List<span class="token generics function"><span class="token punctuation">&lt;</span>Catalog2Vo<span class="token punctuation">&gt;</span></span><span class="token operator">&gt;</span> json <span class="token operator">=</span> null<span class="token punctuation">;</span>

    <span class="token comment">//缓存存在 转换返回</span>
    <span class="token keyword">if</span> <span class="token punctuation">(</span><span class="token operator">!</span>StringUtils<span class="token punctuation">.</span><span class="token function">isEmpty</span><span class="token punctuation">(</span>catalogJson<span class="token punctuation">)</span><span class="token punctuation">)</span> <span class="token punctuation">{</span>
        json <span class="token operator">=</span> JSONObject<span class="token punctuation">.</span><span class="token function">parseObject</span><span class="token punctuation">(</span>catalogJson<span class="token punctuation">,</span> <span class="token keyword">new</span> <span class="token class-name">TypeReference</span><span class="token operator">&lt;</span>Map<span class="token operator">&lt;</span>String<span class="token punctuation">,</span> List<span class="token generics function"><span class="token punctuation">&lt;</span>Catalog2Vo<span class="token punctuation">&gt;</span></span><span class="token operator">&gt;&gt;</span><span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token punctuation">{</span>
        <span class="token punctuation">}</span><span class="token punctuation">)</span><span class="token punctuation">;</span>
        <span class="token keyword">return</span> json<span class="token punctuation">;</span>
    <span class="token punctuation">}</span>
    
    <span class="token comment">//缓存没有从数据查询</span>
    json <span class="token operator">=</span> <span class="token function">getCatalogJsonFromDB</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">;</span>
    <span class="token comment">//转成str 加入缓存</span>
    String jsonString <span class="token operator">=</span> JSON<span class="token punctuation">.</span><span class="token function">toJSONString</span><span class="token punctuation">(</span>json<span class="token punctuation">)</span><span class="token punctuation">;</span>
    jedis<span class="token punctuation">.</span><span class="token function">set</span><span class="token punctuation">(</span><span class="token string">"catalogJson"</span><span class="token punctuation">,</span> jsonString<span class="token punctuation">)</span><span class="token punctuation">;</span>

    <span class="token keyword">return</span> json<span class="token punctuation">;</span>
<span class="token punctuation">}</span>

 /**
     * 从数据库获取数据并封装返回
     *
     * @return
     */
    public Map<String, List<Catalog2Vo>> getCatalogJsonFromDB() {
        //查询出所有分类
        List<CategoryEntity> selectList = baseMapper.selectList(null);
        //先查出所有一级分类
        List<CategoryEntity> level1Categorys = getCategorys(selectList, 0L);
    <span class="token comment">//封装数据 map k,v 结构</span>
    Map<span class="token operator">&lt;</span>String<span class="token punctuation">,</span> List<span class="token generics function"><span class="token punctuation">&lt;</span>Catalog2Vo<span class="token punctuation">&gt;</span></span><span class="token operator">&gt;</span> map <span class="token operator">=</span> level1Categorys<span class="token punctuation">.</span><span class="token function">stream</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">.</span><span class="token function">collect</span><span class="token punctuation">(</span>Collectors<span class="token punctuation">.</span><span class="token function">toMap</span><span class="token punctuation">(</span>k <span class="token operator">-</span><span class="token operator">&gt;</span> k<span class="token punctuation">.</span><span class="token function">getCatId</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">.</span><span class="token function">toString</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">,</span> v <span class="token operator">-</span><span class="token operator">&gt;</span> <span class="token punctuation">{</span>
        <span class="token comment">//每一个的一级分类，查到这个一级分类的二级分类</span>
        List<span class="token generics function"><span class="token punctuation">&lt;</span>CategoryEntity<span class="token punctuation">&gt;</span></span> category2Entities <span class="token operator">=</span> <span class="token function">getCategorys</span><span class="token punctuation">(</span>selectList<span class="token punctuation">,</span> v<span class="token punctuation">.</span><span class="token function">getCatId</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">;</span>
        List<span class="token generics function"><span class="token punctuation">&lt;</span>Catalog2Vo<span class="token punctuation">&gt;</span></span> catelog2Vos <span class="token operator">=</span> null<span class="token punctuation">;</span>

        <span class="token keyword">if</span> <span class="token punctuation">(</span>category2Entities <span class="token operator">!=</span> null<span class="token punctuation">)</span> <span class="token punctuation">{</span>
            catelog2Vos <span class="token operator">=</span> category2Entities<span class="token punctuation">.</span><span class="token function">stream</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">.</span><span class="token function">map</span><span class="token punctuation">(</span>level2 <span class="token operator">-</span><span class="token operator">&gt;</span> <span class="token punctuation">{</span>
                <span class="token comment">//封装catalog2Vo</span>
                Catalog2Vo catalog2Vo <span class="token operator">=</span> <span class="token keyword">new</span> <span class="token class-name">Catalog2Vo</span><span class="token punctuation">(</span>v<span class="token punctuation">.</span><span class="token function">getCatId</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">.</span><span class="token function">toString</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">,</span> null<span class="token punctuation">,</span> level2<span class="token punctuation">.</span><span class="token function">getCatId</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">.</span><span class="token function">toString</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">,</span> level2<span class="token punctuation">.</span><span class="token function">getName</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">;</span>
                <span class="token comment">//每一个二级分类，查到三级分类</span>
                List<span class="token generics function"><span class="token punctuation">&lt;</span>CategoryEntity<span class="token punctuation">&gt;</span></span> category3Entities <span class="token operator">=</span> <span class="token function">getCategorys</span><span class="token punctuation">(</span>selectList<span class="token punctuation">,</span> level2<span class="token punctuation">.</span><span class="token function">getCatId</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">;</span>
                <span class="token keyword">if</span> <span class="token punctuation">(</span>category3Entities <span class="token operator">!=</span> null<span class="token punctuation">)</span> <span class="token punctuation">{</span>
                    List<span class="token generics function"><span class="token punctuation">&lt;</span>Object<span class="token punctuation">&gt;</span></span> catalog3List <span class="token operator">=</span> category3Entities<span class="token punctuation">.</span><span class="token function">stream</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">.</span><span class="token function">map</span><span class="token punctuation">(</span>level3 <span class="token operator">-</span><span class="token operator">&gt;</span> <span class="token punctuation">{</span>
                        <span class="token comment">//封装catalog3Vo</span>
                        Catalog2Vo<span class="token punctuation">.</span>Catalog3Vo catalog3Vo <span class="token operator">=</span> <span class="token keyword">new</span> <span class="token class-name">Catalog2Vo<span class="token punctuation">.</span>Catalog3Vo</span><span class="token punctuation">(</span>level2<span class="token punctuation">.</span><span class="token function">getCatId</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">.</span><span class="token function">toString</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">,</span> level3<span class="token punctuation">.</span><span class="token function">getCatId</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">.</span><span class="token function">toString</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">,</span> level3<span class="token punctuation">.</span><span class="token function">getName</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">;</span>
                        <span class="token keyword">return</span> catalog3Vo<span class="token punctuation">;</span>
                    <span class="token punctuation">}</span><span class="token punctuation">)</span><span class="token punctuation">.</span><span class="token function">collect</span><span class="token punctuation">(</span>Collectors<span class="token punctuation">.</span><span class="token function">toList</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">;</span>
                    <span class="token comment">//封装catalog3Vo到catalog2Vo</span>
                    catalog2Vo<span class="token punctuation">.</span><span class="token function">setCatalog3List</span><span class="token punctuation">(</span>catalog3List<span class="token punctuation">)</span><span class="token punctuation">;</span>
                <span class="token punctuation">}</span>
                <span class="token keyword">return</span> catalog2Vo<span class="token punctuation">;</span>
            <span class="token punctuation">}</span><span class="token punctuation">)</span><span class="token punctuation">.</span><span class="token function">collect</span><span class="token punctuation">(</span>Collectors<span class="token punctuation">.</span><span class="token function">toList</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">;</span>
        <span class="token punctuation">}</span>
        <span class="token comment">//返回v=catalog2Vo</span>
        <span class="token keyword">return</span> catelog2Vos<span class="token punctuation">;</span>
    <span class="token punctuation">}</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">;</span>


    <span class="token keyword">return</span> map<span class="token punctuation">;</span>
<span class="token punctuation">}</span>

测试ok
在这里插入图片描述
但是这样在高并发下缓存是会存在很多问题的，详情参见分布式-Redis-缓存问题

lua脚本：

String script = "if redis.call('get', KEYS[1]) == ARGV[1] then return redis.call('del', KEYS[1]) else return 0 end";

 
 
 
 1

就是加锁设置过期时间保证加锁操作是原子性的，解锁也是同样保持原子性。（原子性简单理解就是1和2两个操作都是要同时ok这整个任务才能算ok）

加锁获取数据

 public Map<String, List<Catalog2Vo>> getCatalogJsonFromDBWithRedisLock() {
        Jedis jedis = redisUtil.getJedis();
        //加锁
        String token = UUID.randomUUID().toString();
        String lock = jedis.set("lock", token, "NX", "EX", 20);
        System.out.println(lock);
    Map<span class="token operator">&lt;</span>String<span class="token punctuation">,</span> List<span class="token generics function"><span class="token punctuation">&lt;</span>Catalog2Vo<span class="token punctuation">&gt;</span></span><span class="token operator">&gt;</span> map <span class="token operator">=</span> null<span class="token punctuation">;</span>
    <span class="token comment">//加锁成功</span>
    <span class="token keyword">if</span> <span class="token punctuation">(</span><span class="token string">"ok"</span><span class="token punctuation">.</span><span class="token function">equals</span><span class="token punctuation">(</span>lock<span class="token punctuation">)</span><span class="token punctuation">)</span> <span class="token punctuation">{</span>
        map <span class="token operator">=</span> <span class="token function">getCatalogJsonFromDB</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">;</span>
        <span class="token comment">//删除锁 lua脚本</span>
        String script <span class="token operator">=</span> <span class="token string">"if redis.call('get', KEYS[1]) == ARGV[1] then return redis.call('del', KEYS[1]) else return 0 end"</span><span class="token punctuation">;</span>
        jedis<span class="token punctuation">.</span><span class="token function">eval</span><span class="token punctuation">(</span>script<span class="token punctuation">,</span> Collections<span class="token punctuation">.</span><span class="token function">singletonList</span><span class="token punctuation">(</span><span class="token string">"lock"</span><span class="token punctuation">)</span><span class="token punctuation">,</span> Collections<span class="token punctuation">.</span><span class="token function">singletonList</span><span class="token punctuation">(</span>token<span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">;</span>
        <span class="token keyword">return</span> map<span class="token punctuation">;</span>
    <span class="token punctuation">}</span> <span class="token keyword">else</span> <span class="token punctuation">{</span>
        <span class="token keyword">try</span> <span class="token punctuation">{</span>
            Thread<span class="token punctuation">.</span><span class="token function">sleep</span><span class="token punctuation">(</span><span class="token number">300</span><span class="token punctuation">)</span><span class="token punctuation">;</span>
        <span class="token punctuation">}</span> <span class="token keyword">catch</span> <span class="token punctuation">(</span><span class="token class-name">InterruptedException</span> e<span class="token punctuation">)</span> <span class="token punctuation">{</span>
            e<span class="token punctuation">.</span><span class="token function">printStackTrace</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">;</span>
        <span class="token punctuation">}</span>
        <span class="token comment">//自旋</span>
        <span class="token keyword">return</span> <span class="token function">getCatalogJsonFromDBWithRedisLock</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">;</span>
    <span class="token punctuation">}</span>

<span class="token punctuation">}</span>

Redis总结

                                </div><div data-report-view="{&quot;mod&quot;:&quot;1585297308_001&quot;,&quot;dest&quot;:&quot;https://blog.csdn.net/D_A_I_H_A_O/article/details/106771416&quot;,&quot;extend1&quot;:&quot;pc&quot;,&quot;ab&quot;:&quot;new&quot;}"><div></div></div>
            <link href="https://csdnimg.cn/release/phoenix/mdeditor/markdown_views-60ecaf1f42.css" rel="stylesheet">
                            </div>

hi163hello

关注

0
点赞
踩
1

收藏

觉得还不错? 一键收藏
0
评论
谷粒商城微服务分布式高级篇八——分布式缓存-Redis-Jedis-分布式锁

文章目录缓存使用SpringBoot使用redis异常Jedis使用jedis缓存使用为了系统性能的提升,我们一般都会将部分数据放入缓存中,加速访问。而db承...
复制链接

扫一扫