redis7(暂缺源码分析和集群相关)

7.0 新特性

image.png
多 aof 文件

安装与配置

image.png
下载 7.0 版本redis
gcc 环境、

yum -y install gcc-c++

建议安装到 6.0.8 及以上的版本
解压编译安装

make & make install

image.png
配置文件修改

mkdir /myredis
cp redis.conf /myredis/
vim /myredis/redis7.conf            

redis.conf配置文件修改

  1. daemonize no ->yes 后台启动
  2. protected-mode yes -> no 同下一条伊奥,配置以用来允许其他机器远程连接
  3. bind 127.0.0.1 注释
  4. requirepass 你自己设置的密码
redis-server /myredis/redis.conf
redis-cli -a 123456  -p 6379

image.png
去除这里的警告
image.png
helloworld 案例
image.png
如何关闭
redis 卸载
image.png
image.png

十大数据类型

image.png
学习
redis 操作手册
英文
Commands
中文
Redis命令中心(Redis commands) – Redis中国用户组(CRUG)
image.png
image.png

学习方法

举出一个数据结构的应用场景(理解数据结构特点),并操作(练习api

string

image.png

image.png
image.png
image.png
[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-L2Qqgs2Y-1685671838136)(https://cdn.nlark.com/yuque/0/2023/png/2166858/1680766224683-71481e2a-87ef-43ad-878b-672361e2fd78.png#averageHue=%23f0f0f0&clientId=u1c49e639-191b-4&from=paste&height=47&id=u554a3e60&originHeight=71&originWidth=674&originalType=binary&ratio=1.5&rotation=0&showTitle=false&size=24256&status=done&style=none&taskId=u0c33f695-41b5-4574-bbc7-09073362dc1&title=&width=449.3333333333333)]
image.png
[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-szCiZHWt-1685671838136)(https://cdn.nlark.com/yuque/0/2023/png/2166858/1680765387549-a9853a9c-88c8-43fd-b088-61746359e2e9.png#averageHue=%23f8f8f8&clientId=u1c49e639-191b-4&from=paste&height=463&id=u9cadd707&originHeight=695&originWidth=1361&originalType=binary&ratio=1.5&rotation=0&showTitle=false&size=92905&status=done&style=none&taskId=udae66ecf-e8b4-41d5-ad81-651be725be6&title=&width=907.3333333333334)]
节点在操作锁资源的时候,setnx lock uuid,操作完毕 del lock
image.png

list

[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-1wvYBZgo-1685671838142)(https://cdn.nlark.com/yuque/0/2023/png/2166858/1680768292675-a3d00b1f-2409-451b-b2c0-93eba0014b9b.png#averageHue=%23f6f6f6&clientId=ufc764910-829e-4&from=paste&height=192&id=u445fd5ca&originHeight=288&originWidth=964&originalType=binary&ratio=1.5&rotation=0&showTitle=false&size=72856&status=done&style=none&taskId=u78473453-90b8-4cb3-a181-af665a8d220&title=&width=642.6666666666666)]
[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-WhUKHcyH-1685671838142)(https://cdn.nlark.com/yuque/0/2023/png/2166858/1680768292831-250779e4-6e6d-4ceb-976b-5a4172e99638.png#averageHue=%23faf9f8&clientId=ufc764910-829e-4&from=paste&height=383&id=uc6371aff&originHeight=575&originWidth=1514&originalType=binary&ratio=1.5&rotation=0&showTitle=false&size=265484&status=done&style=none&taskId=ua6827e8a-ae1c-4869-904d-28eabfacb05&title=&width=1009.3333333333334)]
[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-S4rIOdKa-1685671838143)(https://cdn.nlark.com/yuque/0/2023/png/2166858/1680768514053-c881d8c1-4aa7-46ea-aae7-9cebad247eaa.png#averageHue=%23f3f2f2&clientId=ufc764910-829e-4&from=paste&height=465&id=uedfd61fa&originHeight=697&originWidth=1668&originalType=binary&ratio=1.5&rotation=0&showTitle=false&size=325227&status=done&style=none&taskId=u9e044eda-40e0-4712-b0b4-13190d1bd7c&title=&width=1112)]
[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-VXeHJ3K2-1685671838143)(https://cdn.nlark.com/yuque/0/2023/png/2166858/1680768603720-acd3013f-e347-4507-8890-c967ebbea65b.png#averageHue=%23f9f8f8&clientId=ufc764910-829e-4&from=paste&height=497&id=u565847f4&originHeight=745&originWidth=1191&originalType=binary&ratio=1.5&rotation=0&showTitle=false&size=205322&status=done&style=none&taskId=uffeeb378-cf92-44aa-b55c-4e720f4aead&title=&width=794)]
应用场景
image.png
队列,先进先出,很方便的做分页查询

hash

[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-2LGIxqFJ-1685671838143)(https://cdn.nlark.com/yuque/0/2023/png/2166858/1680769869065-b666bfdb-ab92-4650-a345-50f890bd2b99.png#averageHue=%23f9f5f4&clientId=ufc764910-829e-4&from=paste&height=469&id=ua9546bbc&originHeight=704&originWidth=1446&originalType=binary&ratio=1.5&rotation=0&showTitle=false&size=211429&status=done&style=none&taskId=u08ebfaab-b380-4580-bb0a-a72b361b9da&title=&width=964)]
应用场景:购物车
image.png
天猫Java研发三面:讲讲Redis实现购物车的设计思路! - 腾讯云开发者社区-腾讯云
image.png

# 增加商品
# 用户id: uid001 分别添加 商品id 10025 10065 的两件商品
hset cart:uid001 10025 1
hset cart:uid001 10065 1
# 全选功能
HGETALL cart:uid001
#1) "10025"
#2) "1"
#3) "10065"
#4) "1"
# 查看商品数量
hlen cart:uid001
# 增加(减少hdecrby)商品
hincrby cart:uid001 10025 1
# 删除某个商品
hdel cart:uid001 10065

set

image.png
[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-FwbrSn5y-1685671838144)(https://cdn.nlark.com/yuque/0/2023/png/2166858/1680771775737-5def2ecc-74dd-4005-bfae-892f9868be08.png#averageHue=%23f1f1f1&clientId=ufc764910-829e-4&from=paste&height=340&id=u44334d1c&originHeight=510&originWidth=1644&originalType=binary&ratio=1.5&rotation=0&showTitle=false&size=248389&status=done&style=none&taskId=u73b3aee9-3638-4022-9789-ec226d84007&title=&width=1096)]
image.png
image.png
image.png
image.png
求差集

zset

image.png
image.png
[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-03sXES4E-1685671838146)(https://cdn.nlark.com/yuque/0/2023/png/2166858/1680852126188-9ad8c3e8-16b0-4b5a-8cab-6fc12c382a04.png#averageHue=%23f6f6f5&clientId=u5e37b430-7ea3-4&from=paste&height=515&id=ubb38c05b&originHeight=772&originWidth=1764&originalType=binary&ratio=1.5&rotation=0&showTitle=false&size=439352&status=done&style=none&taskId=u12d5ef03-5bf0-4fdf-9556-5739d95058a&title=&width=1176)]

bitmap

image.png
注意理解,数据结构比较抽象,所以需要多画图理解
image.png
image.png
统计占有字节数,strlen 是八位一扩容,联想 java 集合底层实现
image.png
bitcount 全部键里面含有1的有多少个
image.png

  1. 为什么这里要使用一个映射表:

有一个自增id,节省空间,bitmap 索引是从1开始的

  1. 这里的 bitop 你能说说自己的理解吗

可以理解为两个bit串对应位做与操作

另外一个统计签到的案例
image.png
这里的key使用的是用户id,然后偏移量表示天数,value和之前一致,便于统计某个用户累计签到(与之前统计系统某一段时间用户签到量做对比)
todo: 联想 leetcode 中位操作
估算内存占用
image.png

HyperLogLog

一种只需要占用很小的内存就能计算很多元素集合的基数的数据结构

  • 统计某个网页的UV、某个文章的UV
    • UV Unique Visitor 独立访客,一般理解为客户端IP,需要去重
  • 用户搜索网站关键词数量
  • 统计用户每天搜索不同词条个数

image.png

GEO

image.png
image.png
添加经纬度坐标
image.png
geo底层是 zset
image.png
查看经纬度
image.png
两个距离之间的距离
image.png
处理乱码
image.png

Stream

image.png
image.png
image.png
image.png
image.png
注意这里的消息id是递增的
image.png
image.png
image.png

image.png
image.png

bitfield

没啥人用略过

Redis持久化

image.png
rdb 完整复制
aof 有点像 redo log

rdb

简介

image.png
配置文件

  1. redis 6.0.16 及其以下

image.png

  1. redis 6.2 7.0

image.png

配置说明

image.png
有两种触发方式:手动,自动
image.png

  1. 修改 save 5 2
  2. dir /myredis/dump (储存的文件夹需要提前建立好)
  3. dbfilename dump6379.rdb

进入到 redis-cli 里面 shutdown
补充介绍:cli 里面可以通过 config get xxx(eg requirepass) 获取到配置信息

自动触发

测试触发生成 rdb 文件
image.png
image.png
可以通过使用 ll 来检查 rdb 文件有没有更新
image.png
恢复测试
image.png
执行 flushdb 也会触发产生 rdb 文件,但是里面是空的,无意义
redis 再次启动的时候会从指定位置加载 rdb 文件
生产实践
image.png

手动触发

image.png

  • save:生产不允许使用,会阻塞主程序
  • bgsave(默认):fork 生成一个子进程,来写备份文件

image.png
测试使用
image.png
lastsave 查看上一次备份时间(格式为时间戳),并处理时间戳转换为 时间

优缺点与数据丢失

image.png
image.png
数据丢失的案例很显然

修复 rdb 文件

image.png

触发小节和快照禁用

image.png
image.png

  1. cli 中
  2. 修改配置文件 save “”
rdb 优化参数

基本上不用动
image.png1. stop-write-on-bgsave-error 在 bgsave 发生错误的时候是否停止写入
image.png
image.png
是否压缩
image.png
合法性校验
image.png

aof

简介

image.png

工作流程和写回策略

image.png
写回策略,对应图中的 2
image.png
image.png

aof 配置功能开启

6与7有很大的不同
appendonly yes 开启
image.png
文件结构发生了改变
image.png
multipart-aof
image.png
image.png

正常恢复演示
# 开启 aof

redis-cli
set k1 v1
set k2 v2
keys *

# 备份 aof 文件
mv appendonlydir mv appendonlydir.bak

# 模拟删库
flushdb
keys *
shutdown

# 重新启动
redis-server /myredis/redis.conf
keys *
# 可以看到还是 empty
shutdown

# 做恢复操作
rm -rf appendonlydir
mv appendonlydir.bak appendonlydir
redis-server /myredis/redis.conf
keys *
# 可以观察到已经恢复出来了	

提问1:写一条 kv 后会写到哪个文件 -------> incr
提问2:读取一条 kv 后会写到哪个文件 ------> 不会记录

aof 异常恢复

极端情况,每一秒写入的 aof 存在错误,这一秒钟没有写完整
查看 incr 文件
image.png
手动模拟写入错误
image.png
image.png
修改后发现redis无法启动

redis-check-aof --fix /myredis/appendonlydir/appendonly.aof.1.incr.aof

image.png
修复后正常启动

优劣势

image.png
image.png
优点稳,缺点慢

aof 重写机制

启动 aof 文件的内容压缩,只保留可以恢复数据集的最小指令集
自动触发
image.png
bgrewriteaof 手动触发

RDB - AOF混合持久化

image.png
image.png

纯缓存

关闭 rdb aof

  • save “” 仍然可用 bgsave 手动操作
  • appendonly no 仍然可用 bgrewriteaof 手动生成 aof

redis 事务

对比关系型数据库
image.png
image.png
这块概念比较多,注意理解
image.png
五个案例需要掌握
image.png
case1 正常执行
image.png
case2 放弃事务
image.png
case3 全体连坐
image.png
case4 冤头债主
image.png
类似于 RuntimeException 的时候,对的执行,错的不执行
case5 watch监控
悲观锁
image.png
正常情况
image.png
加塞修改事务中 watch 的数据
整个事务失败
image.png
unwatch 放弃监控
image.png

image.png

redis 管道

来源:优化频繁命令的执行造成的性能瓶颈
image.png
image.png
image.png
也是很多概念,请自行理解
image.png
实操案例
image.png

redis 发布订阅 pub/sub

了解即可
个人觉得还是有必要了解,我自己给出补充,因为在我熟悉的平台中用到了
TODO

主从复制

配从不配主
需要在从机中指定主机的ip 端口 和 密码
image.png
image.png
基本操作命令

  • info replication 查看复制节点的主从关系和配置信息
  • replicaof/slaveof 主库 ip 主库端口 如果该数据库已经某个数据库的从数据库,那么会停止和原主数据库的同步关系转而和新的主数据库同步
  • slaveof no one 停止与主库的连接

案例

  • 主从复制
    • 方案1 配置文件

image.png
补充配置日志文件地址
logfile “/myredis/6379.log”
主机日志
image.png
从机日志
image.png
info replication 从机查看信息
image.png

  • 方案2 手动指定

  • 改换门庭

  • 自立为王

redis哨兵

redis 集群

redis java native

jedis

<!--jedis-->
<dependency>
    <groupId>redis.clients</groupId>
    <artifactId>jedis</artifactId>
    <version>4.3.1</version>
</dependency>

写的过程中复习 redis 数据结构

/**
 * 用来学习jedis的用法,其实也可以用来复习
 */
public class JedisDemo {
    public static void main(String[] args) {
        // 1. connection 连接 指定 ip 和 端口
        Jedis jedis = new Jedis("127.0.0.1", 6379);

        // 2. 指定访问服务器密码
//        jedis.auth("123456");

        // 3. 指定了 Jedis 客户端,可以像访问 jdbc 一样访问 redis
        System.out.println(jedis.ping());

        // 4. string
        jedis.set("k3", "hello-jedis");
        System.out.println(jedis.get("k3"));
        System.out.println(jedis.ttl("k3"));

        // 5. list 双端链表
        jedis.lpush("list", "1", "2", "3");
        List<String> list = jedis.lrange("list", 0, -1);
        for (String s : list) {
            System.out.println(s);
        }
        System.out.println(jedis.rpop("list"));
        System.out.println(jedis.lpop("list"));

        // 6. hash
        jedis.hset("hash1", "k1", "v1");
        Map<String, String> hash = new HashMap<>();
        hash.put("k1", "v1");
        hash.put("k2", "v2");
        hash.put("k3", "v3");
        hash.put("k4", "v4");
        jedis.hmset("hash2", hash);
        System.out.println(jedis.hmget("hash2", "k1", "k2", "k3", "k4"));
        System.out.println(jedis.hget("hash1", "k1"));
        System.out.println(jedis.hexists("hash2", "k1"));
        System.out.println(jedis.hkeys("hash2"));

        // 7. set
        jedis.sadd("set1", "1", "2", "3");
        jedis.sadd("set2", "1");
        System.out.println(jedis.smembers("set1")); // 获取 set1 的成员
        System.out.println(jedis.scard("set1")); // 获取 set1 的成员数量
        System.out.println(jedis.spop("set1")); // 从 set1 中删除一个成员

        // 把 set1 中的 1 从 set1 移动到 set2
        jedis.smove("set1", "set2", "3");
        System.out.println(jedis.smembers("set1"));
        System.out.println(jedis.smembers("set2"));

        System.out.println(jedis.sinter("set1", "set2")); // 交集
        System.out.println(jedis.sunion("set1", "set2")); // 并集

        // 8. zset
        jedis.zadd("zset1", 1.0, "1");
        jedis.zadd("zset1", 2.0, "2");
        jedis.zadd("zset1", 3.0, "3");
        jedis.zadd("zset1", 4.0, "4");
        jedis.zadd("zset1", 5.0, "5");
        jedis.zadd("zset1", 6.0, "6");
        jedis.zadd("zset1", 7.0, "7");
        jedis.zadd("zset1", 8.0, "8");
        jedis.zadd("zset1", 9.0, "9");
        jedis.zadd("zset1", 10.0, "10");

        List<String> zset1 = jedis.zrange("zset1", 0, -1);
        for (String s : zset1) {
            System.out.println(s);
        }

        List<String> zset11 = jedis.zrevrange("zset1", 0, -1);
        zset11.forEach(System.out::println);

    }
}

lettuce

底层用的 netty 比起 jedis 要快

<!--lettuce-->
<dependency>
    <groupId>io.lettuce</groupId>
    <artifactId>lettuce-core</artifactId>
    <version>6.2.1.RELEASE</version>
</dependency>

不做解释,和上面其实差不多

public class LettuceDemo {
    public static void main(String[] args) {
        // 1. 使用构造器链式编程来builder我们的RedisURI
        RedisURI uri = RedisURI.builder()
                .withHost("127.0.0.1")
                .withPort(6379)
//                .withAuthentication("default", "123456")
                .build();

        // 2. 连接客户端
        RedisClient redisClient = RedisClient.create(uri);
        StatefulRedisConnection<String, String> conn = redisClient.connect();

        // 3. 创建操作的command,通过 conn 创建
        RedisCommands<String, String> commands = conn.sync();

        // string
        commands.set("k1", "v1");
        System.out.println(commands.get("k1"));
        System.out.println(commands.mget("k2", "v2"));
        List<String> keys = commands.keys("*");
        for (String key : keys) {
            System.out.println(key);
        }

        // list
        commands.lpush("list01", "1", "2", "3");
        List<String> list01 = commands.lrange("list01", 0, -1);
        for (String s : list01) {
            System.out.println(s);
        }
        System.out.println(commands.rpop("list01", 2));

        // hash
        commands.hset("hash", "k1", "v1");
        commands.hset("hash", "k2", "v2");
        commands.hset("hash", "k3", "v3");
        System.out.println(commands.hgetall("hash"));
        Boolean hexists = commands.hexists("hash", "k2");
        System.out.println(hexists);

        // set
        commands.sadd("set01", "1", "2", "3");
        System.out.println(commands.smembers("set01"));
        commands.sismember("set01", "2");
        System.out.println(commands.scard("set01"));

        // zset
        commands.zadd("zset01", 1.0, "1");
        commands.zadd("zset01", 2.0, "2");
        commands.zadd("zset01", 3.0, "3");
        System.out.println(commands.zrange("zset01", 0, -1));

        // 4. 关闭连接
        conn.close();
        redisClient.shutdown();

    }
}

redis springboot

单机版本

起一个 springboot 模块

<parent>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-parent</artifactId>
        <version>2.7.10</version>
        <relativePath/> <!-- lookup parent from repository -->
    </parent>

<!--SpringBoot与Redis整合依赖-->
<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-data-redis</artifactId>
</dependency>
<dependency>
    <groupId>org.apache.commons</groupId>
    <artifactId>commons-pool2</artifactId>
</dependency>
<!--swagger2-->
<dependency>
    <groupId>io.springfox</groupId>
    <artifactId>springfox-swagger2</artifactId>
    <version>2.9.2</version>
</dependency>
<dependency>
    <groupId>io.springfox</groupId>
    <artifactId>springfox-swagger-ui</artifactId>
    <version>2.9.2</version>
</dependency>

redis starter 中包含 lettuce

server:
  port: 5555

spring:
  # swagger
  swagger2:
    enabled: true
  mvc:
    pathmatch:
      #在springboot2.6.X结合swagger2.9.X会提示documentationPluginsBootstrapper空指针异常,
      #原因是在springboot2.6.X中将SpringMVC默认路径匹配策略从AntPathMatcher更改为PathPatternParser,
      # 导致出错,解决办法是matching-strategy切换回之前ant_path_matcher
      matching-strategy: ant_path_matcher
  redis:
    database: 4
    host: 127.0.0.1
    port: 6379
#    password: 123456

不配置序列化配置类
image.png
image.png

方案一 修改为使用 StringRedisTemplate

image.png
key 的序列化正常了
改成 – raw 重新进入cli
image.png

方案二 手动注入RedisTemplate

image.png

@Configuration
public class RedisConfig {

    @Bean
    public RedisTemplate<String, Object> redisTemplate(LettuceConnectionFactory lettuceConnectionFactory) {
        RedisTemplate<String, Object> redisTemplate = new RedisTemplate<>();

        redisTemplate.setConnectionFactory(lettuceConnectionFactory);

        // 设置key的序列化方式string
        redisTemplate.setKeySerializer(new StringRedisSerializer());
        // 设置value的序列化方式json
        redisTemplate.setValueSerializer(new GenericJackson2JsonRedisSerializer());

        // 设置hash的key的序列化方式string
        redisTemplate.setHashKeySerializer(new StringRedisSerializer());
        // 设置hash的value的序列化方式json
        redisTemplate.setHashValueSerializer(new GenericJackson2JsonRedisSerializer());

        redisTemplate.afterPropertiesSet();

        return redisTemplate;
    }
}

集群版本

补充资料

补充,反向考察自己掌握程度
Redis高频面试题(2023最新)_爱穿背带裤的馫的博客-CSDN博客
稍微简单,拿来入门
20道Redis经典面试题!(珍藏版)
这个复杂一些

redis 单线程 vs 多线程

q: redis 是否支撑多线程
回答切入点 version
image.png
q: 如何理解我们所说的 redis 是单线程
image.png
注意最后一句话,工作线程是单线程,但是到了后面的版本,有些操作是多线程的(eg,aof)
redis 单线程为什么这么快?

  • 基于内存操作:所有数据存在于内存中,所有的运算都在内存中执行
  • 数据结构简单:redis 数据结构大多是专门设计的,这些数据结构的时间复杂度 大部分是 O(1)
  • (最重要)多路复用和非阻塞io:redis 使用io多路复用功能来监听多个 socket 连接客户端,使用一个线程连接来处理多个请求,减少线程切换带来的开销,同时也避免了io阻塞操作
  • 避免上下文切换:单线程模型,避免不必要的上下文切换和多线程竞争,省去了多线程切换带来的时间和性能上的消耗,而且单线程不会导致死锁问题的发生

为什么要引入多线程

  • cpu

q: redis 快的直接原因
io多路复用,epoll函数
q: 如何开启多线程(默认关闭)
image.png
image.png

BigKey

image.png
MoreKey 案例
image.png
批量创建 key

  1. 生成脚本
for((i=1;i<=100*10000;i++)); do echo "set k$i v$i" >> /tmp/redisTest.txt;done;
  1. pipe 批量灌入

image.png
我的环境是 mac,实测脚本生成无法在 mac 执行,我是找了一台 centos 机器上生成下载下到开发机器上使用的
image.png
keys * 由于 redis 单线程,会卡住其他指令,生产禁用
image.png
禁用危险指令

rename-command keys ""
rename-command flushdb ""
rename-command flushall ""

重启后测试
image.png
q: 不能使用 keys * ,使用什么呢
scan
image.png
image.png

bigkey 发现删除策略

image.png
大key判断标准
image.png
image.png
[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-U8RFM8jK-1685671838170)(https://cdn.nlark.com/yuque/0/2023/png/2166858/1681797059934-6fbf2463-d42e-4a70-acef-5c8960b43696.png#averageHue=%23ecebeb&clientId=uffc70029-fcd7-4&from=paste&height=104&id=u2c406f91&originHeight=208&originWidth=1368&originalType=binary&ratio=2&rotation=0&showTitle=false&size=112335&status=done&style=none&taskId=ucd7cde7d-8738-4577-97b6-b19b55ce1fa&title=&width=684)]
[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-dA5I5MsF-1685671838170)(https://cdn.nlark.com/yuque/0/2023/png/2166858/1681797041418-2d5d8359-4c00-4563-b0ca-a12bd592cab3.png#averageHue=%23f4f4f4&clientId=uffc70029-fcd7-4&from=paste&height=109&id=u3d2c1898&originHeight=218&originWidth=1036&originalType=binary&ratio=2&rotation=0&showTitle=false&size=66956&status=done&style=none&taskId=uded57ae0-a3e3-4640-bc18-faf65d03ba5&title=&width=518)]
扫描所有大key

redis-cli --bigkeys

[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-M6Q31rHv-1685671838170)(https://cdn.nlark.com/yuque/0/2023/png/2166858/1681797279561-968fb4f4-082e-4197-9277-53598bee1f39.png#averageHue=%230d0d0d&clientId=uffc70029-fcd7-4&from=paste&height=399&id=ub4bc5851&originHeight=798&originWidth=992&originalType=binary&ratio=2&rotation=0&showTitle=false&size=148726&status=done&style=none&taskId=ubddafc99-a8b5-43fd-bc83-5e37c5b810c&title=&width=496)]
计算每个键值的字节数
[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-lgul5fhC-1685671838170)(https://cdn.nlark.com/yuque/0/2023/png/2166858/1681798571619-d559f78f-3a7e-4a81-ba6c-ef3278a2db96.png#averageHue=%23090909&clientId=uffc70029-fcd7-4&from=paste&height=45&id=u5e8ec278&originHeight=90&originWidth=790&originalType=binary&ratio=2&rotation=0&showTitle=false&size=11517&status=done&style=none&taskId=uc2472c38-67f7-4f6f-88e1-ad3678a55a6&title=&width=395)]
删除方式(渐进式删除)
image.png
下面不用的数据类型

  • String: 一般用 del ,如果过于庞大unlink
  • hash: 使用 hscan 每次获取少量的 field-value,再用 hdel 删除每个 field

[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-GnaRhUaT-1685671838171)(https://cdn.nlark.com/yuque/0/2023/png/2166858/1681798951351-9abee08a-c798-4137-ac28-3bafc269ea58.png#averageHue=%23f3f4f6&clientId=uffc70029-fcd7-4&from=paste&height=414&id=u33db1dbd&originHeight=828&originWidth=1264&originalType=binary&ratio=2&rotation=0&showTitle=false&size=393018&status=done&style=none&taskId=u90a1494c-8acd-441d-ab4c-338fce0e8fe&title=&width=632)]
一个个渐进式删除,hscan 扫描一个属性,hdel删除一个属性,最后删除整个键

  • list: ltrim

[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-YJVHE7S1-1685671838171)(https://cdn.nlark.com/yuque/0/2023/png/2166858/1681799063413-00d397ec-6c04-4227-8de4-4e6857a61c09.png#averageHue=%23e9eeed&clientId=uffc70029-fcd7-4&from=paste&height=379&id=u932a03eb&originHeight=758&originWidth=1164&originalType=binary&ratio=2&rotation=0&showTitle=false&size=242529&status=done&style=none&taskId=u85dc9e59-5cbe-4388-8337-d21c67c8a14&title=&width=582)]
每次 从左边 ltrim 删除掉一百个

  • set

sscan srem
image.png

  • zset

使用 zscan 每次获取部分元素,再使用 zremrangebyrank 删除每个元素
image.png
[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-qfQWjaQ8-1685671838172)(https://cdn.nlark.com/yuque/0/2023/png/2166858/1681799252292-393a1840-47cf-4802-a241-5341c94135a6.png#averageHue=%23f2f3f5&clientId=uffc70029-fcd7-4&from=paste&height=372&id=u3c35b7c4&originHeight=744&originWidth=1112&originalType=binary&ratio=2&rotation=0&showTitle=false&size=343217&status=done&style=none&taskId=u1965cc04-98c7-4b05-98ba-cf99265c005&title=&width=556)]
q: 生产调优
image.png
[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-9CyrE8qC-1685671838172)(https://cdn.nlark.com/yuque/0/2023/png/2166858/1681799437370-ffe35505-29d8-4929-9bc9-d4e0289e2f69.png#averageHue=%23e8dcbf&clientId=uffc70029-fcd7-4&from=paste&height=391&id=u0af3fdd6&originHeight=782&originWidth=1456&originalType=binary&ratio=2&rotation=0&showTitle=false&size=730635&status=done&style=none&taskId=u06dd8602-d5ad-4774-89f5-ce907f24fcf&title=&width=728)]

缓存双写一致性

更新策略

image.png
image.png
image.png
缓存设计要求
缓存分类:

  • 只读缓存:(脚本批量写入,canal 等)
  • 读写缓存
    • 同步直写:vip数据等即时数据
    • 异步缓写:允许延时(仓库,物流),异常出现,有可能需要使用 kafka, rabbitmq 进行弥补,重试重写

[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-9HZWFWzm-1685671838173)(https://cdn.nlark.com/yuque/0/2023/png/2166858/1681800330764-463208b5-3fe8-44e8-af57-93c259042b64.png#averageHue=%23fbf9f8&clientId=uffc70029-fcd7-4&from=paste&height=457&id=ua528a8c3&originHeight=914&originWidth=1772&originalType=binary&ratio=2&rotation=0&showTitle=false&size=491272&status=done&style=none&taskId=u0b573dad-1bbd-427d-aa48-b94f07da24f&title=&width=886)]
3.2 分支
qps 小于一千可以
存在问题:数据库读与redis写不是原子操作
[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-Q0DTgwgc-1685671838173)(https://cdn.nlark.com/yuque/0/2023/png/2166858/1681800485805-43629bb0-a3a7-432a-906b-f119ea212509.png#averageHue=%23b0afae&clientId=uffc70029-fcd7-4&from=paste&height=409&id=u71c18811&originHeight=818&originWidth=1550&originalType=binary&ratio=2&rotation=0&showTitle=false&size=588292&status=done&style=none&taskId=u2dc141ad-3278-42e9-8157-308975ac885&title=&width=775)]
[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-zRFRsMJA-1685671838173)(https://cdn.nlark.com/yuque/0/2023/png/2166858/1681800883898-8a8aa98e-557f-48fb-86fc-5ba62a98bd90.png#averageHue=%23faf8f6&clientId=uffc70029-fcd7-4&from=paste&height=401&id=u2c5ea864&originHeight=802&originWidth=1550&originalType=binary&ratio=2&rotation=0&showTitle=false&size=566570&status=done&style=none&taskId=ud54bf18d-b215-4970-8485-fb53dd8fc9a&title=&width=775)]
数据库和缓存更新的几种策略
=》实现最终一致性
image.png
可以停机
单线程操作
否则,四种更新策略
image.png

  1. 先更新数据库,再更新缓存

image.png
异常问题1
[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-pOwhrwQf-1685671838176)(https://cdn.nlark.com/yuque/0/2023/png/2166858/1681801615944-ed711ffc-f3b3-4e80-b3ab-59dedc3327dc.png#averageHue=%23f6f5f4&clientId=uffc70029-fcd7-4&from=paste&height=388&id=uae99ed28&originHeight=776&originWidth=938&originalType=binary&ratio=2&rotation=0&showTitle=false&size=315410&status=done&style=none&taskId=udb709332-823f-4f1b-9661-55617e6b545&title=&width=469)]
异常问题2
如果停机,可使用此方式

  1. 先更新缓存,再更新数据库

首先业务上不推荐,业务上把 mysql 作为底单数据库,保证最后解释
image.png
自己的理解,只要是写缓存的方式,都会存在线程写入先后导致的数据不一致

  1. 先删除缓存,再更新数据库

[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-rFoNIWIS-1685671838177)(https://cdn.nlark.com/yuque/0/2023/png/2166858/1681801911601-0d0107ae-82b4-491a-858a-4d0f3e538d41.png#averageHue=%23f2f2f2&clientId=uffc70029-fcd7-4&from=paste&height=309&id=ub9f76e86&originHeight=618&originWidth=1550&originalType=binary&ratio=2&rotation=0&showTitle=false&size=229415&status=done&style=none&taskId=u50866123-ad13-487e-8b87-094bdced56e&title=&width=775)]image.png
image.png
image.png
image.png
[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-kfelAddz-1685671838178)(https://cdn.nlark.com/yuque/0/2023/png/2166858/1681802240506-8cbcc615-ddfe-48de-9fea-89077571c03f.png#averageHue=%23f6f5f5&clientId=uffc70029-fcd7-4&from=paste&height=295&id=ub01ac453&originHeight=590&originWidth=1834&originalType=binary&ratio=2&rotation=0&showTitle=false&size=396461&status=done&style=none&taskId=ub61ef02d-1167-4ea1-b3c5-99269b4ed9f&title=&width=917)]
image.png
读缓存会写入旧数据
引出延时双删
[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-7tjF65jd-1685671838179)(https://cdn.nlark.com/yuque/0/2023/png/2166858/1681802399814-59070b1b-274a-4156-b7e9-e45efc081edb.png#averageHue=%23faf7f6&clientId=uffc70029-fcd7-4&from=paste&height=313&id=u1581323b&originHeight=626&originWidth=1732&originalType=binary&ratio=2&rotation=0&showTitle=false&size=320965&status=done&style=none&taskId=u0202a48b-f7a5-4827-8120-0a7c240c708&title=&width=866)]
image.png
[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-kdXRvfv5-1685671838179)(https://cdn.nlark.com/yuque/0/2023/png/2166858/1681802530290-6e287792-66f3-4463-92ec-20f07e8cc25e.png#averageHue=%23f1f1f1&clientId=uffc70029-fcd7-4&from=paste&height=163&id=u97906ce1&originHeight=326&originWidth=1780&originalType=binary&ratio=2&rotation=0&showTitle=false&size=204854&status=done&style=none&taskId=u8625fd5a-6e26-4e90-8388-6d044d492d1&title=&width=890)]
image.png
image.png

  1. 先更新数据库,再删除缓存

存在问题
[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-bYhaShdD-1685671838180)(https://cdn.nlark.com/yuque/0/2023/png/2166858/1681808029228-9d78380d-bf32-437f-8bb0-1755dff6ce93.png#averageHue=%23f7f7f6&clientId=uffc70029-fcd7-4&from=paste&height=350&id=ua51dad94&originHeight=700&originWidth=1832&originalType=binary&ratio=2&rotation=0&showTitle=false&size=351685&status=done&style=none&taskId=uc74bb066-64bf-4303-b6e5-b2e0b5323db&title=&width=916)]
q: 如何实现最终一致性
image.png
image.png
q: 如何取舍
[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-Ke4kW9JQ-1685671838180)(https://cdn.nlark.com/yuque/0/2023/png/2166858/1681808667021-b18dba66-f48a-4d65-96bd-10fa5228431d.png#averageHue=%23f6f5f5&clientId=uffc70029-fcd7-4&from=paste&height=302&id=ue50d41d8&originHeight=604&originWidth=1776&originalType=binary&ratio=2&rotation=0&showTitle=false&size=565810&status=done&style=none&taskId=u1e9843bc-4951-44f3-997d-5d2335b1eae&title=&width=888)]

双写一致性落地案例

mysql -> canal -> redis
Home
canal 选择版本 1.6.1
mysql: 5.7 docker环境安装

1.mysql 准备

docker run -d -p 3306:3306 --privileged=true -v /root/mysql/log:/var/log/mysql -v /root/mysql/data:/var/lib/mysql -v /root/mysql/conf:/etc/mysql/conf.d -e MYSQL_ROOT_PASSWORD=123456  --name mysql mysql:5.7


docker exec -it e8b72b11df2a /bin/bash    //e8b72b11df2a 就是容器id
    
mysql -uroot -p        // 密码 123456

# 查看是否开启 binlog
SHOW VARIABLES LIKE 'log_bin';

# 添加 canal 操作账号
DROP USER IF EXISTS 'canal'@'%';
CREATE USER 'canal'@'%' IDENTIFIED BY 'canal';  
GRANT ALL PRIVILEGES ON *.* TO 'canal'@'%' IDENTIFIED BY 'canal';  
FLUSH PRIVILEGES;
 
SELECT * FROM mysql.user;

# 创建一张表
CREATE TABLE `t_user` 
( 
`id` BIGINT ( 20 ) NOT NULL AUTO_INCREMENT, 
`userName` VARCHAR ( 100 ) NOT NULL, 
PRIMARY KEY ( `id` ) 
) 
ENGINE = INNODB AUTO_INCREMENT = 10 DEFAULT CHARSET = utf8mb4
2.canal 准备

下载 canal.deployer-1.1.6.tar.gz
修改配置文件
[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-wL42omKr-1685671838180)(https://cdn.nlark.com/yuque/0/2023/png/2166858/1681890499797-759ba72f-4ee1-4c43-a2a3-ed561fe7b279.png#averageHue=%23282c32&clientId=uffc70029-fcd7-4&from=paste&height=313&id=ucf06862b&originHeight=626&originWidth=570&originalType=binary&ratio=2&rotation=0&showTitle=false&size=43807&status=done&style=none&taskId=u42a2a126-b306-472f-abe7-8ca3d525bb9&title=&width=285)]

canal.instance.master.address=xxx:xxx:xxx:xxx:3306

# username/password (如果配置的是 canal/canal 则默认无需修改)
canal.instance.dbUsername=canal
canal.instance.dbPassword=canal
./bin/startup.sh

查看启动日志,检查运行状态
[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-VBNjN6ff-1685671838181)(https://cdn.nlark.com/yuque/0/2023/png/2166858/1681891520255-e64a0903-5a49-4a2c-bdfc-bb5387254dc3.png#averageHue=%23040404&clientId=uffc70029-fcd7-4&from=paste&height=68&id=u6364e783&originHeight=136&originWidth=954&originalType=binary&ratio=2&rotation=0&showTitle=false&size=16930&status=done&style=none&taskId=uc975a454-bb30-4e76-8a60-c3806b93bac&title=&width=477)]

3.编写 canal-client

image.png

<parent>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-parent</artifactId>
        <version>2.7.10</version>
        <relativePath/> <!-- lookup parent from repository -->
    </parent>

<dependencies>
        <!--canal-->
        <dependency>
            <groupId>com.alibaba.otter</groupId>
            <artifactId>canal.client</artifactId>
            <version>1.1.0</version>
        </dependency>
        <!--SpringBoot与Redis整合依赖-->
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-data-redis</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-test</artifactId>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>org.projectlombok</groupId>
            <artifactId>lombok</artifactId>
            <version>1.18.26</version>
        </dependency>
    </dependencies>

    <build>
        <plugins>
            <plugin>
                <groupId>org.springframework.boot</groupId>
                <artifactId>spring-boot-maven-plugin</artifactId>
            </plugin>
        </plugins>
    </build>

配置文件(请按需修改)

server:
  port: 5555

spring:
  redis:
    database: 0
    host: 127.0.0.1
    port: 6379
#    password: 123456

config 文件和 springboot 案例中一致

@Configuration
public class RedisConfig {

    @Bean
    public RedisTemplate<String, Object> redisTemplate(LettuceConnectionFactory lettuceConnectionFactory) {
        RedisTemplate<String, Object> redisTemplate = new RedisTemplate<>();

        redisTemplate.setConnectionFactory(lettuceConnectionFactory);

        // 设置key的序列化方式string
        redisTemplate.setKeySerializer(new StringRedisSerializer());
        // 设置value的序列化方式json
        redisTemplate.setValueSerializer(new GenericJackson2JsonRedisSerializer());

        // 设置hash的key的序列化方式string
        redisTemplate.setHashKeySerializer(new StringRedisSerializer());
        // 设置hash的value的序列化方式json
        redisTemplate.setHashValueSerializer(new GenericJackson2JsonRedisSerializer());

        redisTemplate.afterPropertiesSet();

        return redisTemplate;
    }
}

biz 操作封装

@Component
public class RedisCanalClient {

    @Resource
    RedisTemplate redisTemplate;

    /**
     * 读取 canal 数据写入redis
     * @param columns 行
     */
    public void redisInsert(List<CanalEntry.Column> columns) {
        JSONObject jsonObject = new JSONObject();
        for (CanalEntry.Column column : columns) {
            System.out.println(column.getName() + " : " + column.getValue() + "  update=" + column.getUpdated());
            jsonObject.put(column.getName(), column.getValue());
        }
        if (columns.size()>0) {
            redisTemplate.opsForValue().set(columns.get(0).getValue(), jsonObject.toJSONString());
        }
    }

    /**
     * 读取 canal 数据删除 redis 行
     * @param columns 行
     */
    public void redisDelete(List<CanalEntry.Column> columns) {
        JSONObject jsonObject = new JSONObject();
        for (CanalEntry.Column column : columns) {
            System.out.println(column.getName() + " : " + column.getValue() + "  update=" + column.getUpdated());
            jsonObject.put(column.getName(), column.getValue());
        }
        if (columns.size()>0) {
            redisTemplate.delete(columns.get(0).getValue());
        }
    }

    /**
     * 读取 canal 数据修改 redis 行
     * @param columns 行
     */
    public void redisUpdate(List<CanalEntry.Column> columns) {
        JSONObject jsonObject = new JSONObject();
        for (CanalEntry.Column column : columns) {
            System.out.println(column.getName() + " : " + column.getValue() + "  update=" + column.getUpdated());
            jsonObject.put(column.getName(), column.getValue());
        }
        if (columns.size()>0) {

            redisTemplate.opsForValue().set(columns.get(0).getValue(), jsonObject.toJSONString());
            System.out.println("----------------update after: " + redisTemplate.opsForValue().get(columns.get(0).getValue()));

        }
    }

    /**
     * 读取 canal 数据并操作
     * @param entrys 行
     */
    public void printEntry(List<CanalEntry.Entry> entrys) {
        for (CanalEntry.Entry entry : entrys) {
            if (entry.getEntryType() == CanalEntry.EntryType.TRANSACTIONBEGIN || entry.getEntryType() == CanalEntry.EntryType.TRANSACTIONEND) {
                continue;
            }

            CanalEntry.RowChange rowChage = null;
            try {
                rowChage = CanalEntry.RowChange.parseFrom(entry.getStoreValue());
            } catch (Exception e) {
                throw new RuntimeException("ERROR ## parser of eromanga-event has an error , data:" + entry.toString(),
                        e);
            }

            CanalEntry.EventType eventType = rowChage.getEventType();
            System.out.println(String.format("================&gt; binlog[%s:%s] , name[%s,%s] , eventType : %s",
                    entry.getHeader().getLogfileName(), entry.getHeader().getLogfileOffset(),
                    entry.getHeader().getSchemaName(), entry.getHeader().getTableName(),
                    eventType));

            for (CanalEntry.RowData rowData : rowChage.getRowDatasList()) {
                if (eventType == CanalEntry.EventType.DELETE) {
                    redisDelete(rowData.getBeforeColumnsList());
                } else if (eventType == CanalEntry.EventType.INSERT) {
                    redisInsert(rowData.getAfterColumnsList());
                } else {
                    redisUpdate(rowData.getAfterColumnsList());
                }
            }
        }
    }

}

test 测试类

@SpringBootTest
public class CanalClientTest {

    public static final Integer _60SECONDS = 60;

    @Resource
    RedisCanalClient redisCanalClient;

    /**
     * 测试 canal client 监听 server 实现 mysql -> redis
     */
    @Test
    public void startClient() {
        System.out.println("--------------initCanal main()方法------------");

        // ======================================================================================
        CanalConnector connector = CanalConnectors.newSingleConnector(
                new InetSocketAddress("127.0.0.1", 11111), // canal server 地址
                "example",
                "",
                "");

        int batchSize = 1000;
        int emptyCount = 0;
        System.out.println("---------------------canal init OK,开始监听mysql变化------");
        try {
            connector.connect();
//            connector.subscribe(".*\\..*");
            // 设置监听的表

            connector.subscribe("canal.t_user");

            connector.rollback();
            int totalEmptyCount = 10 * _60SECONDS;
            while (emptyCount < totalEmptyCount) {
                System.out.println("我是 canal, 每秒一次正在监听: " + UUID.randomUUID().toString());
                Message message = connector.getWithoutAck(batchSize); // 获取指定数量的数据
                long batchId = message.getId();
                int size = message.getEntries().size();
                if (batchId == -1 || size == 0) {
                    emptyCount++;
                    System.out.println("empty count : " + emptyCount);
                    try {
                        Thread.sleep(1000);
                    } catch (InterruptedException e) {
                    }
                } else {
                    // 计数器置0
                    emptyCount = 0;
                    // System.out.printf("message[batchId=%s,size=%s] \n", batchId, size);
                    redisCanalClient.printEntry(message.getEntries());
                }

                connector.ack(batchId); // 提交确认
                // connector.rollback(batchId); // 处理失败, 回滚数据
            }

            System.out.println("已经监听了"+totalEmptyCount+"秒,无任何消息,请重启重试");
        } finally {
            connector.disconnect();
        }
    }
}

启动测试类

4.测试
  • 测试新增

手动添加一条记录 id: 7 name:666
image.png
查看 redis
[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-tbcfWW5g-1685671838181)(https://cdn.nlark.com/yuque/0/2023/png/2166858/1681891610362-9dce2c47-ee33-4d7b-81a3-c75de9985c3e.png#averageHue=%23040404&clientId=uffc70029-fcd7-4&from=paste&height=219&id=u20e56277&originHeight=438&originWidth=966&originalType=binary&ratio=2&rotation=0&showTitle=false&size=45249&status=done&style=none&taskId=u54ab9354-45cc-4af1-a0a1-312a9713a82&title=&width=483)]

  • 测试修改

修改mysql id:7 的 name 为 777
image.png
[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-NSQyIJO4-1685671838182)(https://cdn.nlark.com/yuque/0/2023/png/2166858/1681891671054-4996e705-b1ef-44f1-8aa1-6b41157713e6.png#averageHue=%230d0d0d&clientId=uffc70029-fcd7-4&from=paste&height=35&id=u71e3d3ba&originHeight=70&originWidth=808&originalType=binary&ratio=2&rotation=0&showTitle=false&size=11301&status=done&style=none&taskId=ue20eaf3d-f4f5-4e3c-9127-4583055399f&title=&width=404)]

  • 测试删除

删除 id:7 的这条数据
[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-A2jgfYtl-1685671838182)(https://cdn.nlark.com/yuque/0/2023/png/2166858/1681891746361-e51f5a12-777f-4ac0-ad52-5ac81920bf41.png#averageHue=%2331353e&clientId=uffc70029-fcd7-4&from=paste&height=75&id=u3742c49a&originHeight=150&originWidth=1358&originalType=binary&ratio=2&rotation=0&showTitle=false&size=34361&status=done&style=none&taskId=u96e8923d-d24b-453f-833a-5d679dca018&title=&width=679)]
image.png

缓存预热雪崩穿透击穿

image.png

缓存预热

[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-JaD3Vhmi-1685671838183)(https://cdn.nlark.com/yuque/0/2023/png/2166858/1681953140107-0bccaed4-21ad-4607-a5fb-4d38045f2c7f.png#averageHue=%23f0ebe9&clientId=uffc70029-fcd7-4&from=paste&height=355&id=u9a69d514&originHeight=710&originWidth=1492&originalType=binary&ratio=2&rotation=0&showTitle=false&size=417710&status=done&style=none&taskId=ud700a0b6-3f9a-42a7-8936-d73e579af3f&title=&width=746)]

缓存雪崩

image.png
有这两种原因

  1. redis key 永不过期or过期时间错开
  2. redis 缓存集群实现高可用
    1. 主从哨兵
    2. Redis Cluster
    3. 开启redis持久化aof,rdb,尽快恢复集群
  3. 多缓存结合预防雪崩:本地缓存 ehcache + redis 缓存
  4. 服务降级:Hystrix 或者 sentinel 限流降级
  5. 人民币玩家:阿里云给了你多少广告?笑

缓存穿透

[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-lnpWzI0I-1685671838183)(https://cdn.nlark.com/yuque/0/2023/png/2166858/1681954314261-b5866deb-811b-47a0-9974-aebc41fe1faa.png#averageHue=%23e6e4e4&clientId=uffc70029-fcd7-4&from=paste&height=237&id=uf332cc98&originHeight=474&originWidth=1678&originalType=binary&ratio=2&rotation=0&showTitle=false&size=327699&status=done&style=none&taskId=u2ef203c0-5324-404b-bf80-a5b6562eef6&title=&width=839)]

[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-aL74UYZe-1685671838183)(https://cdn.nlark.com/yuque/0/2023/png/2166858/1681954347622-ae1c25c1-eac8-4bbc-b83a-5d06ba5f149c.png#averageHue=%23f4f4f4&clientId=uffc70029-fcd7-4&from=paste&height=207&id=u978c1e75&originHeight=414&originWidth=1902&originalType=binary&ratio=2&rotation=0&showTitle=false&size=174435&status=done&style=none&taskId=u3cc9510c-36c0-4a85-96b1-f6c831ed96b&title=&width=951)]
[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-ebDmexs9-1685671838184)(https://cdn.nlark.com/yuque/0/2023/png/2166858/1681954447968-e40550b2-6b71-4a14-9662-b230e440446d.png#averageHue=%23f9f9f9&clientId=uffc70029-fcd7-4&from=paste&height=370&id=uae4f993a&originHeight=740&originWidth=1252&originalType=binary&ratio=2&rotation=0&showTitle=false&size=201838&status=done&style=none&taskId=ub0c87e1f-d6d8-45db-8864-5517f751c21&title=&width=626)]
image.png

  1. guava BloomFilter

image.png

  • 误判问题,但是概率小可以接受,不能从布隆过滤器删除
  • 全部合法的key都需要放入 Guava布隆过滤器+redis里面,不然数据就是返回null

案例

<dependency>
    <groupId>com.google.guava</groupId>
    <artifactId>guava</artifactId>
    <version>31.1-jre</version>
</dependency>
@Service
@Slf4j
public class GuavaBloomFilterService {

    // 1.定义一个常量
    public static final int _1W = 10000;
    // 2.定义我们guava布隆过滤器,初始容量
    public static final int SIZE = 100 * _1W;
    // 3.误判率,它越小误判的个数也越少(思考:是否可以无限小? 没有误判岂不是更好)
    public static double fpp = 0.0000000000003;  // 这个数越小所用的hash函数越多,bitmap占用的位越多  默认的就是0.03,5个hash函数   0.01,7个函数
    // 4.创建guava布隆过滤器
    private static BloomFilter<Integer> bloomFilter = BloomFilter.create(Funnels.integerFunnel(), SIZE, fpp);

    public void guavaBloomFilter() {
        // 1. 往 bloomFilter 中添加数据
        for (int i = 0; i < SIZE; i++) {
            bloomFilter.put(i);
        }
        // 2. 故意取10w个不在范围内的数据进行测试,来进行误判率演示
        List<Integer> list = new ArrayList<>(10 * _1W);

        // 3. 验证
        for (int i = SIZE; i < SIZE + (10 * _1W); i++) {
            if (bloomFilter.mightContain(i)) {
//                log.info("被误判了:{}", i);
                list.add(i);
            }
        }
        log.info("误判总数量:{}", list.size());
        log.info("误判率:{}", list.size() / (10 * _1W));
    }
}

@SpringBootTest
public class GuavaTest {

    @Resource
    GuavaBloomFilterService guavaBloomFilterService;

    /**
     * guava版本布隆过滤器,helloworld 入门级演示
     */
    @Test
    public void testGuavaWithBloomFilter() {
        System.out.println("testGuavaWithBloomFilter");
        // 1. 创建 guava版布隆过滤器
        BloomFilter<Integer> bloomFilter = BloomFilter.create(Funnels.integerFunnel(), 100);

        //2. 判断指定的元素是否存在
        System.out.println(bloomFilter.mightContain(1));
        System.out.println(bloomFilter.mightContain(2));

        // 2. 添加数据
        bloomFilter.put(1);
        bloomFilter.put(2);

        System.out.println(bloomFilter.mightContain(1));
        System.out.println(bloomFilter.mightContain(2));
    }

    @Test
    public void testGuavaWithBloomFilter2() {
        guavaBloomFilterService.guavaBloomFilter();
    }

}

fpp 默认 0.03
[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-CVPOyxjg-1685671838184)(https://cdn.nlark.com/yuque/0/2023/png/2166858/1681959712279-dc24b2ac-860b-44c0-bc73-89e3df9f589d.png#averageHue=%23f6f5f4&clientId=uffc70029-fcd7-4&from=paste&height=180&id=ua48ee1d4&originHeight=360&originWidth=2474&originalType=binary&ratio=2&rotation=0&showTitle=false&size=119338&status=done&style=none&taskId=u5ad5c14a-5d2b-47e5-a25c-8f76be0daaf&title=&width=1237)]
[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-9amjtop2-1685671838185)(https://cdn.nlark.com/yuque/0/2023/png/2166858/1681959677835-96c84336-74ca-44fa-8d8c-abea91999b51.png#averageHue=%23f6f4f4&clientId=uffc70029-fcd7-4&from=paste&height=166&id=uded29c32&originHeight=332&originWidth=2340&originalType=binary&ratio=2&rotation=0&showTitle=false&size=103627&status=done&style=none&taskId=ud99b4aa5-3033-4155-b8f2-b64aeae65bd&title=&width=1170)]
fpp要求越高,bit位数越多,hash函数越多
[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-j1O02QRH-1685671838185)(https://cdn.nlark.com/yuque/0/2023/png/2166858/1681959989180-5fb03ba0-1159-4c46-b569-e7176c393af0.png#averageHue=%23eeeeee&clientId=uffc70029-fcd7-4&from=paste&height=471&id=u8ef1ab3b&originHeight=942&originWidth=1912&originalType=binary&ratio=2&rotation=0&showTitle=false&size=280855&status=done&style=none&taskId=u36189c27-fcf9-4570-9dd6-14375ed76cd&title=&width=956)]
guava 黑名单使用
[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-fCylQWPP-1685671838185)(https://cdn.nlark.com/yuque/0/2023/png/2166858/1681960049527-4b09f97f-25fb-481d-a70e-5083c97a92dc.png#averageHue=%23f5f4eb&clientId=uffc70029-fcd7-4&from=paste&height=381&id=ubd15dd98&originHeight=762&originWidth=1530&originalType=binary&ratio=2&rotation=0&showTitle=false&size=405242&status=done&style=none&taskId=uf34972b8-9bab-4b75-b7ca-2d0ceceb4bd&title=&width=765)]

缓存击穿

image.png
对比穿透和击穿
image.png
[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-qpu5wtga-1685671838185)(https://cdn.nlark.com/yuque/0/2023/png/2166858/1681960730493-a1d6a2bf-f460-4785-b86e-b29f2a19370a.png#averageHue=%23ebe9e9&clientId=uffc70029-fcd7-4&from=paste&height=115&id=u3a298f08&originHeight=230&originWidth=1720&originalType=binary&ratio=2&rotation=0&showTitle=false&size=169792&status=done&style=none&taskId=uda903313-b24a-40cb-ac00-5cec83bad32&title=&width=860)]
image.png
聚划算案例
[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-414svp8C-1685671838186)(https://cdn.nlark.com/yuque/0/2023/png/2166858/1681970846618-281e800a-0c3f-4bbd-b2d8-2122ed860b62.png#averageHue=%23f6f6f5&clientId=uffc70029-fcd7-4&from=paste&height=275&id=ue62bdeeb&originHeight=550&originWidth=1448&originalType=binary&ratio=2&rotation=0&showTitle=false&size=245743&status=done&style=none&taskId=u91f8b902-495c-4539-bc78-40495b7c4e5&title=&width=724)]
功能分析
数据结构使用 list
代码

@ApiModel(value = "聚划算活动product信息")
@AllArgsConstructor
@NoArgsConstructor
@Data
public class Product {
    // 产品id
    private Long id;
    // 产品名称
    private String name;
    // 产品价格
    private Integer price;
    // 产品详情
    private String detail;
}
@Service
@Slf4j
public class JHSTaskService {

    private static final String JHS_KEY = "jhs";
    private static final String JHS_KEY_A = "jhs:a";
    private static final String JHS_KEY_B = "jhs:b";

    @Autowired
    RedisTemplate redisTemplate;

    /**
     * 模拟从数据库读取20件特价商品
     * @return 商品列表
     */
    private List<Product> getProductsFromMysql() {
        List<Product> list = new ArrayList<>();
        for (int i = 0; i < 20; i++) {
            Random random = new Random();
            int id = random.nextInt(1000);
            Product product = new Product((long) id, "product" + i, i, "detail");
            list.add(product);
        }
        log.info("模拟从数据库读取20件特价商品完成{}", list);
        return list;
    }

    @PostConstruct
    public void initJHSAB() {
        log.info("启动AB的定时器 天猫聚划算模拟开始===========");
        new Thread(() -> {
            while (true) {
                // 2.模拟从mysql查到数据,加到 redis 并返回给页面
                List<Product> list = getProductsFromMysql();

                redisTemplate.delete(JHS_KEY);
                redisTemplate.opsForList().leftPushAll(JHS_KEY, list);
                redisTemplate.expire(JHS_KEY, 86410L, TimeUnit.SECONDS);


                // 5.暂停一分钟,间隔1分钟执行一次,模拟聚划算一天执行的参加活动的品牌
                try {
                    Thread.sleep(1000* 60);
                } catch (InterruptedException e) {
                    e.printStackTrace();
                }
            }
        }, "t1").start();
    }

}

测试类

@SpringBootTest
@Slf4j
public class JhsTest {

    private static final String JHS_KEY = "jhs";
    private static final String JHS_KEY_A = "jhs:a";
    private static final String JHS_KEY_B = "jhs:b";

    @Autowired
    private RedisTemplate redisTemplate;

    @Test
    public void find() {
        int page = 1;
        int size = 10;

        List<Product> list = null;

        long start = (page - 1) * size;
        long end = start + size - 1;

        try {
            list = redisTemplate.opsForList().range(JHS_KEY, start, end);

            if (CollectionUtils.isEmpty(list)) {
                // TODO 走 mysql 查询
            }

            log.info("参加活动的商家={}", list);
        } catch (Exception e) {
            // 出异常了,一般 redis 宕机了或者redis网络抖动导致timeout
            log.error("jhs exception{}", e);
            e.printStackTrace();
            //  ..... 重试机制 再次查询 mysql
        }

        log.info(list.toString());

    }

}

测试方法,先跑主启动类(后台更新聚划算商品信息),然后手动执行测试类测试查询
问题分析
image.png
delete 执行间隙,这一瞬间缓存击穿,打到mysql
解决
image.png
[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-iHNGKLUq-1685671838186)(https://cdn.nlark.com/yuque/0/2023/png/2166858/1681974282104-3b51b71c-077b-49b3-9d84-16357fc9e625.png#averageHue=%23f7f5de&clientId=uffc70029-fcd7-4&from=paste&height=276&id=u83a5a017&originHeight=552&originWidth=1650&originalType=binary&ratio=2&rotation=0&showTitle=false&size=299175&status=done&style=none&taskId=ua3176d20-3558-4d6b-9115-cde38267bc0&title=&width=825)]

@PostConstruct
    public void initJHSAB() {
        log.info("启动AB的定时器 天猫聚划算模拟开始===========");
        new Thread(() -> {
            while (true) {
                // 2.模拟从mysql查到数据,加到 redis 并返回给页面
                List<Product> list = getProductsFromMysql();

//                redisTemplate.delete(JHS_KEY);
//                redisTemplate.opsForList().leftPushAll(JHS_KEY, list);
//                redisTemplate.expire(JHS_KEY, 86410L, TimeUnit.SECONDS);

                // 3.先更新B缓存并且让B缓存过期时间超过A时间,如果A突然失效了还有B兜底,防止击穿
                redisTemplate.delete(JHS_KEY_B);
                redisTemplate.opsForList().leftPushAll(JHS_KEY_B, list);
                redisTemplate.expire(JHS_KEY_B, 86410L, TimeUnit.SECONDS);

                // 4.再更新A缓存
                redisTemplate.delete(JHS_KEY_A);
                redisTemplate.opsForList().leftPushAll(JHS_KEY_A, list);
                redisTemplate.expire(JHS_KEY_A, 86400L, TimeUnit.SECONDS);

                // 5.暂停一分钟,间隔1分钟执行一次,模拟聚划算一天执行的参加活动的品牌
                try {
                    Thread.sleep(1000* 60);
                } catch (InterruptedException e) {
                    e.printStackTrace();
                }
            }
        }, "t1").start();
    }
@Test
    public void findAB() {
        int page = 1;
        int size = 10;

        List<Product> list = null;

        long start = (page - 1) * size;
        long end = start + size - 1;

        try {
            list = redisTemplate.opsForList().range(JHS_KEY_A, start, end);

            if (CollectionUtils.isEmpty(list)) {
                log.info("---------A缓存已经过期或活动结束了,记得人工修补,B缓存继续顶着");

                // A 没有来找 B
                list = redisTemplate.opsForList().range(JHS_KEY_B, start, end);

                if (CollectionUtils.isEmpty(list)) {
                    // TODO 走 mysql 查询
                }
            }

            log.info("参加活动的商家={}", list);
        } catch (Exception e) {
            // 出异常了,一般 redis 宕机了或者redis网络抖动导致timeout
            log.error("jhs exception{}", e);
            e.printStackTrace();
            //  ..... 重试机制 再次查询 mysql
        }

        log.info(list.toString());

    }

小总结

[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-OWLjYur9-1685671838187)(https://cdn.nlark.com/yuque/0/2023/png/2166858/1681974241969-d447706b-d158-4c22-91ea-a27c390964ec.png#averageHue=%23d7d7d5&clientId=uffc70029-fcd7-4&from=paste&height=286&id=pAqPF&originHeight=572&originWidth=1510&originalType=binary&ratio=2&rotation=0&showTitle=false&size=405627&status=done&style=none&taskId=uf24cfc2e-3aab-41e7-b4cf-b5a80ee7f17&title=&width=755)]

分布式锁

[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-KZZxiuf2-1685671838187)(https://cdn.nlark.com/yuque/0/2023/png/2166858/1681976292813-dc93414e-4496-4105-b00c-aa2701f715f8.png#averageHue=%23ededed&clientId=uffc70029-fcd7-4&from=paste&height=397&id=u180855b9&originHeight=794&originWidth=1474&originalType=binary&ratio=2&rotation=0&showTitle=false&size=355190&status=done&style=none&taskId=ub4d56170-d61a-4c75-b045-64af4b43d59&title=&width=737)]
q&a
image.png
image.png

分布式锁是什么以及超卖

[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-6Ggzqy5L-1685671838188)(https://cdn.nlark.com/yuque/0/2023/png/2166858/1681976874410-8efc7a31-b44b-4ff5-b952-1688f46e9801.png#averageHue=%23fbede8&clientId=uffc70029-fcd7-4&from=paste&height=96&id=u16cf3e7a&originHeight=192&originWidth=1954&originalType=binary&ratio=2&rotation=0&showTitle=false&size=192655&status=done&style=none&taskId=ubfd993ee-9534-42b7-942e-b19e6019c05&title=&width=977)]
一个分布式锁需要满足的条件和刚需

  • 独占性:任何时间只能有一个线程占有
  • 高可用:
    • 在redis集群环境下,不能因为一个节点挂了而出现获取锁和释放锁失败的情况
    • 高并发请求下,依旧性能 ok
  • 防死锁:杜绝死锁,必须有超时控制机制与撤销操作,有个兜底终止跳出方案
  • 不乱抢:防止张冠李戴,不能私下unlock别人的锁,只能自己加锁自己释放,自己约的锁含着泪也要自己解
  • 重入性:同一个节点的同一个线程,获得锁之后,他也可以再次获取这个锁

复习 setnx 指令
[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-ApiRcNDZ-1685671838188)(https://cdn.nlark.com/yuque/0/2023/png/2166858/1682039566358-afc9c8fd-afeb-46aa-998f-8513bb4ab341.png#averageHue=%23f8f2f1&clientId=uffc70029-fcd7-4&from=paste&height=113&id=ubab744d6&originHeight=226&originWidth=1928&originalType=binary&ratio=2&rotation=0&showTitle=false&size=168708&status=done&style=none&taskId=u936cc972-6a83-4d9e-9f08-5c014b7f0b1&title=&width=964)]
[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-QHUX9fOp-1685671838188)(https://cdn.nlark.com/yuque/0/2023/png/2166858/1682039587933-573a1621-fc73-4349-a426-c64bbb8e1c4b.png#averageHue=%23858282&clientId=uffc70029-fcd7-4&from=paste&height=525&id=uf4a4c580&originHeight=1050&originWidth=1440&originalType=binary&ratio=2&rotation=0&showTitle=false&size=453290&status=done&style=none&taskId=u99c342f7-db0f-4a42-80a0-4f5a7c14c7c&title=&width=720)]
setnx 来获取锁
image.png
之前没接触过 aqs 惭愧了
base案例
使用场景
[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-qm2IOXVz-1685671838188)(https://cdn.nlark.com/yuque/0/2023/png/2166858/1682039720380-0f6804e9-051e-4f03-8731-7f9260723cd5.png#averageHue=%23d2d2d2&clientId=uffc70029-fcd7-4&from=paste&height=41&id=u9c282798&originHeight=82&originWidth=1984&originalType=binary&ratio=2&rotation=0&showTitle=false&size=98571&status=done&style=none&taskId=u84c9d5d9-6683-44ad-b0ca-b98899b7f6e&title=&width=992)]

@Service
@Slf4j
public class InventoryService {

    @Autowired
    private StringRedisTemplate stringRedisTemplate;

    @Value("${server.port}")
    private String port;

    private Lock lock = new ReentrantLock();

    public String sale() {
        String retMessage = "";

        lock.lock();
        try {
            //1 查询库存信息
            String result = stringRedisTemplate.opsForValue().get("inventory001");
            //2 判断库存是否足够
            Integer inventoryNumber = result == null? 0 : Integer.parseInt(result);
            //3 扣除库存,每次减少一个
            if (inventoryNumber > 0) {
                inventoryNumber--;
                stringRedisTemplate.opsForValue().set("inventory001", String.valueOf(inventoryNumber));
                retMessage = "成功卖出一个商品,剩余库存" + inventoryNumber;
                System.out.println(retMessage + ",端口号:" + port);
            }else {
                retMessage = "商品卖完了";
            }
        } finally {
            lock.unlock();
        }

        return retMessage;
    }
}
@RestController
@Api(tags = "redis 分布式锁测试")
public class InventoryController {

    @Autowired
    private InventoryService inventoryService;

    @ApiOperation("扣除库存,一次卖一个")
    @GetMapping("/inventory/sale")
    public String sale() {
        return inventoryService.sale();
    }

}

GET http://localhost:6000/inventory/sale
image.png
拷贝出来一份
nginx 负载均衡

upstream stock {
server 127.0.0.1:6001 weight=1;
server 127.0.0.1:6000 weight=1;
}

server {
listen 9000;
server_name localhost;

location / {
proxy_pass http://stock;
index index.html index.htm;
}
}
GET http://localhost:9000/inventory/sale

image.png
jmeter 100个线程1s执行完毕
执行完毕查看 库存数
image.png
为什么加了synchronized 和 lock 但是没有控制住
image.png
image.png
3.1 版本

public String sale() {
        String retMessage = "";
        String key = "zzyRedisLock";
        String uuidValue = IdUtil.simpleUUID() + Thread.currentThread().getId();

        Boolean flag = stringRedisTemplate.opsForValue().setIfAbsent(key, uuidValue);

        if (!flag) {
            // 暂停20s,进行递归重试
            try {TimeUnit.MICROSECONDS.sleep(20);} catch (InterruptedException e) {e.printStackTrace();}
            sale();
        } else {
            // 抢锁成功的请求线程,进行正常的业务逻辑操作,扣减库存

            try {
                //1 查询库存信息
                String result = stringRedisTemplate.opsForValue().get("inventory001");
                //2 判断库存是否足够
                Integer inventoryNumber = result == null? 0 : Integer.parseInt(result);
                //3 扣除库存,每次减少一个
                if (inventoryNumber > 0) {
                    inventoryNumber--;
                    stringRedisTemplate.opsForValue().set("inventory001", String.valueOf(inventoryNumber));
                    retMessage = "成功卖出一个商品,剩余库存" + inventoryNumber;
                    System.out.println(retMessage + ",端口号:" + port);
                }else {
                    retMessage = "商品卖完了";
                }
            } finally {
                stringRedisTemplate.delete(key);
            }
        }

        return retMessage + ",端口号:" + port;
    }

image.png

public String sale() {
        String retMessage = "";
        String key = "zzyRedisLock";
        String uuidValue = IdUtil.simpleUUID() + Thread.currentThread().getId();


        // 不用递归了,高并发下使用自旋替代递归重试,使用while
        while (!stringRedisTemplate.opsForValue().setIfAbsent(key, uuidValue)) {
            try {
                TimeUnit.MICROSECONDS.sleep(20);
            } catch (InterruptedException e) {
                e.printStackTrace();
            }
        }
        // 抢锁成功的请求线程,进行正常的业务逻辑操作,扣减库存

        try {
            //1 查询库存信息
            String result = stringRedisTemplate.opsForValue().get("inventory001");
            //2 判断库存是否足够
            Integer inventoryNumber = result == null ? 0 : Integer.parseInt(result);
            //3 扣除库存,每次减少一个
            if (inventoryNumber > 0) {
                inventoryNumber--;
                stringRedisTemplate.opsForValue().set("inventory001", String.valueOf(inventoryNumber));
                retMessage = "成功卖出一个商品,剩余库存" + inventoryNumber;
                System.out.println(retMessage + ",端口号:" + port);
            } else {
                retMessage = "商品卖完了";
            }
        } finally {
            stringRedisTemplate.delete(key);
        }

        return retMessage + ",端口号:" + port;
    }

4.1 加锁和过期时间必须在同一行,保证原子性
image.png
锁超时误删除
image.png
[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-N4EMGG01-1685671838190)(https://cdn.nlark.com/yuque/0/2023/png/2166858/1682060980547-d6c10217-21a8-4428-82fe-75b43bfadff2.png#averageHue=%23f2f2f2&clientId=uffc70029-fcd7-4&from=paste&height=156&id=uf8c6b229&originHeight=312&originWidth=1844&originalType=binary&ratio=2&rotation=0&showTitle=false&size=138507&status=done&style=none&taskId=ua5ea7596-cd8a-4403-a8ef-17521c9d80c&title=&width=922)]
调用 lua 脚本,教程

  1. 实现 hello lua

[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-uScknlGR-1685671838190)(https://cdn.nlark.com/yuque/0/2023/png/2166858/1682061537818-1fae472e-66d6-4129-bd4d-afb5f0f59ce5.png#averageHue=%2332363e&clientId=uffc70029-fcd7-4&from=paste&height=36&id=u7af3799c&originHeight=72&originWidth=626&originalType=binary&ratio=2&rotation=0&showTitle=false&size=9706&status=done&style=none&taskId=ua62e025e-6aac-4a13-a0b4-46e5d78a00c&title=&width=313)]

  1. 实现 set expire get

[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-mv1HmVvy-1685671838191)(https://cdn.nlark.com/yuque/0/2023/png/2166858/1682061594948-b0bb3802-9976-4771-af97-36db66440a42.png#averageHue=%23393f42&clientId=uffc70029-fcd7-4&from=paste&height=92&id=ufa15141c&originHeight=184&originWidth=500&originalType=binary&ratio=2&rotation=0&showTitle=false&size=69946&status=done&style=none&taskId=ud1d17b32-5e50-41bd-834a-034b9556e79&title=&width=250)]
image.png

  1. mset 重在掌握 lua脚本中 args传参的用法

image.png

  1. 官网lua脚本
if redis.call("get",KEYS[1]) == ARGV[1] then
    return redis.call("del",KEYS[1])
else
    return 0
end

[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-Z9LrMMz4-1685671838191)(https://cdn.nlark.com/yuque/0/2023/png/2166858/1682063260381-0688fac8-81ee-40ba-be2e-7366c8003035.png#averageHue=%23f9f9f8&clientId=u45758c4c-5b6c-4&from=paste&height=274&id=u1dd49fe9&originHeight=548&originWidth=2090&originalType=binary&ratio=2&rotation=0&showTitle=false&size=367820&status=done&style=none&taskId=ud47efaee-57eb-4c94-8200-ab840814d5a&title=&width=1045)]
补充说明 if elseif else
[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-lTTXlZLU-1685671838191)(https://cdn.nlark.com/yuque/0/2023/png/2166858/1682063604633-8914f640-6b70-4117-8940-a0dd40a7eb1e.png#averageHue=%23e8dac8&clientId=u45758c4c-5b6c-4&from=paste&height=178&id=u19144be3&originHeight=356&originWidth=346&originalType=binary&ratio=2&rotation=0&showTitle=false&size=97515&status=done&style=none&taskId=u5ad6746e-66fd-4eaf-afc8-184ec0fd0b4&title=&width=173)]

// 改进点,修改为 Lua 脚本的 redis 分布式锁调用,必须保证原子性,参考官网脚本案例
String luaScript = "if redis.call('get',KEYS[1]) == ARGV[1] then " +
                    "return redis.call('del',KEYS[1]) " +
                "else " +
                    "return 0 " +
                "end";
stringRedisTemplate.execute(new DefaultRedisScript(luaScript, Boolean.class), Arrays.asList(key), uuidValue);

可重入

image.png

[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-kppAR4zx-1685671838192)(https://cdn.nlark.com/yuque/0/2023/png/2166858/1682221442150-9e9453d9-5a09-4ac7-a814-d9f1ea87770f.png#averageHue=%23efefef&clientId=u6e3390bc-0a0e-4&from=paste&height=81&id=u3a5e0324&originHeight=162&originWidth=1598&originalType=binary&ratio=2&rotation=0&showTitle=false&size=93338&status=done&style=none&taskId=uf6e0fe0e-c7fd-402f-9b6d-65143dffd0f&title=&width=799)]
image.png

image.png
[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-29YxD0E9-1685671838193)(https://cdn.nlark.com/yuque/0/2023/png/2166858/1682232430889-ee89ca97-9b46-49f1-b163-f0ea0f1e68b9.png#averageHue=%23f5f1f1&clientId=uf0783b8b-d4b2-4&from=paste&height=437&id=u73141d99&originHeight=874&originWidth=1536&originalType=binary&ratio=2&rotation=0&showTitle=false&size=438139&status=done&style=none&taskId=u8a16832e-6db6-4d8a-be09-ea3817a396f&title=&width=768)]
setnx只能沟通,但是使用 hset 可以解决可重入问题
下面编写案例
[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-IWhVmRqN-1685671838193)(https://cdn.nlark.com/yuque/0/2023/png/2166858/1682232604036-bcfcc352-e58c-4052-a7c6-4b4b212efddf.png#averageHue=%23f3f2f2&clientId=uf0783b8b-d4b2-4&from=paste&height=194&id=ua9f0f232&originHeight=388&originWidth=1706&originalType=binary&ratio=2&rotation=0&showTitle=false&size=194538&status=done&style=none&taskId=u2d7eedad-aa56-42d6-b5fa-9b98b170eb3&title=&width=853)]

-- 加锁的lua的脚本,对标我们的lock方法
-- 加锁 v1
if redis.call('exists', 'key') == 0 then
    redis.call('hset', 'key', 'uuid:threadid', 1)
    redis.call('expire', 'key', '50')
    return 1
elseif redis.call('hexists', 'key', 'uuid:threadid') == 1 then
    redis.call('hincrby', 'key', 'uuid:threadid', 1)
    redis.call('expire', 'key', '50')
    return 1
else
    return 0
end

-- v2 合并相同的代码,用incrby替代hset 简化代码
if redis.call('exists', 'key') == 0 or redis.call('hexists', 'key', 'uuid:threadid') == 1 then
    redis.call('hincrby', 'key', 'uuid:threadid', 1)
    redis.call('expire', 'key', '50')
    return 1
else
    return 0
end

-- v3 脚本ok,替换参数
if redis.call('exists', KEYS[1]) == 0 or redis.call('hexists', KEYS[1], ARGV[1]) == 1 then
    redis.call('hincrby', KEY[1], ARGV[1], 1)
    redis.call('expire', KEY[1], ARGV[2])
    return 1
else
    return 0
end

if redis.call('exists', KEYS[1]) == 0 or redis.call('hexists', KEYS[1], ARGV[1]) == 1 then redis.call('hincrby', KEY[1], ARGV[1], 1) redis.call('expire', KEY[1], ARGV[2]) return 1 else return 0 end

uuid:线程id
image.png

-- 解锁
if redis.call('hexists', 'key', 'uuid:threadid') == 0 then
    return nil
elseif redis.call('hincrby', 'key', 'uuid:threadid', -1) == 0 then
    return redis.call('del', 'key')
else
    return 0
end

给出测试代码

EVAL "if redis.call('exists', KEYS[1]) == 0 or redis.call('hexists', KEYS[1], ARGV[1]) == 1 then redis.call('hincrby', KEYS[1], ARGV[1], 1) redis.call('expire', KEYS[1], ARGV[2]) return 1 else return 0 end" 1 zzyyRedisLock 001:1 50
EVAL "if redis.call('exists', KEYS[1]) == 0 or redis.call('hexists', KEYS[1], ARGV[1]) == 1 then redis.call('hincrby', KEYS[1], ARGV[1], 1) redis.call('expire', KEYS[1], ARGV[2]) return 1 else return 0 end" 1 zzyyRedisLock 001:1 50
EVAL "if redis.call('exists', KEYS[1]) == 0 or redis.call('hexists', KEYS[1], ARGV[1]) == 1 then redis.call('hincrby', KEYS[1], ARGV[1], 1) redis.call('expire', KEYS[1], ARGV[2]) return 1 else return 0 end" 1 zzyyRedisLock 001:1 50
EVAL "if redis.call('exists', KEYS[1]) == 0 or redis.call('hexists', KEYS[1], ARGV[1]) == 1 then redis.call('hincrby', KEYS[1], ARGV[1], 1) redis.call('expire', KEYS[1], ARGV[2]) return 1 else return 0 end" 1 zzyyRedisLock 001:1 50
ttl zzyyRedisLock
HGET zzyyRedisLock 001:1
EVAL "if redis.call('hexists', KEYS[1], ARGV[1]) == 0 then return nil elseif redis.call('hincrby', KEYS[1], ARGV[1], -1) == 0 then return redis.call('del', KEYS[1]) else return 0 end" 1 zzyyRedisLock 001:1
EVAL "if redis.call('hexists', KEYS[1], ARGV[1]) == 0 then return nil elseif redis.call('hincrby', KEYS[1], ARGV[1], -1) == 0 then return redis.call('del', KEYS[1]) else return 0 end" 1 zzyyRedisLock 001:1
EVAL "if redis.call('hexists', KEYS[1], ARGV[1]) == 0 then return nil elseif redis.call('hincrby', KEYS[1], ARGV[1], -1) == 0 then return redis.call('del', KEYS[1]) else return 0 end" 1 zzyyRedisLock 001:1
EVAL "if redis.call('hexists', KEYS[1], ARGV[1]) == 0 then return nil elseif redis.call('hincrby', KEYS[1], ARGV[1], -1) == 0 then return redis.call('del', KEYS[1]) else return 0 end" 1 zzyyRedisLock 001:1

v7.0 lua 脚本实现 lock / unlock 阳哥准备讲讲设计模式?
[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-sQP8DPbi-1685671838193)(https://cdn.nlark.com/yuque/0/2023/png/2166858/1682237033274-f52367ec-9125-412e-8043-de75013a2e6e.png#averageHue=%23e9e9e9&clientId=uf0783b8b-d4b2-4&from=paste&height=224&id=ucac08c75&originHeight=448&originWidth=1556&originalType=binary&ratio=2&rotation=0&showTitle=false&size=239141&status=done&style=none&taskId=uaefc8ff5-e2b2-4925-b4d2-f855fc92c38&title=&width=778)]
[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-bGlqS0eH-1685671838193)(https://cdn.nlark.com/yuque/0/2023/png/2166858/1682238325380-4acd1dff-858b-4b20-8b7f-d8fd18a77329.png#averageHue=%23f2f1f1&clientId=uf0783b8b-d4b2-4&from=paste&height=278&id=ub053c9a6&originHeight=556&originWidth=1772&originalType=binary&ratio=2&rotation=0&showTitle=false&size=256330&status=done&style=none&taskId=u7038c252-48e0-430b-9081-2c3158d1d12&title=&width=886)]
image.png

image.png
[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-b7thS7Uz-1685671838194)(https://cdn.nlark.com/yuque/0/2023/png/2166858/1682259576566-246ba83f-ebb0-43b6-ae1c-ea01a690fa46.png#averageHue=%23e1d8d1&clientId=u00f4281e-78a3-4&from=paste&height=413&id=u40d3fd57&originHeight=826&originWidth=1760&originalType=binary&ratio=2&rotation=0&showTitle=false&size=807533&status=done&style=none&taskId=ud817b895-dfa2-4c49-9013-dbd55f94315&title=&width=880)]
factory

@Component
public class DistributedLockFactory {

    @Autowired
    private StringRedisTemplate stringRedisTemplate;

    private String lockName;

    private String uuid;

    public DistributedLockFactory() {
        this.uuid = IdUtil.simpleUUID();
    }

    public Lock getDistributedLock(String lockType) {
        if (lockType == null) {
            return null;
        }

        if (lockType.equalsIgnoreCase("REDIS")) {
            this.lockName = "zzyRedisLock";
            return new RedisDistributedLock(stringRedisTemplate, lockName, uuid);
        }else if (lockType.equalsIgnoreCase("ZOOKEEPER")) {
            this.lockName = "zzyZookeeperLock";
            // TODO zookeeper lock
            return null;
        } else if (lockType.equalsIgnoreCase("MYSQL")) {
            // TODO mysql lock
        }
        return null;
    }

}

service

/**
 * 自研分布式锁,实现 Lock 接口
 */
public class RedisDistributedLock implements Lock {

    private StringRedisTemplate stringRedisTemplate;

    private String lockName;

    private String uuidValue;

    private long expireTime;

    /*
    public RedisDistributedLock(StringRedisTemplate stringRedisTemplate, String lockName) {
        this.stringRedisTemplate = stringRedisTemplate;
        this.lockName = lockName;
        this.uuidValue = IdUtil.simpleUUID() + Thread.currentThread().getId();
        this.expireTime = 50L;
    }
     */
    public RedisDistributedLock(StringRedisTemplate stringRedisTemplate, String lockName, String uuid) {
        this.stringRedisTemplate = stringRedisTemplate;
        this.lockName = lockName;
        this.uuidValue = uuid + ":" + Thread.currentThread().getId();
        this.expireTime = 50L;
    }


    @Override
    public void lock() {
        tryLock();
    }

    @Override
    public boolean tryLock() {
        try {
            tryLock(-1L, TimeUnit.SECONDS);
        } catch (InterruptedException e) {
            throw new RuntimeException(e);
        }
        return false;
    }

    @Override
    public boolean tryLock(long time, TimeUnit unit) throws InterruptedException {
        if (time == -1L) {
            String luaScript = "if redis.call('exists', KEYS[1]) == 0 or redis.call('hexists', KEYS[1], ARGV[1]) == 1 then " +
                    "    redis.call('hincrby', KEYS[1], ARGV[1], 1) " +
                    "    redis.call('expire', KEYS[1], ARGV[2]) " +
                    "    return 1 " +
                    "else" +
                    "    return 0 " +
                    "end";
            while (!(Boolean) stringRedisTemplate.execute(new DefaultRedisScript(luaScript, Boolean.class),
                    Arrays.asList(lockName), uuidValue, String.valueOf(expireTime))){
                try {
                    Thread.sleep(60);
                }catch (InterruptedException e) {
                    e.printStackTrace();
                }
            }
            return true;
        }
        return false;
    }

    @Override
    public void unlock() {
        String luaScript = "if redis.call('hexists', KEYS[1], ARGV[1]) == 0 then " +
                "    return nil " +
                "elseif redis.call('hincrby', KEYS[1], ARGV[1], -1) == 0 then " +
                "    return redis.call('del', KEYS[1]) " +
                "else " +
                "    return 0 " +
                "end";
        // nil = false 1 = true 0 = false
        Long flag = (Long) stringRedisTemplate.execute(new DefaultRedisScript(luaScript, Long.class),
                Arrays.asList(lockName), uuidValue);
        if (null == flag) {
            throw new RuntimeException("锁不存在");
        }
    }

    // 下面两个暂时用不到,condition 中断
    @Override
    public Condition newCondition() {
        return null;
    }

    @Override
    public void lockInterruptibly() throws InterruptedException {

    }
}

service

@Service
@Slf4j
public class InventoryService {

    @Autowired
    private StringRedisTemplate stringRedisTemplate;

    @Value("${server.port}")
    private String port;

    @Autowired
    private DistributedLockFactory distributedLockFactory;

//    private Lock lock = new ReentrantLock();

    // v 7.0 lua 脚本
    public String sale() {
        String retMessage = "";
        String key = "zzyRedisLock";

        Lock redisLock = distributedLockFactory.getDistributedLock("redis");
        redisLock.lock();

        try {
            //1 查询库存信息
            String result = stringRedisTemplate.opsForValue().get("inventory001");
            //2 判断库存是否足够
            Integer inventoryNumber = result == null ? 0 : Integer.parseInt(result);
            //3 扣除库存,每次减少一个
            if (inventoryNumber > 0) {
                inventoryNumber--;
                stringRedisTemplate.opsForValue().set("inventory001", String.valueOf(inventoryNumber));
                retMessage = "成功卖出一个商品,剩余库存" + inventoryNumber;
                System.out.println(retMessage + ",端口号:" + port);
                testReEntry();
            } else {
                retMessage = "商品卖完了";
            }
        } finally {
            redisLock.unlock();
        }

        return retMessage + ",端口号:" + port;
    }

    private void testReEntry() {
        Lock redisLock = distributedLockFactory.getDistributedLock("redis");
        redisLock.lock();

        try {
            System.out.println("========测试可重入锁========");
        } finally {
            redisLock.unlock();
        }
    }
}

v 7.1 解决重入后 uuid 不一致问题
前面的 uuid 和机器绑定,后面的线程id绑定线程,二者即可唯一确定
v 8.0 自动实现锁续期
在拿到锁的同时添加一个后台程序,为ttl 的 1/3
续期后再调用自己添加 监视程序,保证能够一直续期

if redis.call('hexists', KEYS[1], ARGV[1]) == 1 then 
  return redis.call('expire', KEYS[1], ARGV[1])
else
  return 0
end
// tryLock 返回之前
// 新建一个后台扫描程序,来监视key目前的ttl,是否到我们规定的 1/2 1/3 来实现续期
resetExpire();

public void resetExpire() {
        String script = "if redis.call('hexists', KEYS[1], ARGV[1]) == 1 then " +
                "return redis.call('expire', KEYS[1], ARGV[2]) " +
                "else " +
                "return 0 " +
                "end";
        new Timer().schedule(new TimerTask() {

            @Override
            public void run() {
                if ((boolean) stringRedisTemplate.execute(new DefaultRedisScript(script, Boolean.class),
                        Arrays.asList(lockName), uuidValue, String.valueOf(expireTime))) {
                    resetExpire();
                }
            }
        }, (this.expireTime * 1000) / 3);
    }

[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-jjgBrPn3-1685671838194)(https://cdn.nlark.com/yuque/0/2023/png/2166858/1682388182488-0cd1bfb3-ed99-45ce-97d0-9cb301c7b475.png#averageHue=%23f8f7f7&clientId=u877eb479-ca93-4&from=paste&height=286&id=u595e1c5b&originHeight=572&originWidth=1808&originalType=binary&ratio=2&rotation=0&showTitle=false&size=244561&status=done&style=none&taskId=u2cbe2ed4-fd77-4786-b5ff-c531849b3a1&title=&width=904)]
[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-dCkwyWmK-1685671838195)(https://cdn.nlark.com/yuque/0/2023/png/2166858/1682388275350-d9904c0d-6021-4f89-ac59-1f21448edb0c.png#averageHue=%23f3f3f3&clientId=u877eb479-ca93-4&from=paste&height=381&id=u774668c0&originHeight=762&originWidth=914&originalType=binary&ratio=2&rotation=0&showTitle=false&size=149900&status=done&style=none&taskId=u438009c2-1dda-4c3c-b0f4-ed2597fa8a1&title=&width=457)]
[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-JdR95O8D-1685671838195)(https://cdn.nlark.com/yuque/0/2023/png/2166858/1682388385902-13a89ca0-b8c4-43a5-8755-dbba34a7551c.png#averageHue=%23f2f2f2&clientId=u877eb479-ca93-4&from=paste&height=378&id=u3f8a0b8a&originHeight=756&originWidth=1770&originalType=binary&ratio=2&rotation=0&showTitle=false&size=420324&status=done&style=none&taskId=ua11212a6-4d0e-4206-bf38-2b1c84eff6b&title=&width=885)]
[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-1PoVNxwF-1685671838195)(https://cdn.nlark.com/yuque/0/2023/png/2166858/1682388559271-abbcc49d-6bda-4a0f-a310-65a5fd4bcb36.png#averageHue=%23f6f6f5&clientId=uf225baa6-2497-4&from=paste&height=333&id=u0642d85d&originHeight=666&originWidth=1104&originalType=binary&ratio=2&rotation=0&showTitle=false&size=224750&status=done&style=none&taskId=u04ef716a-817d-4086-b747-2909f7c530c&title=&width=552)]

小总结

image.png
给自己鼓掌,整个跟下来了

redlock与 redisson

来由
redis 单机故障
[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-9fqcdQRJ-1685671838195)(https://cdn.nlark.com/yuque/0/2023/png/2166858/1682390857891-fbaeaad6-9a28-4dd5-aa89-11595405af01.png#averageHue=%23f4f1e5&clientId=uf225baa6-2497-4&from=paste&height=401&id=u2957a728&originHeight=802&originWidth=1794&originalType=binary&ratio=2&rotation=0&showTitle=false&size=781775&status=done&style=none&taskId=u1d5a464b-200a-4b18-a273-b132dd7e5ce&title=&width=897)]
[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-CdBhUsxF-1685671838196)(https://cdn.nlark.com/yuque/0/2023/png/2166858/1682391680599-63c8b4bc-e8d5-4a79-8206-b4b1fbefbea1.png#averageHue=%23dcdad4&clientId=uf225baa6-2497-4&from=paste&height=454&id=ua7daf018&originHeight=908&originWidth=1832&originalType=binary&ratio=2&rotation=0&showTitle=false&size=1125057&status=done&style=none&taskId=ueb2f5a5a-5afc-49bb-9b74-f7a6b0dcd82&title=&width=916)]
image.png
v9.0
[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-KK1zBuzy-1685671838196)(https://cdn.nlark.com/yuque/0/2023/png/2166858/1682394111373-01b2cc01-56be-4680-b707-68a3eb2ded39.png#averageHue=%23e4e1db&clientId=uf225baa6-2497-4&from=paste&height=236&id=ue4250108&originHeight=472&originWidth=1722&originalType=binary&ratio=2&rotation=0&showTitle=false&size=643336&status=done&style=none&taskId=u8070cbf8-65d6-4063-810e-32ffa31a75f&title=&width=861)]

<dependency>
   <groupId>org.redisson</groupId>
   <artifactId>redisson</artifactId>
   <version>3.20.1</version>
</dependency>  
@Bean
public Redisson redisson() {
    Config config = new Config();
    config.useSingleServer().setAddress("redis://127.0.0.1:6379").setDatabase(0);

    return (Redisson) Redisson.create(config);
}

看门狗,续期

缓存淘汰策略

[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-tlGBYVZX-1685671838196)(https://cdn.nlark.com/yuque/0/2023/png/2166858/1681893359239-27fd45e2-f682-4f12-b733-c7e052112ce0.png#averageHue=%23f3f3f3&clientId=uffc70029-fcd7-4&from=paste&height=382&id=u80933548&originHeight=764&originWidth=1648&originalType=binary&ratio=2&rotation=0&showTitle=false&size=291217&status=done&style=none&taskId=ua6e1c5cf-6d92-4438-979b-074c6b55de9&title=&width=824)]
第 5 章 Redis_OnebyWang的博客-CSDN博客

  1. 设置为最大内存的 3/4
  2. config get maxmemory 查看
    1. 配置文件

[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-arej5aqf-1685671838197)(https://cdn.nlark.com/yuque/0/2023/png/2166858/1681895641051-23fc2218-8e43-4f99-9fda-12c2e79f77aa.png#averageHue=%23000000&clientId=uffc70029-fcd7-4&from=paste&height=456&id=uc7d991b2&originHeight=912&originWidth=1362&originalType=binary&ratio=2&rotation=0&showTitle=false&size=173849&status=done&style=none&taskId=u3724513a-98af-4661-8cd9-e3f9a3cb057&title=&width=681)]

  1. 临时修改,重启后失效 config set maxmemory 104857600
  2. redis 将会报错:(error) OOM command not allowed when used memory > ‘maxmemory’
    1. 定时删除:创建kv的同时cpu会创建定时器,内存友好,cpu占用高
    2. 惰性删除:再次访问这个 key 的时候才会去检查过期时间,内存不友好
    3. 定时删除:每个一段时间去扫描一定数量的key是否过期

8种内存淘汰策略

  • noevition: 不操作
  • allkeys-lru: 对所有 key 使用 lru
  • volatile-lru: 对所有设置了过期时间的key lru
  • allkeys-random
  • volatile-random
  • volatile-ttl: 删除快要过期
  • allkeys-lfu lfu
  • volatile-lfu

修改maxmemory-policy
image.png

  1. lru算法手写,最近最少使用算法

[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-PqxqRM3S-1685671838197)(https://cdn.nlark.com/yuque/0/2023/png/2166858/1681896595166-442c2757-59e5-45b4-a3e2-f0d193769bbb.png#averageHue=%23d4cfca&clientId=uffc70029-fcd7-4&from=paste&height=179&id=ubd5a3cea&originHeight=358&originWidth=1384&originalType=binary&ratio=2&rotation=0&showTitle=false&size=317295&status=done&style=none&taskId=ud23d0767-2289-425e-aa6c-907241b39e5&title=&width=692)]
[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-1rqcfj30-1685671838197)(https://cdn.nlark.com/yuque/0/2023/png/2166858/1681896649275-2eb9b2fe-73a6-4f3f-a925-570667ff466c.png#averageHue=%23f3efee&clientId=uffc70029-fcd7-4&from=paste&height=83&id=uadf9144e&originHeight=166&originWidth=1460&originalType=binary&ratio=2&rotation=0&showTitle=false&size=97600&status=done&style=none&taskId=ufca8921f-9c55-46ac-959b-1bd6c3063c0&title=&width=730)]

redis 源码

redis 基本数据结构
[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-BrhpyPcG-1685671838198)(https://cdn.nlark.com/yuque/0/2023/png/2166858/1682558569676-26b4da94-f73b-4441-951b-d4911cdf831f.png#averageHue=%23eabc58&clientId=u6ffbd194-e80b-4&from=paste&height=338&id=ue3d58a9a&originHeight=676&originWidth=1388&originalType=binary&ratio=2&rotation=0&showTitle=false&size=214381&status=done&style=none&taskId=ude8efb63-3b71-4864-9c05-d944abcef0e&title=&width=694)]
新五大类型的底层类型,使用type测试
[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-eGeEJe2X-1685671838198)(https://cdn.nlark.com/yuque/0/2023/png/2166858/1682558665040-a006e40c-d087-47c6-93de-0ab2b9ae288a.png#averageHue=%23fafafa&clientId=u6ffbd194-e80b-4&from=paste&height=460&id=u6888493f&originHeight=920&originWidth=1094&originalType=binary&ratio=2&rotation=0&showTitle=false&size=165196&status=done&style=none&taskId=ud73cf606-43f2-4595-819c-75963d2b14c&title=&width=547)]
[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-IiTUrCLo-1685671838198)(https://cdn.nlark.com/yuque/0/2023/png/2166858/1682560683815-2579106c-eb77-41b7-a6f7-9f9aa4a670dc.png#averageHue=%23f2f3e1&clientId=u6ffbd194-e80b-4&from=paste&height=387&id=u89289fe6&originHeight=774&originWidth=1646&originalType=binary&ratio=2&rotation=0&showTitle=false&size=355103&status=done&style=none&taskId=u74829b6e-de21-497d-bc8c-17612e28664&title=&width=823)]
[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-UNl0wb34-1685671838198)(https://cdn.nlark.com/yuque/0/2023/png/2166858/1682560784572-525c4716-9d73-4a40-b549-83ecbde957aa.png#averageHue=%23f2f4e2&clientId=u6ffbd194-e80b-4&from=paste&height=379&id=u2c071d7e&originHeight=758&originWidth=1684&originalType=binary&ratio=2&rotation=0&showTitle=false&size=345030&status=done&style=none&taskId=u5d8da4b5-93a8-44c3-b80f-50ee6b5ede3&title=&width=842)]
[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-7Hd9JiYi-1685671838198)(https://cdn.nlark.com/yuque/0/2023/png/2166858/1682565549821-ee4e009e-48c1-4d29-955c-2997aadd4d38.png#averageHue=%23bebcb9&clientId=u6ffbd194-e80b-4&from=paste&height=426&id=ub4ac1fee&originHeight=852&originWidth=1798&originalType=binary&ratio=2&rotation=0&showTitle=false&size=764992&status=done&style=none&taskId=u75a47b68-b810-4c37-9ef0-2cfab8efeac&title=&width=899)]
[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-E7UTl6zl-1685671838199)(https://cdn.nlark.com/yuque/0/2023/png/2166858/1682577198401-2f06248a-cbcc-4a07-a932-2e18b44ee0f2.png#averageHue=%23f7f7f7&clientId=u6ffbd194-e80b-4&from=paste&height=434&id=u19047d26&originHeight=868&originWidth=1176&originalType=binary&ratio=2&rotation=0&showTitle=false&size=153256&status=done&style=none&taskId=u41c583f3-81e6-4f7a-9377-65b88b8aaf4&title=&width=588)]
set age 17 的案例
image.png
[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-fTgUthlv-1685671838199)(https://cdn.nlark.com/yuque/0/2023/png/2166858/1682665230699-df3f8be8-2526-4c2b-a3cd-a8b785412bd7.png#averageHue=%23d8d4d2&clientId=u6ffbd194-e80b-4&from=paste&height=311&id=u06a6249b&originHeight=622&originWidth=1908&originalType=binary&ratio=2&rotation=0&showTitle=false&size=726153&status=done&style=none&taskId=ub149c2cf-20b0-49d2-9486-f45952b6913&title=&width=954)]

string 数据结构三大类型

  • int

image.png
[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-Cyrw7Zzu-1685671838199)(https://cdn.nlark.com/yuque/0/2023/png/2166858/1682666081081-f28ff700-23e0-402d-951d-3523cdd9d6c2.png#averageHue=%230b0b0b&clientId=u6ffbd194-e80b-4&from=paste&height=73&id=uf7df4236&originHeight=146&originWidth=1132&originalType=binary&ratio=2&rotation=0&showTitle=false&size=27380&status=done&style=none&taskId=uaa4cb241-7289-4153-9400-41a683ed2ad&title=&width=566)]

  • embstr

[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-SkY8oPwa-1685671838200)(https://cdn.nlark.com/yuque/0/2023/png/2166858/1682666053015-ff52f4d2-d4c6-46c4-b555-6b850d388ddb.png#averageHue=%23e9e9e9&clientId=u6ffbd194-e80b-4&from=paste&height=118&id=u322b8b56&originHeight=236&originWidth=1646&originalType=binary&ratio=2&rotation=0&showTitle=false&size=145533&status=done&style=none&taskId=u11d3d2e9-8903-44b6-ba47-7d764763821&title=&width=823)]

  • raw

对应 redisObject 中的 encoding
下面简单测试下
[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-QBZAZP54-1685671838200)(https://cdn.nlark.com/yuque/0/2023/png/2166858/1682666146799-926217ea-1e26-4adb-b7f6-6734dd8210f9.png#averageHue=%23f8f1f0&clientId=u6ffbd194-e80b-4&from=paste&height=384&id=ude9eb53c&originHeight=768&originWidth=1204&originalType=binary&ratio=2&rotation=0&showTitle=false&size=418937&status=done&style=none&taskId=u9a74e484-3daf-4575-b733-e0f85ac5b8e&title=&width=602)]

string sds

[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-IAzMt1ZT-1685671838200)(https://cdn.nlark.com/yuque/0/2023/png/2166858/1682669865252-55844b81-3620-4a9a-af2f-660b735ea619.png#averageHue=%23dfdbd9&clientId=u6ffbd194-e80b-4&from=paste&height=436&id=ube57c11f&originHeight=872&originWidth=1866&originalType=binary&ratio=2&rotation=0&showTitle=false&size=641381&status=done&style=none&taskId=u70af7d27-0dab-4fdf-8140-5e07af6915c&title=&width=933)]
image.png
image.png
不同的alloc
image.png
image.png
image.png

[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-fH4ykWS9-1685671838203)(https://cdn.nlark.com/yuque/0/2023/png/2166858/1683165669712-c80b2ae3-9644-45d6-9425-b2ce949789c8.png#averageHue=%23f4f4f4&clientId=u6ffbd194-e80b-4&from=paste&height=552&id=u9173166a&originHeight=1104&originWidth=1088&originalType=binary&ratio=2&rotation=0&showTitle=false&size=329887&status=done&style=none&taskId=ud5e83ee1-7ad9-4867-b40d-b9cb38cf930&title=&width=544)]
暂时跳过下,感觉有点不好办

io多路复用

[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-ctth8nrS-1685671838204)(https://cdn.nlark.com/yuque/0/2023/png/2166858/1683168934650-1bde39e2-9a3a-4eec-b820-3135753d0c23.png#averageHue=%23f1f0f0&clientId=u6ffbd194-e80b-4&from=paste&height=294&id=u482f3d1e&originHeight=588&originWidth=2000&originalType=binary&ratio=2&rotation=0&showTitle=false&size=354590&status=done&style=none&taskId=u37978f66-914c-4f6b-a8d4-d7b113b26ed&title=&width=1000)]
[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-3xUDsAwT-1685671838204)(https://cdn.nlark.com/yuque/0/2023/png/2166858/1683169045118-e6e496e9-3b2f-42e4-aede-16fa14b25490.png#averageHue=%23dedede&clientId=u6ffbd194-e80b-4&from=paste&height=46&id=u51153c41&originHeight=92&originWidth=1542&originalType=binary&ratio=2&rotation=0&showTitle=false&size=68006&status=done&style=none&taskId=uf991c640-c585-425d-b3bc-f179d5d3cc7&title=&width=771)]
Redis高级之IO多路复用和epoll(十二)_晓风残月Lx的博客-CSDN博客

  • 0
    点赞
  • 0
    收藏
    觉得还不错? 一键收藏
  • 0
    评论
评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值