redis 重建主从记录

redis 重建主从变化

说明:监控某个从节点的变化,从节点清空了 appendonlydir/ 目录,删除了 dump.rdb,重启该从节点,监测变化如下。

第一阶段

重新启动该节点期间,监测数据目录下文件变化,首先生成 temp-xxx.rdb 文件,并且该文件一直增大,在此期间日志输出如下

Defaulted container "redis" out of: redis, sentinel, split-brain-fix, redis-exporter, config-init (init)
1:C 20 Nov 2023 11:10:14.899 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
1:C 20 Nov 2023 11:10:14.899 # Redis version=7.0.9, bits=64, commit=00000000, modified=0, pid=1, just started
1:C 20 Nov 2023 11:10:14.899 # Configuration loaded
1:S 20 Nov 2023 11:10:14.899 * monotonic clock: POSIX clock_gettime
1:S 20 Nov 2023 11:10:14.900 * Running mode=standalone, port=6379.
1:S 20 Nov 2023 11:10:14.900 # Server initialized
1:S 20 Nov 2023 11:10:14.901 * Creating AOF base file appendonly.aof.1.base.rdb on server start
1:S 20 Nov 2023 11:10:14.901 * Creating AOF incr file appendonly.aof.1.incr.aof on server start
1:S 20 Nov 2023 11:10:14.901 * Ready to accept connections
1:S 20 Nov 2023 11:10:14.901 * Connecting to MASTER 10.97.31.186:6379
1:S 20 Nov 2023 11:10:14.901 * MASTER <-> REPLICA sync started
1:S 20 Nov 2023 11:10:14.901 * Non blocking connect for SYNC fired the event.
1:S 20 Nov 2023 11:10:14.902 * Master replied to PING, replication can continue...
1:S 20 Nov 2023 11:10:14.902 * Partial resynchronization not possible (no cached master)
1:S 20 Nov 2023 11:10:19.796 * Full resync from master: 1db245adaf12b98d4dea329679a4b03b0a310272:1019321
1:S 20 Nov 2023 11:10:21.616 * MASTER <-> REPLICA sync: receiving streamed RDB from master with EOF to disk
kl logs -f redis-server-2
1:S 20 Nov 2023 10:20:02.717 * MASTER <-> REPLICA sync: Flushing old data
1:S 20 Nov 2023 10:20:02.717 * MASTER <-> REPLICA sync: Loading DB in memory
1:S 20 Nov 2023 10:20:02.718 * Loading RDB produced by version 7.0.9
1:S 20 Nov 2023 10:20:02.718 * RDB age 354 seconds
1:S 20 Nov 2023 10:20:02.718 * RDB memory usage when created 52476.26 Mb

文件变化,先产生 temp-xxx.rdb 文件,并且该文件一直增大

total 22G
drwxrwxrwx 4 root      root 4.0K Nov 20 18:14 ./
drwxr-xr-x 3 root      root 4.0K Nov 20 17:50 ../
drwxr-xr-x 2 openbayes root 4.0K Nov 20 18:14 appendonlydir/
drwxr-xr-x 2 openbayes root 4.0K Nov 20 18:14 conf/
-rw-r--r-- 1 openbayes root  22G Nov 20 18:20 temp-1700475247.1.rdb

等到跟主库的 dump.rdb 文件一样大的时候,瞬间生成 dump.rdb 文件

total 22G
drwxrwxrwx 4 root      root 4.0K Nov 20 18:20 ./
drwxr-xr-x 3 root      root 4.0K Nov 20 17:50 ../
drwxr-xr-x 2 openbayes root 4.0K Nov 20 18:14 appendonlydir/
drwxr-xr-x 2 openbayes root 4.0K Nov 20 18:14 conf/
-rw-r--r-- 1 openbayes root  22G Nov 20 18:20 dump.rdb

第二阶段

生成 dump.rbd 文件后内存用量会增加,这期间日志输出如下,文件不会有变化。

1:S 20 Nov 2023 11:16:16.342 * MASTER <-> REPLICA sync: Flushing old data
1:S 20 Nov 2023 11:16:16.342 * MASTER <-> REPLICA sync: Loading DB in memory
1:S 20 Nov 2023 11:16:16.343 * Loading RDB produced by version 7.0.9
1:S 20 Nov 2023 11:16:16.343 * RDB age 356 seconds
1:S 20 Nov 2023 11:16:16.343 * RDB memory usage when created 52477.00 Mb

第三阶段

生成 dump.rbd 文件后内存用量会增加,当内存用量增加到一定极值后,日志输出如下,开始第三阶段

1:S 20 Nov 2023 11:20:33.582 * Done loading RDB, keys loaded: 136197566, keys expired: 0.
1:S 20 Nov 2023 11:20:33.582 * MASTER <-> REPLICA sync: Finished with success
1:S 20 Nov 2023 11:20:33.582 * Creating AOF incr file temp-appendonly.aof.incr on background rewrite
1:S 20 Nov 2023 11:20:34.166 * Background append only file rewriting started by pid 579

生成 dump.rbd 文件后内存用量会增加,当内存用量增加到一定极值后开始在 appendonlydir 目录下产生 temp-appendonly.aof.incr 文件,该文件不增加

ll -h appendonlydir/
total 16K
drwxr-xr-x 2 openbayes root 4.0K Nov 20 18:24 ./
drwxrwxrwx 4 root      root 4.0K Nov 20 18:24 ../
-rw-r--r-- 1 openbayes root   88 Nov 20 18:14 appendonly.aof.1.base.rdb
-rw-r--r-- 1 openbayes root    0 Nov 20 18:14 appendonly.aof.1.incr.aof
-rw-r--r-- 1 openbayes root   88 Nov 20 18:14 appendonly.aof.manifest
-rw-r--r-- 1 openbayes root    0 Nov 20 18:24 temp-appendonly.aof.incr

同时在 appendonlydir 同级目录下产生 temp-rewriteaof-578.aof 文件,该文件一直增大

total 28G
drwxrwxrwx 4 root      root 4.0K Nov 20 18:24 ./
drwxr-xr-x 3 root      root 4.0K Nov 20 17:50 ../
drwxr-xr-x 2 openbayes root 4.0K Nov 20 18:24 appendonlydir/
drwxr-xr-x 2 openbayes root 4.0K Nov 20 18:14 conf/
-rw-r--r-- 1 openbayes root  22G Nov 20 18:20 dump.rdb
-rw-r--r-- 1 openbayes root 6.3G Nov 20 18:26 temp-rewriteaof-578.aof

第四阶段

当 temp-rewriteaof-578.aof 文件达到跟 dump.rbd 文件大一样大的时候, temp-rewriteaof-578.aof 文件消失,然后在 appendonlydir 同级目录下生成一个 temp-rewriteaof-bg-579.aof 文件

ll -h appendonlydir/
total 16
drwxr-xr-x 2 openbayes root 4096 Nov 20 19:20 ./
drwxrwxrwx 4 root      root 4096 Nov 20 19:27 ../
-rw-r--r-- 1 openbayes root   88 Nov 20 19:10 appendonly.aof.1.base.rdb
-rw-r--r-- 1 openbayes root    0 Nov 20 19:10 appendonly.aof.1.incr.aof
-rw-r--r-- 1 openbayes root   88 Nov 20 19:10 appendonly.aof.manifest
-rw-r--r-- 1 openbayes root    0 Nov 20 19:20 temp-appendonly.aof.incr
total 43G
drwxrwxrwx 4 root      root 4.0K Nov 20 19:27 ./
drwxr-xr-x 3 root      root 4.0K Nov 20 17:50 ../
drwxr-xr-x 2 openbayes root 4.0K Nov 20 19:20 appendonlydir/
drwxr-xr-x 2 openbayes root 4.0K Nov 20 19:10 conf/
-rw-r--r-- 1 openbayes root  22G Nov 20 19:16 dump.rdb
-rw-r--r-- 1 openbayes root  22G Nov 20 19:27 temp-rewriteaof-bg-579.aof

生成 temp-rewriteaof-bg-579.aof 文件的日志输出

579:C 20 Nov 2023 11:27:04.566 * Successfully created the temporary AOF base file temp-rewriteaof-bg-579.aof
579:C 20 Nov 2023 11:27:05.051 * Fork CoW for AOF rewrite: current 2 MB, peak 2 MB, average 2 MB
1:S 20 Nov 2023 11:27:05.753 * Background AOF rewrite terminated with success

片刻间,temp-rewriteaof-bg-579.aof 文件消失变成 appendonlydir 目录下 appendonly.aof.2.base.rdb 文件,temp-appendonly.aof.incr 变成 appendonly.aof.2.incr.aof 文件。并且清理原来的 appendonly.aof.1.base.rdb 和 appendonly.aof.1.incr.aof 文件

drwxrwxrwx 4 root      root 4.0K Nov 20 18:05 ./
drwxr-xr-x 3 root      root 4.0K Nov 20 17:28 ../
drwxr-xr-x 2 openbayes root 4.0K Nov 20 17:34 appendonlydir/
drwxr-xr-x 2 openbayes root 4.0K Nov 20 17:48 conf/
-rw-r--r-- 1 openbayes root  22G Nov 20 18:05 dump.rdb
ll -h appendonlydir/
total 22G
drwxr-xr-x 2 openbayes root 4.0K Nov 20 19:27 ./
drwxrwxrwx 4 root      root 4.0K Nov 20 19:27 ../
-rw-r--r-- 1 openbayes root  22G Nov 20 19:27 appendonly.aof.2.base.rdb
-rw-r--r-- 1 openbayes root    0 Nov 20 19:20 appendonly.aof.2.incr.aof
-rw-r--r-- 1 openbayes root   88 Nov 20 19:27 appendonly.aof.manifest

文件变化日志

1:S 20 Nov 2023 11:27:05.753 * Successfully renamed the temporary AOF base file temp-rewriteaof-bg-579.aof into appendonly.aof.2.base.rdb
1:S 20 Nov 2023 11:27:05.753 * Successfully renamed the temporary AOF incr file temp-appendonly.aof.incr into appendonly.aof.2.incr.aof

清理原来的 appendonly.aof.1.base.rdb 和 appendonly.aof.1.incr.aof 文件日志输出

1:S 20 Nov 2023 11:27:05.753 * Removing the history file appendonly.aof.1.incr.aof in the background
1:S 20 Nov 2023 11:27:05.754 * Removing the history file appendonly.aof.1.base.rdb in the background

该阶段日志输出如下

579:C 20 Nov 2023 11:27:04.566 * Successfully created the temporary AOF base file temp-rewriteaof-bg-579.aof
579:C 20 Nov 2023 11:27:05.051 * Fork CoW for AOF rewrite: current 2 MB, peak 2 MB, average 2 MB
1:S 20 Nov 2023 11:27:05.753 * Background AOF rewrite terminated with success
1:S 20 Nov 2023 11:27:05.753 * Successfully renamed the temporary AOF base file temp-rewriteaof-bg-579.aof into appendonly.aof.2.base.rdb
1:S 20 Nov 2023 11:27:05.753 * Successfully renamed the temporary AOF incr file temp-appendonly.aof.incr into appendonly.aof.2.incr.aof
1:S 20 Nov 2023 11:27:05.753 * Removing the history file appendonly.aof.1.incr.aof in the background
1:S 20 Nov 2023 11:27:05.754 * Removing the history file appendonly.aof.1.base.rdb in the background
1:S 20 Nov 2023 11:27:05.754 * Background AOF rewrite finished successfully

从库恢复期间,完整的日志输出如下

578:C 20 Nov 2023 10:30:50.987 * Successfully created the temporary AOF base file temp-rewriteaof-bg-578.aof
578:C 20 Nov 2023 10:30:51.486 * Fork CoW for AOF rewrite: current 2 MB, peak 2 MB, average 2 MB
1:S 20 Nov 2023 10:30:52.228 * Background AOF rewrite terminated with success
1:S 20 Nov 2023 10:30:52.228 * Successfully renamed the temporary AOF base file temp-rewriteaof-bg-578.aof into appendonly.aof.2.base.rdb
1:S 20 Nov 2023 10:30:52.228 * Successfully renamed the temporary AOF incr file temp-appendonly.aof.incr into appendonly.aof.2.incr.aof
1:S 20 Nov 2023 10:30:52.229 * Removing the history file appendonly.aof.1.incr.aof in the background
1:S 20 Nov 2023 10:30:52.229 * Removing the history file appendonly.aof.1.base.rdb in the background
1:S 20 Nov 2023 10:30:52.229 * Background AOF rewrite finished successfully

重建主从期间日志变化

kl logs -f redis-server-2
Defaulted container "redis" out of: redis, sentinel, split-brain-fix, redis-exporter, config-init (init)
1:C 20 Nov 2023 11:10:14.899 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
1:C 20 Nov 2023 11:10:14.899 # Redis version=7.0.9, bits=64, commit=00000000, modified=0, pid=1, just started
1:C 20 Nov 2023 11:10:14.899 # Configuration loaded
1:S 20 Nov 2023 11:10:14.899 * monotonic clock: POSIX clock_gettime
1:S 20 Nov 2023 11:10:14.900 * Running mode=standalone, port=6379.
1:S 20 Nov 2023 11:10:14.900 # Server initialized
1:S 20 Nov 2023 11:10:14.901 * Creating AOF base file appendonly.aof.1.base.rdb on server start
1:S 20 Nov 2023 11:10:14.901 * Creating AOF incr file appendonly.aof.1.incr.aof on server start
1:S 20 Nov 2023 11:10:14.901 * Ready to accept connections
1:S 20 Nov 2023 11:10:14.901 * Connecting to MASTER 10.97.31.186:6379
1:S 20 Nov 2023 11:10:14.901 * MASTER <-> REPLICA sync started
1:S 20 Nov 2023 11:10:14.901 * Non blocking connect for SYNC fired the event.
1:S 20 Nov 2023 11:10:14.902 * Master replied to PING, replication can continue...
1:S 20 Nov 2023 11:10:14.902 * Partial resynchronization not possible (no cached master)
1:S 20 Nov 2023 11:10:19.796 * Full resync from master: 1db245adaf12b98d4dea329679a4b03b0a310272:1019321
1:S 20 Nov 2023 11:10:21.616 * MASTER <-> REPLICA sync: receiving streamed RDB from master with EOF to disk


1:S 20 Nov 2023 11:16:16.342 * MASTER <-> REPLICA sync: Flushing old data
1:S 20 Nov 2023 11:16:16.342 * MASTER <-> REPLICA sync: Loading DB in memory
1:S 20 Nov 2023 11:16:16.343 * Loading RDB produced by version 7.0.9
1:S 20 Nov 2023 11:16:16.343 * RDB age 356 seconds
1:S 20 Nov 2023 11:16:16.343 * RDB memory usage when created 52477.00 Mb


1:S 20 Nov 2023 11:20:33.582 * Done loading RDB, keys loaded: 136197566, keys expired: 0.
1:S 20 Nov 2023 11:20:33.582 * MASTER <-> REPLICA sync: Finished with success
1:S 20 Nov 2023 11:20:33.582 * Creating AOF incr file temp-appendonly.aof.incr on background rewrite
1:S 20 Nov 2023 11:20:34.166 * Background append only file rewriting started by pid 579


579:C 20 Nov 2023 11:27:04.566 * Successfully created the temporary AOF base file temp-rewriteaof-bg-579.aof

579:C 20 Nov 2023 11:27:05.051 * Fork CoW for AOF rewrite: current 2 MB, peak 2 MB, average 2 MB
1:S 20 Nov 2023 11:27:05.753 * Background AOF rewrite terminated with success

1:S 20 Nov 2023 11:27:05.753 * Successfully renamed the temporary AOF base file temp-rewriteaof-bg-579.aof into appendonly.aof.2.base.rdb
1:S 20 Nov 2023 11:27:05.753 * Successfully renamed the temporary AOF incr file temp-appendonly.aof.incr into appendonly.aof.2.incr.aof

1:S 20 Nov 2023 11:27:05.753 * Removing the history file appendonly.aof.1.incr.aof in the background
1:S 20 Nov 2023 11:27:05.754 * Removing the history file appendonly.aof.1.base.rdb in the background
1:S 20 Nov 2023 11:27:05.754 * Background AOF rewrite finished successfully
  • 0
    点赞
  • 0
    收藏
    觉得还不错? 一键收藏
  • 打赏
    打赏
  • 0
    评论
Redis主从复制是一种常用的数据复制和高可用性方案,它通过将一个Redis实例(主节点)的数据复制到其他Redis实例(从节点)来实现数据的备份和读写分离。下面是Redis配置主从复制的步骤: 1. 配置主节点: - 打开主节点的配置文件redis.conf。 - 将配置项`bind`设置为主节点的IP地址。 - 将配置项`port`设置为主节点的端口号。 - 将配置项`daemonize`设置为yes,表示以守护进程方式运行。 - 将配置项`logfile`设置为日志文件路径。 - 将配置项`dir`设置为持久化文件的存储路径。 - 将配置项`appendonly`设置为yes,开启AOF持久化方式(可选)。 - 保存并关闭配置文件。 2. 启动主节点: - 打开终端,进入Redis安装目录。 - 执行命令`redis-server redis.conf`启动主节点。 3. 配置从节点: - 复制主节点的配置文件redis.conf到从节点。 - 打开从节点的配置文件redis.conf。 - 将配置项`bind`设置为从节点的IP地址。 - 将配置项`port`设置为从节点的端口号。 - 将配置项`daemonize`设置为yes,表示以守护进程方式运行。 - 将配置项`logfile`设置为日志文件路径。 - 将配置项`dir`设置为持久化文件的存储路径。 - 将配置项`appendonly`设置为yes,开启AOF持久化方式(可选)。 - 将配置项`replicaof`设置为主节点的IP地址和端口号,格式为`replicaof <masterip> <masterport>`。 - 保存并关闭配置文件。 4. 启动从节点: - 打开终端,进入Redis安装目录。 - 执行命令`redis-server redis.conf`启动从节点。 至此,Redis主从复制配置完成。主节点会将数据同步到从节点,从节点可以处理读请求,提高系统的读取性能和可用性。
评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包

打赏作者

时空无限

你的鼓励将是我创作的最大动力

¥1 ¥2 ¥4 ¥6 ¥10 ¥20
扫码支付:¥1
获取中
扫码支付

您的余额不足,请更换扫码支付或充值

打赏作者

实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值