Redis5.0相关问题及哨兵模式和cluster集群搭建

最新推荐文章于 2023-06-14 14:11:18 发布

無賴

最新推荐文章于 2023-06-14 14:11:18 发布

阅读量635

点赞数 2

文章标签： redis

本文链接：https://blog.csdn.net/weixin_43976886/article/details/109261713

版权

1、RDB和AOF的优缺点

RDB

优点

RDB 是一个非常紧凑（compact）的文件，体积小，传输速度快，适合灾备。
RDB 可以最大化 Redis 的性能：父进程在保存 RDB 文件时唯一要做的就是 fork 出一个子进程，然后这个子进程就会处理接下来的所有保存工作，父进程无须执行任何磁盘 I/O 操作。
RDB 在恢复大数据集时的速度比 AOF 的恢复速度要快很多。

缺点

RDB 丢数据将会丢掉两次持久化之间的所有数据～
当redis中数据集比较大时候，RDB由于RDB方式需要对数据进行完成拷贝并生成快照文件，fork的子进程会耗CPU，并且数据越大，RDB快照生成会越耗时。
RDB文件是特定的格式，阅读性差，由于格式固定，并且多版本之间存在不兼容情况。很难受～

AOF

优点

数据更完整，秒级数据丢失(取决于设置fsync策略)
兼容性较高，由于是基于redis通讯协议而形成的命令追加方式，无论何种版本的redis都兼容，再者aof文件是明文的，可阅读性较好。

缺点

数据文件体积较大,即使有重写机制，但是在相同的数据集情况下，AOF文件通常比RDB文件大。
相对RDB方式，AOF速度慢于RDB，并且在数据量大时候，恢复速度AOF速度也是慢于RDB。
由于频繁地将命令同步到文件中，AOF持久化对性能的影响相对RDB较大，但是对于我们来说是可以接受的。

2、master和slave同步过程

Redis的主从结构可以采用一主多从或者级联结构，Redis主从复制可以根据是否是全量分为全量同步和增量同步。下图为级联结构。

在这里插入图片描述

全量同步
Redis全量复制一般发生在Slave初始化阶段，这时Slave需要将Master上的所有数据都复制一份。具体步骤如下：

从服务器连接主服务器，发送 SYNC 命令
主服务器接收到 SYNC 命令后，开始执行 bgsave 命令生成 RDB 文件，并使用缓冲区记录此后执行的所有写命令
主服务 bgsave 执行完成后，向所有从服务器发送快照文件，并在发送期间继续记录被执行的写命令
从服务器收到快照文件后丢弃所有旧数据，载入收到快照
主服务器快照发送完成后开始向从服务器发送缓冲区中的写命令
从服务器完成对快照的载入，开始接收命令请求，并执行来自主服务器缓冲区的写命令

在这里插入图片描述

完成上面几个步骤后就完成了从服务器数据初始化的所有操作，从服务器此时可以接收来自用户的读请求。

增量同步
Redis增量复制是指Slave初始化后开始正常工作时主服务器发生的写操作同步到从服务器的过程。
增量复制的过程主要是主服务器每执行一个写命令就会向从服务器发送相同的写命令，从服务器接收并执行收到的写命令。

Redis主从同步策略
主从刚刚连接的时候，进行全量同步；全同步结束后，进行增量同步。当然，如果有需要，slave 在任何时候都可以发起全量同步。redis 策略是，无论如何，首先会尝试进行增量同步，如不成功，要求从机进行全量同步。

redis 的主从结构可以采用一主多从或者级联结构，redis 主从复制可以根据是否是全量分为全量同步和增量同步。

3、哨兵的使用和实现机制

哨兵机制

Redis的哨兵(sentinel) 系统用于管理多个 Redis 服务器,该系统执行以下三个任务:

监控(Monitoring)
哨兵(sentinel) 会不断地检查你的Master和Slave是否运作正常。
提醒(Notification)
- 当被监控的某个 Redis出现问题时, 哨兵(sentinel) 可以通过 API 向管理员或者其他应用程序发送通知。
自动故障迁移(Automatic failover)
- 当一个Master不能正常工作时，哨兵(sentinel) 会开始一次自动故障迁移操作,它会将失效Master的其中一个Slave升级为新的Master, 并让失效Master的其他Slave改为复制新的Master; 当客户端试图连接失效的Master时,集群也会向客户端返回新Master的地址,使得集群可以使用Master代替失效Master。
哨兵(sentinel) 是一个分布式系统,你可以在一个架构中运行多个哨兵(sentinel) 进程,这些进程使用流言协议(gossipprotocols)来接收关于Master是否下线的信息,并使用投票协议(agreement protocols)来决定是否执行自动故障迁移,以及选择哪个Slave作为新的Master。
每个哨兵(sentinel) 会向其它哨兵(sentinel)、master、slave定时发送消息,以确认对方是否”活”着,如果发现对方在指定时间(可配置)内未回应,则暂时认为对方已挂(所谓的”主观认为宕机” Subjective Down,简称sdown).
若“哨兵群”中的多数sentinel,都报告某一master没响应,系统才认为该master"彻底死亡"(即:客观上的真正down机,Objective Down,简称odown),通过一定的vote算法,从剩下的slave节点中,选一台提升为master,然后自动修改相关配置。
虽然哨兵(sentinel) 释出为一个单独的可执行文件 redis-sentinel ,但实际上它只是一个运行在特殊模式下的 Redis 服务器，你可以在启动一个普通 Redis 服务器时通过给定 --sentinel 选项来启动哨兵(sentinel)。

sentinel哨兵实战

1.环境准备

角色	主机	IP	端口
主库	master	10.0.0.81	6379
从库	slave1	10.0.0.82	6379
从库	slave2	10.0.0.83	6379

2.准备主从状态

# 从库slave1
[root@82 redis]# redis-cli 
127.0.0.1:6379> replicaof 10.0.0.81 6379
OK
127.0.0.1:6379> config set masterauth 123456
OK
127.0.0.1:6379> info replication
# Replication
role:slave
master_host:10.0.0.81
master_port:6379
master_link_status:up
master_last_io_seconds_ago:6
master_sync_in_progress:0
slave_repl_offset:2520
slave_priority:100
slave_read_only:1
connected_slaves:0
master_replid:c07e62736773826b3ae58dd8aa96fadfc99ff7a2
master_replid2:0000000000000000000000000000000000000000
master_repl_offset:2520
second_repl_offset:-1
repl_backlog_active:1
repl_backlog_size:1048576
repl_backlog_first_byte_offset:2507
repl_backlog_histlen:14


# 从库slave2
[root@83 redis]# redis-cli 
127.0.0.1:6379> replicaof 10.0.0.81 6379
OK
127.0.0.1:6379> config set masterauth 123456
OK
127.0.0.1:6379> info replication
# Replication
role:slave
master_host:10.0.0.81
master_port:6379
master_link_status:up
master_last_io_seconds_ago:8
master_sync_in_progress:0
slave_repl_offset:2590
slave_priority:100
slave_read_only:1
connected_slaves:0
master_replid:c07e62736773826b3ae58dd8aa96fadfc99ff7a2
master_replid2:0000000000000000000000000000000000000000
master_repl_offset:2590
second_repl_offset:-1
repl_backlog_active:1
repl_backlog_size:1048576
repl_backlog_first_byte_offset:2577
repl_backlog_histlen:14

#主库master查看
[root@81 redis]# redis-cli -a 123456
Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
127.0.0.1:6379> info replication
# Replication
role:master
connected_slaves:2
slave0:ip=10.0.0.82,port=6379,state=online,offset=2772,lag=0
slave1:ip=10.0.0.83,port=6379,state=online,offset=2772,lag=0
master_replid:c07e62736773826b3ae58dd8aa96fadfc99ff7a2
master_replid2:0000000000000000000000000000000000000000
master_repl_offset:2772
second_repl_offset:-1
repl_backlog_active:1
repl_backlog_size:1048576
repl_backlog_first_byte_offset:1
repl_backlog_histlen:2772

3.配置sentinel哨兵

# master
# sentinel.conf此配置文件是在官网下载的压缩包里
[root@81 ~]# cd /usr/local/src/redis-5.0.9/
[root@81 redis-5.0.9]# ls
00-RELEASENOTES  deps       README.md        runtest-moduleapi  tests
BUGS             INSTALL    redis.conf       runtest-sentinel   utils
CONTRIBUTING     Makefile   runtest          sentinel.conf
COPYING          MANIFESTO  runtest-cluster  src

# 拷贝到redis安装的目录下
[root@81 redis-5.0.9]# cp sentinel.conf /usr/local/redis/etc/
[root@81 redis-5.0.9]# cd /usr/local/redis/
[root@81 redis]# tree
.
├── bin
│   ├── redis-benchmark
│   ├── redis-check-aof
│   ├── redis-check-rdb
│   ├── redis-cli
│   ├── redis-sentinel -> redis-server
│   └── redis-server
├── data
│   └── dump_6379.rdb
├── dump.rdb
├── etc
│   ├── redis_6379.conf
│   └── sentinel.conf
├── logs
│   └── redis_6379.log
└── run
    └── redis_6379.pid


#编辑sentinel配置文件
[root@81 redis]# vim etc/sentinel.conf 
[root@81 redis]# egrep "^[^#| ]"  etc/sentinel.conf 
bind 0.0.0.0			# 监听 IP
port 26379				# 监听端口
daemonize yes			# 后台启用
pidfile /usr/local/redis/run/redis-sentinel_26379.pid	# pid文件路径
logfile /usr/local/redis/logs/sentinel_26379.log		# 日志文件路径
dir /usr/local/redis/data/								# 工作目录
sentinel monitor mymaster 10.0.0.81 6379 2				# 指定主服务器地址和端口，1为sentinel数量，需为服务器数量的n/2+1,但这里只有一台服务器做测试，所以直接写1即可
sentinel auth-pass mymaster 123456		# 此处是master的密码,注意此行要在上面行的下面
sentinel down-after-milliseconds mymaster 10000			# 当主服务器5秒中内没有回复，则认为主库宕机
sentinel parallel-syncs mymaster 1						# 当故障转移后，每次同时向新主同时发起复制操作节点个数，如果这个值较大，虽然一般不会阻塞主节点，但是会给网络和磁盘IO带来开销。
sentinel failover-timeout mymaster 180000				# 故障转移时，检测从库是否在线时间，如果在180秒内从库没有回应，则表示从库宕机，不参与新主的选举
sentinel deny-scripts-reconfig yes						# 禁止修改脚本




# slave1
[root@82 redis]# cp /usr/local/src/redis-5.0.9/sentinel.conf /usr/local/redis/etc/
[root@82 redis]# vim etc/sentinel.conf 
[root@82 redis]# egrep "^[^#| ]" etc/sentinel.conf 
bind 0.0.0.0
port 26379
daemonize yes
pidfile /usr/local/redis/run/redis-sentinel_26379.pid
logfile /usr/local/redis/logs/sentinel_26379.log
dir /usr/local/redis/data/
sentinel monitor mymaster 10.0.0.81 6379 2
sentinel auth-pass mymaster 123456		# 此处是master的密码,注意此行要在上面行的下面
sentinel down-after-milliseconds mymaster 10000
sentinel parallel-syncs mymaster 1
sentinel failover-timeout mymaster 180000
sentinel deny-scripts-reconfig yes


# slave2
[root@83 redis]# cp /usr/local/src/redis-5.0.9/sentinel.conf /usr/local/redis/etc/
[root@83 redis]# vim etc/sentinel.conf
[root@83 redis]# egrep "^[^#| ]" etc/sentinel.conf
bind 0.0.0.0
port 26379
daemonize yes
pidfile /usr/local/redis/run/redis-sentinel_26379.pid
logfile /usr/local/redis/logs/sentinel_26379.log
dir /usr/local/redis/data/
sentinel monitor mymaster 10.0.0.81 6379 2
sentinel auth-pass mymaster 123456
sentinel down-after-milliseconds mymaster 10000
sentinel parallel-syncs mymaster 1
sentinel failover-timeout mymaster 180000
sentinel deny-scripts-reconfig yes

4.启动sentinel

# 启动sentinel
[root@81 redis]# redis-sentinel /usr/local/redis/etc/sentinel.conf 
[root@82 redis]# redis-sentinel /usr/local/redis/etc/sentinel.conf 
[root@83 redis]# redis-sentinel /usr/local/redis/etc/sentinel.conf 

# 验证哨兵端口
[root@81 redis]# ss -ntl
State        Recv-Q       Send-Q              Local Address:Port                Peer Address:Port       
LISTEN       0            128                       0.0.0.0:26379                    0.0.0.0:*          
LISTEN       0            128                       0.0.0.0:6379                     0.0.0.0:*          
LISTEN       0            128                       0.0.0.0:5355                     0.0.0.0:*          
LISTEN       0            128                       0.0.0.0:22                       0.0.0.0:*          
LISTEN       0            128                          [::]:5355                        [::]:*          
LISTEN       0            128                          [::]:22                          [::]:*          

# 查看sentinel日志
# master
[root@81 redis]# tail -f /usr/local/redis/logs/sentinel_26379.log 
21748:X 23 Oct 2020 16:57:13.660 # +monitor master mymaster 10.0.0.81 6379 quorum 2
21755:X 23 Oct 2020 16:58:19.813 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
21755:X 23 Oct 2020 16:58:19.813 # Redis version=5.0.9, bits=64, commit=00000000, modified=0, pid=21755, just started
21755:X 23 Oct 2020 16:58:19.813 # Configuration loaded
21756:X 23 Oct 2020 16:58:19.968 * Increased maximum number of open files to 10032 (it was originally set to 1024).
21756:X 23 Oct 2020 16:58:19.968 # Could not create server TCP listening socket 0.0.0.0:26379: bind: Address already in use
21748:X 23 Oct 2020 16:58:56.613 * +sentinel sentinel 4314eea5b8ba37434ad38cec2a790c2d7bdfb2ce 10.0.0.82 26379 @ mymaster 10.0.0.81 6379
21748:X 23 Oct 2020 16:59:39.919 * +sentinel sentinel 25e0eac692c5b2e85c95dfdcc78daf31b4c1bcb2 10.0.0.83 26379 @ mymaster 10.0.0.81 6379
21748:X 23 Oct 2020 17:02:54.963 * +slave slave 10.0.0.82:6379 10.0.0.82 6379 @ mymaster 10.0.0.81 6379
21748:X 23 Oct 2020 17:04:55.509 * +slave slave 10.0.0.83:6379 10.0.0.83 6379 @ mymaster 10.0.0.81 6379



# slave1
[root@82 redis]# tail -f /usr/local/redis/logs/sentinel_26379.log 
9521:X 23 Oct 2020 16:58:54.746 # Configuration loaded
9522:X 23 Oct 2020 16:58:54.748 * Increased maximum number of open files to 10032 (it was originally set to 1024).
9522:X 23 Oct 2020 16:58:54.748 * Running mode=sentinel, port=26379.
9522:X 23 Oct 2020 16:58:54.748 # WARNING: The TCP backlog setting of 511 cannot be enforced because /proc/sys/net/core/somaxconn is set to the lower value of 128.
9522:X 23 Oct 2020 16:58:54.748 # Sentinel ID is 4314eea5b8ba37434ad38cec2a790c2d7bdfb2ce
9522:X 23 Oct 2020 16:58:54.748 # +monitor master mymaster 10.0.0.81 6379 quorum 2
9522:X 23 Oct 2020 16:58:55.482 * +sentinel sentinel 09cdd00f39c357c328f3a5d70a3f87bb5897173e 10.0.0.81 26379 @ mymaster 10.0.0.81 6379
9522:X 23 Oct 2020 16:59:04.808 # +sdown sentinel 25e0eac692c5b2e85c95dfdcc78daf31b4c1bcb2 10.0.0.83 26379 @ mymaster 10.0.0.81 6379
9522:X 23 Oct 2020 16:59:38.360 # -sdown sentinel 25e0eac692c5b2e85c95dfdcc78daf31b4c1bcb2 10.0.0.83 26379 @ mymaster 10.0.0.81 6379
9522:X 23 Oct 2020 17:00:29.814 * +reboot slave 10.0.0.82:6379 10.0.0.82 6379 @ mymaster 10.0.0.81 6379


# slave2
[root@83 redis]# tail -f /usr/local/redis/logs/sentinel_26379.log 
8609:X 23 Oct 2020 16:59:38.085 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
8609:X 23 Oct 2020 16:59:38.085 # Redis version=5.0.9, bits=64, commit=00000000, modified=0, pid=8609, just started
8609:X 23 Oct 2020 16:59:38.085 # Configuration loaded
8610:X 23 Oct 2020 16:59:38.087 * Increased maximum number of open files to 10032 (it was originally set to 1024).
8610:X 23 Oct 2020 16:59:38.088 * Running mode=sentinel, port=26379.
8610:X 23 Oct 2020 16:59:38.088 # WARNING: The TCP backlog setting of 511 cannot be enforced because /proc/sys/net/core/somaxconn is set to the lower value of 128.
8610:X 23 Oct 2020 16:59:38.088 # Sentinel ID is 25e0eac692c5b2e85c95dfdcc78daf31b4c1bcb2
8610:X 23 Oct 2020 16:59:38.088 # +monitor master mymaster 10.0.0.81 6379 quorum 2
8610:X 23 Oct 2020 16:59:38.252 * +sentinel sentinel 09cdd00f39c357c328f3a5d70a3f87bb5897173e 10.0.0.81 26379 @ mymaster 10.0.0.81 6379
8610:X 23 Oct 2020 17:00:29.803 * +reboot slave 10.0.0.82:6379 10.0.0.82 6379 @ mymaster 10.0.0.81 6379



#启动之后配置文件会发生改变
# master
[root@81 redis]# egrep "^[^#| ]" /usr/local/redis/etc/sentinel.conf 
bind 0.0.0.0
port 26379
daemonize yes
pidfile "/usr/local/redis/run/redis-sentinel_26379.pid"
logfile "/usr/local/redis/logs/sentinel_26379.log"
dir "/tmp"
sentinel myid 09cdd00f39c357c328f3a5d70a3f87bb5897173e
sentinel deny-scripts-reconfig yes
sentinel monitor mymaster 10.0.0.81 6379 2
sentinel down-after-milliseconds mymaster 10000
sentinel auth-pass mymaster 123456
sentinel config-epoch mymaster 0
protected-mode no
sentinel leader-epoch mymaster 0
sentinel known-replica mymaster 10.0.0.83 6379
sentinel known-replica mymaster 10.0.0.82 6379
sentinel known-sentinel mymaster 10.0.0.83 26379 25e0eac692c5b2e85c95dfdcc78daf31b4c1bcb2
sentinel known-sentinel mymaster 10.0.0.82 26379 4314eea5b8ba37434ad38cec2a790c2d7bdfb2ce
sentinel current-epoch 0


# slave1
[root@82 redis]# egrep "^[^#| ]" /usr/local/redis/etc/sentinel.conf 
bind 0.0.0.0
port 26379
daemonize yes
pidfile "/usr/local/redis/run/redis-sentinel_26379.pid"
logfile "/usr/local/redis/logs/sentinel_26379.log"
dir "/tmp"
sentinel myid 4314eea5b8ba37434ad38cec2a790c2d7bdfb2ce
sentinel deny-scripts-reconfig yes
sentinel monitor mymaster 10.0.0.81 6379 2
sentinel down-after-milliseconds mymaster 10000
sentinel auth-pass mymaster 123456
sentinel config-epoch mymaster 0
protected-mode no
sentinel leader-epoch mymaster 0
sentinel known-replica mymaster 10.0.0.83 6379
sentinel known-replica mymaster 10.0.0.82 6379
sentinel known-sentinel mymaster 10.0.0.83 26379 25e0eac692c5b2e85c95dfdcc78daf31b4c1bcb2
sentinel known-sentinel mymaster 10.0.0.81 26379 09cdd00f39c357c328f3a5d70a3f87bb5897173e
sentinel current-epoch 0


# slave2
[root@83 redis]# egrep "^[^#| ]" /usr/local/redis/etc/sentinel.conf 
bind 0.0.0.0
port 26379
daemonize yes
pidfile "/usr/local/redis/run/redis-sentinel_26379.pid"
logfile "/usr/local/redis/logs/sentinel_26379.log"
dir "/tmp"
sentinel myid 25e0eac692c5b2e85c95dfdcc78daf31b4c1bcb2
sentinel deny-scripts-reconfig yes
sentinel monitor mymaster 10.0.0.81 6379 2
sentinel down-after-milliseconds mymaster 10000
sentinel auth-pass mymaster 123456
sentinel config-epoch mymaster 0
protected-mode no
sentinel leader-epoch mymaster 0
sentinel known-replica mymaster 10.0.0.82 6379
sentinel known-replica mymaster 10.0.0.83 6379
sentinel known-sentinel mymaster 10.0.0.81 26379 09cdd00f39c357c328f3a5d70a3f87bb5897173e
sentinel known-sentinel mymaster 10.0.0.82 26379 4314eea5b8ba37434ad38cec2a790c2d7bdfb2ce
sentinel current-epoch 0

5.查看sentinel状态

# master
[root@81 redis]# redis-cli -p 26379
127.0.0.1:26379> info sentinel
# Sentinel
sentinel_masters:1
sentinel_tilt:0
sentinel_running_scripts:0
sentinel_scripts_queue_length:0
sentinel_simulate_failure_flags:0
master0:name=mymaster,status=ok,address=10.0.0.81:6379,slaves=2,sentinels=3


# slave1
[root@82 redis]# redis-cli -p 26379
127.0.0.1:26379> info sentinel
# Sentinel
sentinel_masters:1
sentinel_tilt:0
sentinel_running_scripts:0
sentinel_scripts_queue_length:0
sentinel_simulate_failure_flags:0
master0:name=mymaster,status=ok,address=10.0.0.81:6379,slaves=2,sentinels=3

6.停止sentinel

[root@db01 ~]# redis-cli -p 26379 shutdown

7.测试sentinel

# 关闭主库master的redis
127.0.0.1:6379> shutdown
not connected> 

# 查看其它从库主从状态
# slave1
127.0.0.1:6379> info replication
# Replication
role:master
connected_slaves:0
master_replid:a54d790b1933a55361a0b4724d71c1fa523dba23
master_replid2:542dbdb57d6cef7c21201e1aea318533d5aa75de
master_repl_offset:170505
second_repl_offset:103983
repl_backlog_active:1
repl_backlog_size:1048576
repl_backlog_first_byte_offset:1
repl_backlog_histlen:170505


# slave2
127.0.0.1:6379> info replication
# Replication
role:slave
master_host:10.0.0.82
master_port:6379
master_link_status:down
master_last_io_seconds_ago:-1
master_sync_in_progress:0
slave_repl_offset:103982
master_link_down_since_seconds:402
slave_priority:100
slave_read_only:1
connected_slaves:0
master_replid:542dbdb57d6cef7c21201e1aea318533d5aa75de
master_replid2:0000000000000000000000000000000000000000
master_repl_offset:103982
second_repl_offset:-1
repl_backlog_active:1
repl_backlog_size:1048576
repl_backlog_first_byte_offset:22921
repl_backlog_histlen:81062

# 查看sentinel日志
# slave1
[root@82 redis]# tail -f /usr/local/redis/logs/sentinel_26379.log 
9522:X 23 Oct 2020 17:11:52.722 # +sdown master mymaster 10.0.0.81 6379
9522:X 23 Oct 2020 17:11:52.751 # +new-epoch 1
9522:X 23 Oct 2020 17:11:52.754 # +vote-for-leader 09cdd00f39c357c328f3a5d70a3f87bb5897173e 1
9522:X 23 Oct 2020 17:11:52.794 # +odown master mymaster 10.0.0.81 6379 #quorum 3/2
9522:X 23 Oct 2020 17:11:52.794 # Next failover delay: I will not start a failover before Fri Oct 23 17:17:52 2020
9522:X 23 Oct 2020 17:11:53.353 # +config-update-from sentinel 09cdd00f39c357c328f3a5d70a3f87bb5897173e 10.0.0.81 26379 @ mymaster 10.0.0.81 6379
9522:X 23 Oct 2020 17:11:53.353 # +switch-master mymaster 10.0.0.81 6379 10.0.0.82 6379
9522:X 23 Oct 2020 17:11:53.353 * +slave slave 10.0.0.83:6379 10.0.0.83 6379 @ mymaster 10.0.0.82 6379
9522:X 23 Oct 2020 17:11:53.353 * +slave slave 10.0.0.81:6379 10.0.0.81 6379 @ mymaster 10.0.0.82 6379
9522:X 23 Oct 2020 17:12:03.362 # +sdown slave 10.0.0.81:6379 10.0.0.81 6379 @ mymaster 10.0.0.82 6379



# slave2
[root@83 redis]# tail -f /usr/local/redis/logs/sentinel_26379.log 
8610:X 23 Oct 2020 16:59:38.252 * +sentinel sentinel 09cdd00f39c357c328f3a5d70a3f87bb5897173e 10.0.0.81 26379 @ mymaster 10.0.0.81 6379
8610:X 23 Oct 2020 17:00:29.803 * +reboot slave 10.0.0.82:6379 10.0.0.82 6379 @ mymaster 10.0.0.81 6379
8610:X 23 Oct 2020 17:11:52.672 # +sdown master mymaster 10.0.0.81 6379
8610:X 23 Oct 2020 17:11:52.789 # +new-epoch 1
8610:X 23 Oct 2020 17:11:52.792 # +vote-for-leader 09cdd00f39c357c328f3a5d70a3f87bb5897173e 1
8610:X 23 Oct 2020 17:11:53.390 # +config-update-from sentinel 09cdd00f39c357c328f3a5d70a3f87bb5897173e 10.0.0.81 26379 @ mymaster 10.0.0.81 6379
8610:X 23 Oct 2020 17:11:53.390 # +switch-master mymaster 10.0.0.81 6379 10.0.0.82 6379
8610:X 23 Oct 2020 17:11:53.391 * +slave slave 10.0.0.83:6379 10.0.0.83 6379 @ mymaster 10.0.0.82 6379
8610:X 23 Oct 2020 17:11:53.391 * +slave slave 10.0.0.81:6379 10.0.0.81 6379 @ mymaster 10.0.0.82 6379
8610:X 23 Oct 2020 17:12:03.478 # +sdown slave 10.0.0.81:6379 10.0.0.81 6379 @ mymaster 10.0.0.82 6379

8.恢复故障节点

#修复故障节点
[root@81 redis]# systemctl start redis
[root@81 redis]# redis-cli -p 6379 -a 123456
Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
127.0.0.1:6379> info replication
# Replication
role:slave
master_host:10.0.0.82
master_port:6379
master_link_status:up
master_last_io_seconds_ago:1
master_sync_in_progress:0
slave_repl_offset:224407
slave_priority:100
slave_read_only:1
connected_slaves:0
master_replid:a54d790b1933a55361a0b4724d71c1fa523dba23
master_replid2:0000000000000000000000000000000000000000
master_repl_offset:224407
second_repl_offset:-1
repl_backlog_active:1
repl_backlog_size:1048576
repl_backlog_first_byte_offset:221977
repl_backlog_histlen:2431
127.0.0.1:6379> 

#查看日志，显示原主变成从
[root@81 redis]# tail -f /usr/local/redis/logs/sentinel_26379.log
21748:X 23 Oct 2020 17:21:54.318 * +convert-to-slave slave 10.0.0.81:6379 10.0.0.81 6379 @ mymaster 10.0.0.82 6379

9.sentinel管理命令（不常用）

#连接sentinel管理端口
[root@db01 ~]# redis-cli -p 26379

#检测状态，返回PONG
127.0.0.1:26379> ping
PONG

#列出所有被监视的主服务器
127.0.0.1:26379> SENTINEL masters

#列出所有被监视的从服务器
127.0.0.1:26379> SENTINEL slaves mymaster

#返回给定名字的主服务器的IP地址和端口号
127.0.0.1:26379> SENTINEL get-master-addr-by-name mymaster
1) "10.0.0.82"
2) "6379"

#重置所有名字和给定模式
127.0.0.1:26379> SENTINEL reset mymaster

#当主服务器失效时，在不询问其他Sentinel意见的情况下，强制开始一次自动故障迁移。
127.0.0.1:26379> SENTINEL failover mymaster

10.设置权重，指定主库的优先级

# 恢复之前的主从关系

#登录redis,查看slave1的权重
127.0.0.1:6379> CONFIG GET slave-priority
1) "slave-priority"
2) "100"

#修改salve1的权重值为0
127.0.0.1:6379> CONFIG set slave-priority 0
OK

127.0.0.1:6379> CONFIG GET slave-priority
1) "slave-priority"
2) "0"
#权重值越低越不会优先切换为主库，如果设置为0，则永远不会选举成为新master


#在master上登录sentinel，强制开始一次自动故障迁移
127.0.0.1:26379> SENTINEL failover mymaster
OK


#再次查看，主库为slave2
# 登录数据库模式下
[root@83 redis]# redis-cli -p 6379
127.0.0.1:6379> info replication
# Replication
role:master
connected_slaves:1
slave0:ip=10.0.0.81,port=6379,state=online,offset=510243,lag=0
master_replid:3a9f17ff026236b73c06d449975b1fac0464da3d
master_replid2:052103610ff1b742a30a6f932175ae1b81ed5402
master_repl_offset:510376
second_repl_offset:431847
repl_backlog_active:1
repl_backlog_size:1048576
repl_backlog_first_byte_offset:391644
repl_backlog_histlen:118733

# 登录sentinel模式下
[root@83 redis]# redis-cli -p 26379
127.0.0.1:26379> info sentinel
# Sentinel
sentinel_masters:1
sentinel_tilt:0
sentinel_running_scripts:0
sentinel_scripts_queue_length:0
sentinel_simulate_failure_flags:0
master0:name=mymaster,status=ok,address=10.0.0.83:6379,slaves=2,sentinels=3


# 查看日志
[root@83 redis]# tail -f /usr/local/redis/logs/sentinel_26379.log 
8610:X 23 Oct 2020 17:33:16.611 # +config-update-from sentinel 09cdd00f39c357c328f3a5d70a3f87bb5897173e 10.0.0.81 26379 @ mymaster 10.0.0.82 6379
8610:X 23 Oct 2020 17:33:16.611 # +switch-master mymaster 10.0.0.82 6379 10.0.0.81 6379
8610:X 23 Oct 2020 17:33:16.612 * +slave slave 10.0.0.83:6379 10.0.0.83 6379 @ mymaster 10.0.0.81 6379
8610:X 23 Oct 2020 17:33:16.612 * +slave slave 10.0.0.82:6379 10.0.0.82 6379 @ mymaster 10.0.0.81 6379
8610:X 23 Oct 2020 17:39:41.877 # +new-epoch 3
8610:X 23 Oct 2020 17:39:42.516 # +config-update-from sentinel 09cdd00f39c357c328f3a5d70a3f87bb5897173e 10.0.0.81 26379 @ mymaster 10.0.0.81 6379
8610:X 23 Oct 2020 17:39:42.517 # +switch-master mymaster 10.0.0.81 6379 10.0.0.83 6379
8610:X 23 Oct 2020 17:39:42.517 * +slave slave 10.0.0.82:6379 10.0.0.82 6379 @ mymaster 10.0.0.83 6379
8610:X 23 Oct 2020 17:39:42.517 * +slave slave 10.0.0.81:6379 10.0.0.81 6379 @ mymaster 10.0.0.83 6379
8610:X 23 Oct 2020 17:46:49.250 * +reboot slave 10.0.0.82:6379 10.0.0.82 6379 @ mymaster 10.0.0.83 6379



# 重新启动master
[root@81 redis]# systemctl start redis
[root@81 redis]# redis-cli -p 6379 -a 123456
Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
127.0.0.1:6379> info replication
# Replication
role:master
connected_slaves:0
master_replid:4fcfeea5e6d183b8b3c0c0f12724dfb34cd5f392
master_replid2:0000000000000000000000000000000000000000
master_repl_offset:0
second_repl_offset:-1
repl_backlog_active:0
repl_backlog_size:1048576
repl_backlog_first_byte_offset:0
repl_backlog_histlen:0
127.0.0.1:6379> info replication
# Replication
role:slave
master_host:10.0.0.83
master_port:6379
master_link_status:up
master_last_io_seconds_ago:1
master_sync_in_progress:0
slave_repl_offset:79523
slave_priority:100
slave_read_only:1
connected_slaves:0
master_replid:f051eb0e01ab51c6f25c65c7dea57282edc31457
master_replid2:0000000000000000000000000000000000000000
master_repl_offset:79523
second_repl_offset:-1
repl_backlog_active:1
repl_backlog_size:1048576
repl_backlog_first_byte_offset:78437
repl_backlog_histlen:1087


# 查看slave2日志
[root@83 redis]# tail -f /usr/local/redis/logs/sentinel_26379.log 
8610:X 23 Oct 2020 18:48:20.595 # -odown master mymaster 10.0.0.81 6379
8610:X 23 Oct 2020 18:49:29.327 * +reboot master mymaster 10.0.0.81 6379
8610:X 23 Oct 2020 18:49:29.388 # -sdown master mymaster 10.0.0.81 6379
8610:X 23 Oct 2020 18:50:09.390 # +sdown master mymaster 10.0.0.81 6379
8610:X 23 Oct 2020 18:51:19.599 # +failover-end-for-timeout master mymaster 10.0.0.81 6379
8610:X 23 Oct 2020 18:51:19.599 # +failover-end master mymaster 10.0.0.81 6379
8610:X 23 Oct 2020 18:51:19.599 * +slave-reconf-sent-be slave 10.0.0.82:6379 10.0.0.82 6379 @ mymaster 10.0.0.81 6379
8610:X 23 Oct 2020 18:51:19.599 # +switch-master mymaster 10.0.0.81 6379 10.0.0.83 6379
8610:X 23 Oct 2020 18:51:19.599 * +slave slave 10.0.0.82:6379 10.0.0.82 6379 @ mymaster 10.0.0.83 6379
8610:X 23 Oct 2020 18:51:19.600 * +slave slave 10.0.0.81:6379 10.0.0.81 6379 @ mymaster 10.0.0.83 6379

4、redis cluster集群创建和使用

Redis5.0.9 集群搭建

准备工作

本次集群准备六台服务器（虚拟机也可以），每台服务器上面安装一个 Redis 实例，端口是 6379，如下：

IP	角色	端口	虚拟机
10.0.0.82	Redis_master1	6379	centos82
10.0.0.83	Redis_master2	6379	centos83
10.0.0.84	Redis_master3	6379	centos84
10.0.0.85	Redis_slave1	6379	centos85
10.0.0.86	Redis_slave2	6379	centos86
10.0.0.87	Redis_slave3	6379	centos87
10.0.0.81	Ansible		centos81
10.0.0.88	Redis_master4（备用）	6379	centos87
10.0.0.89	Redis_slave4（备用）	6379	centos87

Redis 安装

Redis 官网下载安装包

wget https://download.redis.io/releases/redis-5.0.9.tar.gz
tar xf redis-5.0.9.tar.gz
cd redis-5.0.9
make install

集群操作

1、Redis 配置文件

# centos 82
[root@82 redis]# vim etc/redis.conf 
[root@82 redis]# egrep "^[^#| ]" etc/redis.conf 
bind  0.0.0.0			---->监听IP
protected-mode yes
port 6379				---->监听端口
tcp-backlog 511
timeout 0
tcp-keepalive 300
daemonize  yes			---->开启后台运行
supervised no
pidfile  /usr/local/redis/run/redis_6379.pid
loglevel notice
logfile /usr/local/redis/logs/redis_6379.log
databases 16
always-show-logo yes
save 900 1
save 300 10
save 60 10000
stop-writes-on-bgsave-error yes
rdbcompression yes
rdbchecksum yes
dbfilename dump_6379.rdb
dir /usr/local/redis/data/
replica-serve-stale-data yes
replica-read-only yes
repl-diskless-sync no
repl-diskless-sync-delay 5
repl-disable-tcp-nodelay no
replica-priority 100
lazyfree-lazy-eviction no
lazyfree-lazy-expire no
lazyfree-lazy-server-del no
replica-lazy-flush no
appendonly no
appendfilename "appendonly.aof"
appendfsync everysec
no-appendfsync-on-rewrite no
auto-aof-rewrite-percentage 100
auto-aof-rewrite-min-size 64mb
aof-load-truncated yes
aof-use-rdb-preamble yes
lua-time-limit 5000
cluster-enabled yes			---->开启cluster集群模式
slowlog-log-slower-than 10000
slowlog-max-len 128
latency-monitor-threshold 0
notify-keyspace-events ""
hash-max-ziplist-entries 512
hash-max-ziplist-value 64
list-max-ziplist-size -2
list-compress-depth 0
set-max-intset-entries 512
zset-max-ziplist-entries 128
zset-max-ziplist-value 64
hll-sparse-max-bytes 3000
stream-node-max-bytes 4096
stream-node-max-entries 100
activerehashing yes
client-output-buffer-limit normal 0 0 0
client-output-buffer-limit replica 256mb 64mb 60
client-output-buffer-limit pubsub 32mb 8mb 60
hz 10
dynamic-hz yes
aof-rewrite-incremental-fsync yes
rdb-save-incremental-fsync yes

将 Redis 目录下的 redis.conf 配置文件 copy 到每台虚拟机相关目录下

[root@82 redis]# scp /usr/local/redis/etc/redis.conf 10.0.0.83:/usr/local/redis/etc/
[root@82 redis]# scp /usr/local/redis/etc/redis.conf 10.0.0.84:/usr/local/redis/etc/
[root@82 redis]# scp /usr/local/redis/etc/redis.conf 10.0.0.85:/usr/local/redis/etc/
[root@82 redis]# scp /usr/local/redis/etc/redis.conf 10.0.0.86:/usr/local/redis/etc/
[root@82 redis]# scp /usr/local/redis/etc/redis.conf 10.0.0.87:/usr/local/redis/etc/

2、启动Redis

[root@82 redis]# systemctl start redis
[root@82 redis]# ss -ntl
State      Recv-Q     Send-Q         Local Address:Port           Peer Address:Port     
LISTEN     0          128                  0.0.0.0:22                  0.0.0.0:*        
LISTEN     0          128                  0.0.0.0:16379               0.0.0.0:*        
LISTEN     0          128                  0.0.0.0:6379                0.0.0.0:*        
LISTEN     0          128                  0.0.0.0:5355                0.0.0.0:*        
LISTEN     0          128                     [::]:22                     [::]:*        
LISTEN     0          128                     [::]:5355                   [::]:*

3、查看生成的node文件

Redis Cluster集群相关命令，执行下面命令需要先登录redis：

集群

cluster info ：打印集群的信息
cluster nodes ：列出集群当前已知的所有节点（ node），以及这些节点的相关信息。

节点

cluster meet ：将 ip 和 port 所指定的节点添加到集群当中，让它成为集群的一份子。
cluster forget <node_id> ：从集群中移除 node_id 指定的节点。
cluster replicate <master_node_id> ：将当前从节点设置为 node_id 指定的master节点的slave节点。只能针对slave节点操作。
cluster saveconfig ：将节点的配置文件保存到硬盘里面。

槽(slot)

cluster addslots [slot …] ：将一个或多个槽（ slot）指派（ assign）给当前节点。
cluster delslots [slot …] ：移除一个或多个槽对当前节点的指派。
cluster flushslots ：移除指派给当前节点的所有槽，让当前节点变成一个没有指派任何槽的节点。
cluster setslot node <node_id> ：将槽 slot 指派给 node_id 指定的节点，如果槽已经指派给另一个节点，那么先让另一个节点删除该槽>，然后再进行指派。
cluster setslot migrating <node_id> ：将本节点的槽 slot 迁移到 node_id 指定的节点中。
cluster setslot importing <node_id> ：从 node_id 指定的节点中导入槽 slot 到本节点。
cluster setslot stable ：取消对槽 slot 的导入（ import）或者迁移（ migrate）。

键

cluster keyslot ：计算键 key 应该被放置在哪个槽上。
cluster countkeysinslot ：返回槽 slot 目前包含的键值对数量。
cluster getkeysinslot ：返回 count 个 slot 槽中的键。

cluster info：打印集群的信息。

cluster nodes：列出集群当前已知的所有节点（node）的相关信息。

cluster meet <ip> <port>：将ip和port所指定的节点添加到集群当中。

cluster addslots <slot> [slot ...]：将一个或多个槽（slot）指派（assign）给当前节点。

cluster delslots <slot> [slot ...]：移除一个或多个槽对当前节点的指派。

cluster slots：列出槽位、节点信息。

cluster slaves <node_id>：列出指定节点下面的从节点信息。

cluster replicate <node_id>：将当前节点设置为指定节点的从节点。

cluster saveconfig：手动执行命令保存保存集群的配置文件，集群默认在配置修改的时候会自动保存配置文件。

cluster keyslot <key>：列出key被放置在哪个槽上。

cluster flushslots：移除指派给当前节点的所有槽，让当前节点变成一个没有指派任何槽的节点。

cluster countkeysinslot <slot>：返回槽目前包含的键值对数量。

cluster getkeysinslot <slot> <count>：返回count个槽中的键。

cluster setslot <slot> node <node_id> 将槽指派给指定的节点，如果槽已经指派给另一个节点，那么先让另一个节点删除该槽，然后再进行指派。 

cluster setslot <slot> migrating <node_id> 将本节点的槽迁移到指定的节点中。  

cluster setslot <slot> importing <node_id> 从 node_id 指定的节点中导入槽 slot 到本节点。 

cluster setslot <slot> stable 取消对槽 slot 的导入（import）或者迁移（migrate）。 

cluster failover：手动进行故障转移。

cluster forget <node_id>：从集群中移除指定的节点，这样就无法完成握手，过期时为60s，60s后两节点又会继续完成握手。

cluster reset [HARD|SOFT]：重置集群信息，soft是清空其他节点的信息，但不修改自己的id，hard还会修改自己的id，不传该参数则使用soft方式。

cluster count-failure-reports <node_id>：列出某个节点的故障报告的长度。

cluster SET-CONFIG-EPOCH：设置节点epoch，只有在节点加入集群前才能设置。

节点启动后会在相应的目录生成节点集群配置信息cluster-config，即nodes.conf文件
# centos82
[root@82 redis]# tree
.
├── bin
│   ├── redis-benchmark
│   ├── redis-check-aof
│   ├── redis-check-rdb
│   ├── redis-cli
│   ├── redis-sentinel -> redis-server
│   └── redis-server
├── data
│   └── nodes.conf
├── etc
│   └── redis.conf
├── logs
│   └── redis_6379.log
└── run
    └── redis_6379.pid

5 directories, 10 files


[root@82 redis]# cat data/nodes.conf 
44a9e9147d3746703514b694c7432d630c339010 :0@0 myself,master - 0 0 0 connected
vars currentEpoch 0 lastVoteEpoch 0

内容中最重要的是节点 ID（44a9e9147d3746703514b694c7432d630c339010），他是一个 40 位 16 进制的字符串，用来唯一标识集群内的一个节点。节点 ID 在集群初始化时只创建一次，节点重启时会加载集群配置文件进行重用。

6 个 Redis 实例都已经启动成功了，但是 Redis 集群并没有搭建完成，这些实例都不知道其他实例的存在，都是孤立的并没有形成一个集群。

6 个节点相互独立，如下：

[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-U57UB8U6-1603528295471)(../../../%E9%A9%AC%E5%93%A5%E6%95%99%E8%82%B2%E5%AD%A6%E4%B9%A0%E7%AC%94%E8%AE%B0/typora_img/image-20201023210105373.png)]

4、节点握手

可以通过节点握手让 6 个 Redis 节点建立联系，从而形成一个集群。

节点握手是指一批运行在集群模式下的节点通过 Gossip 协议彼此通信，达到感知对方的过程。

通过在客户端执行命令 cluster meet{ip}{port} 就可以建立两个节点之间的握手，该命令是一个异步命令，执行后立刻返回，内部执行握手的过程。过程如下：

centos82节点 6379 创建 centos83节点6379 信息对象，并发送 meet 消息。
centos83节点6379 接受到 meet 消息后，保存 6379 节点信息并回复 pong 消息。
之后节点 centos82 和 centos83 彼此定期通过 ping/pong 消息进行正常的节点通信。

在节点 10.0.0.82:6379 节点执行如下命令：

[root@82 redis]# redis-cli -h 127.0.0.1 -p 6379 CLUSTER MEET 10.0.0.83 6379
OK

这样 6379 节点就与 6380 之间建立了联系，如下：

[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-CwQqOCPK-1603528295474)(../../../%E9%A9%AC%E5%93%A5%E6%95%99%E8%82%B2%E5%AD%A6%E4%B9%A0%E7%AC%94%E8%AE%B0/typora_img/image-20201023210922395.png)]

继续在该节点执行如下命令：

[root@82 redis]# redis-cli -h 127.0.0.1 -p 6379 CLUSTER MEET 10.0.0.84 6379
OK
[root@82 redis]# redis-cli -h 127.0.0.1 -p 6379 CLUSTER MEET 10.0.0.85 6379
OK
[root@82 redis]# redis-cli -h 127.0.0.1 -p 6379 CLUSTER MEET 10.0.0.86 6379
OK
[root@82 redis]# redis-cli -h 127.0.0.1 -p 6379 CLUSTER MEET 10.0.0.87 6379
OK

上面命令是可以在任意节点上面执行，握手状态会通过消息在集群内传播，这样其他节点会自动发现新节点并发起握手流程，这样 6 个节点都互相感知了，建立了一个集群，如下图：

[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-K5Gz06Qa-1603528295476)(../../../%E9%A9%AC%E5%93%A5%E6%95%99%E8%82%B2%E5%AD%A6%E4%B9%A0%E7%AC%94%E8%AE%B0/typora_img/image-20201023211056530.png)]

查看集群节点信息

[root@82 redis]# redis-cli -h 127.0.0.1 -p 6379 cluster nodes
78182c222b3bf256e9a687bda0ec97bde3d7e520 10.0.0.87:6379@16379 master - 0 1603458683441 5 connected
10dd4f6089bd39d000ffeb64abd58378accd0f7b 10.0.0.83:6379@16379 master - 0 1603458682000 1 connected
8b113086a78c3e0bb4fbe0af8c73b2fe7bb1a05a 10.0.0.85:6379@16379 master - 0 1603458681000 4 connected
44a9e9147d3746703514b694c7432d630c339010 10.0.0.82:6379@16379 myself,master - 0 1603458681000 2 connected
ef6ee356ad6c66be29e2d734b0b17fd7eec2071d 10.0.0.86:6379@16379 master - 0 1603458683000 0 connected
6a18be154b06347bf2ccf51e8c59a074cc3dacca 10.0.0.84:6379@16379 master - 0 1603458682430 3 connected

在其余 5 个节点查看都可以得到相同的信息。

6 个节点建立握手后，集群还是不能正常工作，这是因为还有槽没有分配。Redis 集群一共 16384 个槽点，只有这些槽点全部分配节点后，集群才会进入在线状态。

centos84

[root@84 redis]# redis-cli -h 127.0.0.1 -p 6379 cluster nodes
ef6ee356ad6c66be29e2d734b0b17fd7eec2071d 10.0.0.86:6379@16379 master - 0 1603458789000 0 connected
6a18be154b06347bf2ccf51e8c59a074cc3dacca 10.0.0.84:6379@16379 myself,master - 0 1603458785000 3 connected
44a9e9147d3746703514b694c7432d630c339010 10.0.0.82:6379@16379 master - 0 1603458790332 2 connected
10dd4f6089bd39d000ffeb64abd58378accd0f7b 10.0.0.83:6379@16379 master - 0 1603458787000 1 connected
78182c222b3bf256e9a687bda0ec97bde3d7e520 10.0.0.87:6379@16379 master - 0 1603458788312 5 connected
8b113086a78c3e0bb4fbe0af8c73b2fe7bb1a05a 10.0.0.85:6379@16379 master - 0 1603458789321 4 connected

centos86

[root@86 redis]# redis-cli -h 127.0.0.1 -p 6379 cluster nodes
8b113086a78c3e0bb4fbe0af8c73b2fe7bb1a05a 10.0.0.85:6379@16379 master - 0 1603458793000 4 connected
ef6ee356ad6c66be29e2d734b0b17fd7eec2071d 10.0.0.86:6379@16379 myself,master - 0 1603458792000 0 connected
6a18be154b06347bf2ccf51e8c59a074cc3dacca 10.0.0.84:6379@16379 master - 0 1603458793450 3 connected
10dd4f6089bd39d000ffeb64abd58378accd0f7b 10.0.0.83:6379@16379 master - 0 1603458792000 1 connected
44a9e9147d3746703514b694c7432d630c339010 10.0.0.82:6379@16379 master - 0 1603458792000 2 connected
78182c222b3bf256e9a687bda0ec97bde3d7e520 10.0.0.87:6379@16379 master - 0 1603458791431 5 connected

5、分配槽点

分配槽点的命令如下：

redis-cli -h IP -p 端口 cluster addslots { begin .. end }

Redis 集群分为主从，其中首次启动的节点和被分配槽的节点都是主节点，从节点负责复制主节点槽信息和相关的数据。这里我们配置的三主三从，主节点为三台服务器centos82~84的6379 节点，从节点为centos85~87的6379 节点。所以整个命令如下：

[root@82 redis]# redis-cli -h 10.0.0.82 -p 6379 cluster addslots {0..5460}
OK
[root@82 redis]# redis-cli -h 10.0.0.83 -p 6379 cluster addslots {5461..10922}
OK
[root@82 redis]# redis-cli -h 10.0.0.84 -p 6379 cluster addslots {10923..16383}
OK

这样主节点就已经配置完成了，下面则是配置从节点。cluster replicate {nodeId} 可以让某个节点成为从节点，在三台服务器执行如下三个命令:

cluster replicate 
# 查看nodeid
[root@85 redis]# cat data/nodes.conf 
ef6ee356ad6c66be29e2d734b0b17fd7eec2071d 10.0.0.86:6379@16379 master - 0 1603459123000 0 connected
44a9e9147d3746703514b694c7432d630c339010 10.0.0.82:6379@16379 master - 0 1603459123000 2 connected 0-5460
78182c222b3bf256e9a687bda0ec97bde3d7e520 10.0.0.87:6379@16379 master - 0 1603459123814 5 connected
6a18be154b06347bf2ccf51e8c59a074cc3dacca 10.0.0.84:6379@16379 master - 0 1603459122000 3 connected 10923-16383
10dd4f6089bd39d000ffeb64abd58378accd0f7b 10.0.0.83:6379@16379 master - 0 1603459122805 1 connected 5461-10922
8b113086a78c3e0bb4fbe0af8c73b2fe7bb1a05a 10.0.0.85:6379@16379 myself,master - 0 1603459124000 4 connected
vars currentEpoch 5 lastVoteEpoch 0




# 设置从节点,连接从节点 cluster replicate 指向master的nodeid
# centos82
[root@82 redis]# redis-cli -h 10.0.0.85 -p 6379 cluster replicate 44a9e9147d3746703514b694c7432d630c339010
OK

[root@82 redis]# redis-cli -h 10.0.0.86 -p 6379 cluster replicate 10dd4f6089bd39d000ffeb64abd58378accd0f7b
OK
[root@82 redis]# redis-cli -h 10.0.0.87 -p 6379 cluster replicate 6a18be154b06347bf2ccf51e8c59a074cc3dacca
OK

再次查看集群节点信息：

# centos82
[root@82 redis]# redis-cli -h 127.0.0.1 -p 6379 cluster nodes
78182c222b3bf256e9a687bda0ec97bde3d7e520 10.0.0.87:6379@16379 slave 6a18be154b06347bf2ccf51e8c59a074cc3dacca 0 1603459986000 5 connected
10dd4f6089bd39d000ffeb64abd58378accd0f7b 10.0.0.83:6379@16379 master - 0 1603459987444 1 connected 5461-10922
8b113086a78c3e0bb4fbe0af8c73b2fe7bb1a05a 10.0.0.85:6379@16379 slave 44a9e9147d3746703514b694c7432d630c339010 0 1603459986000 4 connected
44a9e9147d3746703514b694c7432d630c339010 10.0.0.82:6379@16379 myself,master - 0 1603459986000 2 connected 0-5460
ef6ee356ad6c66be29e2d734b0b17fd7eec2071d 10.0.0.86:6379@16379 slave 10dd4f6089bd39d000ffeb64abd58378accd0f7b 0 1603459985425 1 connected
6a18be154b06347bf2ccf51e8c59a074cc3dacca 10.0.0.84:6379@16379 master - 0 1603459986435 3 connected 10923-16383

到这里我们就已经完成了 Redis 集群的搭建，整个集群图例如下：

[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-TYUK9yl9-1603528295478)(../../../%E9%A9%AC%E5%93%A5%E6%95%99%E8%82%B2%E5%AD%A6%E4%B9%A0%E7%AC%94%E8%AE%B0/typora_img/image-20201023213401366.png)]

测试

# master：centos82,用集群模式连接,加参数：-c
[root@82 redis]# redis-cli -c -h 10.0.0.82 -p 6379
10.0.0.82:6379> keys *
(empty list or set)
10.0.0.82:6379> set name wang 
-> Redirected to slot [5798] located at 10.0.0.83:6379
OK


# slave：centos83
[root@83 redis]# redis-cli 
127.0.0.1:6379> keys *
(empty list or set)
127.0.0.1:6379> keys *
1) "name"
127.0.0.1:6379> get name
"wang"

验证成功！！

6、redis-cli 操作集群

1) redis-cluster架构图

[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-7U1u8xvt-1603528295480)(../../../%E9%A9%AC%E5%93%A5%E6%95%99%E8%82%B2%E5%AD%A6%E4%B9%A0%E7%AC%94%E8%AE%B0/typora_img/image-20201024103646697.png)]

架构细节:

(1)所有的redis节点彼此互联(PING-PONG机制),内部使用二进制协议优化传输速度和带宽.
(2)节点的fail是通过集群中超过半数的master节点检测失效时才生效.
(3)客户端与redis节点直连,不需要中间proxy层.客户端不需要连接集群所有节点,连接集群中任何一个可用节点即可
(4)redis-cluster把所有的物理节点映射到[0-16383]slot上,cluster 负责维护node<->slot<->key

2) redis-cluster选举:容错

在这里插入图片描述

(1)选举过程是集群中所有master参与,如果半数以上master节点与故障节点通信超过(cluster-node-timeout),认为该节点故障，自动触发故障转移操作.

(2):什么时候整个集群不可用(cluster_state:fail)?

a:如果集群任意master挂掉,且当前master没有slave.集群进入fail状态,也可以理解成集群的slot映射[0-16383]不完整时进入fail状态.

ps : redis-3.0.0.rc1加入cluster-require-full-coverage参数,默认关闭,打开集群兼容部分失败.

b:如果集群超过半数以上master挂掉，无论是否有slave集群进入fail状态.

ps:当集群不可用时,所有对集群的操作做都不可用，收到((error) CLUSTERDOWN The cluster is down)错误

所有配置依然采用上面的 Redis 配置，启动所有 Redis 服务（如果是复用上面的配置，需要删除掉所有的 node-xx.conf）。

说明：edis 5 不支持 redis-trib.rb 来搭建集群了，推荐使用 redis-cli 来操作

查看利用 redis-cli 操作集群的命令帮助文档

[root@82 redis]# redis-cli --cluster help
Cluster Manager Commands:
  create         host1:port1 ... hostN:portN
                 --cluster-replicas <arg>
  check          host:port
                 --cluster-search-multiple-owners
  info           host:port
  fix            host:port
                 --cluster-search-multiple-owners
  reshard        host:port
                 --cluster-from <arg>
                 --cluster-to <arg>
                 --cluster-slots <arg>
                 --cluster-yes
                 --cluster-timeout <arg>
                 --cluster-pipeline <arg>
                 --cluster-replace
  rebalance      host:port
                 --cluster-weight <node1=w1...nodeN=wN>
                 --cluster-use-empty-masters
                 --cluster-timeout <arg>
                 --cluster-simulate
                 --cluster-pipeline <arg>
                 --cluster-threshold <arg>
                 --cluster-replace
  add-node       new_host:new_port existing_host:existing_port
                 --cluster-slave
                 --cluster-master-id <arg>
  del-node       host:port node_id
  call           host:port command arg arg .. arg
  set-timeout    host:port milliseconds
  import         host:port
                 --cluster-from <arg>
                 --cluster-copy
                 --cluster-replace
  help           

For check, fix, reshard, del-node, set-timeout you can specify the host and port of any working node in the cluster.

将上面配置全部恢复到启动redis之前状态

# dump.rdb
dump.rdb是由Redis服务器自动生成的 默认情况下 每隔一段时间redis服务器程序会自动对数据库做一次遍历，把内存快照写在一个叫做“dump.rdb”的文件里，这个持久化机制叫做SNAPSHOT。有了SNAPSHOT后，如果服务器宕机，重新启动redis服务器程序时redis会自动加载dump.rdb，将数据库状态恢复到上一次做SNAPSHOT时的状态。

# appendonly.aof（数据持久化）
默认情况下Redis会异步的将数据导出到磁盘上。这种模式对许多应用程序已经足够了，但是如果断电或者redis进程出问题就会导致一段时间内的更新数据丢失（取决与配置项）；
这种只增文件是可选的能够提供更好的体验的数据持久化策略。
举个例子，如果使用默认的配置数据fsync策略，在服务器意外断电的情况下redis只会丢失一秒中内的更新数据，或者当redis进程出问题但操作系统运转正常时，redis只会丢失一个数据更新操作。
AOF 和 RDB 持久化方式可以同时启动并且无冲突。
如果AOF开启，启动redis时会加载aof文件，这些文件能够提供更好的保证。

 
# nodes.conf
每个集群节点都有一个集群配置文件。它是由Redis节点自动创建和更新的。每个Redis集群节点都需要一个不同的集群配置文件。

注：确保在同一系统中运行的实例没有重叠的集群配置文件名。集群的配置,配置文件首次启动自动生成。

# 利用xshell快捷方式全部恢复到初始状态
[root@82 redis]# redis-cli 
127.0.0.1:6379> flushall
OK
127.0.0.1:6379> flushall 
OK
127.0.0.1:6379> flushall
OK
127.0.0.1:6379> keys *
(empty list or set)
127.0.0.1:6379> 
[root@82 redis]# systemctl stop redis
[root@82 redis]# rm -rf data/*
[root@82 redis]# rm -rf run/*
[root@82 redis]# rm -rf logs/*

7、搭建集群

搭建集群的命令如下：

create host1:port1 ... hostN:portN --cluster-replicas <arg>

启动Redis

# 利用xshell启动全部redis
[root@82 redis]# systemctl restart redis
[root@82 redis]# tree
.
├── bin
│   ├── redis-benchmark
│   ├── redis-check-aof
│   ├── redis-check-rdb
│   ├── redis-cli
│   ├── redis-sentinel -> redis-server
│   └── redis-server
├── data
│   ├── dump_6379.rdb
│   └── nodes.conf
├── etc
│   └── redis.conf
├── logs
│   └── redis_6379.log
└── run
    └── redis_6379.pid

5 directories, 11 files

创建集群

[root@82 redis]# redis-cli --cluster create 10.0.0.82:6379 10.0.0.83:6379 10.0.0.84:6379 10.0.0.85:6379 10.0.0.86:6379 10.0.0.87:6379 --cluster-replicas 1 
# --cluster-replicas 1  表示主从配置比，1表示的是1:1，前三个是主，后三个是从
# 若配置文件中设置的密码，则还需要加上-a passwod

>>> Performing hash slots allocation on 6 nodes...
Master[0] -> Slots 0 - 5460
Master[1] -> Slots 5461 - 10922
Master[2] -> Slots 10923 - 16383
Adding replica 10.0.0.86:6379 to 10.0.0.82:6379
Adding replica 10.0.0.87:6379 to 10.0.0.83:6379
Adding replica 10.0.0.85:6379 to 10.0.0.84:6379
M: 4af24b2e09524dc2542b267468ff207922ad89f3 10.0.0.82:6379
   slots:[0-5460] (5461 slots) master
M: 4657c4b5fd7067568e6367cb52fbeaccb447a38b 10.0.0.83:6379
   slots:[5461-10922] (5462 slots) master
M: 23e761b4f8fa1107062b1f6ce42d40ca04bcd12d 10.0.0.84:6379
   slots:[10923-16383] (5461 slots) master
S: 34ed5c19d7c5b1a142bd33c7c70bd0b1639958bc 10.0.0.85:6379
   replicates 23e761b4f8fa1107062b1f6ce42d40ca04bcd12d
S: c6b935f53eeffdd29fb393283628946d87296b1b 10.0.0.86:6379
   replicates 4af24b2e09524dc2542b267468ff207922ad89f3
S: fd815c656855c2862a08975e6ac488a113fcfe32 10.0.0.87:6379
   replicates 4657c4b5fd7067568e6367cb52fbeaccb447a38b
Can I set the above configuration? (type 'yes' to accept): yes
>>> Nodes configuration updated
>>> Assign a different config epoch to each node
>>> Sending CLUSTER MEET messages to join the cluster
Waiting for the cluster to join
.....
>>> Performing Cluster Check (using node 10.0.0.82:6379)
M: 4af24b2e09524dc2542b267468ff207922ad89f3 10.0.0.82:6379
   slots:[0-5460] (5461 slots) master
   1 additional replica(s)
S: fd815c656855c2862a08975e6ac488a113fcfe32 10.0.0.87:6379
   slots: (0 slots) slave
   replicates 4657c4b5fd7067568e6367cb52fbeaccb447a38b
S: c6b935f53eeffdd29fb393283628946d87296b1b 10.0.0.86:6379
   slots: (0 slots) slave
   replicates 4af24b2e09524dc2542b267468ff207922ad89f3
M: 23e761b4f8fa1107062b1f6ce42d40ca04bcd12d 10.0.0.84:6379
   slots:[10923-16383] (5461 slots) master
   1 additional replica(s)
S: 34ed5c19d7c5b1a142bd33c7c70bd0b1639958bc 10.0.0.85:6379
   slots: (0 slots) slave
   replicates 23e761b4f8fa1107062b1f6ce42d40ca04bcd12d
M: 4657c4b5fd7067568e6367cb52fbeaccb447a38b 10.0.0.83:6379
   slots:[5461-10922] (5462 slots) master
   1 additional replica(s)
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.

8、查询集群信息

centos82

[root@82 redis]# redis-cli --cluster info 10.0.0.82 6379
10.0.0.82:6379 (4af24b2e...) -> 0 keys | 5461 slots | 1 slaves.
10.0.0.84:6379 (23e761b4...) -> 0 keys | 5461 slots | 1 slaves.
10.0.0.83:6379 (4657c4b5...) -> 0 keys | 5462 slots | 1 slaves.
[OK] 0 keys in 3 masters.
0.00 keys per slot on average.

centos82

# 更加详细信息

[root@82 redis]# redis-cli -h 127.0.0.1 -p 6379 -c cluster nodes
fd815c656855c2862a08975e6ac488a113fcfe32 10.0.0.87:6379@16379 slave 4657c4b5fd7067568e6367cb52fbeaccb447a38b 0 1603462834439 6 connected
c6b935f53eeffdd29fb393283628946d87296b1b 10.0.0.86:6379@16379 slave 4af24b2e09524dc2542b267468ff207922ad89f3 0 1603462836457 5 connected
23e761b4f8fa1107062b1f6ce42d40ca04bcd12d 10.0.0.84:6379@16379 master - 0 1603462833000 3 connected 10923-16383
4af24b2e09524dc2542b267468ff207922ad89f3 10.0.0.82:6379@16379 myself,master - 0 1603462833000 1 connected 0-5460
34ed5c19d7c5b1a142bd33c7c70bd0b1639958bc 10.0.0.85:6379@16379 slave 23e761b4f8fa1107062b1f6ce42d40ca04bcd12d 0 1603462835447 4 connected
4657c4b5fd7067568e6367cb52fbeaccb447a38b 10.0.0.83:6379@16379 master - 0 1603462834000 2 connected 5461-10922

注意：查询集群信息需要加上-c参数

参数说明：

-c：表示以集群方式连接惹redis
-h：指定IP地址
-p：指定端口
cluster nodes：查询集群节点信息
cluster info：查询集群状态信息

[root@82 redis]# redis-cli -h 127.0.0.1 -p 6379 -c cluster info
cluster_state:ok
cluster_slots_assigned:16384
cluster_slots_ok:16384
cluster_slots_pfail:0
cluster_slots_fail:0
cluster_known_nodes:6
cluster_size:3
cluster_current_epoch:6
cluster_my_epoch:1
cluster_stats_messages_ping_sent:497
cluster_stats_messages_pong_sent:521
cluster_stats_messages_sent:1018
cluster_stats_messages_ping_received:516
cluster_stats_messages_pong_received:497
cluster_stats_messages_meet_received:5
cluster_stats_messages_received:1018

centos83

[root@83 redis]# redis-cli --cluster check 127.0.0.1:6379
127.0.0.1:6379 (4657c4b5...) -> 0 keys | 5462 slots | 1 slaves.
10.0.0.84:6379 (23e761b4...) -> 0 keys | 5461 slots | 1 slaves.
10.0.0.82:6379 (4af24b2e...) -> 0 keys | 5461 slots | 1 slaves.
[OK] 0 keys in 3 masters.
0.00 keys per slot on average.
>>> Performing Cluster Check (using node 127.0.0.1:6379)
M: 4657c4b5fd7067568e6367cb52fbeaccb447a38b 127.0.0.1:6379
   slots:[5461-10922] (5462 slots) master
   1 additional replica(s)
S: fd815c656855c2862a08975e6ac488a113fcfe32 10.0.0.87:6379
   slots: (0 slots) slave
   replicates 4657c4b5fd7067568e6367cb52fbeaccb447a38b
M: 23e761b4f8fa1107062b1f6ce42d40ca04bcd12d 10.0.0.84:6379
   slots:[10923-16383] (5461 slots) master
   1 additional replica(s)
M: 4af24b2e09524dc2542b267468ff207922ad89f3 10.0.0.82:6379
   slots:[0-5460] (5461 slots) master
   1 additional replica(s)
S: c6b935f53eeffdd29fb393283628946d87296b1b 10.0.0.86:6379
   slots: (0 slots) slave
   replicates 4af24b2e09524dc2542b267468ff207922ad89f3
S: 34ed5c19d7c5b1a142bd33c7c70bd0b1639958bc 10.0.0.85:6379
   slots: (0 slots) slave
   replicates 23e761b4f8fa1107062b1f6ce42d40ca04bcd12d
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.

centos85：可以看看 Redis 启动信息

[root@85 redis]# redis-cli --cluster check 127.0.0.1:6379
10.0.0.84:6379 (23e761b4...) -> 0 keys | 5461 slots | 1 slaves.
10.0.0.83:6379 (4657c4b5...) -> 0 keys | 5462 slots | 1 slaves.
10.0.0.82:6379 (4af24b2e...) -> 0 keys | 5461 slots | 1 slaves.
[OK] 0 keys in 3 masters.
0.00 keys per slot on average.
>>> Performing Cluster Check (using node 127.0.0.1:6379)
S: 34ed5c19d7c5b1a142bd33c7c70bd0b1639958bc 127.0.0.1:6379
   slots: (0 slots) slave
   replicates 23e761b4f8fa1107062b1f6ce42d40ca04bcd12d
M: 23e761b4f8fa1107062b1f6ce42d40ca04bcd12d 10.0.0.84:6379
   slots:[10923-16383] (5461 slots) master
   1 additional replica(s)
M: 4657c4b5fd7067568e6367cb52fbeaccb447a38b 10.0.0.83:6379
   slots:[5461-10922] (5462 slots) master
   1 additional replica(s)
S: c6b935f53eeffdd29fb393283628946d87296b1b 10.0.0.86:6379
   slots: (0 slots) slave
   replicates 4af24b2e09524dc2542b267468ff207922ad89f3
S: fd815c656855c2862a08975e6ac488a113fcfe32 10.0.0.87:6379
   slots: (0 slots) slave
   replicates 4657c4b5fd7067568e6367cb52fbeaccb447a38b
M: 4af24b2e09524dc2542b267468ff207922ad89f3 10.0.0.82:6379
   slots:[0-5460] (5461 slots) master
   1 additional replica(s)
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.

centos87

[root@87 redis]# redis-cli --cluster info 127.0.0.1:6379
10.0.0.84:6379 (23e761b4...) -> 0 keys | 5461 slots | 1 slaves.
10.0.0.82:6379 (4af24b2e...) -> 0 keys | 5461 slots | 1 slaves.
10.0.0.83:6379 (4657c4b5...) -> 0 keys | 5462 slots | 1 slaves.
[OK] 0 keys in 3 masters.
0.00 keys per slot on average.

9、添加主节点

添加一个master主节点

准备一个干净的redis节点。按上面集群版修改redis配置文件并开启该redis节点

# centos 88
[root@88 redis]# egrep "^[^#| ]" etc/redis.conf
bind 0.0.0.0
protected-mode yes
port 6379
tcp-backlog 511
timeout 0
tcp-keepalive 300
daemonize yes
supervised no
pidfile /usr/local/redis/run/redis_6379.pid
loglevel notice
logfile /usr/local/redis/logs/redis_6379.log
databases 16
always-show-logo yes
save 900 1
save 300 10
save 60 10000
stop-writes-on-bgsave-error yes
rdbcompression yes
rdbchecksum yes
dbfilename dump.rdb
dir /usr/local/redis/data/
replica-serve-stale-data yes
replica-read-only yes
repl-diskless-sync no
repl-diskless-sync-delay 5
repl-disable-tcp-nodelay no
replica-priority 100
lazyfree-lazy-eviction no
lazyfree-lazy-expire no
lazyfree-lazy-server-del no
replica-lazy-flush no
appendonly no
appendfilename "appendonly.aof"
appendfsync everysec
no-appendfsync-on-rewrite no
auto-aof-rewrite-percentage 100
auto-aof-rewrite-min-size 64mb
aof-load-truncated yes
aof-use-rdb-preamble yes
lua-time-limit 5000
cluster-enabled yes
slowlog-log-slower-than 10000
slowlog-max-len 128
latency-monitor-threshold 0
notify-keyspace-events ""
hash-max-ziplist-entries 512
hash-max-ziplist-value 64
list-max-ziplist-size -2
list-compress-depth 0
set-max-intset-entries 512
zset-max-ziplist-entries 128
zset-max-ziplist-value 64
hll-sparse-max-bytes 3000
stream-node-max-bytes 4096
stream-node-max-entries 100
activerehashing yes
client-output-buffer-limit normal 0 0 0
client-output-buffer-limit replica 256mb 64mb 60
client-output-buffer-limit pubsub 32mb 8mb 60
hz 10
dynamic-hz yes
aof-rewrite-incremental-fsync yes
rdb-save-incremental-fsync yes



# 启动服务
[root@88 redis]# systemctl restart redis
[root@88 redis]# ss -ntl
State      Recv-Q     Send-Q         Local Address:Port           Peer Address:Port     
LISTEN     0          128                  0.0.0.0:22                  0.0.0.0:*        
LISTEN     0          128                  0.0.0.0:16379               0.0.0.0:*        
LISTEN     0          128                  0.0.0.0:6379                0.0.0.0:*        
LISTEN     0          128                  0.0.0.0:5355                0.0.0.0:*        
LISTEN     0          128                     [::]:22                     [::]:*        
LISTEN     0          128                     [::]:5355                   [::]:*  


[root@88 redis]# tree /usr/local/redis/
/usr/local/redis/
├── bin
│   ├── redis-benchmark
│   ├── redis-check-aof
│   ├── redis-check-rdb
│   ├── redis-cli
│   ├── redis-sentinel -> redis-server
│   └── redis-server
├── data
│   └── nodes.conf
├── etc
│   └── redis.conf
├── logs
│   └── redis_6379.log
└── run
    └── redis_6379.pid

5 directories, 10 files


# 查看进程
[root@88 redis]# ps -ef | grep redis
redis       8154       1  0 07:54 ?        00:00:00 /usr/local/redis/bin/redis-server 0.0.0.0:6379 [cluster]
root        8178    8126  0 08:01 pts/0    00:00:00 grep --color=auto redis

添加新主节点命令：

redis-cli --cluster add-node 新节点IP:端口 原节点任意IP:端口 -a 密码
# 如果没有密码，则不用-a

# 登录10.0.0.88的集群节点，查看节点信息
[root@88 redis]# redis-cli -h 10.0.0.88 -p 6379 -c
10.0.0.88:6379> cluster nodes
9ce04c3b13c4de557dd21e24fda37e39e7485c16 :6379@16379 myself,master - 0 0 0 connected
# 注意到:此节点只有节点本身，孤儿节点


# 进入集群中任意节点，查看节点信息
[root@88 redis]# redis-cli -h 10.0.0.82 -p 6379 -c
10.0.0.82:6379> cluster nodes
fd815c656855c2862a08975e6ac488a113fcfe32 10.0.0.87:6379@16379 slave 4657c4b5fd7067568e6367cb52fbeaccb447a38b 0 1603500955000 6 connected
c6b935f53eeffdd29fb393283628946d87296b1b 10.0.0.86:6379@16379 slave 4af24b2e09524dc2542b267468ff207922ad89f3 0 1603500956318 5 connected
23e761b4f8fa1107062b1f6ce42d40ca04bcd12d 10.0.0.84:6379@16379 master - 0 1603500953274 8 connected 0-332 5461-5794 10923-16383
4af24b2e09524dc2542b267468ff207922ad89f3 10.0.0.82:6379@16379 myself,master - 0 1603500954000 1 connected 333-5460
34ed5c19d7c5b1a142bd33c7c70bd0b1639958bc 10.0.0.85:6379@16379 slave 23e761b4f8fa1107062b1f6ce42d40ca04bcd12d 0 1603500955307 8 connected
4657c4b5fd7067568e6367cb52fbeaccb447a38b 10.0.0.83:6379@16379 master - 0 1603500954289 2 connected 5795-10922

# 此时没有新节点加入
# 添加新节点
[root@88 redis]# redis-cli --cluster add-node 10.0.0.88:6379 10.0.0.82:6379
>>> Adding node 10.0.0.88:6379 to cluster 10.0.0.82:6379
>>> Performing Cluster Check (using node 10.0.0.82:6379)
M: 4af24b2e09524dc2542b267468ff207922ad89f3 10.0.0.82:6379
   slots:[333-5460] (5128 slots) master
   1 additional replica(s)
S: fd815c656855c2862a08975e6ac488a113fcfe32 10.0.0.87:6379
   slots: (0 slots) slave
   replicates 4657c4b5fd7067568e6367cb52fbeaccb447a38b
S: c6b935f53eeffdd29fb393283628946d87296b1b 10.0.0.86:6379
   slots: (0 slots) slave
   replicates 4af24b2e09524dc2542b267468ff207922ad89f3
M: 23e761b4f8fa1107062b1f6ce42d40ca04bcd12d 10.0.0.84:6379
   slots:[0-332],[5461-5794],[10923-16383] (6128 slots) master
   1 additional replica(s)
S: 34ed5c19d7c5b1a142bd33c7c70bd0b1639958bc 10.0.0.85:6379
   slots: (0 slots) slave
   replicates 23e761b4f8fa1107062b1f6ce42d40ca04bcd12d
M: 4657c4b5fd7067568e6367cb52fbeaccb447a38b 10.0.0.83:6379
   slots:[5795-10922] (5128 slots) master
   1 additional replica(s)
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
>>> Send CLUSTER MEET to node 10.0.0.88:6379 to make it join the cluster.
[OK] New node added correctly.


# 登录集群，查看节点
[root@88 redis]# redis-cli -h 10.0.0.82 -p 6379 -c
10.0.0.82:6379> cluster nodes
fd815c656855c2862a08975e6ac488a113fcfe32 10.0.0.87:6379@16379 slave 4657c4b5fd7067568e6367cb52fbeaccb447a38b 0 1603501462000 6 connected
c6b935f53eeffdd29fb393283628946d87296b1b 10.0.0.86:6379@16379 slave 4af24b2e09524dc2542b267468ff207922ad89f3 0 1603501464000 5 connected
23e761b4f8fa1107062b1f6ce42d40ca04bcd12d 10.0.0.84:6379@16379 master - 0 1603501464996 8 connected 0-332 5461-5794 10923-16383
4af24b2e09524dc2542b267468ff207922ad89f3 10.0.0.82:6379@16379 myself,master - 0 1603501462000 1 connected 333-5460
34ed5c19d7c5b1a142bd33c7c70bd0b1639958bc 10.0.0.85:6379@16379 slave 23e761b4f8fa1107062b1f6ce42d40ca04bcd12d 0 1603501466006 8 connected
9ce04c3b13c4de557dd21e24fda37e39e7485c16 10.0.0.88:6379@16379 master - 0 1603501462000 0 connected
4657c4b5fd7067568e6367cb52fbeaccb447a38b 10.0.0.83:6379@16379 master - 0 1603501463000 2 connected 5795-10922

# 注意到10.0.0.88是master，但是没有数据，没有slot槽点

# 分配槽点：redis-cli --cluster reshard 原集群中任意节点IP:端口
[root@88 redis]# redis-cli --cluster reshard 10.0.0.82 6379
>>> Performing Cluster Check (using node 10.0.0.82:6379)
M: 4af24b2e09524dc2542b267468ff207922ad89f3 10.0.0.82:6379
   slots:[333-5460] (5128 slots) master
   1 additional replica(s)
S: fd815c656855c2862a08975e6ac488a113fcfe32 10.0.0.87:6379
   slots: (0 slots) slave
   replicates 4657c4b5fd7067568e6367cb52fbeaccb447a38b
S: c6b935f53eeffdd29fb393283628946d87296b1b 10.0.0.86:6379
   slots: (0 slots) slave
   replicates 4af24b2e09524dc2542b267468ff207922ad89f3
M: 23e761b4f8fa1107062b1f6ce42d40ca04bcd12d 10.0.0.84:6379
   slots:[0-332],[5461-5794],[10923-16383] (6128 slots) master
   1 additional replica(s)
S: 34ed5c19d7c5b1a142bd33c7c70bd0b1639958bc 10.0.0.85:6379
   slots: (0 slots) slave
   replicates 23e761b4f8fa1107062b1f6ce42d40ca04bcd12d
M: 9ce04c3b13c4de557dd21e24fda37e39e7485c16 10.0.0.88:6379
   slots: (0 slots) master
M: 4657c4b5fd7067568e6367cb52fbeaccb447a38b 10.0.0.83:6379
   slots:[5795-10922] (5128 slots) master
   1 additional replica(s)
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
How many slots do you want to move (from 1 to 16384)? 1000 # 分配的节点数
What is the receiving node ID? 9ce04c3b13c4de557dd21e24fda37e39e7485c16  # 要分配的节点nodeID
Please enter all the source node IDs.
  Type 'all' to use all the nodes as source nodes for the hash slots.
  Type 'done' once you entered all the source nodes IDs.
Source node #1: all  # 设置slot的迁出方，可以选择all，也可以选择哪个节点的nodeID

Ready to move 1000 slots.
  Source nodes:
    M: 4af24b2e09524dc2542b267468ff207922ad89f3 10.0.0.82:6379
       slots:[333-5460] (5128 slots) master
       1 additional replica(s)
    M: 23e761b4f8fa1107062b1f6ce42d40ca04bcd12d 10.0.0.84:6379
       slots:[0-332],[5461-5794],[10923-16383] (6128 slots) master
       1 additional replica(s)
    M: 4657c4b5fd7067568e6367cb52fbeaccb447a38b 10.0.0.83:6379
       slots:[5795-10922] (5128 slots) master
       1 additional replica(s)
  Destination node:
    M: 9ce04c3b13c4de557dd21e24fda37e39e7485c16 10.0.0.88:6379
       slots: (0 slots) master
  Resharding plan:
    Moving slot 0 from 23e761b4f8fa1107062b1f6ce42d40ca04bcd12d
    Moving slot 1 from 23e761b4f8fa1107062b1f6ce42d40ca04bcd12d
    Moving slot 2 from 23e761b4f8fa1107062b1f6ce42d40ca04bcd12d
    Moving slot 3 from 23e761b4f8fa1107062b1f6ce42d40ca04bcd12d
    Moving slot 4 from 23e761b4f8fa1107062b1f6ce42d40ca04bcd12d
    Moving slot 5497 from 23e761b4f8fa1107062b1f6ce42d40ca04bcd12d
    Moving slot 5498 from 23e761b4f8fa1107062b1f6ce42d40ca04bcd12d
    Moving slot 5499 from 23e761b4f8fa1107062b1f6ce42d40ca04bcd12d
    Moving slot 5500 from 23e761b4f8fa1107062b1f6ce42d40ca04bcd12d
    Moving slot 5501 from 23e761b4f8fa1107062b1f6ce42d40ca04bcd12d
    Moving slot 5502 from 23e761b4f8fa1107062b1f6ce42d40ca04bcd12d
    Moving slot 333 from 4af24b2e09524dc2542b267468ff207922ad89f3
    Moving slot 334 from 4af24b2e09524dc2542b267468ff207922ad89f3
    Moving slot 335 from 4af24b2e09524dc2542b267468ff207922ad89f3
    Moving slot 336 from 4af24b2e09524dc2542b267468ff207922ad89f3
    Moving slot 337 from 4af24b2e09524dc2542b267468ff207922ad89f3
    Moving slot 338 from 4af24b2e09524dc2542b267468ff207922ad89f3
    Moving slot 339 from 4af24b2e09524dc2542b267468ff207922ad89f3
    Moving slot 340 from 4af24b2e09524dc2542b267468ff207922ad89f3
    Moving slot 341 from 4af24b2e09524dc2542b267468ff207922ad89f3
.............
    Moving slot 639 from 4af24b2e09524dc2542b267468ff207922ad89f3
    Moving slot 640 from 4af24b2e09524dc2542b267468ff207922ad89f3
    Moving slot 641 from 4af24b2e09524dc2542b267468ff207922ad89f3
    Moving slot 642 from 4af24b2e09524dc2542b267468ff207922ad89f3
    Moving slot 643 from 4af24b2e09524dc2542b267468ff207922ad89f3
    Moving slot 644 from 4af24b2e09524dc2542b267468ff207922ad89f3
    Moving slot 5795 from 4657c4b5fd7067568e6367cb52fbeaccb447a38b
    Moving slot 5796 from 4657c4b5fd7067568e6367cb52fbeaccb447a38b
    Moving slot 5797 from 4657c4b5fd7067568e6367cb52fbeaccb447a38b
    Moving slot 5798 from 4657c4b5fd7067568e6367cb52fbeaccb447a38b
    Moving slot 5799 from 4657c4b5fd7067568e6367cb52fbeaccb447a38b
    Moving slot 5800 from 4657c4b5fd7067568e6367cb52fbeaccb447a38b
    Moving slot 5801 from 4657c4b5fd7067568e6367cb52fbeaccb447a38b
    Moving slot 5802 from 4657c4b5fd7067568e6367cb52fbeaccb447a38b
    Moving slot 5803 from 4657c4b5fd7067568e6367cb52fbeaccb447a38b
    Moving slot 5804 from 4657c4b5fd7067568e6367cb52fbeaccb447a38b
    Moving slot 5805 from 4657c4b5fd7067568e6367cb52fbeaccb447a38b
.........
    Moving slot 6103 from 4657c4b5fd7067568e6367cb52fbeaccb447a38b
    Moving slot 6104 from 4657c4b5fd7067568e6367cb52fbeaccb447a38b
    Moving slot 6105 from 4657c4b5fd7067568e6367cb52fbeaccb447a38b
    Moving slot 6106 from 4657c4b5fd7067568e6367cb52fbeaccb447a38b
Do you want to proceed with the proposed reshard plan (yes/no)? yes  # 是否按照计划分配,输入yes
.........
Moving slot 6102 from 10.0.0.83:6379 to 10.0.0.88:6379: 
Moving slot 6103 from 10.0.0.83:6379 to 10.0.0.88:6379: 
Moving slot 6104 from 10.0.0.83:6379 to 10.0.0.88:6379: 
Moving slot 6105 from 10.0.0.83:6379 to 10.0.0.88:6379: 
Moving slot 6106 from 10.0.0.83:6379 to 10.0.0.88:6379: 
[root@88 redis]# 

# 查看集群节点信息
[root@88 redis]# redis-cli -h 10.0.0.88 -p 6379 -c cluster nodes
fd815c656855c2862a08975e6ac488a113fcfe32 10.0.0.87:6379@16379 slave 4657c4b5fd7067568e6367cb52fbeaccb447a38b 0 1603503209770 2 connected
c6b935f53eeffdd29fb393283628946d87296b1b 10.0.0.86:6379@16379 slave 4af24b2e09524dc2542b267468ff207922ad89f3 0 1603503205000 1 connected
34ed5c19d7c5b1a142bd33c7c70bd0b1639958bc 10.0.0.85:6379@16379 slave 23e761b4f8fa1107062b1f6ce42d40ca04bcd12d 0 1603503208764 8 connected
23e761b4f8fa1107062b1f6ce42d40ca04bcd12d 10.0.0.84:6379@16379 master - 0 1603503208000 8 connected 5503-5794 10923-16383
4af24b2e09524dc2542b267468ff207922ad89f3 10.0.0.82:6379@16379 master - 0 1603503206000 1 connected 645-5460
9ce04c3b13c4de557dd21e24fda37e39e7485c16 10.0.0.88:6379@16379 myself,master - 0 1603503207000 9 connected 0-644 5461-5502 5795-6106  # myself表示当前连接的节点
4657c4b5fd7067568e6367cb52fbeaccb447a38b 10.0.0.83:6379@16379 master - 0 1603503207754 2 connected 6107-10922

10、添加从节点

# 准备一个新的节点10.0.0.89，配置跟之前一致
# 启动redis服务
[root@89 redis]# systemctl restart redis
[root@89 redis]# ps -ef | grep redis
redis       8175       1  0 09:40 ?        00:00:00 /usr/local/redis/bin/redis-server 0.0.0.0:6379 [cluster]
root        8182    7749  0 09:40 pts/0    00:00:00 grep --color=auto redis

# 查看节点ID
[root@89 redis]# redis-cli --cluster check 10.0.0.88 6379
10.0.0.88:6379 (9ce04c3b...) -> 0 keys | 999 slots | 0 slaves.
10.0.0.84:6379 (23e761b4...) -> 0 keys | 5753 slots | 1 slaves.
10.0.0.82:6379 (4af24b2e...) -> 0 keys | 4816 slots | 1 slaves.
10.0.0.83:6379 (4657c4b5...) -> 0 keys | 4816 slots | 1 slaves.
[OK] 0 keys in 4 masters.
0.00 keys per slot on average.
>>> Performing Cluster Check (using node 10.0.0.88:6379)
M: 9ce04c3b13c4de557dd21e24fda37e39e7485c16 10.0.0.88:6379
   slots:[0-644],[5461-5502],[5795-6106] (999 slots) master
S: fd815c656855c2862a08975e6ac488a113fcfe32 10.0.0.87:6379
   slots: (0 slots) slave
   replicates 4657c4b5fd7067568e6367cb52fbeaccb447a38b
S: c6b935f53eeffdd29fb393283628946d87296b1b 10.0.0.86:6379
   slots: (0 slots) slave
   replicates 4af24b2e09524dc2542b267468ff207922ad89f3
S: 34ed5c19d7c5b1a142bd33c7c70bd0b1639958bc 10.0.0.85:6379
   slots: (0 slots) slave
   replicates 23e761b4f8fa1107062b1f6ce42d40ca04bcd12d
M: 23e761b4f8fa1107062b1f6ce42d40ca04bcd12d 10.0.0.84:6379
   slots:[5503-5794],[10923-16383] (5753 slots) master
   1 additional replica(s)
M: 4af24b2e09524dc2542b267468ff207922ad89f3 10.0.0.82:6379
   slots:[645-5460] (4816 slots) master
   1 additional replica(s)
M: 4657c4b5fd7067568e6367cb52fbeaccb447a38b 10.0.0.83:6379
   slots:[6107-10922] (4816 slots) master
   1 additional replica(s)
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.


# 添加slave节点
# 随机分配master: redis-cli --cluster add-node 新节点:端口 任意节点:端口 --cluster-slave
# 指定分配master: redis-cli --cluster add-node 新节点:端口 指定master:端口 --cluster-slave --cluster-master-id 相应的master的nodeID

[root@89 redis]# redis-cli --cluster add-node 10.0.0.89:6379 10.0.0.88:6379 --cluster-slave --cluster-master-id 9ce04c3b13c4de557dd21e24fda37e39e7485c16
>>> Adding node 10.0.0.89:6379 to cluster 10.0.0.88:6379
>>> Performing Cluster Check (using node 10.0.0.88:6379)
M: 9ce04c3b13c4de557dd21e24fda37e39e7485c16 10.0.0.88:6379
   slots:[0-644],[5461-5502],[5795-6106] (999 slots) master
S: fd815c656855c2862a08975e6ac488a113fcfe32 10.0.0.87:6379
   slots: (0 slots) slave
   replicates 4657c4b5fd7067568e6367cb52fbeaccb447a38b
S: c6b935f53eeffdd29fb393283628946d87296b1b 10.0.0.86:6379
   slots: (0 slots) slave
   replicates 4af24b2e09524dc2542b267468ff207922ad89f3
S: 34ed5c19d7c5b1a142bd33c7c70bd0b1639958bc 10.0.0.85:6379
   slots: (0 slots) slave
   replicates 23e761b4f8fa1107062b1f6ce42d40ca04bcd12d
M: 23e761b4f8fa1107062b1f6ce42d40ca04bcd12d 10.0.0.84:6379
   slots:[5503-5794],[10923-16383] (5753 slots) master
   1 additional replica(s)
M: 4af24b2e09524dc2542b267468ff207922ad89f3 10.0.0.82:6379
   slots:[645-5460] (4816 slots) master
   1 additional replica(s)
M: 4657c4b5fd7067568e6367cb52fbeaccb447a38b 10.0.0.83:6379
   slots:[6107-10922] (4816 slots) master
   1 additional replica(s)
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
>>> Send CLUSTER MEET to node 10.0.0.89:6379 to make it join the cluster.
Waiting for the cluster to join

>>> Configure node as replica of 10.0.0.88:6379.
[OK] New node added correctly.


# 查看集群节点信息
[root@89 redis]# redis-cli --cluster check 10.0.0.88 6379
10.0.0.88:6379 (9ce04c3b...) -> 0 keys | 999 slots | 1 slaves.
10.0.0.84:6379 (23e761b4...) -> 0 keys | 5753 slots | 1 slaves.
10.0.0.82:6379 (4af24b2e...) -> 0 keys | 4816 slots | 1 slaves.
10.0.0.83:6379 (4657c4b5...) -> 0 keys | 4816 slots | 1 slaves.
[OK] 0 keys in 4 masters.
0.00 keys per slot on average.
>>> Performing Cluster Check (using node 10.0.0.88:6379)
M: 9ce04c3b13c4de557dd21e24fda37e39e7485c16 10.0.0.88:6379
   slots:[0-644],[5461-5502],[5795-6106] (999 slots) master
   1 additional replica(s)
S: fd815c656855c2862a08975e6ac488a113fcfe32 10.0.0.87:6379
   slots: (0 slots) slave
   replicates 4657c4b5fd7067568e6367cb52fbeaccb447a38b
S: c6b935f53eeffdd29fb393283628946d87296b1b 10.0.0.86:6379
   slots: (0 slots) slave
   replicates 4af24b2e09524dc2542b267468ff207922ad89f3
S: 34ed5c19d7c5b1a142bd33c7c70bd0b1639958bc 10.0.0.85:6379
   slots: (0 slots) slave
   replicates 23e761b4f8fa1107062b1f6ce42d40ca04bcd12d
M: 23e761b4f8fa1107062b1f6ce42d40ca04bcd12d 10.0.0.84:6379
   slots:[5503-5794],[10923-16383] (5753 slots) master
   1 additional replica(s)
M: 4af24b2e09524dc2542b267468ff207922ad89f3 10.0.0.82:6379
   slots:[645-5460] (4816 slots) master
   1 additional replica(s)
S: ea8867084f8038b492a15b071f9651cae5e4b6d7 10.0.0.89:6379
   slots: (0 slots) slave
   replicates 9ce04c3b13c4de557dd21e24fda37e39e7485c16
M: 4657c4b5fd7067568e6367cb52fbeaccb447a38b 10.0.0.83:6379
   slots:[6107-10922] (4816 slots) master
   1 additional replica(s)
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.


# 查看集群中的主从关系
[root@89 redis]# redis-cli -h 10.0.0.89 -p 6379 -c cluster nodes
4af24b2e09524dc2542b267468ff207922ad89f3 10.0.0.82:6379@16379 master - 0 1603505269000 1 connected 645-5460
23e761b4f8fa1107062b1f6ce42d40ca04bcd12d 10.0.0.84:6379@16379 master - 0 1603505270398 8 connected 5503-5794 10923-16383
c6b935f53eeffdd29fb393283628946d87296b1b 10.0.0.86:6379@16379 slave 4af24b2e09524dc2542b267468ff207922ad89f3 0 1603505266366 1 connected
ea8867084f8038b492a15b071f9651cae5e4b6d7 10.0.0.89:6379@16379 myself,slave 9ce04c3b13c4de557dd21e24fda37e39e7485c16 0 1603505267000 0 connected
9ce04c3b13c4de557dd21e24fda37e39e7485c16 10.0.0.88:6379@16379 master - 0 1603505271410 9 connected 0-644 5461-5502 5795-6106
34ed5c19d7c5b1a142bd33c7c70bd0b1639958bc 10.0.0.85:6379@16379 slave 23e761b4f8fa1107062b1f6ce42d40ca04bcd12d 0 1603505269389 8 connected
fd815c656855c2862a08975e6ac488a113fcfe32 10.0.0.87:6379@16379 slave 4657c4b5fd7067568e6367cb52fbeaccb447a38b 0 1603505268000 2 connected
4657c4b5fd7067568e6367cb52fbeaccb447a38b 10.0.0.83:6379@16379 master - 0 1603505272423 2 connected 6107-10922

11、删除从节点

# 命令
redis-cli --cluster del-node 从节点IP:端口 相应的从节点nodeID

# 删除10.0.0.89从节点
[root@89 redis]# redis-cli --cluster del-node 10.0.0.89:6379 ea8867084f8038b492a15b071f9651cae5e4b6d7
>>> Removing node ea8867084f8038b492a15b071f9651cae5e4b6d7 from cluster 10.0.0.89:6379
>>> Sending CLUSTER FORGET messages to the cluster...
>>> SHUTDOWN the node.

# 查看集群节点信息
[root@89 redis]# redis-cli -h 10.0.0.88 -p 6379 -c cluster nodes
fd815c656855c2862a08975e6ac488a113fcfe32 10.0.0.87:6379@16379 slave 4657c4b5fd7067568e6367cb52fbeaccb447a38b 0 1603505532000 2 connected
c6b935f53eeffdd29fb393283628946d87296b1b 10.0.0.86:6379@16379 slave 4af24b2e09524dc2542b267468ff207922ad89f3 0 1603505533332 1 connected
34ed5c19d7c5b1a142bd33c7c70bd0b1639958bc 10.0.0.85:6379@16379 slave 23e761b4f8fa1107062b1f6ce42d40ca04bcd12d 0 1603505534341 8 connected
23e761b4f8fa1107062b1f6ce42d40ca04bcd12d 10.0.0.84:6379@16379 master - 0 1603505531313 8 connected 5503-5794 10923-16383
4af24b2e09524dc2542b267468ff207922ad89f3 10.0.0.82:6379@16379 master - 0 1603505532322 1 connected 645-5460
9ce04c3b13c4de557dd21e24fda37e39e7485c16 10.0.0.88:6379@16379 myself,master - 0 1603505529000 9 connected 0-644 5461-5502 5795-6106
4657c4b5fd7067568e6367cb52fbeaccb447a38b 10.0.0.83:6379@16379 master - 0 1603505531000 2 connected 6107-10922
# 注意到，已经没有10.0.0.89的信息

12、删除主节点

# 删除主节点10.0.0.88
# 因为主节点分配了槽点，所以先将槽点转移到其他主节点上，然后再删除该节点
命令：redis-cli --cluster reshard 要删除的节点IP:端口

[root@89 redis]# redis-cli --cluster reshard 10.0.0.88:6379		# 删除指定的节点
>>> Performing Cluster Check (using node 10.0.0.88:6379)
M: 9ce04c3b13c4de557dd21e24fda37e39e7485c16 10.0.0.88:6379
   slots:[0-644],[5461-5502],[5795-6106] (999 slots) master  # 删除的节点槽点数slots
S: fd815c656855c2862a08975e6ac488a113fcfe32 10.0.0.87:6379
   slots: (0 slots) slave
   replicates 4657c4b5fd7067568e6367cb52fbeaccb447a38b
S: c6b935f53eeffdd29fb393283628946d87296b1b 10.0.0.86:6379
   slots: (0 slots) slave
   replicates 4af24b2e09524dc2542b267468ff207922ad89f3
S: 34ed5c19d7c5b1a142bd33c7c70bd0b1639958bc 10.0.0.85:6379
   slots: (0 slots) slave
   replicates 23e761b4f8fa1107062b1f6ce42d40ca04bcd12d
M: 23e761b4f8fa1107062b1f6ce42d40ca04bcd12d 10.0.0.84:6379
   slots:[5503-5794],[10923-16383] (5753 slots) master
   1 additional replica(s)
M: 4af24b2e09524dc2542b267468ff207922ad89f3 10.0.0.82:6379
   slots:[645-5460] (4816 slots) master
   1 additional replica(s)
M: 4657c4b5fd7067568e6367cb52fbeaccb447a38b 10.0.0.83:6379
   slots:[6107-10922] (4816 slots) master
   1 additional replica(s)
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
How many slots do you want to move (from 1 to 16384)? 999	# 删除的槽点数
What is the receiving node ID? 4af24b2e09524dc2542b267468ff207922ad89f3  # 将删除的节点槽点slots转移到新的节点nodeID
Please enter all the source node IDs.
  Type 'all' to use all the nodes as source nodes for the hash slots.
  Type 'done' once you entered all the source nodes IDs.
Source node #1: 9ce04c3b13c4de557dd21e24fda37e39e7485c16	# 设置slot的迁出方，就是要删除节点的nodeID
Source node #2: done
.......
    Moving slot 6098 from 9ce04c3b13c4de557dd21e24fda37e39e7485c16
    Moving slot 6099 from 9ce04c3b13c4de557dd21e24fda37e39e7485c16
    Moving slot 6100 from 9ce04c3b13c4de557dd21e24fda37e39e7485c16
    Moving slot 6101 from 9ce04c3b13c4de557dd21e24fda37e39e7485c16
    Moving slot 6102 from 9ce04c3b13c4de557dd21e24fda37e39e7485c16
    Moving slot 6103 from 9ce04c3b13c4de557dd21e24fda37e39e7485c16
    Moving slot 6104 from 9ce04c3b13c4de557dd21e24fda37e39e7485c16
    Moving slot 6105 from 9ce04c3b13c4de557dd21e24fda37e39e7485c16
    Moving slot 6106 from 9ce04c3b13c4de557dd21e24fda37e39e7485c16
Do you want to proceed with the proposed reshard plan (yes/no)? yes
.......
Moving slot 6098 from 10.0.0.88:6379 to 10.0.0.82:6379: 
Moving slot 6099 from 10.0.0.88:6379 to 10.0.0.82:6379: 
Moving slot 6100 from 10.0.0.88:6379 to 10.0.0.82:6379: 
Moving slot 6101 from 10.0.0.88:6379 to 10.0.0.82:6379: 
Moving slot 6102 from 10.0.0.88:6379 to 10.0.0.82:6379: 
Moving slot 6103 from 10.0.0.88:6379 to 10.0.0.82:6379: 
Moving slot 6104 from 10.0.0.88:6379 to 10.0.0.82:6379: 
Moving slot 6105 from 10.0.0.88:6379 to 10.0.0.82:6379: 
Moving slot 6106 from 10.0.0.88:6379 to 10.0.0.82:6379: 

# 查看集群节点信息
[root@89 redis]# redis-cli -h 10.0.0.88 -p 6379 -c cluster nodes
fd815c656855c2862a08975e6ac488a113fcfe32 10.0.0.87:6379@16379 slave 4657c4b5fd7067568e6367cb52fbeaccb447a38b 0 1603505995515 2 connected
c6b935f53eeffdd29fb393283628946d87296b1b 10.0.0.86:6379@16379 slave 4af24b2e09524dc2542b267468ff207922ad89f3 0 1603505996526 10 connected
34ed5c19d7c5b1a142bd33c7c70bd0b1639958bc 10.0.0.85:6379@16379 slave 23e761b4f8fa1107062b1f6ce42d40ca04bcd12d 0 1603505996000 8 connected
23e761b4f8fa1107062b1f6ce42d40ca04bcd12d 10.0.0.84:6379@16379 master - 0 1603505998544 8 connected 5503-5794 10923-16383
4af24b2e09524dc2542b267468ff207922ad89f3 10.0.0.82:6379@16379 master - 0 1603505997533 10 connected 0-5502 5795-6106
9ce04c3b13c4de557dd21e24fda37e39e7485c16 10.0.0.88:6379@16379 myself,master - 0 1603505998000 9 connected
4657c4b5fd7067568e6367cb52fbeaccb447a38b 10.0.0.83:6379@16379 master - 0 1603505999550 2 connected 6107-10922


# 删除主节点：redis-cli --cluster del-node 删除节点的IP:端口 nodeID
[root@89 redis]# redis-cli --cluster del-node 10.0.0.88:6379 9ce04c3b13c4de557dd21e24fda37e39e7485c16
>>> Removing node 9ce04c3b13c4de557dd21e24fda37e39e7485c16 from cluster 10.0.0.88:6379
>>> Sending CLUSTER FORGET messages to the cluster...
>>> SHUTDOWN the node.


# 查看集群节点信息
[root@89 redis]# redis-cli -h 10.0.0.82 -p 6379 -c cluster nodes
fd815c656855c2862a08975e6ac488a113fcfe32 10.0.0.87:6379@16379 slave 4657c4b5fd7067568e6367cb52fbeaccb447a38b 0 1603506126364 6 connected
c6b935f53eeffdd29fb393283628946d87296b1b 10.0.0.86:6379@16379 slave 4af24b2e09524dc2542b267468ff207922ad89f3 0 1603506124000 10 connected
23e761b4f8fa1107062b1f6ce42d40ca04bcd12d 10.0.0.84:6379@16379 master - 0 1603506121000 8 connected 5503-5794 10923-16383
4af24b2e09524dc2542b267468ff207922ad89f3 10.0.0.82:6379@16379 myself,master - 0 1603506122000 10 connected 0-5502 5795-6106
34ed5c19d7c5b1a142bd33c7c70bd0b1639958bc 10.0.0.85:6379@16379 slave 23e761b4f8fa1107062b1f6ce42d40ca04bcd12d 0 1603506123337 8 connected
4657c4b5fd7067568e6367cb52fbeaccb447a38b 10.0.0.83:6379@16379 master - 0 1603506125357 2 connected 6107-10922

# 此时集群节点已经没有10.0.0.88的信息
# 而且，10.0.0.88的redis服务已停止
[root@88 redis]# ss -ntl
State       Recv-Q      Send-Q           Local Address:Port             Peer Address:Port      
LISTEN      0           128                    0.0.0.0:22                    0.0.0.0:*         
LISTEN      0           128                    0.0.0.0:5355                  0.0.0.0:*         
LISTEN      0           128                       [::]:22                       [::]:*         
LISTEN      0           128                       [::]:5355                     [::]:*

13、删除节点

从集群中删除一个节点命令格式：

redis-cli --cluster del-node 127.0.0.1:6379 <node-id>

“127.0.0.1:6379”为集群中任意一个非待删除节点，“node-id”为待删除节点的ID。如果待删除的是master节点，则在删除之前需要将该master负责的slots先全部迁到其它master。

# 删除10.0.0.89从节点
[root@89 redis]# redis-cli --cluster del-node 10.0.0.89:6379 ea8867084f8038b492a15b071f9651cae5e4b6d7
>>> Removing node ea8867084f8038b492a15b071f9651cae5e4b6d7 from cluster 10.0.0.89:6379
>>> Sending CLUSTER FORGET messages to the cluster...
>>> SHUTDOWN the node.

如果删除后，其它节点还看得到这个被删除的节点，则可通过FORGET命令解决，需要在所有还看得到的其它节点上执行：

CLUSTER FORGET <node-id>

FORGET做两件事：

从节点表剔除节点；
在60秒的时间内，阻止相同ID的节点加进来。

14. 检查节点状态

以检查节点“10.0.0.82”的状态为例：

redis-cli --cluster check 10.0.0.82 6379

如发现如下这样的错误：

[WARNING] Node 10.0.0.82 6379 has slots in migrating state (5461).[WARNING] The following slots are open: 5461

可以使用redis命令取消slots迁移（5461为slot的ID）：

cluster setslot 5461 stable

需要注意，须登录到10.0.0.82 6379上执行redis的setslot子命令。

15. slots相关命令

CLUSTER ADDSLOTS slot1 [slot2] ... [slotN]
CLUSTER DELSLOTS slot1 [slot2] ... [slotN]
CLUSTER KEYSLOT key # 查看指定key的slot值

 
CLUSTER SETSLOT slot IMPORTING node # 在目标节点上执行，将slot设置为importing状态

CLUSTER SETSLOT slot MIGRATING node # 在源节点执行，将slot设置为migrating状态

CLUSTER SETSLOT slot STABLE # 清除slot的importing / migrating状态

CLUSTER SETSLOT slot NODE # 将slot和node关系绑定，清除importing / migrating状态

16、测试

1、验证slave的读写功能

# 查看复制信息
[root@83 redis]# redis-cli --cluster check 10.0.0.82 6379
10.0.0.82:6379 (4af24b2e...) -> 1 keys | 5815 slots | 1 slaves.
10.0.0.84:6379 (23e761b4...) -> 0 keys | 5753 slots | 1 slaves.
10.0.0.83:6379 (4657c4b5...) -> 0 keys | 4816 slots | 1 slaves.
[OK] 1 keys in 3 masters.
0.00 keys per slot on average.
>>> Performing Cluster Check (using node 10.0.0.82:6379)
M: 4af24b2e09524dc2542b267468ff207922ad89f3 10.0.0.82:6379
   slots:[0-5502],[5795-6106] (5815 slots) master
   1 additional replica(s)
S: fd815c656855c2862a08975e6ac488a113fcfe32 10.0.0.87:6379
   slots: (0 slots) slave
   replicates 4657c4b5fd7067568e6367cb52fbeaccb447a38b # 87复制83
S: c6b935f53eeffdd29fb393283628946d87296b1b 10.0.0.86:6379
   slots: (0 slots) slave
   replicates 4af24b2e09524dc2542b267468ff207922ad89f3 # 86复制82
M: 23e761b4f8fa1107062b1f6ce42d40ca04bcd12d 10.0.0.84:6379
   slots:[5503-5794],[10923-16383] (5753 slots) master
   1 additional replica(s)
S: 34ed5c19d7c5b1a142bd33c7c70bd0b1639958bc 10.0.0.85:6379
   slots: (0 slots) slave
   replicates 23e761b4f8fa1107062b1f6ce42d40ca04bcd12d # 85复制84
M: 4657c4b5fd7067568e6367cb52fbeaccb447a38b 10.0.0.83:6379
   slots:[6107-10922] (4816 slots) master
   1 additional replica(s)
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.


# 在从节点测试set
[root@85 redis]# redis-cli set age 18
(error) MOVED 741 10.0.0.82:6379
# 提示你要到对应的节点里面去执行set命令

# 在对应的主节点set
[root@82 redis]# redis-cli set name wang
OK

# 在从节点测试get
[root@85 redis]# redis-cli get name
(error) MOVED 5798 10.0.0.82:6379

# 说明：未使用集群模式，从节点redis上无法进行set等操作


# 进入集群模式，在任意从节点测试set
[root@85 redis]# redis-cli -c set age 18
OK
[root@85 redis]# redis-cli -c get age
"18"

# 再测试另一个从节点
[root@87 redis]# redis-cli -c get age
"18"
# 注意：并不是从set数据，而是它自动帮你转移到了相对应的redis主节点去set数据

2、模拟 master 故障，对应的slave节点自动提升为新master

# master：10.0.0.83
[root@83 redis]# redis-cli --cluster check 10.0.0.83 6379
10.0.0.83:6379 (4657c4b5...) -> 0 keys | 4816 slots | 1 slaves.
10.0.0.84:6379 (23e761b4...) -> 0 keys | 5753 slots | 1 slaves.
10.0.0.82:6379 (4af24b2e...) -> 2 keys | 5815 slots | 1 slaves.
[OK] 2 keys in 3 masters.
0.00 keys per slot on average.
>>> Performing Cluster Check (using node 10.0.0.83:6379)
M: 4657c4b5fd7067568e6367cb52fbeaccb447a38b 10.0.0.83:6379
   slots:[6107-10922] (4816 slots) master
   1 additional replica(s)
S: fd815c656855c2862a08975e6ac488a113fcfe32 10.0.0.87:6379
   slots: (0 slots) slave
   replicates 4657c4b5fd7067568e6367cb52fbeaccb447a38b
M: 23e761b4f8fa1107062b1f6ce42d40ca04bcd12d 10.0.0.84:6379
   slots:[5503-5794],[10923-16383] (5753 slots) master
   1 additional replica(s)
M: 4af24b2e09524dc2542b267468ff207922ad89f3 10.0.0.82:6379
   slots:[0-5502],[5795-6106] (5815 slots) master
   1 additional replica(s)
S: c6b935f53eeffdd29fb393283628946d87296b1b 10.0.0.86:6379
   slots: (0 slots) slave
   replicates 4af24b2e09524dc2542b267468ff207922ad89f3
S: 34ed5c19d7c5b1a142bd33c7c70bd0b1639958bc 10.0.0.85:6379
   slots: (0 slots) slave
   replicates 23e761b4f8fa1107062b1f6ce42d40ca04bcd12d
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.

# 停止redis：master-10.0.0.83
[root@83 redis]# systemctl stop redis

# 查看节点信息
[root@83 redis]# redis-cli -h 10.0.0.82 -p 6379 --cluster check 10.0.0.82 6379
Could not connect to Redis at 10.0.0.83:6379: Connection refused
*** WARNING: 10.0.0.87:6379 claims to be slave of unknown node ID 4657c4b5fd7067568e6367cb52fbeaccb447a38b.
10.0.0.82:6379 (4af24b2e...) -> 2 keys | 5815 slots | 1 slaves.
10.0.0.84:6379 (23e761b4...) -> 0 keys | 5753 slots | 1 slaves.
[OK] 2 keys in 2 masters.
0.00 keys per slot on average.
>>> Performing Cluster Check (using node 10.0.0.82:6379)
M: 4af24b2e09524dc2542b267468ff207922ad89f3 10.0.0.82:6379
   slots:[0-5502],[5795-6106] (5815 slots) master
   1 additional replica(s)
S: fd815c656855c2862a08975e6ac488a113fcfe32 10.0.0.87:6379
   slots: (0 slots) slave
   replicates 4657c4b5fd7067568e6367cb52fbeaccb447a38b
S: c6b935f53eeffdd29fb393283628946d87296b1b 10.0.0.86:6379
   slots: (0 slots) slave
   replicates 4af24b2e09524dc2542b267468ff207922ad89f3
M: 23e761b4f8fa1107062b1f6ce42d40ca04bcd12d 10.0.0.84:6379
   slots:[5503-5794],[10923-16383] (5753 slots) master
   1 additional replica(s)
S: 34ed5c19d7c5b1a142bd33c7c70bd0b1639958bc 10.0.0.85:6379
   slots: (0 slots) slave
   replicates 23e761b4f8fa1107062b1f6ce42d40ca04bcd12d
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[ERR] Not all 16384 slots are covered by nodes.

# 再次查看节点信息,此时slave：10.0.0.87已经提升为master
[root@83 redis]# redis-cli -h 10.0.0.82 -p 6379 --cluster check 10.0.0.82 6379
Could not connect to Redis at 10.0.0.83:6379: Connection refused
10.0.0.82:6379 (4af24b2e...) -> 2 keys | 5815 slots | 1 slaves.
10.0.0.87:6379 (fd815c65...) -> 0 keys | 4816 slots | 0 slaves.
10.0.0.84:6379 (23e761b4...) -> 0 keys | 5753 slots | 1 slaves.
[OK] 2 keys in 3 masters.
0.00 keys per slot on average.
>>> Performing Cluster Check (using node 10.0.0.82:6379)
M: 4af24b2e09524dc2542b267468ff207922ad89f3 10.0.0.82:6379
   slots:[0-5502],[5795-6106] (5815 slots) master
   1 additional replica(s)
M: fd815c656855c2862a08975e6ac488a113fcfe32 10.0.0.87:6379
   slots:[6107-10922] (4816 slots) master
S: c6b935f53eeffdd29fb393283628946d87296b1b 10.0.0.86:6379
   slots: (0 slots) slave
   replicates 4af24b2e09524dc2542b267468ff207922ad89f3
M: 23e761b4f8fa1107062b1f6ce42d40ca04bcd12d 10.0.0.84:6379
   slots:[5503-5794],[10923-16383] (5753 slots) master
   1 additional replica(s)
S: 34ed5c19d7c5b1a142bd33c7c70bd0b1639958bc 10.0.0.85:6379
   slots: (0 slots) slave
   replicates 23e761b4f8fa1107062b1f6ce42d40ca04bcd12d
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.


[root@83 redis]# redis-cli -h 10.0.0.82 -p 6379 --cluster info 10.0.0.82 6379
Could not connect to Redis at 10.0.0.83:6379: Connection refused
10.0.0.82:6379 (4af24b2e...) -> 2 keys | 5815 slots | 1 slaves.
10.0.0.87:6379 (fd815c65...) -> 0 keys | 4816 slots | 0 slaves.
10.0.0.84:6379 (23e761b4...) -> 0 keys | 5753 slots | 1 slaves.
[OK] 2 keys in 3 masters.
0.00 keys per slot on average.


# 再次重启10.0.0.83
[root@83 redis]# systemctl restart redis
[root@83 redis]# redis-cli -h 10.0.0.82 -p 6379 --cluster check 10.0.0.82 6379
10.0.0.82:6379 (4af24b2e...) -> 2 keys | 5815 slots | 1 slaves.
10.0.0.87:6379 (fd815c65...) -> 0 keys | 4816 slots | 1 slaves.
10.0.0.84:6379 (23e761b4...) -> 0 keys | 5753 slots | 1 slaves.
[OK] 2 keys in 3 masters.
0.00 keys per slot on average.
>>> Performing Cluster Check (using node 10.0.0.82:6379)
M: 4af24b2e09524dc2542b267468ff207922ad89f3 10.0.0.82:6379
   slots:[0-5502],[5795-6106] (5815 slots) master
   1 additional replica(s)
M: fd815c656855c2862a08975e6ac488a113fcfe32 10.0.0.87:6379
   slots:[6107-10922] (4816 slots) master
   1 additional replica(s)
S: c6b935f53eeffdd29fb393283628946d87296b1b 10.0.0.86:6379
   slots: (0 slots) slave
   replicates 4af24b2e09524dc2542b267468ff207922ad89f3
M: 23e761b4f8fa1107062b1f6ce42d40ca04bcd12d 10.0.0.84:6379
   slots:[5503-5794],[10923-16383] (5753 slots) master
   1 additional replica(s)
S: 34ed5c19d7c5b1a142bd33c7c70bd0b1639958bc 10.0.0.85:6379
   slots: (0 slots) slave
   replicates 23e761b4f8fa1107062b1f6ce42d40ca04bcd12d
S: 4657c4b5fd7067568e6367cb52fbeaccb447a38b 10.0.0.83:6379
   slots: (0 slots) slave
   replicates fd815c656855c2862a08975e6ac488a113fcfe32
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.

# 此时发现10.0.0.83已经变成10.0.0.87的slave

無賴

关注

2
点赞
踩
1

收藏

觉得还不错? 一键收藏
0
评论
Redis5.0相关问题及哨兵模式和cluster集群搭建

1、RDB和AOF的优缺点RDB优点RDB 是一个非常紧凑（compact）的文件，体积小，传输速度快，适合灾备。RDB 可以最大化 Redis 的性能：父进程在保存 RDB 文件时唯一要做的就是 fork 出一个子进程，然后这个子进程就会处理接下来的所有保存工作，父进程无须执行任何磁盘 I/O 操作。RDB 在恢复大数据集时的速度比 AOF 的恢复速度要快很多。缺点RDB 丢数据将会丢掉两次持久化之间的所有数据～当redis中数据集比较大时候，RDB由于RDB方式需
复制链接

扫一扫