一、环境
CentOS 7.9
redis 6.2.11
3台服务器集群,分别装的主和从。
场景:服务器1:63,服务器被重置了。
主IP | 从复制谁 (不是自己) | 端口 | 集群端口 | 备注 |
|
|
| ||
10.10.10.64 | 10.10.10.63 | 主:6479 从:6579 | 16479 16579 | 主节点 从节点 |
10.10.10.65 | 10.10.10.64 | 主:6479 从:6579 | 16479 16579 | 主节点 从节点 |
二、检查集群
redis-cli --cluster help
[root@cespprod1 redis-6.2.11]# redis-cli --cluster check 10.10.10.64:6479 -a myPassword
Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
10.10.10.64:6479 (05f48182...) -> 28 keys | 5462 slots | 1 slaves.
10.10.10.65:6479 (903a7fc4...) -> 32 keys | 5461 slots | 0 slaves.
10.10.10.64:6579 (da133856...) -> 25 keys | 5461 slots | 0 slaves.
[OK] 85 keys in 3 masters.
0.01 keys per slot on average.
>>> Performing Cluster Check (using node 10.10.10.64:6479)
M: 05f481827fc5d3a0362a3d7a06eb1fbd52c73303 10.10.10.64:6479
slots:[5461-10922] (5462 slots) master
1 additional replica(s)
M: 903a7fc4ee63c8cde8276fa5d7be5d4a258e2005 10.10.10.65:6479
slots:[10923-16383] (5461 slots) master
M: da133856054e4e160d962782389bc461af21a704 10.10.10.64:6579
slots:[0-5460] (5461 slots) master
S: 9b11853186c0c4d48b4210ddab042755ba68fd06 10.10.10.65:6579
slots: (0 slots) slave
replicates 05f481827fc5d3a0362a3d7a06eb1fbd52c73303
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
原来的三主三从,故障转移,有个一从自动升成主了。
三、添加新主节点,新从节点
1、添加主节点
redis-cli --cluster add-node 10.10.10.63:6479 10.10.10.64:6479 -a myPassword
前面是要添加的节点,后面的是集群中某一个节点
2、添加从节点
redis-cli --cluster add-node 10.10.10.63:6579 10.10.10.64:6479 --cluster-slave --cluster-master-id be7a6314fa34e2e3957d68368c99cb2c24c22b01 -a myPassword
前面是要添加的节点,后面的是集群中某一个节点
--cluster-slave 添加从节点
--cluster-master-id 指定主节点的ID
3、添加主节点
[root@cespprod1 redis-6.2.11]# redis-cli --cluster add-node 10.10.10.63:6479 10.10.10.64:6479 -a myPassword
Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
>>> Adding node 10.10.10.63:6479 to cluster 10.10.10.64:6479
>>> Performing Cluster Check (using node 10.10.10.64:6479)
M: 05f481827fc5d3a0362a3d7a06eb1fbd52c73303 10.10.10.64:6479
slots:[5461-10922] (5462 slots) master
1 additional replica(s)
M: 903a7fc4ee63c8cde8276fa5d7be5d4a258e2005 10.10.10.65:6479
slots:[10923-16383] (5461 slots) master
M: da133856054e4e160d962782389bc461af21a704 10.10.10.64:6579
slots:[0-5460] (5461 slots) master
S: 9b11853186c0c4d48b4210ddab042755ba68fd06 10.10.10.65:6579
slots: (0 slots) slave
replicates 05f481827fc5d3a0362a3d7a06eb1fbd52c73303
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
>>> Send CLUSTER MEET to node 10.10.10.63:6479 to make it join the cluster.
[OK] New node added correctly.
4、查看主节点
[root@cespprod1 redis-6.2.11]# redis-cli --cluster check 10.10.10.64:6479 -a myPassword
Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
10.10.10.64:6479 (05f48182...) -> 27 keys | 5462 slots | 1 slaves.
10.10.10.63:6479 (be7a6314...) -> 0 keys | 0 slots | 0 slaves.
10.10.10.65:6479 (903a7fc4...) -> 30 keys | 5461 slots | 0 slaves.
10.10.10.64:6579 (da133856...) -> 25 keys | 5461 slots | 0 slaves.
[OK] 82 keys in 4 masters.
0.01 keys per slot on average.
>>> Performing Cluster Check (using node 10.10.10.64:6479)
M: 05f481827fc5d3a0362a3d7a06eb1fbd52c73303 10.10.10.64:6479
slots:[5461-10922] (5462 slots) master
1 additional replica(s)
M: be7a6314fa34e2e3957d68368c99cb2c24c22b01 10.10.10.63:6479
slots: (0 slots) master
M: 903a7fc4ee63c8cde8276fa5d7be5d4a258e2005 10.10.10.65:6479
slots:[10923-16383] (5461 slots) master
M: da133856054e4e160d962782389bc461af21a704 10.10.10.64:6579
slots:[0-5460] (5461 slots) master
S: 9b11853186c0c4d48b4210ddab042755ba68fd06 10.10.10.65:6579
slots: (0 slots) slave
replicates 05f481827fc5d3a0362a3d7a06eb1fbd52c73303
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
5、添加从节点
如果主节点指定错了,可以删除从节点,重新添加,也可以改变从的主节点。
[root@cespprod1 redis-6.2.11]# redis-cli --cluster add-node 10.10.10.63:6579 10.10.10.64:6479 --cluster-slave --cluster-master-id be7a6314fa34e2e3957d68368c99cb2c24c22b01 -a myPassword
Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
>>> Adding node 10.10.10.63:6579 to cluster 10.10.10.64:6479
>>> Performing Cluster Check (using node 10.10.10.64:6479)
M: 05f481827fc5d3a0362a3d7a06eb1fbd52c73303 10.10.10.64:6479
slots:[5461-10922] (5462 slots) master
1 additional replica(s)
M: be7a6314fa34e2e3957d68368c99cb2c24c22b01 10.10.10.63:6479
slots: (0 slots) master
M: 903a7fc4ee63c8cde8276fa5d7be5d4a258e2005 10.10.10.65:6479
slots:[10923-16383] (5461 slots) master
M: da133856054e4e160d962782389bc461af21a704 10.10.10.64:6579
slots:[0-5460] (5461 slots) master
S: 9b11853186c0c4d48b4210ddab042755ba68fd06 10.10.10.65:6579
slots: (0 slots) slave
replicates 05f481827fc5d3a0362a3d7a06eb1fbd52c73303
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
>>> Send CLUSTER MEET to node 10.10.10.63:6579 to make it join the cluster.
Waiting for the cluster to join
>>> Configure node as replica of 10.10.10.63:6479.
[OK] New node added correctly.
6、查看从节点
[root@cespprod1 redis-6.2.11]# redis-cli --cluster check 10.10.10.64:6479 -a myPassword
Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
10.10.10.64:6479 (05f48182...) -> 27 keys | 5462 slots | 1 slaves.
10.10.10.63:6479 (be7a6314...) -> 0 keys | 0 slots | 1 slaves.
10.10.10.65:6479 (903a7fc4...) -> 30 keys | 5461 slots | 0 slaves.
10.10.10.64:6579 (da133856...) -> 25 keys | 5461 slots | 0 slaves.
[OK] 82 keys in 4 masters.
0.01 keys per slot on average.
>>> Performing Cluster Check (using node 10.10.10.64:6479)
M: 05f481827fc5d3a0362a3d7a06eb1fbd52c73303 10.10.10.64:6479
slots:[5461-10922] (5462 slots) master
1 additional replica(s)
M: be7a6314fa34e2e3957d68368c99cb2c24c22b01 10.10.10.63:6479
slots: (0 slots) master
1 additional replica(s)
M: 903a7fc4ee63c8cde8276fa5d7be5d4a258e2005 10.10.10.65:6479
slots:[10923-16383] (5461 slots) master
M: da133856054e4e160d962782389bc461af21a704 10.10.10.64:6579
slots:[0-5460] (5461 slots) master
S: 9b11853186c0c4d48b4210ddab042755ba68fd06 10.10.10.65:6579
slots: (0 slots) slave
replicates 05f481827fc5d3a0362a3d7a06eb1fbd52c73303
S: ec2a04d625c62cee5efd7c6de38873ea4b043702 10.10.10.63:6579
slots: (0 slots) slave
replicates be7a6314fa34e2e3957d68368c99cb2c24c22b01
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
7、改变从的主
方式一
[root@ ~]# redis-cli -h 127.0.0.1 -p 6579 -a myPassword
127.0.0.1:6579> replicaof 10.10.10.64 6479 (新版本)
127.0.0.1:6579> slaveof 10.10.10.64 6479 (老版本)
方式二
改变当前从的主
CLUSTER REPLICATE 主的ID
[root@ ~]# redis-cli -h 127.0.0.1 -p 6579 -a myPassword
Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
127.0.0.1:6579> CLUSTER REPLICATE 903a7fc4ee63c8cde8276fa5d7be5d4a258e2005
OK
127.0.0.1:6579> CLUSTER SAVECONFIG
OK
8、查看改变的从
[root@cespprod1 redis-6.2.11]# redis-cli --cluster check 10.10.10.64:6479 -a myPassword
Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
10.10.10.64:6479 (05f48182...) -> 28 keys | 5462 slots | 1 slaves.
10.10.10.63:6479 (be7a6314...) -> 0 keys | 0 slots | 0 slaves.
10.10.10.65:6479 (903a7fc4...) -> 31 keys | 5461 slots | 1 slaves.
10.10.10.64:6579 (da133856...) -> 25 keys | 5461 slots | 0 slaves.
[OK] 84 keys in 4 masters.
0.01 keys per slot on average.
>>> Performing Cluster Check (using node 10.10.10.64:6479)
M: 05f481827fc5d3a0362a3d7a06eb1fbd52c73303 10.10.10.64:6479
slots:[5461-10922] (5462 slots) master
1 additional replica(s)
M: be7a6314fa34e2e3957d68368c99cb2c24c22b01 10.10.10.63:6479
slots: (0 slots) master
M: 903a7fc4ee63c8cde8276fa5d7be5d4a258e2005 10.10.10.65:6479
slots:[10923-16383] (5461 slots) master
1 additional replica(s)
M: da133856054e4e160d962782389bc461af21a704 10.10.10.64:6579
slots:[0-5460] (5461 slots) master
S: 9b11853186c0c4d48b4210ddab042755ba68fd06 10.10.10.65:6579
slots: (0 slots) slave
replicates 05f481827fc5d3a0362a3d7a06eb1fbd52c73303
S: ec2a04d625c62cee5efd7c6de38873ea4b043702 10.10.10.63:6579
slots: (0 slots) slave
replicates 903a7fc4ee63c8cde8276fa5d7be5d4a258e2005
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
四、迁移slot
1、迁移slot
host:最好写局域网IP,不写127.0.0.1(可能容易失败)
--cluster-slots:迁移的solt数量,可通过check查询,如果迁移部分失败后再次迁移,这个数量会变 。
--cluster-timeout:超时时间(毫秒),最好写上,要不默认60s,容易失败
迁移slot,中途失败,被迁移的原主节点可能会从master降为slave,被迁移的原从节点可能升级为master,后续可以调整。
redis-cli --cluster reshard 10.10.10.64:6479 --cluster-from da133856054e4e160d962782389bc461af21a704 --cluster-to be7a6314fa34e2e3957d68368c99cb2c24c22b01 --cluster-slots 5461 --cluster-yes --cluster-timeout 3600000 -a myPassword
Moving slot 5458 from 10.10.10.64:6579 to 10.10.10.63:6479:
Moving slot 5459 from 10.10.10.64:6579 to 10.10.10.63:6479:
Moving slot 5460 from 10.10.10.64:6579 to 10.10.10.63:6479:
Node 10.10.10.64:6579 replied with error:
ERR Please use SETSLOT only with masters.
这个错误,可忽略,是因为迁移完slot后,节点从主降到从了。
2、检查集群
[root@cespprod1 redis-6.2.11]# redis-cli --cluster check 10.10.10.64:6479 -a myPassword
Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
10.10.10.64:6479 (05f48182...) -> 28 keys | 5462 slots | 1 slaves.
10.10.10.63:6479 (be7a6314...) -> 25 keys | 5461 slots | 1 slaves.
10.10.10.65:6479 (903a7fc4...) -> 30 keys | 5461 slots | 1 slaves.
[OK] 83 keys in 3 masters.
0.01 keys per slot on average.
>>> Performing Cluster Check (using node 10.10.10.64:6479)
M: 05f481827fc5d3a0362a3d7a06eb1fbd52c73303 10.10.10.64:6479
slots:[5461-10922] (5462 slots) master
1 additional replica(s)
M: be7a6314fa34e2e3957d68368c99cb2c24c22b01 10.10.10.63:6479
slots:[0-5460] (5461 slots) master
1 additional replica(s)
M: 903a7fc4ee63c8cde8276fa5d7be5d4a258e2005 10.10.10.65:6479
slots:[10923-16383] (5461 slots) master
1 additional replica(s)
S: da133856054e4e160d962782389bc461af21a704 10.10.10.64:6579
slots: (0 slots) slave
replicates be7a6314fa34e2e3957d68368c99cb2c24c22b01
S: 9b11853186c0c4d48b4210ddab042755ba68fd06 10.10.10.65:6579
slots: (0 slots) slave
replicates 05f481827fc5d3a0362a3d7a06eb1fbd52c73303
S: ec2a04d625c62cee5efd7c6de38873ea4b043702 10.10.10.63:6579
slots: (0 slots) slave
replicates 903a7fc4ee63c8cde8276fa5d7be5d4a258e2005
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
参考
报错处理,修复集群