Docker Image Version
redis:7.0.11-alpine
方案意义
为了解决 Redis 高性能高可用的问题,Redis 推出了自带的 Cluster 模式。他通过将一整个集群分成 16384 哈希槽位,再将槽位平均分配到每个主节点上面。同时主节点采用 Redis Replication 主从模式(自带 Sentinel 哨兵机制)防止主节点故障。同时 Redis 作者强烈不建议集群数目大于 1000 台节点,因为通信需要占用很大的带宽。
三主三从 Docker Compose 基础配置
version: '3.8'
services:
redis-cluster1:
container_name: redis-cluster1
image: redis:7.0.11-alpine
hostname: redis-cluster1
restart: unless-stopped
ports:
- "6379:6379"
networks:
redis-cluster:
logging:
driver: json-file
options:
max-size: 100m
max-file: '3'
volumes:
- redis-cluster1:/data
command:
- redis-server
- --bind
- "0.0.0.0"
- --port
- "6379"
- --requirepass
- "ChangeMe"
- --appendonly
- "yes"
- --masterauth
- "ChangeMe"
- --cluster-enabled
- "yes"
- --cluster-config-file
- "redis-cluster1.conf"
redis-cluster2:
container_name: redis-cluster2
image: redis:7.0.11-alpine
hostname: redis-cluster2
restart: unless-stopped
ports:
- "6380:6379"
networks:
redis-cluster:
logging:
driver: json-file
options:
max-size: 100m
max-file: '3'
volumes:
- redis-cluster2:/data
command:
- redis-server
- --bind
- "0.0.0.0"
- --port
- "6379"
- --requirepass
- "ChangeMe"
- --appendonly
- "yes"
- --masterauth
- "ChangeMe"
- --cluster-enabled
- "yes"
- --cluster-config-file
- "redis-cluster2.conf"
redis-cluster3:
container_name: redis-cluster3
image: redis:7.0.11-alpine
hostname: redis-cluster3
restart: unless-stopped
ports:
- "6381:6379"
networks:
redis-cluster:
logging:
driver: json-file
options:
max-size: 100m
max-file: '3'
volumes:
- redis-cluster3:/data
command:
- redis-server
- --bind
- "0.0.0.0"
- --port
- "6379"
- --requirepass
- "ChangeMe"
- --appendonly
- "yes"
- --masterauth
- "ChangeMe"
- --cluster-enabled
- "yes"
- --cluster-config-file
- "redis-cluster3.conf"
redis-cluster4:
container_name: redis-cluster4
image: redis:7.0.11-alpine
hostname: redis-cluster4
restart: unless-stopped
ports:
- "6382:6379"
networks:
redis-cluster:
logging:
driver: json-file
options:
max-size: 100m
max-file: '3'
volumes:
- redis-cluster4:/data
command:
- redis-server
- --bind
- "0.0.0.0"
- --port
- "6379"
- --requirepass
- "ChangeMe"
- --appendonly
- "yes"
- --masterauth
- "ChangeMe"
- --cluster-enabled
- "yes"
- --cluster-config-file
- "redis-cluster4.conf"
redis-cluster5:
container_name: redis-cluster5
image: redis:7.0.11-alpine
hostname: redis-cluster5
restart: unless-stopped
ports:
- "6383:6379"
networks:
redis-cluster:
logging:
driver: json-file
options:
max-size: 100m
max-file: '3'
volumes:
- redis-cluster5:/data
command:
- redis-server
- --bind
- "0.0.0.0"
- --port
- "6379"
- --requirepass
- "ChangeMe"
- --appendonly
- "yes"
- --masterauth
- "ChangeMe"
- --cluster-enabled
- "yes"
- --cluster-config-file
- "redis-cluster5.conf"
redis-cluster6:
container_name: redis-cluster6
image: redis:7.0.11-alpine
hostname: redis-cluster6
restart: unless-stopped
ports:
- "6384:6379"
networks:
redis-cluster:
logging:
driver: json-file
options:
max-size: 100m
max-file: '3'
volumes:
- redis-cluster6:/data
command:
- redis-server
- --bind
- "0.0.0.0"
- --port
- "6379"
- --requirepass
- "ChangeMe"
- --appendonly
- "yes"
- --masterauth
- "ChangeMe"
- --cluster-enabled
- "yes"
- --cluster-config-file
- "redis-cluster6.conf"
volumes:
redis-cluster1:
redis-cluster2:
redis-cluster3:
redis-cluster4:
redis-cluster5:
redis-cluster6:
networks:
redis-cluster:
driver: bridge
ipam:
config:
- subnet: 172.21.0.0/16
创建 Cluster 集群
docker exec -it redis-cluster1 redis-cli -a ChangeMe --cluster create --cluster-replicas 1 redis-cluster1:6379 redis-cluster2:6379 redis-cluster3:6379 redis-cluster4:6379 redis-cluster5:6379 redis-cluster6:6379
>>> Performing hash slots allocation on 6 nodes...
Master[0] -> Slots 0 - 5460
Master[1] -> Slots 5461 - 10922
Master[2] -> Slots 10923 - 16383
Adding replica redis-cluster5:6379 to redis-cluster1:6379
Adding replica redis-cluster6:6379 to redis-cluster2:6379
Adding replica redis-cluster4:6379 to redis-cluster3:6379
M: 5e262c16f666affcd260d5688242d993194b86d7 redis-cluster1:6379
slots:[0-5460] (5461 slots) master
M: 0f9955ecb806141e8149bf63a7ca9ffb6bc73918 redis-cluster2:6379
slots:[5461-10922] (5462 slots) master
M: 63b7152d0af00bcd55a4b7ac3811104ae810fef5 redis-cluster3:6379
slots:[10923-16383] (5461 slots) master
S: 8d50b5b6e61e2eebdf80ca7b3a3d75858fc2b3a0 redis-cluster4:6379
replicates 63b7152d0af00bcd55a4b7ac3811104ae810fef5
S: 0b7aa8453ab1607fdc4c4a893945f239b4da92fa redis-cluster5:6379
replicates 5e262c16f666affcd260d5688242d993194b86d7
S: 210dceb99e1e3a0508ee9a0262afcdaaddfe46a9 redis-cluster6:6379
replicates 0f9955ecb806141e8149bf63a7ca9ffb6bc73918
Can I set the above configuration? (type 'yes' to accept): yes
>>> Nodes configuration updated
>>> Assign a different config epoch to each node
>>> Sending CLUSTER MEET messages to join the cluster
Waiting for the cluster to join
.
>>> Performing Cluster Check (using node redis-cluster1:6379)
M: 5e262c16f666affcd260d5688242d993194b86d7 redis-cluster1:6379
slots:[0-5460] (5461 slots) master
1 additional replica(s)
M: 0f9955ecb806141e8149bf63a7ca9ffb6bc73918 172.21.0.5:6379
slots:[5461-10922] (5462 slots) master
1 additional replica(s)
S: 8d50b5b6e61e2eebdf80ca7b3a3d75858fc2b3a0 172.21.0.7:6379
slots: (0 slots) slave
replicates 63b7152d0af00bcd55a4b7ac3811104ae810fef5
S: 210dceb99e1e3a0508ee9a0262afcdaaddfe46a9 172.21.0.2:6379
slots: (0 slots) slave
replicates 0f9955ecb806141e8149bf63a7ca9ffb6bc73918
M: 63b7152d0af00bcd55a4b7ac3811104ae810fef5 172.21.0.4:6379
slots:[10923-16383] (5461 slots) master
1 additional replica(s)
S: 0b7aa8453ab1607fdc4c4a893945f239b4da92fa 172.21.0.6:6379
slots: (0 slots) slave
replicates 5e262c16f666affcd260d5688242d993194b86d7
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
集群查询
docker exec -it redis-cluster1 redis-cli -a ChangeMe -p 6379 cluster nodes
0f9955ecb806141e8149bf63a7ca9ffb6bc73918 172.21.0.5:6379@16379 master - 0 1687404017000 2 connected 5461-10922
8d50b5b6e61e2eebdf80ca7b3a3d75858fc2b3a0 172.21.0.7:6379@16379 slave 63b7152d0af00bcd55a4b7ac3811104ae810fef5 0 1687404018111 3 connected
210dceb99e1e3a0508ee9a0262afcdaaddfe46a9 172.21.0.2:6379@16379 slave 0f9955ecb806141e8149bf63a7ca9ffb6bc73918 0 1687404018000 2 connected
63b7152d0af00bcd55a4b7ac3811104ae810fef5 172.21.0.4:6379@16379 master - 0 1687404019136 3 connected 10923-16383
0b7aa8453ab1607fdc4c4a893945f239b4da92fa 172.21.0.6:6379@16379 slave 5e262c16f666affcd260d5688242d993194b86d7 0 1687404020165 1 connected
5e262c16f666affcd260d5688242d993194b86d7 172.21.0.3:6379@16379 myself,master - 0 1687404017000 1 connected 0-5460
集群水平扩展
启动redis-cluster7
和redis-cluster8
两台容器
docker run -d --hostname redis-cluster7 --name redis-cluster7 --log-driver=json-file \
--log-opt max-size=100m --log-opt max-file=3 --restart=unless-stopped \
-v redis-cluster7:/data -p 6385:6379 --network=redis_redis-cluster redis:7.0.11-alpine \
redis-server --requirepass ChangeMe --appendonly yes --cluster-enabled yes \
--cluster-config-file redis-cluster7.conf && \
docker run -d --hostname redis-cluster8 --name redis-cluster8 --log-driver=json-file \
--log-opt max-size=100m --log-opt max-file=3 --restart=unless-stopped \
-v redis-cluster8:/data -p 6386:6379 --network=redis_redis-cluster redis:7.0.11-alpine \
redis-server --requirepass ChangeMe --appendonly yes --cluster-enabled yes \
--cluster-config-file redis-cluster8.conf
将redis-cluster7
作为主节点,redis-cluster8
作为redis-cluster7
的副节点加入集群
docker exec -it redis-cluster1 redis-cli -a ChangeMe -p 6379 --cluster add-node redis-cluster7:6379 redis-cluster1:6379
>>> Adding node redis-cluster7:6379 to cluster redis-cluster1:6379
>>> Performing Cluster Check (using node redis-cluster1:6379)
M: 5e262c16f666affcd260d5688242d993194b86d7 redis-cluster1:6379
slots:[0-5460] (5461 slots) master
1 additional replica(s)
M: 0f9955ecb806141e8149bf63a7ca9ffb6bc73918 172.21.0.5:6379
slots:[5461-10922] (5462 slots) master
1 additional replica(s)
S: 8d50b5b6e61e2eebdf80ca7b3a3d75858fc2b3a0 172.21.0.7:6379
slots: (0 slots) slave
replicates 63b7152d0af00bcd55a4b7ac3811104ae810fef5
S: 210dceb99e1e3a0508ee9a0262afcdaaddfe46a9 172.21.0.2:6379
slots: (0 slots) slave
replicates 0f9955ecb806141e8149bf63a7ca9ffb6bc73918
M: 63b7152d0af00bcd55a4b7ac3811104ae810fef5 172.21.0.4:6379
slots:[10923-16383] (5461 slots) master
1 additional replica(s)
S: 0b7aa8453ab1607fdc4c4a893945f239b4da92fa 172.21.0.6:6379
slots: (0 slots) slave
replicates 5e262c16f666affcd260d5688242d993194b86d7
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
>>> Getting functions from cluster
>>> Send FUNCTION LIST to redis-cluster7:6379 to verify there is no functions in it
>>> Send FUNCTION RESTORE to redis-cluster7:6379
>>> Send CLUSTER MEET to node redis-cluster7:6379 to make it join the cluster.
[OK] New node added correctly.
查询redis-cluster7
的master id
为3194b5c5820004232b351a9a9170a1481c34e31f
docker exec -it redis-cluster7 redis-cli -a ChangeMe -p 6379 cluster nodes
5e262c16f666affcd260d5688242d993194b86d7 172.21.0.3:6379@16379 master - 0 1687406681000 1 connected 0-5460
3194b5c5820004232b351a9a9170a1481c34e31f 172.21.0.8:6379@16379 myself,master - 0 1687406682000 0 connected
8d50b5b6e61e2eebdf80ca7b3a3d75858fc2b3a0 172.21.0.7:6379@16379 slave 63b7152d0af00bcd55a4b7ac3811104ae810fef5 0 1687406684006 3 connected
0f9955ecb806141e8149bf63a7ca9ffb6bc73918 172.21.0.5:6379@16379 master - 0 1687406680000 2 connected 5461-10922
210dceb99e1e3a0508ee9a0262afcdaaddfe46a9 172.21.0.2:6379@16379 slave 0f9955ecb806141e8149bf63a7ca9ffb6bc73918 0 1687406681946 2 connected
0b7aa8453ab1607fdc4c4a893945f239b4da92fa 172.21.0.6:6379@16379 slave 5e262c16f666affcd260d5688242d993194b86d7 0 1687406682972 1 connected
63b7152d0af00bcd55a4b7ac3811104ae810fef5 172.21.0.4:6379@16379 master - 0 1687406680920 3 connected 10923-16383
加入副节点redis-cluster8
docker exec -it redis-cluster1 redis-cli -a ChangeMe --cluster add-node redis-cluster8:6379 redis-cluster1:6379 --cluster-slave --cluster-master-id 3194b5c5820004232b351a9a9170a1481c34e31f
>>> Adding node redis-cluster8:6379 to cluster redis-cluster1:6379
>>> Performing Cluster Check (using node redis-cluster1:6379)
M: 5e262c16f666affcd260d5688242d993194b86d7 redis-cluster1:6379
slots:[0-5460] (5461 slots) master
1 additional replica(s)
M: 3194b5c5820004232b351a9a9170a1481c34e31f 172.21.0.8:6379
slots: (0 slots) master
M: 0f9955ecb806141e8149bf63a7ca9ffb6bc73918 172.21.0.5:6379
slots:[5461-10922] (5462 slots) master
1 additional replica(s)
S: 8d50b5b6e61e2eebdf80ca7b3a3d75858fc2b3a0 172.21.0.7:6379
slots: (0 slots) slave
replicates 63b7152d0af00bcd55a4b7ac3811104ae810fef5
S: 210dceb99e1e3a0508ee9a0262afcdaaddfe46a9 172.21.0.2:6379
slots: (0 slots) slave
replicates 0f9955ecb806141e8149bf63a7ca9ffb6bc73918
M: 63b7152d0af00bcd55a4b7ac3811104ae810fef5 172.21.0.4:6379
slots:[10923-16383] (5461 slots) master
1 additional replica(s)
S: 0b7aa8453ab1607fdc4c4a893945f239b4da92fa 172.21.0.6:6379
slots: (0 slots) slave
replicates 5e262c16f666affcd260d5688242d993194b86d7
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
>>> Send CLUSTER MEET to node redis-cluster8:6379 to make it join the cluster.
Waiting for the cluster to join
>>> Configure node as replica of 172.21.0.8:6379.
[OK] New node added correctly.
重新分配卡槽
docker exec -it redis-cluster1 redis-cli -a ChangeMe --cluster reshard redis-cluster1:6379
>>> Performing Cluster Check (using node redis-cluster1:6379)
M: 5e262c16f666affcd260d5688242d993194b86d7 redis-cluster1:6379
slots:[1365-5460] (4096 slots) master
1 additional replica(s)
M: 3194b5c5820004232b351a9a9170a1481c34e31f 172.21.0.8:6379
slots:[0-1364],[5461-6826],[10923-12287] (4096 slots) master
1 additional replica(s)
M: 0f9955ecb806141e8149bf63a7ca9ffb6bc73918 172.21.0.5:6379
slots:[6827-10922] (4096 slots) master
1 additional replica(s)
S: 8d50b5b6e61e2eebdf80ca7b3a3d75858fc2b3a0 172.21.0.7:6379
slots: (0 slots) slave
replicates 63b7152d0af00bcd55a4b7ac3811104ae810fef5
S: 210dceb99e1e3a0508ee9a0262afcdaaddfe46a9 172.21.0.2:6379
slots: (0 slots) slave
replicates 0f9955ecb806141e8149bf63a7ca9ffb6bc73918
S: 5ca1b4875a404334a729266ad9e9ffa9a361135e 172.21.0.9:6379
slots: (0 slots) slave
replicates 3194b5c5820004232b351a9a9170a1481c34e31f
M: 63b7152d0af00bcd55a4b7ac3811104ae810fef5 172.21.0.4:6379
slots:[12288-16383] (4096 slots) master
1 additional replica(s)
S: 0b7aa8453ab1607fdc4c4a893945f239b4da92fa 172.21.0.6:6379
slots: (0 slots) slave
replicates 5e262c16f666affcd260d5688242d993194b86d7
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
How many slots do you want to move (from 1 to 16384)? 4096
What is the receiving node ID? 3194b5c5820004232b351a9a9170a1481c34e31f
Please enter all the source node IDs.
Type 'all' to use all the nodes as source nodes for the hash slots.
Type 'done' once you entered all the source nodes IDs.
Source node #1:
...
这里可以指定固定的源节点,然后回车输入done
,同时也可以输入all
指定所有的源节点进行迁移。