Docker 部署 redis cluster

简介

        Redis的官方多机部署方案，Redis Cluster。一组Redis Cluster是由多个Redis实例组成，官方推荐我们使用6个实例，其中3个为主节点，3个为从节点。一旦主节点发生故障的时候，Redis Cluster可以选举出对应的从节点成为新的主节点，继续对外服务，从而保证服务的高可用性。

        对于客户端来说，怎么知道对应的key要路由到那个节点？Redis Cluster把所有数据划分成16384个不同槽位，可以根据机器性能把不同的槽位分配给不同的Redis实例。对于Redis实例来说，它们只会存储部分Redis数据。当然，槽的数据是可以迁移的，不同的实例之间，可以通过一定的协议，进行数据迁移。

部署Redis Cluster

1. 创建6个节点的Redis集群，为了方便，使用脚本创建。

for port in $(seq 7001 7006); \
do \
mkdir -p /mydata/redis/node-${port}/conf
touch /mydata/redis/node-${port}/conf/redis.conf
cat << EOF >/mydata/redis/node-${port}/conf/redis.conf
port ${port}
cluster-enabled yes
cluster-config-file nodes.conf
cluster-node-timeout 5000
# 192.168.94.133  为ip，根据实际情况修改
cluster-announce-ip 192.168.94.133  
cluster-announce-port ${port}
cluster-announce-bus-port 1${port}
appendonly yes
EOF
docker run -p ${port}:${port} -p 1${port}:1${port} --name redis-${port} \
-v /mydata/redis/node-${port}/data:/data \
-v /mydata/redis/node-${port}/conf/redis.conf:/etc/redis/redis.conf \
-d redis:5.0.7 redis-server /etc/redis/redis.conf; \
done

停止redis container

docker stop $(docker ps -a |grep redis-700 | awk '{ print $1}')

删除redis container

docker rm $(docker ps -a |grep redis-700 | awk '{ print $1}')

2. 使用redis创建集群

docker exec -it redis-7001 bash
redis-cli --cluster create 192.168.94.133:7001 192.168.94.133:7002 192.168.94.133:7003
192.168.94.133:7004 192.168.94.133:7005 192.168.94.133:7006 --cluster-replicas 1

执行结果：

>>> Performing hash slots allocation on 6 nodes...
Master[0] -> Slots 0 - 5460
Master[1] -> Slots 5461 - 10922
Master[2] -> Slots 10923 - 16383
Adding replica 192.168.94.133:7005 to 192.168.94.133:7001
Adding replica 192.168.94.133:7006 to 192.168.94.133:7002
Adding replica 192.168.94.133:7004 to 192.168.94.133:7003
>>> Trying to optimize slaves allocation for anti-affinity
[WARNING] Some slaves are in the same host as their master
M: 13a1653c2ed5988b0fd2a061ef63ff10428152d8 192.168.94.133:7001
   slots:[0-5460] (5461 slots) master
M: 5c13aabf66d720dbbd26aa008a9655f826b18bbc 192.168.94.133:7002
   slots:[5461-10922] (5462 slots) master
M: 66c72ef869158c438c533aba76c528549fb64a66 192.168.94.133:7003
   slots:[10923-16383] (5461 slots) master
S: c192aaaf20c159ae7a89e40832fb999a69944b67 192.168.94.133:7004
   replicates 66c72ef869158c438c533aba76c528549fb64a66
S: 4a227a068eafcb9fd026d0a201cc9493ab2b6318 192.168.94.133:7005
   replicates 13a1653c2ed5988b0fd2a061ef63ff10428152d8
S: 403994e0b16d3651766f04b76836494aa28a1bd5 192.168.94.133:7006
   replicates 5c13aabf66d720dbbd26aa008a9655f826b18bbc
Can I set the above configuration? (type 'yes' to accept): yes
>>> Nodes configuration updated
>>> Assign a different config epoch to each node
>>> Sending CLUSTER MEET messages to join the cluster
Waiting for the cluster to join
...
>>> Performing Cluster Check (using node 192.168.94.133:7001)
M: 13a1653c2ed5988b0fd2a061ef63ff10428152d8 192.168.94.133:7001
   slots:[0-5460] (5461 slots) master
   1 additional replica(s)
M: 66c72ef869158c438c533aba76c528549fb64a66 192.168.94.133:7003
   slots:[10923-16383] (5461 slots) master
   1 additional replica(s)
M: 5c13aabf66d720dbbd26aa008a9655f826b18bbc 192.168.94.133:7002
   slots:[5461-10922] (5462 slots) master
   1 additional replica(s)
S: 4a227a068eafcb9fd026d0a201cc9493ab2b6318 192.168.94.133:7005
   slots: (0 slots) slave
   replicates 13a1653c2ed5988b0fd2a061ef63ff10428152d8
S: 403994e0b16d3651766f04b76836494aa28a1bd5 192.168.94.133:7006
   slots: (0 slots) slave
   replicates 5c13aabf66d720dbbd26aa008a9655f826b18bbc
S: c192aaaf20c159ae7a89e40832fb999a69944b67 192.168.94.133:7004
   slots: (0 slots) slave
   replicates 66c72ef869158c438c533aba76c528549fb64a66
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.

3. 测试集群效果

• 使用redis-cli的cluster方式进行连接

# -c 表示cluster
redis-cli -c -h 192.168.94.133 -p 7001

执行结果：

192.168.94.133:7001>

• 获取集群信息

192.168.94.133:7001> cluster info

执行结果：

cluster_state:ok
cluster_slots_assigned:16384
cluster_slots_ok:16384
cluster_slots_pfail:0
cluster_slots_fail:0
cluster_known_nodes:6
cluster_size:3
cluster_current_epoch:6
cluster_my_epoch:1
cluster_stats_messages_ping_sent:210
cluster_stats_messages_pong_sent:219
cluster_stats_messages_sent:429
cluster_stats_messages_ping_received:214
cluster_stats_messages_pong_received:210
cluster_stats_messages_meet_received:5
cluster_stats_messages_received:429

• 获取节点信息

192.168.94.133:7001> cluster nodes

执行结果：

66c72ef869158c438c533aba76c528549fb64a66 192.168.94.133:7003@17003 master - 0 1625909426497 3 connected 10923-16383
5c13aabf66d720dbbd26aa008a9655f826b18bbc 192.168.94.133:7002@17002 master - 0 1625909426000 2 connected 5461-10922
4a227a068eafcb9fd026d0a201cc9493ab2b6318 192.168.94.133:7005@17005 slave 13a1653c2ed5988b0fd2a061ef63ff10428152d8 0 1625909426600 5 connected
13a1653c2ed5988b0fd2a061ef63ff10428152d8 192.168.94.133:7001@17001 myself,master - 0 1625909427000 1 connected 0-5460
403994e0b16d3651766f04b76836494aa28a1bd5 192.168.94.133:7006@17006 slave 5c13aabf66d720dbbd26aa008a9655f826b18bbc 0 1625909427002 6 connected
c192aaaf20c159ae7a89e40832fb999a69944b67 192.168.94.133:7004@17004 slave 66c72ef869158c438c533aba76c528549fb64a66 0 1625909427508 4 connected