docker+redis+cluster+intellij redis分片集群搭建

使用 redis+master+slave+cluster 的架构搭建redis数据分片集群。不使用sentinel的情况下完成主从的自动切换、intellij的读写分离

在linux中创建目录8001 -8006  分别是3个master以及3个slave

每个目录中都有data目录、conf目录，其中conf目录中新增redis.conf 文件

目录结构如下：

8001/conf/redis.conf

配置文件

内容如下：

8001/conf/redis.conf
内容如下：
port 8001
#redis 访问密码
#requirepass password
#redis 访问Master节点密码
#masterauth password
# 关闭保护模式
protected-mode no
# 开启集群
cluster-enabled yes
# 集群节点配置
cluster-config-file nodes.conf
# 超时
cluster-node-timeout 5000
# 集群节点IP host模式为宿主机IP
cluster-announce-ip 192.168.0.3
# 集群节点端口 8001 - 8006
cluster-announce-port 8001
cluster-announce-bus-port 18001
# 开启 appendonly 备份模式
appendonly yes
# 每秒钟备份
appendfsync everysec
# 对aof文件进行压缩时，是否执行同步操作
no-appendfsync-on-rewrite no
# 当目前aof文件大小超过上一次重写时的aof文件大小的100%时会再次进行重写
auto-aof-rewrite-percentage 100
# 重写前AOF文件的大小最小值 默认 64mb
auto-aof-rewrite-min-size 64mb

# 日志配置
# debug:会打印生成大量信息，适用于开发/测试阶段
# verbose:包含很多不太有用的信息，但是不像debug级别那么混乱
# notice:适度冗长，适用于生产环境
# warning:仅记录非常重要、关键的警告消息
loglevel notice
# 日志文件路径
logfile "/data/redis.log"

 

依次修改8002~8006的redis.conf

创建容器并配置集群

然后使用docker run 命令创建容器

三主

docker run -d --name redis-master8001 -p 8001:8001 -p 18001:18001 -v /home/redis-cluster/8001/conf/redis.conf:/etc/local/redis/redis.conf -v /home/redis-cluster/8001/data:/data redis /etc/local/redis/redis.conf

docker run -d --name redis-master8002 -p 8002:8002 -p 18002:18002 -v /home/redis-cluster/8002/conf/redis.conf:/etc/local/redis/redis.conf -v /home/redis-cluster/8002/data:/data redis /etc/local/redis/redis.conf

docker run -d --name redis-master8003 -p 8003:8003 -p 18003:18003 -v /home/redis-cluster/8003/conf/redis.conf:/etc/local/redis/redis.conf -v /home/redis-cluster/8003/data:/data redis /etc/local/redis/redis.conf

三从

docker run -d --name redis-slave8004 -p 8004:8004 -p 18004:18004 -v /home/redis-cluster/8004/conf/redis.conf:/etc/local/redis/redis.conf -v /home/redis-cluster/8004/data:/data redis /etc/local/redis/redis.conf

docker run -d --name redis-slave8005 -p 8005:8005 -p 18005:18005 -v /home/redis-cluster/8005/conf/redis.conf:/etc/local/redis/redis.conf -v /home/redis-cluster/8005/data:/data redis /etc/local/redis/redis.conf

docker run -d --name redis-slave8006 -p 8006:8006 -p 18006:18006 -v /home/redis-cluster/8006/conf/redis.conf:/etc/local/redis/redis.conf -v /home/redis-cluster/8006/data:/data redis /etc/local/redis/redis.conf

接下来配置主从关系 

使用docker exec -it 命令进入redis-master8001容器内部

执行如下命令进行主从集群开启

redis-cli -p 8001 --cluster create --cluster-replicas 1 192.168.0.3:8001 192.168.0.3:8002 192.168.0.3:8003 192.168.0.3:8004 192.168.0.3:8005 192.168.0.3:8006

其中 1 代表着 当前的集群中么个主机有一个从机，后面一共是6个ip:port  ，其中前面的3个会默认被算法标识为master 后面的是slave

Master[0] -> Slots 0 - 5460

Master[1] -> Slots 5461 - 10922

Master[2] -> Slots 10923 - 16383

Adding replica 192.168.0.3:8005 to 192.168.0.3:8001

Adding replica 192.168.0.3:8006 to 192.168.0.3:8002

Adding replica 192.168.0.3:8004 to 192.168.0.3:8003

>>> Trying to optimize slaves allocation for anti-affinity

[WARNING] Some slaves are in the same host as their master

M: 4d90426310fdefb7f85f248b934c50cba0a6a85b 192.168.0.3:8001

   slots:[0-5460] (5461 slots) master

M: ea7007c24f7ae0adb4ce4f165e6bc68d70910fc5 192.168.0.3:8002

   slots:[5461-10922] (5462 slots) master

M: 9c6c0e8a4de4da3e2190b183836c6924901c9a11 192.168.0.3:8003

   slots:[10923-16383] (5461 slots) master

S: 83fd65647c66f2714e4612928760a3d8a3aedddd 192.168.0.3:8004

   replicates 4d90426310fdefb7f85f248b934c50cba0a6a85b

S: d81ec65e981e4b53577c16d51e5e21e1b9e3c5b2 192.168.0.3:8005

   replicates ea7007c24f7ae0adb4ce4f165e6bc68d70910fc5

S: ec315906a73b78bdaae3d93a3a5a65789a0f5192 192.168.0.3:8006

   replicates 9c6c0e8a4de4da3e2190b183836c6924901c9a11

下方提示的位置输入yes

Can I set the above configuration? (type 'yes' to accept): yes

>>> Nodes configuration updated

>>> Assign a different config epoch to each node

>>> Sending CLUSTER MEET messages to join the cluster

Waiting for the cluster to join

.

>>> Performing Cluster Check (using node 192.168.0.3:8001)

M: 4d90426310fdefb7f85f248b934c50cba0a6a85b 192.168.0.3:8001

   slots:[0-5460] (5461 slots) master

   1 additional replica(s)

S: ec315906a73b78bdaae3d93a3a5a65789a0f5192 192.168.0.3:8006

   slots: (0 slots) slave

   replicates 9c6c0e8a4de4da3e2190b183836c6924901c9a11

S: 83fd65647c66f2714e4612928760a3d8a3aedddd 192.168.0.3:8004

   slots: (0 slots) slave

   replicates 4d90426310fdefb7f85f248b934c50cba0a6a85b

M: 9c6c0e8a4de4da3e2190b183836c6924901c9a11 192.168.0.3:8003

   slots:[10923-16383] (5461 slots) master

   1 additional replica(s)

S: d81ec65e981e4b53577c16d51e5e21e1b9e3c5b2 192.168.0.3:8005

   slots: (0 slots) slave

   replicates ea7007c24f7ae0adb4ce4f165e6bc68d70910fc5

M: ea7007c24f7ae0adb4ce4f165e6bc68d70910fc5 192.168.0.3:8002

   slots:[5461-10922] (5462 slots) master

   1 additional replica(s)

[OK] All nodes agree about slots configuration.

>>> Check for open slots...

>>> Check slots coverage...

[OK] All 16384 slots covered.

redis会自动给每一个master分配插槽（slot）

容器内使用redis-cli 命令查看当前集群的状态

redis-cli -h 192.168.0.3 -p 8001 cluster nodes

ec315906a73b78bdaae3d93a3a5a65789a0f5192 192.168.0.3:8006@18006 slave 9c6c0e8a4de4da3e2190b183836c6924901c9a11 0 1681789249733 3 connected

4d90426310fdefb7f85f248b934c50cba0a6a85b 192.168.0.3:8001@18001 myself,master - 0 1681789249000 1 connected 0-5460

83fd65647c66f2714e4612928760a3d8a3aedddd 192.168.0.3:8004@18004 slave 4d90426310fdefb7f85f248b934c50cba0a6a85b 0 1681789250737 1 connected

9c6c0e8a4de4da3e2190b183836c6924901c9a11 192.168.0.3:8003@18003 master - 0 1681789249531 3 connected 10923-16383

d81ec65e981e4b53577c16d51e5e21e1b9e3c5b2 192.168.0.3:8005@18005 slave ea7007c24f7ae0adb4ce4f165e6bc68d70910fc5 0 1681789249000 2 connected

ea7007c24f7ae0adb4ce4f165e6bc68d70910fc5 192.168.0.3:8002@18002 master - 0 1681789249531 2 connected 5461-10922

想要在docker容器中测试集群的重定向功能，使用redis-cli命令的时候一定要加上 -c 也就是集群模式  cluster

未加的情况：redis-cli -p 8001

127.0.0.1:8001> set name 1

(error) MOVED 5798 192.168.0.3:8002

加上的情况：redis-cli -c  -p 8001

127.0.0.1:8001> set name 1

-> Redirected to slot [5798] located at 192.168.0.3:8002

OK

192.168.0.3:8002>

redis集群会自动找到该插槽的位置并重定向过去。因为key是跟插槽slot绑定的。所以会经过路由重定向到对应的实例中

固定将某一类型的业务数据存储在同一台服务器中，减少redis集群重定向导致的性能消耗可以控制固定有效部分的方式进行存储

例如：num 这个key是存储于8001上的，name这个key是存储于8002上的，那么想要将name在8001上存储，就需要如下

set {num}name 1122 那么就会存储在8001上了

127.0.0.1:8001> set {num}name 111

OK

127.0.0.1:8001> keys *

1) "{num}name"

2) "num"

测试集群的故障转移

使用docker stop 任意一个master

ec315906a73b78bdaae3d93a3a5a65789a0f5192 192.168.0.3:8006@18006 slave 9c6c0e8a4de4da3e2190b183836c6924901c9a11 0 1681793301516 3 connected

4d90426310fdefb7f85f248b934c50cba0a6a85b 192.168.0.3:8001@18001 myself,master - 0 1681793299000 1 connected 0-5460

83fd65647c66f2714e4612928760a3d8a3aedddd 192.168.0.3:8004@18004 slave 4d90426310fdefb7f85f248b934c50cba0a6a85b 0 1681793302521 1 connected

9c6c0e8a4de4da3e2190b183836c6924901c9a11 192.168.0.3:8003@18003 master,fail? - 1681793296485 1681793294000 3 disconnected 10923-16383

d81ec65e981e4b53577c16d51e5e21e1b9e3c5b2 192.168.0.3:8005@18005 slave ea7007c24f7ae0adb4ce4f165e6bc68d70910fc5 0 1681793302000 2 connected

ea7007c24f7ae0adb4ce4f165e6bc68d70910fc5 192.168.0.3:8002@18002 master - 0 1681793301000 2 connected 5461-10922

之后在重新start

ec315906a73b78bdaae3d93a3a5a65789a0f5192 192.168.0.3:8006@18006 master - 0 1681793329678 7 connected 10923-16383

4d90426310fdefb7f85f248b934c50cba0a6a85b 192.168.0.3:8001@18001 myself,master - 0 1681793328000 1 connected 0-5460

83fd65647c66f2714e4612928760a3d8a3aedddd 192.168.0.3:8004@18004 slave 4d90426310fdefb7f85f248b934c50cba0a6a85b 0 1681793328572 1 connected

9c6c0e8a4de4da3e2190b183836c6924901c9a11 192.168.0.3:8003@18003 slave ec315906a73b78bdaae3d93a3a5a65789a0f5192 0 1681793328673 7 connected

d81ec65e981e4b53577c16d51e5e21e1b9e3c5b2 192.168.0.3:8005@18005 slave ea7007c24f7ae0adb4ce4f165e6bc68d70910fc5 0 1681793328000 2 connected

ea7007c24f7ae0adb4ce4f165e6bc68d70910fc5 192.168.0.3:8002@18002 master - 0 1681793329577 2 connected 5461-10922

可以看出来，当前的8003变成了slave，而它所属的slave变更为master了。

redis集群的伸缩功能

接下来测试redis集群的伸缩功能

新增节点

先新增一个节点master8007

别忘记更改端口。

使用docker run 命令运行

docker run -d --name redis-master8007 -p 8007:8007 -p 18007:18007 -v /home/redis-cluster/8007/conf/redis.conf:/etc/local/redis/redis.conf -v /home/redis-cluster/8007/data:/data redis /etc/local/redis/redis.conf

进入8007的容器内部，执行add-node命令

redis-cli --cluster add-node 192.168.0.3:8007 192.168.0.3:8001
>>> Adding node 192.168.0.3:8007 to cluster 192.168.0.3:8001
>>> Performing Cluster Check (using node 192.168.0.3:8001)
M: 4d90426310fdefb7f85f248b934c50cba0a6a85b 192.168.0.3:8001
   slots:[0-5460] (5461 slots) master
   1 additional replica(s)
M: ec315906a73b78bdaae3d93a3a5a65789a0f5192 192.168.0.3:8006
   slots:[10923-16383] (5461 slots) master
   1 additional replica(s)
S: 83fd65647c66f2714e4612928760a3d8a3aedddd 192.168.0.3:8004
   slots: (0 slots) slave
   replicates 4d90426310fdefb7f85f248b934c50cba0a6a85b
S: 9c6c0e8a4de4da3e2190b183836c6924901c9a11 192.168.0.3:8003
   slots: (0 slots) slave
   replicates ec315906a73b78bdaae3d93a3a5a65789a0f5192
S: d81ec65e981e4b53577c16d51e5e21e1b9e3c5b2 192.168.0.3:8005
   slots: (0 slots) slave
   replicates ea7007c24f7ae0adb4ce4f165e6bc68d70910fc5
M: ea7007c24f7ae0adb4ce4f165e6bc68d70910fc5 192.168.0.3:8002
   slots:[5461-10922] (5462 slots) master
   1 additional replica(s)
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
>>> Send CLUSTER MEET to node 192.168.0.3:8007 to make it join the cluster.
[OK] New node added correctly.
已经成功加入到集群中，
4d90426310fdefb7f85f248b934c50cba0a6a85b 192.168.0.3:8001@18001 master - 0 1681804273617 1 connected 0-5460
d81ec65e981e4b53577c16d51e5e21e1b9e3c5b2 192.168.0.3:8005@18005 slave ea7007c24f7ae0adb4ce4f165e6bc68d70910fc5 0 1681804274119 2 connected
ec315906a73b78bdaae3d93a3a5a65789a0f5192 192.168.0.3:8006@18006 master - 0 1681804274119 7 connected 10923-16383
83fd65647c66f2714e4612928760a3d8a3aedddd 192.168.0.3:8004@18004 slave 4d90426310fdefb7f85f248b934c50cba0a6a85b 0 1681804273517 1 connected
9c6c0e8a4de4da3e2190b183836c6924901c9a11 192.168.0.3:8003@18003 slave ec315906a73b78bdaae3d93a3a5a65789a0f5192 0 1681804272000 7 connected
ea7007c24f7ae0adb4ce4f165e6bc68d70910fc5 192.168.0.3:8002@18002 master - 0 1681804273114 2 connected 5461-10922
bf3c979d52ec314b5dcb78d9b518b95f1b5b13ad 192.168.0.3:8007@18007 myself,master - 0 1681804272000 0 connected

通过查询节点信息，发现现在还没有分配插槽。接下来将key  num 的插槽数据分配给8007

root@ae97763f8266:/data# redis-cli -c -p 8007
127.0.0.1:8007> get num
-> Redirected to slot [2765] located at 192.168.0.3:8001
"1234111"
通过get num 发现重定向到了8001  那么就需要将8001 的插槽分配给8007
通过命令reshard 重新分片命令
root@ae97763f8266:/data# redis-cli --cluster reshard 192.168.0.3:8001
>>> Performing Cluster Check (using node 192.168.0.3:8001)
M: 4d90426310fdefb7f85f248b934c50cba0a6a85b 192.168.0.3:8001
   slots:[0-5460] (5461 slots) master
   1 additional replica(s)
M: ec315906a73b78bdaae3d93a3a5a65789a0f5192 192.168.0.3:8006
   slots:[10923-16383] (5461 slots) master
   1 additional replica(s)
S: 83fd65647c66f2714e4612928760a3d8a3aedddd 192.168.0.3:8004
   slots: (0 slots) slave
   replicates 4d90426310fdefb7f85f248b934c50cba0a6a85b
S: 9c6c0e8a4de4da3e2190b183836c6924901c9a11 192.168.0.3:8003
   slots: (0 slots) slave
   replicates ec315906a73b78bdaae3d93a3a5a65789a0f5192
M: bf3c979d52ec314b5dcb78d9b518b95f1b5b13ad 192.168.0.3:8007
   slots: (0 slots) master
S: d81ec65e981e4b53577c16d51e5e21e1b9e3c5b2 192.168.0.3:8005
   slots: (0 slots) slave
   replicates ea7007c24f7ae0adb4ce4f165e6bc68d70910fc5
M: ea7007c24f7ae0adb4ce4f165e6bc68d70910fc5 192.168.0.3:8002
   slots:[5461-10922] (5462 slots) master
   1 additional replica(s)
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
How many slots do you want to move (from 1 to 16384)?
通过之前查询到的num所在插槽为2765，所以这里输入3000 就可以
What is the receiving node ID? bf3c979d52ec314b5dcb78d9b518b95f1b5b13ad
这里是设置接收的节点的id  输入8007的就可以。
Please enter all the source node IDs.
  Type 'all' to use all the nodes as source nodes for the hash slots.
  Type 'done' once you entered all the source nodes IDs.
Source node #1:
这里是需要输入源节点id，输入8001的id就可以。
Source node #2: 由于只从一个节点进行copy，所以这里输入done就可以
Do you want to proceed with the proposed reshard plan (yes/no)? yes 这是确认操作
再查询节点信息
4d90426310fdefb7f85f248b934c50cba0a6a85b 192.168.0.3:8001@18001 master - 0 1681805139589 1 connected 3000-5460
d81ec65e981e4b53577c16d51e5e21e1b9e3c5b2 192.168.0.3:8005@18005 slave ea7007c24f7ae0adb4ce4f165e6bc68d70910fc5 0 1681805138585 2 connected
ec315906a73b78bdaae3d93a3a5a65789a0f5192 192.168.0.3:8006@18006 master - 0 1681805139589 7 connected 10923-16383
83fd65647c66f2714e4612928760a3d8a3aedddd 192.168.0.3:8004@18004 slave 4d90426310fdefb7f85f248b934c50cba0a6a85b 0 1681805139589 1 connected
9c6c0e8a4de4da3e2190b183836c6924901c9a11 192.168.0.3:8003@18003 slave ec315906a73b78bdaae3d93a3a5a65789a0f5192 0 1681805138000 7 connected
ea7007c24f7ae0adb4ce4f165e6bc68d70910fc5 192.168.0.3:8002@18002 master - 0 1681805139000 2 connected 5461-10922
bf3c979d52ec314b5dcb78d9b518b95f1b5b13ad 192.168.0.3:8007@18007 myself,master - 0 1681805139000 8 connected 0-2999

再使用get num 发现已经不需要重定向了

root@ae97763f8266:/data# redis-cli -c -p 8007

127.0.0.1:8007> get num

"1234111"

删除节点

接下来将插槽重新分配给8001，并将8007删除。总的来说就是将之前的操作反向来一次

redis-cli --cluster reshard 192.168.0.3:8007
。。。
How many slots do you want to move (from 1 to 16384)? 3000
这里还是输入3000
接下来的接收者变为了8001的id 源变更为8007的id
通过查询发现8007中的插槽已经消失了
root@ae97763f8266:/data# redis-cli -p 8007 cluster nodes
4d90426310fdefb7f85f248b934c50cba0a6a85b 192.168.0.3:8001@18001 master - 0 1681805784235 9 connected 0-5460
d81ec65e981e4b53577c16d51e5e21e1b9e3c5b2 192.168.0.3:8005@18005 slave ea7007c24f7ae0adb4ce4f165e6bc68d70910fc5 0 1681805785239 2 connected
ec315906a73b78bdaae3d93a3a5a65789a0f5192 192.168.0.3:8006@18006 master - 0 1681805784537 7 connected 10923-16383
83fd65647c66f2714e4612928760a3d8a3aedddd 192.168.0.3:8004@18004 slave 4d90426310fdefb7f85f248b934c50cba0a6a85b 0 1681805785000 9 connected
9c6c0e8a4de4da3e2190b183836c6924901c9a11 192.168.0.3:8003@18003 slave ec315906a73b78bdaae3d93a3a5a65789a0f5192 0 1681805784034 7 connected
ea7007c24f7ae0adb4ce4f165e6bc68d70910fc5 192.168.0.3:8002@18002 master - 0 1681805785000 2 connected 5461-10922
bf3c979d52ec314b5dcb78d9b518b95f1b5b13ad 192.168.0.3:8007@18007 myself,master - 0 1681805784000 8 connected
通过del-node命令删除节点
root@ae97763f8266:/data# redis-cli --cluster del-node 192.168.0.3:8007 bf3c979d52ec314b5dcb78d9b518b95f1b5b13ad
>>> Removing node bf3c979d52ec314b5dcb78d9b518b95f1b5b13ad from cluster 192.168.0.3:8007
>>> Sending CLUSTER FORGET messages to the cluster...
>>> Sending CLUSTER RESET SOFT to the deleted node.
到集群中的任意节点中查询集群信息，发现8007已经消失了
root@db66b4efcede:/data# redis-cli -p 8001 cluster nodes
ec315906a73b78bdaae3d93a3a5a65789a0f5192 192.168.0.3:8006@18006 master - 0 1681805966558 7 connected 10923-16383
4d90426310fdefb7f85f248b934c50cba0a6a85b 192.168.0.3:8001@18001 myself,master - 0 1681805965000 9 connected 0-5460
83fd65647c66f2714e4612928760a3d8a3aedddd 192.168.0.3:8004@18004 slave 4d90426310fdefb7f85f248b934c50cba0a6a85b 0 1681805964149 9 connected
9c6c0e8a4de4da3e2190b183836c6924901c9a11 192.168.0.3:8003@18003 slave ec315906a73b78bdaae3d93a3a5a65789a0f5192 0 1681805966457 7 connected
d81ec65e981e4b53577c16d51e5e21e1b9e3c5b2 192.168.0.3:8005@18005 slave ea7007c24f7ae0adb4ce4f165e6bc68d70910fc5 0 1681805964451 2 connected
ea7007c24f7ae0adb4ce4f165e6bc68d70910fc5 192.168.0.3:8002@18002 master - 0 1681805965454 2 connected 5461-10922

到此，节点的删除操作就完成了！

 intellij中使用

接下来在intellij中使用，并实现读写分离

spring:

 redis:

  cluster:

   nodes:

    - 192.168.0.3:8001

    - 192.168.0.3:8002

    - 192.168.0.3:8003

    - 192.168.0.3:8004

    - 192.168.0.3:8005

    - 192.168.0.3:8006

在启动类或者配置类中

@Bean

public LettuceClientConfigurationBuilderCustomizer clientConfigurationBuilderCustomizer(){

return clientConfigurationBuilder -> clientConfigurationBuilder.readFrom(ReadFrom.REPLICA_PREFERRED);

}

然后就可以实现redis的读写分离了。