高可用Redis-cluster搭建（完整必成版）

最新推荐文章于 2024-04-26 20:43:31 发布

肉装法师

最新推荐文章于 2024-04-26 20:43:31 发布

阅读量303

点赞数 1

分类专栏： redis 文章标签： redis

本文链接：https://blog.csdn.net/weixin_41772761/article/details/103730757

版权

redis 专栏收录该内容

5 篇文章 0 订阅

订阅专栏

环境：centos 7
Ruby：ruby-2.5.3
Redis：redis-3.2.11
安装包git地址
本次案例以测试为主，redis集群理论此处不再赘述。本次模拟集群搭建，只使用了一台虚拟机。搭建6个redis，3主3从，实现Redis集群高可用，当主节点宕机时，从节点替换成其主节点。真实环境可将一主一从放在不通节点上。
本人将安装包上传至虚机 /usr/local/src/ 目录下，各位可按喜好放置。redis版本也可以是其他版本，但必须是3.0以上。

1. 上传rudy包

1.1 先安装所需依赖

yum install -y openssh openssh-clients openssl gcc openssh-server

1.2 解压ruby

[root@localhost src]# tar -xvf ruby-2.5.3.tar.gz

1.3 进入到ruby 目录下编译安装(时间较长)

[root@localhost src]# cd ruby-2.5.3
[root@localhost ruby-2.5.3]# ./configure  --with-openssl-dir=/usr/local/ssl
[root@localhost ruby-2.5.3]# make && make instal

1.4 yum安装gems

RubyGems（简称 gems）是一个用于对 Ruby组件进行打包的 Ruby 打包系统。它提供一个分发 Ruby 程序和库的标准格式，还提供一个管理程序包安装的工具。简单理解就是ruby运行时,需要的各种插件都在gems里; 是一种技术支持;

yum -y install rubygems

1.5 下载支持redis的插件内容

如果报错安装失败查看1.1中依赖是否安装或者1.3步骤中./configure 后是否指定

[root@localhost ruby-2.5.3]# gem install redis
Fetching: redis-4.1.3.gem (100%)
Successfully installed redis-4.1.3
Parsing documentation for redis-4.1.3
Installing ri documentation for redis-4.1.3
Done installing documentation for redis after 0 seconds
1 gem installed

1.6 检查ruby版本

[root@localhost ruby-2.5.3]# ruby -v 
ruby 2.5.3p105 (2018-10-18 revision 65156) [x86_64-linux]

2. 上传redis包到环境上

2.1 解压 redis

[root@localhost src]# tar -xvf redis-3.2.11.tar.gz

2.2 编译安装 redis

[root@localhost src]# cd redis-3.2.11
[root@localhost redis-3.2.11]# make && make install

2.3 创建集群节点并查看

不同的节点创建对应的目录管理配置文件

[root@localhost redis-3.2.11]# mkdir 8000 8001 8002 8003 8004 8005
[root@localhost redis-3.2.11]# ll
总用量 212
-rw-rw-r--.  1 root root 92766 9月  21 2017 00-RELEASENOTES
drwxr-xr-x.  2 root root     6 12月 27 14:40 8000
drwxr-xr-x.  2 root root     6 12月 27 14:40 8001
drwxr-xr-x.  2 root root     6 12月 27 14:40 8002
drwxr-xr-x.  2 root root     6 12月 27 14:40 8003
drwxr-xr-x.  2 root root     6 12月 27 14:40 8004
drwxr-xr-x.  2 root root     6 12月 27 14:40 8005
......

2.4 根据实际情况修改配置文件

将redis.conf文件中的8000替换成8001放入8001目录下

[root@localhost 8000]cp redis.conf ../8001/
[root@localhost 8000]vim ../8001/redis.conf
:%s/8000/8001/g
#5 次替换，共 5 行  
:wq

以此类推完成其余4个redis节点配置，检查确保无误

[root@localhost redis-3.2.11]# ls ./800*
./8000:
redis.conf

./8001:
redis.conf

./8002:
redis.conf

./8003:
redis.conf

./8004:
redis.conf

./8005:
redis.conf

2.5 检查无误后启动各个节点

[root@localhost redis-3.2.11]# cp ./src/redis-cli ./
[root@localhost redis-3.2.11]# cp ./src/redis-server ./
[root@localhost redis-3.2.11]# redis-server 8000/redis.conf 
[root@localhost redis-3.2.11]# redis-server 8001/redis.conf 
[root@localhost redis-3.2.11]# redis-server 8002/redis.conf 
[root@localhost redis-3.2.11]# redis-server 8003/redis.conf 
[root@localhost redis-3.2.11]# redis-server 8004/redis.conf 
[root@localhost redis-3.2.11]# redis-server 8005/redis.conf

启动完成查看各节点启动状态

[root@localhost redis-3.2.11]# ps -ef | grep redis
root      70929      1  0 15:54 ?        00:00:00 ./redis-server *:8000 [cluster]
root      70933      1  0 15:54 ?        00:00:00 ./redis-server *:8001 [cluster]
root      70937      1  0 15:54 ?        00:00:00 ./redis-server *:8002 [cluster]
root      70941      1  0 15:54 ?        00:00:00 ./redis-server *:8003 [cluster]
root      70945      1  0 15:54 ?        00:00:00 ./redis-server *:8004 [cluster]
root      70949      1  0 15:54 ?        00:00:00 ./redis-server *:8005 [cluster]
root      70953  10977  0 15:54 pts/0    00:00:00 grep --color=auto redis
[root@localhost redis-3.2.11]#

如果启动不成功，查看对应节点 800X目录下/redis.log文件报错信息。以8000为例，若8000redis节点为启动，查看日志。

[root@localhost 8000]# ls
redis.conf  redis.log
[root@localhost 8000]# cat redis.log

2.6 登录集群的客户端命令

登录集群的客户端命令

redis-cli -c -p 8000

-p 表示端口连接, -c 以集群状态登录节点(本质上是对单个节点的命令封装); 但是这个时候所有的节点并不是集群状态,我们单独登录节点一样无法实现集群的操作。

[root@localhost redis-3.2.11]# redis-cli -p 8000
127.0.0.1:8000> set name RumCoke
(error) CLUSTERDOWN Hash slot not served
127.0.0.1:8000>

3. 创建集群

3.1 进入到redis-3.2.11/src/ 目录下查看是否存在文件 redis-trib.rb

[root@localhost redis-3.2.11]# cd src/
[root@localhost src]# ll redis-trib.rb 
-rwxrwxr-x. 1 root root 60852 9月  21 2017 redis-trib.rb

3.2 将8000 8001 8002 作为主节点 8003 8004 8005 作为从节点（暂时不用）执行命令并输入yes

[root@localhost src]# ./redis-trib.rb  create  172.8.8.184:8000 172.8.8.184:8001 172.8.8.184:8002 
>>> Creating cluster
>>> Performing hash slots allocation on 3 nodes...
Using 3 masters:
172.8.8.184:8000
172.8.8.184:8001
172.8.8.184:8002
M: cc2e011296ad35388eed1d7263140b820acd9be1 172.8.8.184:8000
   slots:0-5460 (5461 slots) master
M: d7f0752c7f20a6c0d72f648234f6b6aa7038ec20 172.8.8.184:8001
   slots:5461-10922 (5462 slots) master
M: 54a64c2856670e14e66e76a48af70d82b631dee1 172.8.8.184:8002
   slots:10923-16383 (5461 slots) master
Can I set the above configuration? (type 'yes' to accept): yes
>>> Nodes configuration updated
>>> Assign a different config epoch to each node
>>> Sending CLUSTER MEET messages to join the cluster
Waiting for the cluster to join.
>>> Performing Cluster Check (using node 172.8.8.184:8000)
M: cc2e011296ad35388eed1d7263140b820acd9be1 172.8.8.184:8000
   slots:0-5460 (5461 slots) master
   0 additional replica(s)
M: d7f0752c7f20a6c0d72f648234f6b6aa7038ec20 172.8.8.184:8001
   slots:5461-10922 (5462 slots) master
   0 additional replica(s)
M: 54a64c2856670e14e66e76a48af70d82b631dee1 172.8.8.184:8002
   slots:10923-16383 (5461 slots) master
   0 additional replica(s)
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
[root@localhost src]#

也可以直接

redis-trib.rb create --replicas 1 ip1:port1 ip2:port2

–replicas 1 表示自动为每一个master节点分配至少一个slave节点上面有6个节点，程序会按照一定规则生成 3个master（主）3个slave（从）

3.3 完成集群搭建

上一步最后显示如下
在这里插入图片描述
表示集群搭建成功,槽道分配完毕;

3.4 登陆集群

查看集群状态

[root@localhost src]# redis-cli -c -p 8000 
127.0.0.1:8000> cluster info
cluster_state:ok
cluster_slots_assigned:16384
cluster_slots_ok:16384
cluster_slots_pfail:0
cluster_slots_fail:0
cluster_known_nodes:3
cluster_size:3
cluster_current_epoch:3
cluster_my_epoch:1
cluster_stats_messages_sent:1295
cluster_stats_messages_received:1295
127.0.0.1:8000>

查看集群节点信息

127.0.0.1:8000> cluster nodes 
d7f0752c7f20a6c0d72f648234f6b6aa7038ec20 172.8.8.184:8001 master - 0 1577435297840 2 connected 5461-10922
54a64c2856670e14e66e76a48af70d82b631dee1 172.8.8.184:8002 master - 0 1577435296831 3 connected 10923-16383
cc2e011296ad35388eed1d7263140b820acd9be1 172.8.8.184:8000 myself,master - 0 0 1 connected 0-5460
127.0.0.1:8000>

测试set和get
集群底层计算判断逻辑:name的哈希取模,5798,判断是否归 8000管理;转向了8001

127.0.0.1:8000> set name RumCoke
-> Redirected to slot [5798] located at 172.8.8.184:8001
OK
172.8.8.184:8001>

3.5 动态添加节点

上面3.2步骤没有将所有6个节点自动添加至集群，现在使用ruby动态添加节点。
前面是新节点,后面是存在的任意一个节点。像是在某个旧节点后插入一个新节点。
插入成功如下：

[root@localhost src]# ./redis-trib.rb add-node 172.8.8.184:8003 172.8.8.184:8000
>>> Adding node 172.8.8.184:8003 to cluster 172.8.8.184:8000
>>> Performing Cluster Check (using node 172.8.8.184:8000)
M: cc2e011296ad35388eed1d7263140b820acd9be1 172.8.8.184:8000
   slots:0-5460 (5461 slots) master
   0 additional replica(s)
M: d7f0752c7f20a6c0d72f648234f6b6aa7038ec20 172.8.8.184:8001
   slots:5461-10922 (5462 slots) master
   0 additional replica(s)
M: 54a64c2856670e14e66e76a48af70d82b631dee1 172.8.8.184:8002
   slots:10923-16383 (5461 slots) master
   0 additional replica(s)
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
>>> Send CLUSTER MEET to node 172.8.8.184:8003 to make it join the cluster.
[OK] New node added correctly.
[root@localhost src]#

若当前新节点已经被集群知道,或者新节点有数据
在这里插入图片描述
登录到新增节点，执行


[root@localhost src]# redis-cli -c -p 8004
127.0.0.1:8004>cluster nodes 
127.0.0.1:8004> flushdb
OK
127.0.0.1:8004> keys *
(empty list or set)

把新增节点的nodes文件删除，重启服务，重新添加将节点中的数据清空

[root@localhost src]# ./redis-trib.rb del-node 172.8.8.184:8003 480d5eb679e2355a4e40c688c58c121291b392ba
>>> Removing node 480d5eb679e2355a4e40c688c58c121291b392ba from cluster 172.8.8.184:8003
>>> Sending CLUSTER FORGET messages to the cluster...
>>> SHUTDOWN the node.

进入集群查看节点状态(默认情况是master,不管理槽道)

3.6 动态添加从节点

–slave 和–master-id 必须同时配置就可以指定给一个 master添加从节点,新增节点就是从节点的角色
先进入集群查看主节点id

127.0.0.1:8000> cluster nodes
d7f0752c7f20a6c0d72f648234f6b6aa7038ec20 172.8.8.184:8001 master - 0 1577437107930 2 connected 5461-10922
d691b3dc79a4a10b902a117d6007bacfa83616d2 172.8.8.184:8005 master - 0 1577437109444 0 connected
54a64c2856670e14e66e76a48af70d82b631dee1 172.8.8.184:8002 master - 0 1577437109948 3 connected 10923-16383
a247a177e7c3a7cbc520e730f487ab4bfcd5af47 172.8.8.184:8004 master - 0 1577437110956 5 connected
cc2e011296ad35388eed1d7263140b820acd9be1 172.8.8.184:8000 myself,master - 0 0 1 connected 0-5460
480d5eb679e2355a4e40c688c58c121291b392ba 172.8.8.184:8003 master - 0 1577437111964 4 connected
127.0.0.1:8000>

获取一个主节点，此处我们获取一个8000节点 id cc2e011296ad35388eed1d7263140b820acd9be1
到src 目录下通过ruby执行增加从节点命令

[root@localhost src]#./redis-trib.rb add-node --slave --master-id cc2e011296ad35388eed1d7263140b820acd9be1 172.8.8.184:8003 172.8.8.184:8000

4. 高可用测试

查看redis进程杀掉一个主节点

[root@localhost redis-3.2.11]# ps -aux | grep redis
root      71087  0.2  0.2 141072  9848 ?        Ssl  17:38   0:13 ./redis-server *:8000 [cluster]
root      71091  0.2  0.2 141072  9852 ?        Ssl  17:38   0:13 ./redis-server *:8001 [cluster]
root      71128  0.2  0.2 141072  9852 ?        Ssl  17:50   0:09 ./redis-server *:8002 [cluster]
root      71187  0.2  0.2 136976  7760 ?        Ssl  18:00   0:07 ./redis-server *:8003 [cluster]
root      71191  0.2  0.2 136976  7760 ?        Ssl  18:00   0:07 ./redis-server *:8004 [cluster]
root      71195  0.2  0.2 136976  7760 ?        Ssl  18:00   0:07 ./redis-server *:8005 [cluster]
root      71327  0.0  0.0 112728   972 pts/1    S+   18:53   0:00 grep --color=auto redis
[root@localhost redis-3.2.11]# kill -9 71128

查看集群可以看到集群正在选举

[root@localhost redis-3.2.11]# ./redis-cli -c -p 8000
127.0.0.1:8000> cluster nodes
d7f0752c7f20a6c0d72f648234f6b6aa7038ec20 172.8.8.184:8001 master - 0 1577444081084 2 connected 5461-10922
23aa17fe4b326dae97d27411d9dff453fe51eb18 172.8.8.184:8004 slave d7f0752c7f20a6c0d72f648234f6b6aa7038ec20 0 1577444082092 2 connected
241c3872122417e97d40e526389f05db52388d9b 172.8.8.184:8003 slave cc2e011296ad35388eed1d7263140b820acd9be1 0 1577444083104 1 connected
cc2e011296ad35388eed1d7263140b820acd9be1 172.8.8.184:8000 myself,master - 0 0 1 connected 0-5460
ca7d0774760204fe213adcc40074a93afac9af1d 172.8.8.184:8005 master - 0 1577444084113 6 connected 10923-16383
54a64c2856670e14e66e76a48af70d82b631dee1 172.8.8.184:8002 master,fail - 1577444052211 1577444051807 3 disconnected
127.0.0.1:8000>

过一会查看
8002对应的从8005已成为master

127.0.0.1:8000> cluster nodes
d7f0752c7f20a6c0d72f648234f6b6aa7038ec20 172.8.8.184:8001 master - 0 1577444242700 2 connected 5461-10922
23aa17fe4b326dae97d27411d9dff453fe51eb18 172.8.8.184:8004 slave d7f0752c7f20a6c0d72f648234f6b6aa7038ec20 0 1577444243712 2 connected
241c3872122417e97d40e526389f05db52388d9b 172.8.8.184:8003 slave cc2e011296ad35388eed1d7263140b820acd9be1 0 1577444241690 1 connected
cc2e011296ad35388eed1d7263140b820acd9be1 172.8.8.184:8000 myself,master - 0 0 1 connected 0-5460
ca7d0774760204fe213adcc40074a93afac9af1d 172.8.8.184:8005 master - 0 1577444242195 6 connected 10923-16383
54a64c2856670e14e66e76a48af70d82b631dee1 172.8.8.184:8002 master,fail - 1577444052211 1577444051807 3 disconnected
127.0.0.1:8000>

再将其恢复，8002已成为8005的slave节点，不会再恢复成master.

127.0.0.1:8000> CLUSTER NODES
d7f0752c7f20a6c0d72f648234f6b6aa7038ec20 172.8.8.184:8001 master - 0 1577444367956 2 connected 5461-10922
23aa17fe4b326dae97d27411d9dff453fe51eb18 172.8.8.184:8004 slave d7f0752c7f20a6c0d72f648234f6b6aa7038ec20 0 1577444366948 2 connected
241c3872122417e97d40e526389f05db52388d9b 172.8.8.184:8003 slave cc2e011296ad35388eed1d7263140b820acd9be1 0 1577444368966 1 connected
cc2e011296ad35388eed1d7263140b820acd9be1 172.8.8.184:8000 myself,master - 0 0 1 connected 0-5460
ca7d0774760204fe213adcc40074a93afac9af1d 172.8.8.184:8005 master - 0 1577444362912 6 connected 10923-16383
54a64c2856670e14e66e76a48af70d82b631dee1 172.8.8.184:8002 slave ca7d0774760204fe213adcc40074a93afac9af1d 0 1577444364931 6 connected
127.0.0.1:8000>