构建Redis集群环境

环境信息

• 操作系统：Ubuntu 18.04.3
• Redis版本：redis-5.0.7
• 集群环境
  2台机器：work01、work02
  6个redis节点：node1、node2、node3、node4、node5、node6
• 节点角色：3主3从

安装Redis

1. 下载redis-5.0.7，解压后安装

$ mkdir redis-cluster
$ cd redis-cluster
$ wget http://download.redis.io/releases/redis-5.0.7.tar.gz
$ tar xzf redis-5.0.7.tar.gz
$ cp redis-5.0.7 -r redis1
$ cp redis-5.0.7 -r redis2
$ cp redis-5.0.7 -r redis3
$ cd redis1
$ make
$ cd redis2
$ make
$ cd redis3
$ make

2. 创建redis节点的启动配置文件，并修改集群配置信息、持久化路径、日志路径、守护进程方式启动等

$ touch redis1.conf redis2.conf redis3.conf
$ vim redis1.conf

#端口
port 7001
# 守护进程
daemonize yes
protected-mode no
databases 16
cluster-enable yes
cluster-config-file /usr/local/redis-cluster/nodes_1.conf
cluster-node-timeout 5000
pidfile /var/run/redis_1.pid
dbfilename dump.rdb
# 持久化路径
dir /data/redis1
# 日志路径
logfile "/var/log/redis-cluster/redis1/redis.log"

其他两个配置文件依次配置即可；记得把持久化和日志路径文件夹创建出来。

3. 创建启动redis集群脚本，并添加执行权限

$ touch start-all.sh
$ sudo chmod +x start-all.sh
$ vim start-all.sh

#启动redis集群
/usr/local/redis-cluster/redis1/src/redis-server /usr/local/redis-cluster/redis1.conf
/usr/local/redis-cluster/redis2/src/redis-server /usr/local/redis-cluster/redis2.conf
/usr/local/redis-cluster/redis3/src/redis-server /usr/local/redis-cluster/redis3.conf

4. 创建启动redis集群脚本，并添加执行权限

$ touch stop-all.sh
$ sudo chmod +x stop-all.sh
$ vim stop-all.sh

#停止redis集群
/usr/local/redis-cluster/redis1/src/redis-cli -p 7001 shutdown
/usr/local/redis-cluster/redis2/src/redis-cli -p 7002  shutdown
/usr/local/redis-cluster/redis3/src/redis-cli -p 7003  shutdown

5. 启动redis

$ ./start-all.sh

创建集群

1. 创建集群 在其中一个redis节点上执行命令：

➜  src ./redis-cli --cluster create 192.168.1.20:7001 192.168.1.20:7002 192.168.1.20:7003 192.168.1.21:7001 192.168.1.21:7002 192.168.1.21:7003 --cluster-replicas 1
>>> Performing hash slots allocation on 6 nodes...
Master[0] -> Slots 0 - 5460
Master[1] -> Slots 5461 - 10922
Master[2] -> Slots 10923 - 16383
Adding replica 192.168.1.21:7003 to 192.168.1.20:7001
Adding replica 192.168.1.20:7003 to 192.168.1.21:7001
Adding replica 192.168.1.21:7002 to 192.168.1.20:7002
M: f89d8033205a54ac40c36eb253a2d8483349f208 192.168.1.20:7001
   slots:[0-5460] (5461 slots) master
M: cdd06cf6514e390fab3c0979ed86cbfa72f0dc5f 192.168.1.20:7002
   slots:[10923-16383] (5461 slots) master
S: 2044d80c3de097c4452990ac78cf491472255566 192.168.1.20:7003
   replicates 592e5b4ec828dc9704d1f226b50dcede24df65a5
M: 592e5b4ec828dc9704d1f226b50dcede24df65a5 192.168.1.21:7001
   slots:[5461-10922] (5462 slots) master
S: f377ecfbcab24470349761c3374e22eee8385e93 192.168.1.21:7002
   replicates cdd06cf6514e390fab3c0979ed86cbfa72f0dc5f
S: 5a068d57dab5c30d64f85398c6190956cb2a5d32 192.168.1.21:7003
   replicates f89d8033205a54ac40c36eb253a2d8483349f208
Can I set the above configuration? (type 'yes' to accept):yes

>>> Nodes configuration updated
>>> Assign a different config epoch to each node
>>> Sending CLUSTER MEET messages to join the cluster
Waiting for the cluster to join
....
>>> Performing Cluster Check (using node 192.168.1.20:7001)
M: f89d8033205a54ac40c36eb253a2d8483349f208 192.168.1.20:7001
   slots:[0-5460] (5461 slots) master
   1 additional replica(s)
S: 5a068d57dab5c30d64f85398c6190956cb2a5d32 192.168.1.21:7003
   slots: (0 slots) slave
   replicates f89d8033205a54ac40c36eb253a2d8483349f208
S: 2044d80c3de097c4452990ac78cf491472255566 192.168.1.20:7003
   slots: (0 slots) slave
   replicates 592e5b4ec828dc9704d1f226b50dcede24df65a5
M: 592e5b4ec828dc9704d1f226b50dcede24df65a5 192.168.1.21:7001
   slots:[5461-10922] (5462 slots) master
   1 additional replica(s)
M: cdd06cf6514e390fab3c0979ed86cbfa72f0dc5f 192.168.1.20:7002
   slots:[10923-16383] (5461 slots) master
   1 additional replica(s)
S: f377ecfbcab24470349761c3374e22eee8385e93 192.168.1.21:7002
   slots: (0 slots) slave
   replicates cdd06cf6514e390fab3c0979ed86cbfa72f0dc5f
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.

2. 以集群的方式登录：

$ ./redis-cli -p 7001 -c

3. 测试集群可用性，kill其中一个master

ps aux |grep redis
root     15659  0.2  0.0  72072  5840 ?        Ssl  14:20   0:07 /usr/local/redis-cluster/redis1/src/redis-server *:7001 [cluster]
root     15665  0.2  0.0  67976  5220 ?        Ssl  14:20   0:07 /usr/local/redis-cluster/redis2/src/redis-server *:7002 [cluster]
root     15671  0.2  0.0  70536  5968 ?        Ssl  14:20   0:07 /usr/local/redis-cluster/redis3/src/redis-server *:7003 [cluster]
huangyi+ 16282  0.0  0.0  21532  1084 pts/1    S+   15:12   0:00 grep --color=auto --exclude-dir=.bzr --exclude-dir=CVS --exclude-dir=.git --exclude-dir=.hg --exclude-dir=.svn redis
➜  src sudo kill -9 15659

检查集群状态：发现从节点192.168.1.21:7003已经切换成了主节点。

sudo ./redis-cli --cluster check 127.0.0.1:7002
Could not connect to Redis at 192.168.1.20:7001: Connection refused
127.0.0.1:7002 (cdd06cf6...) -> 2 keys | 5461 slots | 1 slaves.
192.168.1.21:7003 (5a068d57...) -> 2 keys | 5461 slots | 0 slaves.
192.168.1.21:7001 (592e5b4e...) -> 1 keys | 5462 slots | 1 slaves.
[OK] 5 keys in 3 masters.
0.00 keys per slot on average.
>>> Performing Cluster Check (using node 127.0.0.1:7002)
M: cdd06cf6514e390fab3c0979ed86cbfa72f0dc5f 127.0.0.1:7002
   slots:[10923-16383] (5461 slots) master
   1 additional replica(s)
M: 5a068d57dab5c30d64f85398c6190956cb2a5d32 192.168.1.21:7003
   slots:[0-5460] (5461 slots) master
S: 2044d80c3de097c4452990ac78cf491472255566 192.168.1.20:7003
   slots: (0 slots) slave
   replicates 592e5b4ec828dc9704d1f226b50dcede24df65a5
M: 592e5b4ec828dc9704d1f226b50dcede24df65a5 192.168.1.21:7001
   slots:[5461-10922] (5462 slots) master
   1 additional replica(s)
S: f377ecfbcab24470349761c3374e22eee8385e93 192.168.1.21:7002
   slots: (0 slots) slave
   replicates cdd06cf6514e390fab3c0979ed86cbfa72f0dc5f
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.

启动刚刚被kill掉的节点，再查看节点状态，此节点变成了21:7003的从节点。

127.0.0.1:7001> cluster nodes
f377ecfbcab24470349761c3374e22eee8385e93 192.168.1.21:7002@17002 slave cdd06cf6514e390fab3c0979ed86cbfa72f0dc5f 0 1577085453000 5 connected
5a068d57dab5c30d64f85398c6190956cb2a5d32 192.168.1.21:7003@17003 master - 0 1577085454586 7 connected 0-5460
f89d8033205a54ac40c36eb253a2d8483349f208 192.168.1.20:7001@17001 myself,slave 5a068d57dab5c30d64f85398c6190956cb2a5d32 0 1577085454000 1 connected
cdd06cf6514e390fab3c0979ed86cbfa72f0dc5f 192.168.1.20:7002@17002 master - 0 1577085453000 2 connected 10923-16383
2044d80c3de097c4452990ac78cf491472255566 192.168.1.20:7003@17003 slave 592e5b4ec828dc9704d1f226b50dcede24df65a5 0 1577085454000 4 connected
592e5b4ec828dc9704d1f226b50dcede24df65a5 192.168.1.21:7001@17001 master - 0 1577085454686 4 connected 5461-10922

至此3主3从的redis集群搭建完成。

设置Redis开机自动启动

1. 拷贝 redis1 安装目前下的 /usr/local/redis-cluster/redis1/utils/redis_init_script 到 /etc/init.d/redis1文件中；

$ cp /usr/local/redis-cluster/redis1/utils/redis_init_script /etc/init.d/redis1

2. 修改/etc/init.d/redis1 文件，修改redis安装的相关文件安装目录

REDISPORT=7001
#注意自己安装的redis根目录
REDISPATH=/usr/local/redis-cluster/redis1
EXEC=${REDISPATH}/src/redis-server
CLIEXEC=${REDISPATH}/src/redis-cli

PIDFILE=/var/run/redis_1.pid
CONF="/usr/local/redis-cluster/redis1.conf"

3. 对redis2、redis3重复执行上述1 2操作
4. 设置为开机启动项

$ sysv-rc-conf redis1 on
$ sysv-rc-conf redis2 on
$ sysv-rc-conf redis3 on

5. 服务验证

$ service redis1 start
$ service redis1 stop
$ service redis2 start
$ service redis2 stop
$ service redis3 start
$ service redis3 stop

如果启动报如下错误，使用 sudo systemctl daemon-reload刷新：

$ sudo service redis3 start
Failed to start redis3.service: Unit redis3.service not found.

$ sudo systemctl daemon-reload

6. 重启系统后查看redis服务

$ reboot

$ ps aux |grep redis
root       734  0.1  0.0  64392  4852 ?        Ssl  13:34   0:00 /usr/local/redis-cluster/redis1/src/redis-server *:7001 [cluster]
root       740  0.1  0.0  64392  4740 ?        Ssl  13:34   0:00 /usr/local/redis-cluster/redis2/src/redis-server *:7002 [cluster]
root       749  0.1  0.0  64392  4744 ?        Ssl  13:34   0:00 /usr/local/redis-cluster/redis3/src/redis-server *:7003 [cluster]

7. 设置密码：修改所有Redis集群中的redis.conf文件加入，然后重启所有redis。

masterauth 123456 
requirepass 123456 

8. 带密码访问集群

./redis-cli -c -p 7001 -a 123456

9. 修复redis集群

sudo ./redis-cli --cluster fix 127.0.0.1:7001 -a 123456 --cluster-search-multiple-owners