Redis集群部署（哨兵模式——Docker部署）

Redis集群部署（哨兵模式——Docker部署）

1、环境准备

• 我这里使用的是虚拟机，虚拟机安装的在这里【Vmware上搭建CentOS 7.9版本系统及网络配置】
• 2核2G及30G的磁盘空间
• Docker环境：3个sentinel节点、1个master节点及2个slave节点

2、开始搭建哨兵集群

2.1、启动3个redis节点

先创建如下目录，并在各个目录下配置redis.conf文件。

创建文件目录
[root@node1 ~]# mkdir -p redis/rd1 redis/rd2 redis/rd3

3个redis配置文件需要调整这几个参数：

protected-mode no #关闭保护模式
bind * -::* #接受所有ip请求该redis服务

使用Docker容器启动3个redis，操作指令如下：

docker运行3个节点
[root@node1 ~]# docker run -d -v /root/redis/rd1/redis.conf:/usr/local/etc/redis/redis.conf --name redis1 redis redis-server /usr/local/etc/redis/redis.conf
[root@node1 ~]# docker run -d -v /root/redis/rd2/redis.conf:/usr/local/etc/redis/redis.conf --name redis2 redis redis-server /usr/local/etc/redis/redis.conf
[root@node1 ~]# docker run -d -v /root/redis/rd3/redis.conf:/usr/local/etc/redis/redis.conf --name redis3 redis redis-server /usr/local/etc/redis/redis.conf

查看容器运行情况
[root@node1 ~]# docker ps
CONTAINER ID        IMAGE               COMMAND                  CREATED             STATUS              PORTS               NAMES
a40474e05d81        redis               "docker-entrypoint.s…"   2 seconds ago       Up 2 seconds        6379/tcp            redis3
6e513af6bab0        redis               "docker-entrypoint.s…"   8 seconds ago       Up 7 seconds        6379/tcp            redis2
277780aa9cb3        redis               "docker-entrypoint.s…"   32 seconds ago      Up 32 seconds       6379/tcp            redis1


将redis2与redis3设置为redis1的从节点，查看redis1的ip地址
[root@node1 ~]# docker inspect redis1|grep IPAddress
            "SecondaryIPAddresses": null,
            "IPAddress": "172.17.0.2",
                    "IPAddress": "172.17.0.2",

分别进入redis2与redis3容器，执行如下命令
[root@node1 ~]# docker exec -it redis2  redis-cli REPLICAOF 172.17.0.2 6379
OK
[root@node1 ~]# docker exec -it redis3  redis-cli REPLICAOF 172.17.0.2 6379
OK
[root@node1 ~]# 
 
查看redis1节点状态
[root@node1 ~]# docker exec -it redis1 redis-cli info Replication
# Replication
role:master
connected_slaves:2
slave0:ip=172.17.0.3,port=6379,state=online,offset=84,lag=1
slave1:ip=172.17.0.4,port=6379,state=online,offset=84,lag=0
master_failover_state:no-failover
master_replid:46afa405f4a4444587a3f45f9d3ae7e95554fcbd
master_replid2:0000000000000000000000000000000000000000
master_repl_offset:84
second_repl_offset:-1
repl_backlog_active:1
repl_backlog_size:1048576
repl_backlog_first_byte_offset:1
repl_backlog_histlen:84
[root@node1 ~]# 

2.2、启动3个sentinel节点

哨兵主要是用来监控master节点的，当master宕机时，哨兵会先主观认为master下线（此时master还未下线），然后该哨兵会跟其他哨兵通信说“我联系不上master了你们看下”，如果其他master也联系不上，那么master就会被标记客观下线，然后其中一个哨兵就会去处理故障转移操作（谁发现水处理原则）。
哨兵判断master是否下线跟下面配置有关：

sentinel monitor mymaster 172.17.0.2 6379 2 #这里的2就是需要2个sentinel都认为master下线才是真的下线

创建3个与sentinel相关的文档，分别在这3个目录下各配置sentinel.conf的配置文件。

[root@node1 ~]# mkdir -p sentinel/sn1 sentinel/sn2 sentinel/sn3 

3个sentinel的配置文件如下：

port 26379
dir /tmp
sentinel monitor mymaster 172.17.0.2 6379 2
sentinel down-after-milliseconds mymaster 30000
sentinel parallel-syncs mymaster 1
sentinel failover-timeout mymaster 180000

使用Docker启动3个sentinel的容器

[root@node1 ~]# docker run -d --name sentinel1 -v /root/sentinel/sn1/sentinel.conf:/usr/local/etc/redis/sentinel/sentinel.conf redis redis-sentinel /usr/local/etc/redis/sentinel/sentinel.conf
[root@node1 ~]# docker run -d --name sentinel2 -v /root/sentinel/sn2/sentinel.conf:/usr/local/etc/redis/sentinel/sentinel.conf redis redis-sentinel /usr/local/etc/redis/sentinel/sentinel.conf
[root@node1 ~]# docker run -d --name sentinel3 -v /root/sentinel/sn3/sentinel.conf:/usr/local/etc/redis/sentinel/sentinel.conf redis redis-sentinel /usr/local/etc/redis/sentinel/sentinel.conf

检查哨兵集群是否搭建成功，发现已经有3台sentinel启动了，到此3个sentinel搭建完成
[root@node1 ~]# docker exec -it redis1 /bin/bash
root@1a9e87f2cd12:/data# redis-cli SUBSCRIBE __sentinel__:hello
Reading messages... (press Ctrl-C to quit)
1) "subscribe"
2) "__sentinel__:hello"
3) (integer) 1
1) "message"
2) "__sentinel__:hello"
3) "172.17.0.5,26379,1603492da1d237aa59d4f69f2fe49e0086b858bc,0,mymaster,172.17.0.2,6379,0"
1) "message"
2) "__sentinel__:hello"
3) "172.17.0.6,26379,c4fef119ee566ce5202b11e59e2cfb8a264108a2,0,mymaster,172.17.0.2,6379,0"
1) "message"
2) "__sentinel__:hello"
3) "172.17.0.7,26379,813076c78201fb58bb8b092b6621687c45f00d72,0,mymaster,172.17.0.2,6379,0"
^Z
[1]+  Stopped                 redis-cli SUBSCRIBE __sentinel__:hello
root@1a9e87f2cd12:/data# exit
exit
There are stopped jobs.
root@1a9e87f2cd12:/data# exit
exit

3、哨兵集群故障转移

3.1、故障转移实战

停止掉redis1节点，然后等待30s左右

[root@node1 ~]# docker stop redis1
redis1
[root@node1 ~]# 

查看sentinel日志
[root@node1 ~]# docker logs --tail=20 sentinel1

这里保错的原因是因为挂载的配置路径没有写的权限，执行chmod 777 -R 加目录即可
1:X 29 Jun 2022 03:21:24.353 # Could not rename tmp config file (Device or resource busy)
1:X 29 Jun 2022 03:21:24.353 # WARNING: Sentinel was not able to save the new configuration on disk!!!: Device or resource busy

哨兵集群发现172.17.0.2这台机器的redis宕机了
1:X 29 Jun 2022 03:39:46.232 # +sdown master mymaster 172.17.0.2 6379
1:X 29 Jun 2022 03:39:46.313 # Could not rename tmp config file (Device or resource busy)
1:X 29 Jun 2022 03:39:46.313 # WARNING: Sentinel was not able to save the new configuration on disk!!!: Device or resource busy

纪元+1
1:X 29 Jun 2022 03:39:46.313 # +new-epoch 1
1:X 29 Jun 2022 03:39:46.314 # Could not rename tmp config file (Device or resource busy)
1:X 29 Jun 2022 03:39:46.314 # WARNING: Sentinel was not able to save the new configuration on disk!!!: Device or resource busy
1:X 29 Jun 2022 03:39:46.314 # +vote-for-leader c4fef119ee566ce5202b11e59e2cfb8a264108a2 1
1:X 29 Jun 2022 03:39:47.144 # +config-update-from sentinel c4fef119ee566ce5202b11e59e2cfb8a264108a2 172.17.0.6 26379 @ mymaster 172.17.0.2 6379

redis主从切花，提拔172.17.0.3机器作为redis服务新的master节点
1:X 29 Jun 2022 03:39:47.144 # +switch-master mymaster 172.17.0.2 6379 172.17.0.3 6379

从节点加入新的master节点
1:X 29 Jun 2022 03:39:47.145 * +slave slave 172.17.0.4:6379 172.17.0.4 6379 @ mymaster 172.17.0.3 6379
1:X 29 Jun 2022 03:39:47.145 * +slave slave 172.17.0.2:6379 172.17.0.2 6379 @ mymaster 172.17.0.3 6379

查看172.17.0.3（redis2）发现role已经从slave变成了master，说明哨兵集群运行ok
[root@node1 ~]# docker exec -it redis2 redis-cli info Replication
# Replication
role:master
connected_slaves:1
slave0:ip=172.17.0.4,port=6379,state=online,offset=361364,lag=1
master_failover_state:no-failover
master_replid:712789c9b75cca7f19529c0339c6c8bcc72f8ec9
master_replid2:d68b5f7e3366440f58fb73d94e112e21f154ad06
master_repl_offset:361364
second_repl_offset:227848
repl_backlog_active:1
repl_backlog_size:1048576
repl_backlog_first_byte_offset:1
repl_backlog_histlen:361364
[root@node1 ~]# 

查看redis1重启后的状态，发现redis1自动加入了redis2作为其从节点。
[root@node1 ~]# docker start redis1
redis1
[root@node1 ~]# docker exec -it redis1 redis-cli info Replication
# Replication
role:slave
master_host:172.17.0.3
master_port:6379
master_link_status:up
master_last_io_seconds_ago:0
master_sync_in_progress:0
slave_read_repl_offset:378914
slave_repl_offset:378914
slave_priority:100
slave_read_only:1
replica_announced:1
connected_slaves:0
master_failover_state:no-failover
master_replid:712789c9b75cca7f19529c0339c6c8bcc72f8ec9
master_replid2:0000000000000000000000000000000000000000
master_repl_offset:378914
second_repl_offset:-1
repl_backlog_active:1
repl_backlog_size:1048576
repl_backlog_first_byte_offset:378487
repl_backlog_histlen:428

到此哨兵集群搭建完成。
上篇：【redis主从集群部署】
下篇：【Redis集群部署（cluster模式——Docker）】