Docker下redis与springboot三部曲之二：安装redis主从和哨兵

在上一章《 Docker下redis与springboot三部曲之一：极速体验》我们快速体验了springboot访问单机版redis，今天的实战中我们在Docker下搭建redis主从和哨兵服务；

原文地址：http://blog.csdn.net/boling_cavalry/article/details/78995407

实战环境

1. 本次实战的环境是Ubuntu16，安装的Docker版本是17.03.2-ce；
2. 当前电脑创建文件夹/usr/local/work/share，后面会用来做共享文件夹；
3. 本次实战要创建八个容器，redis相关的6个(1主2从3哨兵)，再用一个redis客户端用于远程连接前面6个机器做验证，还有一个用来部署springboot的机器，如下表格所示：
   | 容器名 | ip | 功能|
   | :-------- | :----| :----|
   | blog_master_1 | 172.18.0.2 | redis主 |
   | blog_slave_1 | 172.17.0.4 | redis从一 |
   | blog_slave_2 | 172.18.0.7 | redis从二 |
   | blog_sentinel_1 | 172.18.0.5 | redis哨兵一 |
   | blog_sentinel_2 | 172.18.0.8 | redis哨兵二 |
   | blog_sentinel_3 | 172.18.0.9 | redis哨兵三 |
   | blog_client_1 | 172.18.0.3 | redis客户端，远程连接主、从、哨兵验证工作情况 |
   | blog_java_1 | 172.18.0.6 | springboot工程运行环境 |

制作哨兵镜像

redis主、从所用镜像同为官方redis:3，而哨兵的容器不能直接用这个镜像，需要在这个镜像的基础上做一定的加工方可使用；
制作哨兵镜像需要制作下面这三个文本文件：

1. sentinel.conf；
2. sentinel-entrypoint.sh；
3. Dockerfile；

下面依次说明这三个文件的内容和作用：

• 本机创建一个文件夹，名为sentinel；
• sentinel文件夹下新增一个sentinel.conf文件，哨兵相关的配置都在这里，制作镜像时会用到，内容如下：

# Example sentinel.conf can be downloaded from http://download.redis.io/redis-stable/sentinel.conf

port 26379

dir /tmp

sentinel monitor mymaster redis-master 6379 $SENTINEL_QUORUM

sentinel down-after-milliseconds mymaster $SENTINEL_DOWN_AFTER

sentinel parallel-syncs mymaster 1

sentinel failover-timeout mymaster $SENTINEL_FAILOVER

上面的配置中，redis-master对应的是docker-compose.yml中redis主的link参数；

• sentinel文件夹下新增一个sentinel-entrypoint.sh文件，容器启动的时候会执行该脚本，内容如下：

#!/bin/sh
#替换原有的配置参数
sed -i "s/\$SENTINEL_QUORUM/$SENTINEL_QUORUM/g" /etc/redis/sentinel.conf
sed -i "s/\$SENTINEL_DOWN_AFTER/$SENTINEL_DOWN_AFTER/g" /etc/redis/sentinel.conf
sed -i "s/\$SENTINEL_FAILOVER/$SENTINEL_FAILOVER/g" /etc/redis/sentinel.conf
#再执行另一个脚本
exec docker-entrypoint.sh redis-server /etc/redis/sentinel.conf --sentinel

• sentinel文件夹下新增一个Dockerfile文件，内容如下：

# Docker image for redis sentinel
# VERSION 0.0.1
# Author: bolingcavalry

#基础镜像使用官方redis:3
FROM redis:3

#作者
MAINTAINER BolingCavalry <zq2599@gmail.com>
#对外暴露26379接口
EXPOSE 26379
#替换原有的sentinel.conf文件
ADD sentinel.conf /etc/redis/sentinel.conf
RUN chown redis:redis /etc/redis/sentinel.conf
#定义环境变量
ENV SENTINEL_QUORUM 2
ENV SENTINEL_DOWN_AFTER 30000
ENV SENTINEL_FAILOVER 180000
#把容器启动时要执行的脚本复制到镜像中
COPY sentinel-entrypoint.sh /usr/local/bin/
#给脚本添加可执行权限
RUN chmod +x /usr/local/bin/sentinel-entrypoint.sh
#定义启动容器时自动执行的脚本
ENTRYPOINT ["sentinel-entrypoint.sh"]

• 打开命令行，在sentinel文件夹下执行以下命令构建一个镜像：

docker build -t bolingcavalry/redis3sentinel:0.0.1 .

这样哨兵镜像就做好了；

通过docker-compose命令启动

哨兵镜像做好了，我们就可以启容器了，多个容器的启动还是用docker-compose比较方便，新建一个目录名叫blog，在blog目录下制作docker-compose.yml文件，内容如下：

master:
  image: redis:3
  ports:
    - "6379:6379"
slave:
  image: redis:3
  command: redis-server --slaveof redis-master 6379
  links:
    - master:redis-master
sentinel:
  image: bolingcavalry/redis3sentinel:0.0.1
  environment:
    - SENTINEL_DOWN_AFTER=5000
    - SENTINEL_FAILOVER=5000    
  links:
    - master:redis-master
    - slave
java:
  image: bolingcavalry/springbootrun:0.0.1
  links:
    - sentinel:redis-sentinel
    - master:redis-master
  volumes:
     - /usr/local/work/share:/usr/Downloads
  ports:
    - "8080:8080"
  tty:true
client:
  image: redis:3

用docker ps命令看到五个容器启动成功，如下图：

查看信息

• 执行以下命令查看master的IP：

root@rabbitmq:~# docker exec blog_java_1 cat /etc/hosts
127.0.0.1       localhost
::1     localhost ip6-localhost ip6-loopback
fe00::0 ip6-localnet
ff00::0 ip6-mcastprefix
ff02::1 ip6-allnodes
ff02::2 ip6-allrouters
172.17.0.2      master_1 b3191e43f27e blog_master_1
172.17.0.2      redis-master b3191e43f27e blog_master_1
172.17.0.5      blog_sentinel_1 a85a4096837b
172.17.0.5      redis-sentinel a85a4096837b blog_sentinel_1
172.17.0.5      sentinel_1 a85a4096837b blog_sentinel_1
172.17.0.2      blog_master_1 b3191e43f27e
172.17.0.6      337f68cf3f25

可以看到master的ip是172.17.0.2；

• 执行以下命令查看哨兵的IP：

root@rabbitmq:/usr/local/work/blog# docker exec blog_sentinel_1 cat /etc/hosts
127.0.0.1       localhost
::1     localhost ip6-localhost ip6-loopback
fe00::0 ip6-localnet
ff00::0 ip6-mcastprefix
ff02::1 ip6-allnodes
ff02::2 ip6-allrouters
172.17.0.2      blog_master_1 b3191e43f27e
172.17.0.2      master_1 b3191e43f27e blog_master_1
172.17.0.2      redis-master b3191e43f27e blog_master_1
172.17.0.4      blog_slave_1 f1cb5e268c14
172.17.0.4      slave f1cb5e268c14 blog_slave_1
172.17.0.4      slave_1 f1cb5e268c14 blog_slave_1
172.17.0.5      a85a4096837b

可以看到哨兵的ip是172.17.0.5(a85a4096837b是容器ID)；

• 执行以下命令进入client容器：

docker exec -it blog_client_1 /bin/bash

• 在client容器中执行以下命令查看master的情况：

root@942deeb36265:/data# redis-cli -h 172.17.0.2 -p 6379 info Replication
# Replication
role:master
connected_slaves:1
slave0:ip=172.17.0.4,port=6379,state=online,offset=99305,lag=0
master_repl_offset:99440
repl_backlog_active:1
repl_backlog_size:1048576
repl_backlog_first_byte_offset:2
repl_backlog_histlen:99439

可以看见只有一个slave：172.17.0.4

• 在client容器中执行以下命令查看哨兵的情况：

root@942deeb36265:/data# redis-cli -h 172.17.0.5 -p 26379 info Sentinel
# Sentinel
sentinel_masters:1
sentinel_tilt:0
sentinel_running_scripts:0
sentinel_scripts_queue_length:0
sentinel_simulate_failure_flags:0
master0:name=mymaster,status=ok,address=172.17.0.2:6379,slaves=2,sentinels=1

可以看见只有一个哨兵；

slave扩容

之前我们规划的是两个slave、三个哨兵，目前只有一个slave和一个哨兵，接下来我们试试docker-compose提供的横向扩容的命令，快速增加slave和哨兵的容器数；

执行以下命令，增加一个slave:

docker-compose scale slave=2

再执行命令docker ps看一下容器情况，如下：

root@rabbitmq:/usr/local/work/blog# docker-compose scale slave=2
WARNING: The scale command is deprecated. Use the up command with the --scale flag instead.
Starting blog_slave_1 ... done
Creating blog_slave_2 ... done
root@rabbitmq:/usr/local/work/blog# docker ps
CONTAINER ID        IMAGE                                COMMAND                  CREATED             STATUS              PORTS                                NAMES
56a10f1fc70b        redis:3                              "docker-entrypoint..."   11 seconds ago      Up 10 seconds       6379/tcp                             blog_slave_2
337f68cf3f25        bolingcavalry/springbootrun:0.0.1    "/bin/bash"              29 minutes ago      Up 29 minutes       0.0.0.0:8080->8080/tcp               blog_java_1
a85a4096837b        bolingcavalry/redis3sentinel:0.0.1   "sentinel-entrypoi..."   29 minutes ago      Up 29 minutes       6379/tcp, 26379/tcp   blog_sentinel_1
f1cb5e268c14        redis:3                              "docker-entrypoint..."   29 minutes ago      Up 29 minutes       6379/tcp                             blog_slave_1
942deeb36265        redis:3                              "docker-entrypoint..."   29 minutes ago      Up 29 minutes       6379/tcp                             blog_client_1
b3191e43f27e        redis:3                              "docker-entrypoint..."   29 minutes ago      Up 29 minutes       6379/tcp               blog_master_1

• 在client容器上执行命令看看主从的信息：

root@942deeb36265:/data# redis-cli -h 172.17.0.2 -p 6379 info Replication
# Replication
role:master
connected_slaves:2
slave0:ip=172.17.0.4,port=6379,state=online,offset=127060,lag=1
slave1:ip=172.17.0.7,port=6379,state=online,offset=127060,lag=0
master_repl_offset:127195
repl_backlog_active:1
repl_backlog_size:1048576
repl_backlog_first_byte_offset:2
repl_backlog_histlen:127194

哨兵扩容

执行以下命令，增加两个哨兵:

docker-compose scale sentinel=3

再执行命令docker ps看一下容器情况，如下：

root@rabbitmq:/usr/local/work/blog# docker-compose scale sentinel=3
WARNING: The scale command is deprecated. Use the up command with the --scale flag instead.
Starting blog_sentinel_1 ... done
Creating blog_sentinel_2 ... done
Creating blog_sentinel_3 ... done
root@rabbitmq:/usr/local/work/blog# docker ps
CONTAINER ID        IMAGE                                COMMAND                  CREATED              STATUS              PORTS                    NAMES
45ede729f216        bolingcavalry/redis3sentinel:0.0.1   "sentinel-entrypoi..."   About a minute ago   Up About a minute   6379/tcp, 26379/tcp      blog_sentinel_3
f99d0501b137        bolingcavalry/redis3sentinel:0.0.1   "sentinel-entrypoi..."   About a minute ago   Up About a minute   6379/tcp, 26379/tcp      blog_sentinel_2
abcc206ff31f        redis:3                              "docker-entrypoint..."   About a minute ago   Up About a minute   6379/tcp                 blog_slave_2
f7527e8d9b98        bolingcavalry/springbootrun:0.0.1    "/bin/bash"              About a minute ago   Up About a minute   0.0.0.0:8080->8080/tcp   blog_java_1
f3b7e3e8e820        bolingcavalry/redis3sentinel:0.0.1   "sentinel-entrypoi..."   About a minute ago   Up About a minute   6379/tcp, 26379/tcp      blog_sentinel_1
3a42ac696e7b        redis:3                              "docker-entrypoint..."   About a minute ago   Up About a minute   6379/tcp                 blog_slave_1
11c4bc32062d        redis:3                              "docker-entrypoint..."   About a minute ago   Up About a minute   6379/tcp                 blog_master_1
07a2be2bf114        redis:3                              "docker-entrypoint..."   About a minute ago   Up About a minute   6379/tcp                 blog_client_1

• 在client容器上执行命令redis-cli -h 172.17.0.5 -p 26379 info Sentinel看看哨兵的信息：

root@07a2be2bf114:/data# redis-cli -h 172.17.0.5 -p 26379 info Sentinel
# Sentinel
sentinel_masters:1
sentinel_tilt:0
sentinel_running_scripts:0
sentinel_scripts_queue_length:0
sentinel_simulate_failure_flags:0
master0:name=mymaster,status=ok,address=172.17.0.3:6379,slaves=2,sentinels=3

可以看到哨兵数量为3；

验证扩容结果

进入master容器写一条记录，再进入slave容器读这个记录，看看主从同步是否生效：

root@rabbitmq:/usr/local/work/blog# docker exec -it blog_master_1 /bin/bash
root@11c4bc32062d:/data# redis-cli
127.0.0.1:6379> set abc 123456
OK
127.0.0.1:6379> quit
root@11c4bc32062d:/data# exit
exit
root@rabbitmq:/usr/local/work/blog# docker exec -it blog_slave_2 /bin/bash
root@abcc206ff31f:/data# redis-cli
127.0.0.1:6379> get abc
"123456"
127.0.0.1:6379>

验证高可用

• 执行命令docker stop blog_master_1将master停掉
• 执行命令docker logs -f blog_sentinel_1看见哨兵容器的日志如下：

1:X 10 Jan 04:37:43.358 # +sdown master mymaster 172.17.0.2 6379
1:X 10 Jan 04:37:43.458 # +new-epoch 1
1:X 10 Jan 04:37:43.458 # +vote-for-leader dae600f31e27dc2b5eef79cdb6154f4365676a04 1
1:X 10 Jan 04:37:43.865 # +config-update-from sentinel dae600f31e27dc2b5eef79cdb6154f4365676a04 172.17.0.9 26379 @ mymaster 172.17.0.2 6379
1:X 10 Jan 04:37:43.865 # +switch-master mymaster 172.17.0.2 6379 172.17.0.4 6379
1:X 10 Jan 04:37:43.865 * +slave slave 172.17.0.7:6379 172.17.0.7 6379 @ mymaster 172.17.0.4 6379
1:X 10 Jan 04:37:43.866 * +slave slave 172.17.0.2:6379 172.17.0.2 6379 @ mymaster 172.17.0.4 6379
1:X 10 Jan 04:37:48.903 # +sdown slave 172.17.0.2:6379 172.17.0.2 6379 @ mymaster 172.17.0.4 6379

可以看见发生了切换，redis集群的master由之前的 172.17.0.2 6379变成了172.17.0.4 6379；

• 进入blog_client_1容器，执行命令redis-cli -h 172.17.0.4 -p 6379 info Replication 看看最新的master信息如下：

root@c451d8ecb672:/data# redis-cli -h 172.17.0.4 -p 6379 info Replication 
# Replication
role:master
connected_slaves:1
slave0:ip=172.17.0.7,port=6379,state=online,offset=22318,lag=0
master_repl_offset:22453
repl_backlog_active:1
repl_backlog_size:1048576
repl_backlog_first_byte_offset:2
repl_backlog_histlen:22452

• 进入blog_client_1容器，执行命令redis-cli -h 172.17.0.7 -p 6379 info Replication 看看最另一个slave信息如下，role依然是slave：

root@c451d8ecb672:/data# redis-cli -h 172.17.0.7 -p 6379 info Replication 
# Replication
role:slave
master_host:172.17.0.4
master_port:6379
master_link_status:up
master_last_io_seconds_ago:1
master_sync_in_progress:0
slave_repl_offset:28570
slave_priority:100
slave_read_only:1
connected_slaves:0
master_repl_offset:0
repl_backlog_active:0
repl_backlog_size:1048576
repl_backlog_first_byte_offset:0
repl_backlog_histlen:0

以上就是在Docker下搭建redis主从和哨兵的整个实战过程，接下来的实战，我们会用springboot访问redis高可用环境，本章的环境在下一个章的实战中会继续用到；

欢迎关注我的公众号：程序员欣宸