Docker初识：Redis服务的搭建（集群）

最新推荐文章于 2024-04-18 18:33:14 发布

滴水可藏海

最新推荐文章于 2024-04-18 18:33:14 发布

阅读量411

点赞数

分类专栏： # Redis

本文链接：https://blog.csdn.net/wngpenghao/article/details/106664477

版权

Redis 专栏收录该内容

6 篇文章 0 订阅

订阅专栏

版本说明：

系统：centos 7

docker：18

redis：6.0.3

参考博客：

Docker Compose搭建Redis一主二从三哨兵高可用集群

参考以上博客，在进行集群搭建，可能仅为软件版本的原因导致配置文件存在差异。以下过程为在适应当前系统软件版本的情况下，记录整个集群搭建过程，并说明其中区别。

一、Redis-Cluster

通过Docker来搭建Redis-Cluster集群，只需要分配6个不同端端口即可。

IP	PORT
172.17.0.6	7001
172.17.0.6	7002
172.17.0.6	7003
172.17.0.6	7004
172.17.0.6	7005
172.17.0.6	7006

1、编写配置文件，并且构建集群

构建Redis相关目录

redis-cluster.tmpl

# redis端口
port ${PORT}
# 关闭保护模式
protected-mode no
#监听
bind 0.0.0.0
# 开启集群
cluster-enabled yes
# 集群节点配置
cluster-config-file nodes.conf
# 超时
cluster-node-timeout 5000
# 集群节点IP host模式为宿主机IP
cluster-announce-ip 172.17.0.6
# 集群节点端口 7001 - 7006
cluster-announce-port ${PORT}
cluster-announce-bus-port 1${PORT}
# RDB
# #表示900 秒内如果至少有 1 个 key 的值变化，则保存
save 900 1
# #表示300 秒内如果至少有 10 个 key 的值变化，则保存
save 300 10
# #表示60 秒内如果至少有 10000 个 key 的值变化，则保存
save 60 10000
#AOF
# 开启 appendonly 备份模式
appendonly yes
# 每秒钟备份
appendfsync everysec
# 对aof文件进行压缩时，是否执行同步操作
no-appendfsync-on-rewrite no
# 当目前aof文件大小超过上一次重写时的aof文件大小的100%时会再次进行重写
auto-aof-rewrite-percentage 100
# 重写前AOF文件的大小最小值 默认 64mb
auto-aof-rewrite-min-size 64mb
# 此处绑定ip 可以是阿里内网ip 和 本地ip 也可以直接注释掉该项
#bind 172.17.0.6
#设置redis密码 各个节点请保持密码一致
requirepass 123456
masterauth 123456

新增属性：

#设置redis密码各个节点请保持密码一致
requirepass 密码

执行文件：build.sh

for port in `seq 7001 7006`; do \
  mkdir -p ./redis-cluster/${port}/conf \
  && PORT=${port} envsubst < ./redis-cluster.tmpl > ./redis-cluster/${port}/conf/redis.conf \
  && mkdir -p ./redis-cluster/${port}/data; \
done

[root@VM_0_6_centos cluster1]# tree
.
|-- build-cluster.sh
|-- build.sh
|-- docker-compose-redis-cluster.yml
|-- down.sh
|-- redis-cluster
|   |-- 7001
|   |   |-- conf
|   |   |   `-- redis.conf
|   |   `-- data
|   |       |-- appendonly.aof
|   |       `-- nodes.conf
|   |-- 7002
|   |   |-- conf
|   |   |   `-- redis.conf
|   |   `-- data
|   |       |-- appendonly.aof
|   |       `-- nodes.conf
|   |-- 7003
|   |   |-- conf
|   |   |   `-- redis.conf
|   |   `-- data
|   |       |-- appendonly.aof
|   |       `-- nodes.conf
|   |-- 7004
|   |   |-- conf
|   |   |   `-- redis.conf
|   |   `-- data
|   |       |-- appendonly.aof
|   |       `-- nodes.conf
|   |-- 7005
|   |   |-- conf
|   |   |   `-- redis.conf
|   |   `-- data
|   |       |-- appendonly.aof
|   |       `-- nodes.conf
|   `-- 7006
|       |-- conf
|       |   `-- redis.conf
|       `-- data
|           |-- appendonly.aof
|           `-- nodes.conf
|-- redis-cluster.tmpl
`-- up.sh

19 directories, 24 files

docker-compose.yml文件

docker-compose-redis-cluster.yml

version: '3.3'
services:
  redis7001:
    image: 'redis'
    container_name: redis7001
    command:
      ["redis-server", "/usr/local/etc/redis/redis.conf"]
    volumes:
      - ./redis-cluster/7001/conf/redis.conf:/usr/local/etc/redis/redis.conf
      - ./redis-cluster/7001/data:/data
    ports:
      - "7001:7001"
      - "17001:17001"
    environment:
      - TZ=Asia/Shanghai
  redis7002:
    image: 'redis'
    container_name: redis7002
    command:
      ["redis-server", "/usr/local/etc/redis/redis.conf"]
    volumes:
      - ./redis-cluster/7002/conf/redis.conf:/usr/local/etc/redis/redis.conf
      - ./redis-cluster/7002/data:/data
    ports:
      - "7002:7002"
      - "17002:17002"
    environment:
      - TZ=Asia/Shanghai
  redis7003:
    image: 'redis'
    container_name: redis7003
    command:
      ["redis-server", "/usr/local/etc/redis/redis.conf"]
    volumes:
      - ./redis-cluster/7003/conf/redis.conf:/usr/local/etc/redis/redis.conf
      - ./redis-cluster/7003/data:/data
    ports:
      - "7003:7003"
      - "17003:17003"
    environment:
      - TZ=Asia/Shanghai
  redis7004:
    image: 'redis'
    container_name: redis7004
    command:
      ["redis-server", "/usr/local/etc/redis/redis.conf"]
    volumes:
      - ./redis-cluster/7004/conf/redis.conf:/usr/local/etc/redis/redis.conf
      - ./redis-cluster/7004/data:/data
    ports:
      - "7004:7004"
      - "17004:17004"
    environment:
      - TZ=Asia/Shanghai
  redis7005:
    image: 'redis'
    container_name: redis7005
    command:
      ["redis-server", "/usr/local/etc/redis/redis.conf"]
    volumes:
      - ./redis-cluster/7005/conf/redis.conf:/usr/local/etc/redis/redis.conf
      - ./redis-cluster/7005/data:/data
    ports:
      - "7005:7005"
      - "17005:17005"
    environment:
      - TZ=Asia/Shanghai
  redis7006:
    image: 'redis'
    container_name: redis7006
    command:
      ["redis-server", "/usr/local/etc/redis/redis.conf"]
    volumes:
      - ./redis-cluster/7006/conf/redis.conf:/usr/local/etc/redis/redis.conf
      - ./redis-cluster/7006/data:/data
    ports:
      - "7006:7006"
      - "17006:17006"
    environment:
      - TZ=Asia/Shanghai

up.sh文件：

[root@VM_0_6_centos cluster1]# cat up.sh 
docker-compose -f docker-compose-redis-cluster.yml up -d

执行up.sh，启动集群

[root@VM_0_6_centos cluster1]# ./up.sh 
Creating network "cluster1_default" with the default driver
Creating redis7002 ... done
Creating redis7003 ... 
Creating redis7006 ... 
Creating redis7002 ... 
Creating redis7005 ... 
Creating redis7004 ...

执行结果：

[root@VM_0_6_centos cluster1]# docker ps -a
CONTAINER ID        IMAGE                        COMMAND                  CREATED             STATUS                       PORTS                                                                                                                     NAMES
c30c7ff84360        redis                        "docker-entrypoint.s…"   4 seconds ago       Up 2 seconds                 0.0.0.0:7002->7002/tcp, 6379/tcp, 0.0.0.0:17002->17002/tcp                                                                redis7002
7bb15a15c68f        redis                        "docker-entrypoint.s…"   4 seconds ago       Up 2 seconds                 0.0.0.0:7004->7004/tcp, 6379/tcp, 0.0.0.0:17004->17004/tcp                                                                redis7004
68b7d1209034        redis                        "docker-entrypoint.s…"   4 seconds ago       Up 3 seconds                 0.0.0.0:7006->7006/tcp, 6379/tcp, 0.0.0.0:17006->17006/tcp                                                                redis7006
17096f501e76        redis                        "docker-entrypoint.s…"   4 seconds ago       Up 2 seconds                 0.0.0.0:7005->7005/tcp, 6379/tcp, 0.0.0.0:17005->17005/tcp                                                                redis7005
de6d7ee843ab        redis                        "docker-entrypoint.s…"   4 seconds ago       Up 3 seconds                 0.0.0.0:7001->7001/tcp, 6379/tcp, 0.0.0.0:17001->17001/tcp                                                                redis7001
45551baee2e9        redis                        "docker-entrypoint.s…"   4 seconds ago       Up 3 seconds                 0.0.0.0:7003->7003/tcp, 6379/tcp, 0.0.0.0:17003->17003/tcp                                                                redis7003

下面进行集群配置，配置文件build-cluster.sh

docker exec -it redis7001 redis-cli -p 7001 -a 123456 --cluster create 172.17.0.6:7001 172.17.0.6:7002 172.17.0.6:7003 172.17.0.6:7004 172.17.0.6:7005 172.17.0.6:7006 --cluster-replicas 1

执行结果：

[root@VM_0_6_centos cluster1]# ./build-cluster.sh 
Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
>>> Performing hash slots allocation on 6 nodes...
Master[0] -> Slots 0 - 5460
Master[1] -> Slots 5461 - 10922
Master[2] -> Slots 10923 - 16383
Adding replica 172.17.0.6:7005 to 172.17.0.6:7001
Adding replica 172.17.0.6:7006 to 172.17.0.6:7002
Adding replica 172.17.0.6:7004 to 172.17.0.6:7003
>>> Trying to optimize slaves allocation for anti-affinity
[WARNING] Some slaves are in the same host as their master
M: 4931ee71903cf1e5f284fdcc7207687c1896043c 172.17.0.6:7001
   slots:[0-5460] (5461 slots) master
M: 992226de65c5eaa36fd6d2c8845903f9ff8fac33 172.17.0.6:7002
   slots:[5461-10922] (5462 slots) master
M: a957432b7333a28b51fe6475963841d2e420bf78 172.17.0.6:7003
   slots:[10923-16383] (5461 slots) master
S: 1a7c29136a0d3a42e79eab44d7cd435963a25325 172.17.0.6:7004
   replicates 992226de65c5eaa36fd6d2c8845903f9ff8fac33
S: dd4cebc1ccde3907b07234e868c617d07f9c4002 172.17.0.6:7005
   replicates a957432b7333a28b51fe6475963841d2e420bf78
S: e5adfdd62fa00d8aef4ec5361d7179dba1d12d66 172.17.0.6:7006
   replicates 4931ee71903cf1e5f284fdcc7207687c1896043c
Can I set the above configuration? (type 'yes' to accept): yes
>>> Nodes configuration updated
>>> Assign a different config epoch to each node
>>> Sending CLUSTER MEET messages to join the cluster
Waiting for the cluster to join
...
>>> Performing Cluster Check (using node 172.17.0.6:7001)
M: 4931ee71903cf1e5f284fdcc7207687c1896043c 172.17.0.6:7001
   slots:[0-5460] (5461 slots) master
   1 additional replica(s)
M: a957432b7333a28b51fe6475963841d2e420bf78 172.17.0.6:7003
   slots:[10923-16383] (5461 slots) master
   1 additional replica(s)
S: e5adfdd62fa00d8aef4ec5361d7179dba1d12d66 172.17.0.6:7006
   slots: (0 slots) slave
   replicates 4931ee71903cf1e5f284fdcc7207687c1896043c
M: 992226de65c5eaa36fd6d2c8845903f9ff8fac33 172.17.0.6:7002
   slots:[5461-10922] (5462 slots) master
   1 additional replica(s)
S: dd4cebc1ccde3907b07234e868c617d07f9c4002 172.17.0.6:7005
   slots: (0 slots) slave
   replicates a957432b7333a28b51fe6475963841d2e420bf78
S: 1a7c29136a0d3a42e79eab44d7cd435963a25325 172.17.0.6:7004
   slots: (0 slots) slave
   replicates 992226de65c5eaa36fd6d2c8845903f9ff8fac33
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
[root@VM_0_6_centos cluster1]#

2、测试集群

2.1 测试集群通信是否正常

redis7001主节点对它的副本节点redis7005进行ping操作。

[root@VM_0_6_centos cluster1]# docker exec -it redis7001 redis-cli -h 172.17.0.6 -p 7005 -a 123456 ping
Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
PONG

2.2、测试简单存储

redis7001主节点客户端操作redis7003主节点。

[root@VM_0_6_centos cluster1]# docker exec -it redis7001 redis-cli -h 192.168.124.5 -p 7003 -a 123456
Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
^C[root@VM_0_6_centos cluster1]# docker exec -it redis7001 redis-cli -h 172.17.0.6 -p 7003 -a 123456
Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
172.17.0.6:7003>  set name admin
(error) MOVED 5798 172.17.0.6:7002
172.17.0.6:7003> exit

由于Redis Cluster会根据key进行hash运算，然后将key分散到不同slots，name的hash运算结果在redis7002节点上的slots中。所以我们操作redis7003写操作会自动路由到7002。然而error提示无法路由？没关系，差一个 -c 参数而已。

再次运行查看结果如下:

[root@VM_0_6_centos cluster1]# docker exec -it redis7001 redis-cli -h 172.17.0.6 -p 7003 -a 123456 -c
Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
172.17.0.6:7003> set name admin
-> Redirected to slot [5798] located at 172.17.0.6:7002
OK
172.17.0.6:7002> get name
"admin"
172.17.0.6:7002>

2.3、查看集群节点

172.17.0.6:7002> cluster nodes
dd4cebc1ccde3907b07234e868c617d07f9c4002 172.17.0.6:7005@17005 slave a957432b7333a28b51fe6475963841d2e420bf78 0 1622556724904 5 connected
4931ee71903cf1e5f284fdcc7207687c1896043c 172.17.0.6:7001@17001 master - 0 1622556724000 1 connected 0-5460
a957432b7333a28b51fe6475963841d2e420bf78 172.17.0.6:7003@17003 master - 0 1622556723502 3 connected 10923-16383
e5adfdd62fa00d8aef4ec5361d7179dba1d12d66 172.17.0.6:7006@17006 slave 4931ee71903cf1e5f284fdcc7207687c1896043c 0 1622556724102 6 connected
992226de65c5eaa36fd6d2c8845903f9ff8fac33 172.17.0.6:7002@17002 myself,master - 0 1622556723000 2 connected 5461-10922
1a7c29136a0d3a42e79eab44d7cd435963a25325 172.17.0.6:7004@17004 slave 992226de65c5eaa36fd6d2c8845903f9ff8fac33 0 1622556724504 4 connected
172.17.0.6:7002>

2.4、查看slots分片

172.17.0.6:7002> cluster slots
1) 1) (integer) 0
   2) (integer) 5460
   3) 1) "172.17.0.6"
      2) (integer) 7001
      3) "4931ee71903cf1e5f284fdcc7207687c1896043c"
   4) 1) "172.17.0.6"
      2) (integer) 7006
      3) "e5adfdd62fa00d8aef4ec5361d7179dba1d12d66"
2) 1) (integer) 10923
   2) (integer) 16383
   3) 1) "172.17.0.6"
      2) (integer) 7003
      3) "a957432b7333a28b51fe6475963841d2e420bf78"
   4) 1) "172.17.0.6"
      2) (integer) 7005
      3) "dd4cebc1ccde3907b07234e868c617d07f9c4002"
3) 1) (integer) 5461
   2) (integer) 10922
   3) 1) "172.17.0.6"
      2) (integer) 7002
      3) "992226de65c5eaa36fd6d2c8845903f9ff8fac33"
   4) 1) "172.17.0.6"
      2) (integer) 7004
      3) "1a7c29136a0d3a42e79eab44d7cd435963a25325"
172.17.0.6:7002>

2.5、查看集群信息

172.17.0.6:7002> cluster info
cluster_state:ok
cluster_slots_assigned:16384
cluster_slots_ok:16384
cluster_slots_pfail:0
cluster_slots_fail:0
cluster_known_nodes:6
cluster_size:3
cluster_current_epoch:6
cluster_my_epoch:2
cluster_stats_messages_ping_sent:6112
cluster_stats_messages_pong_sent:6044
cluster_stats_messages_meet_sent:4
cluster_stats_messages_sent:12160
cluster_stats_messages_ping_received:6042
cluster_stats_messages_pong_received:6116
cluster_stats_messages_meet_received:2
cluster_stats_messages_received:12160
172.17.0.6:7002>

2.6、压测

选项	描述
-t	指定命令
-c	客户端连接数
-n	总请求数
-d	set、get的value大小(单位byte)

测试如下：

[root@VM_0_6_centos cluster1]# docker exec -it redis7001 /bin/bash
root@de6d7ee843ab:/data# redis-benchmark -h 172.17.0.6 -p 7001 -t set -c 100 -n 50000 -d 20
ERROR: NOAUTH Authentication required.
ERROR: failed to fetch CONFIG from 172.17.0.6:7001
WARN: could not fetch server CONFIG
====== SET ======
  50000 requests completed in 1.71 seconds
  100 parallel clients
  20 bytes payload
  keep alive: 1
  multi-thread: no

0.00% <= 1.0 milliseconds
0.01% <= 1.1 milliseconds
0.05% <= 1.2 milliseconds
0.10% <= 1.3 milliseconds
0.19% <= 1.4 milliseconds
0.55% <= 1.5 milliseconds
2.30% <= 1.6 milliseconds
8.71% <= 1.7 milliseconds
17.95% <= 1.8 milliseconds
26.94% <= 1.9 milliseconds
35.17% <= 2 milliseconds
84.14% <= 3 milliseconds
96.12% <= 4 milliseconds
99.09% <= 5 milliseconds
99.69% <= 6 milliseconds
99.90% <= 7 milliseconds
99.95% <= 8 milliseconds
99.97% <= 10 milliseconds
99.98% <= 11 milliseconds
100.00% <= 17 milliseconds
100.00% <= 17 milliseconds
29256.88 requests per second


root@de6d7ee843ab:/data#

2.7、容灾测试

杀掉主节点redis7001，看从节点redis7005是否会接替它的位置。

[root@VM_0_6_centos cluster1]# docker exec -it redis7005 redis-cli -h 172.17.0.6 -p 7005 -a 123456
Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
172.17.0.6:7005> cluster nodes
a957432b7333a28b51fe6475963841d2e420bf78 172.17.0.6:7003@17003 master - 0 1622557497000 3 connected 10923-16383
992226de65c5eaa36fd6d2c8845903f9ff8fac33 172.17.0.6:7002@17002 master - 0 1622557498719 2 connected 5461-10922
1a7c29136a0d3a42e79eab44d7cd435963a25325 172.17.0.6:7004@17004 slave 992226de65c5eaa36fd6d2c8845903f9ff8fac33 0 1622557498217 4 connected
4931ee71903cf1e5f284fdcc7207687c1896043c 172.17.0.6:7001@17001 master,fail - 1622557419076 1622557417975 1 disconnected 0-5460
e5adfdd62fa00d8aef4ec5361d7179dba1d12d66 172.17.0.6:7006@17006 slave 4931ee71903cf1e5f284fdcc7207687c1896043c 0 1622557497213 6 connected
dd4cebc1ccde3907b07234e868c617d07f9c4002 172.17.0.6:7005@17005 myself,slave a957432b7333a28b51fe6475963841d2e420bf78 0 1622557497000 5 connected
172.17.0.6:7005> cluster nodes
a957432b7333a28b51fe6475963841d2e420bf78 172.17.0.6:7003@17003 master - 0 1622557584529 3 connected 10923-16383
992226de65c5eaa36fd6d2c8845903f9ff8fac33 172.17.0.6:7002@17002 master - 0 1622557583524 2 connected 5461-10922
1a7c29136a0d3a42e79eab44d7cd435963a25325 172.17.0.6:7004@17004 slave 992226de65c5eaa36fd6d2c8845903f9ff8fac33 0 1622557583524 4 connected
4931ee71903cf1e5f284fdcc7207687c1896043c 172.17.0.6:7001@17001 master,fail - 1622557419076 1622557417975 1 disconnected 0-5460
e5adfdd62fa00d8aef4ec5361d7179dba1d12d66 172.17.0.6:7006@17006 slave 4931ee71903cf1e5f284fdcc7207687c1896043c 0 1622557583000 6 connected
dd4cebc1ccde3907b07234e868c617d07f9c4002 172.17.0.6:7005@17005 myself,slave a957432b7333a28b51fe6475963841d2e420bf78 0 1622557583000 5 connected
172.17.0.6:7005> exit
[root@VM_0_6_centos cluster1]# docker start de6d7ee843ab
de6d7ee843ab
[root@VM_0_6_centos cluster1]# docker exec -it redis7005 redis-cli -h 172.17.0.6 -p 7005 -a 123456
Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
172.17.0.6:7005> cluster nodes
a957432b7333a28b51fe6475963841d2e420bf78 172.17.0.6:7003@17003 master - 0 1622557622652 3 connected 10923-16383
992226de65c5eaa36fd6d2c8845903f9ff8fac33 172.17.0.6:7002@17002 master - 0 1622557621000 2 connected 5461-10922
1a7c29136a0d3a42e79eab44d7cd435963a25325 172.17.0.6:7004@17004 slave 992226de65c5eaa36fd6d2c8845903f9ff8fac33 0 1622557622551 4 connected
4931ee71903cf1e5f284fdcc7207687c1896043c 172.17.0.6:7001@17001 master - 0 1622557621000 1 connected 0-5460
e5adfdd62fa00d8aef4ec5361d7179dba1d12d66 172.17.0.6:7006@17006 slave 4931ee71903cf1e5f284fdcc7207687c1896043c 0 1622557622000 6 connected
dd4cebc1ccde3907b07234e868c617d07f9c4002 172.17.0.6:7005@17005 myself,slave a957432b7333a28b51fe6475963841d2e420bf78 0 1622557621000 5 connected
172.17.0.6:7005>

实际测试过程中，7005并没有成为主节点。原因是什么呢？待探究？

查看从节点日志可以发现：从节点连接主节点失败，检查配置文件发现未设置masterauth，从而导致无法连接。

1:S 16 Jun 2021 15:39:53.278 * Connecting to MASTER 172.17.0.6:7001
1:S 16 Jun 2021 15:39:53.278 * MASTER <-> REPLICA sync started
1:S 16 Jun 2021 15:39:53.278 * Non blocking connect for SYNC fired the event.
1:S 16 Jun 2021 15:39:53.278 * Master replied to PING, replication can continue...
1:S 16 Jun 2021 15:39:53.278 * (Non critical) Master does not understand REPLCONF listening-port: -NOAUTH Authentication required.
1:S 16 Jun 2021 15:39:53.279 * (Non critical) Master does not understand REPLCONF capa: -NOAUTH Authentication required.
1:S 16 Jun 2021 15:39:53.279 * Partial resynchronization not possible (no cached master)
1:S 16 Jun 2021 15:39:53.279 # Unexpected reply to PSYNC from master: -NOAUTH Authentication required.
1:S 16 Jun 2021 15:39:53.279 * Retrying with SYNC...
1:S 16 Jun 2021 15:39:53.279 # MASTER aborted replication with an error: NOAUTH Authentication required.

修改配置文件后（上面集群配置文件中masterauth为发现错误后增加），重新启动测试。

[root@VM_0_6_centos ~]# docker stop 35b71161ad74
35b71161ad74

查看redis7005节点日志可以发现，日志新增了几条信息：

[root@VM_0_6_centos cluster1]# docker logs -f 2cb4107f3849
1:S 16 Jun 2021 15:49:40.509 * FAIL message received from bbb8871b895d55d9db7d4b7960abce8f3710d307 about 2a57d563dfb9d1f3091fb09f1d3393f706ec7f81
1:S 16 Jun 2021 15:49:40.509 # Cluster state changed: fail
1:S 16 Jun 2021 15:49:41.294 # Cluster state changed: ok

这时在查看集群信息，会发现已经重新选举7006为主节点。

root@57be8ef8c5a9:/data# redis-cli -p 7002 -a 123456
Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
127.0.0.1:7002> get name
(nil)
127.0.0.1:7002> set name admin
OK
127.0.0.1:7002> get nage
(error) MOVED 14701 172.17.0.6:7003
127.0.0.1:7002> get name
"admin"
127.0.0.1:7002> cluster nodes
2db63b6ea82c538005d5cdb7e77cc13c15418066 172.17.0.6:7006@17006 master - 0 1623829803090 7 connected 0-5460
bbef93cba563619e2939d51cf7efe47f582eb103 172.17.0.6:7003@17003 master - 0 1623829802086 3 connected 10923-16383
bbb8871b895d55d9db7d4b7960abce8f3710d307 172.17.0.6:7002@17002 myself,master - 0 1623829801000 2 connected 5461-10922
2a57d563dfb9d1f3091fb09f1d3393f706ec7f81 172.17.0.6:7001@17001 master,fail - 1623829775380 1623829774077 1 disconnected
d9724cee129f31240cdd1c1e1f9bbd4c6314d441 172.17.0.6:7005@17005 slave bbef93cba563619e2939d51cf7efe47f582eb103 0 1623829803000 5 connected
02ded0630fe3d3b365e36b4b4cf14e1fa75611e8 172.17.0.6:7004@17004 slave bbb8871b895d55d9db7d4b7960abce8f3710d307 0 1623829804095 4 connected

至此，完成测试。将7001节点重新启动，可以发现集群信息有了新的变化。

root@57be8ef8c5a9:/data# redis-cli -p 7002 -a 123456
Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
127.0.0.1:7002> get name
(nil)
127.0.0.1:7002> set name admin
OK
127.0.0.1:7002> get nage
(error) MOVED 14701 172.17.0.6:7003
127.0.0.1:7002> get name
"admin"
127.0.0.1:7002> cluster nodes
2db63b6ea82c538005d5cdb7e77cc13c15418066 172.17.0.6:7006@17006 master - 0 1623829803090 7 connected 0-5460
bbef93cba563619e2939d51cf7efe47f582eb103 172.17.0.6:7003@17003 master - 0 1623829802086 3 connected 10923-16383
bbb8871b895d55d9db7d4b7960abce8f3710d307 172.17.0.6:7002@17002 myself,master - 0 1623829801000 2 connected 5461-10922
2a57d563dfb9d1f3091fb09f1d3393f706ec7f81 172.17.0.6:7001@17001 master,fail - 1623829775380 1623829774077 1 disconnected
d9724cee129f31240cdd1c1e1f9bbd4c6314d441 172.17.0.6:7005@17005 slave bbef93cba563619e2939d51cf7efe47f582eb103 0 1623829803000 5 connected
02ded0630fe3d3b365e36b4b4cf14e1fa75611e8 172.17.0.6:7004@17004 slave bbb8871b895d55d9db7d4b7960abce8f3710d307 0 1623829804095 4 connected
127.0.0.1:7002> cluster nodes
2db63b6ea82c538005d5cdb7e77cc13c15418066 172.17.0.6:7006@17006 master - 0 1623829886864 7 connected 0-5460
bbef93cba563619e2939d51cf7efe47f582eb103 172.17.0.6:7003@17003 master - 0 1623829886000 3 connected 10923-16383
bbb8871b895d55d9db7d4b7960abce8f3710d307 172.17.0.6:7002@17002 myself,master - 0 1623829884000 2 connected 5461-10922
2a57d563dfb9d1f3091fb09f1d3393f706ec7f81 172.17.0.6:7001@17001 slave 2db63b6ea82c538005d5cdb7e77cc13c15418066 0 1623829885561 7 connected
d9724cee129f31240cdd1c1e1f9bbd4c6314d441 172.17.0.6:7005@17005 slave bbef93cba563619e2939d51cf7efe47f582eb103 0 1623829886000 5 connected
02ded0630fe3d3b365e36b4b4cf14e1fa75611e8 172.17.0.6:7004@17004 slave bbb8871b895d55d9db7d4b7960abce8f3710d307 0 1623829886363 4 connected
127.0.0.1:7002>

2.8、测试完成后，看配置文件

[root@VM_0_6_centos cluster1]# tree
.
|-- build-cluster.sh
|-- build.sh
|-- docker-compose-redis-cluster.yml
|-- down.sh
|-- redis-cluster
|   |-- 7001
|   |   |-- conf
|   |   |   `-- redis.conf
|   |   `-- data
|   |       |-- appendonly.aof
|   |       `-- nodes.conf
|   |-- 7002
|   |   |-- conf
|   |   |   `-- redis.conf
|   |   `-- data
|   |       |-- appendonly.aof
|   |       `-- nodes.conf
|   |-- 7003
|   |   |-- conf
|   |   |   `-- redis.conf
|   |   `-- data
|   |       |-- appendonly.aof
|   |       `-- nodes.conf
|   |-- 7004
|   |   |-- conf
|   |   |   `-- redis.conf
|   |   `-- data
|   |       |-- appendonly.aof
|   |       `-- nodes.conf
|   |-- 7005
|   |   |-- conf
|   |   |   `-- redis.conf
|   |   `-- data
|   |       |-- appendonly.aof
|   |       `-- nodes.conf
|   `-- 7006
|       |-- conf
|       |   `-- redis.conf
|       `-- data
|           |-- appendonly.aof
|           `-- nodes.conf
|-- redis-cluster.tmpl
`-- up.sh

19 directories, 24 files

二、Redis一主二从三哨兵服务

主从复制：主节点负责写数据，从节点负责读数据，主节点定期把数据同步到从节点保证数据的一致性。从性能方面，redis复制功能增加了读的性能，理论上说，每增加一倍的服务器，整个系统的读能力就增加一倍。

服务器端口分布

	IP	端口
master	172.17.0.6	6379
salve1	172.17.0.6	6380
salve2	172.17.0.6	6381
sentinel1	172.17.0.6	26379
sentinel2	172.17.0.6	26380
sentinel3	172.17.0.6	26381

1、一主二从

首先构建目录结构，并编写配置文件。

[root@VM_0_6_centos cluster2]# pwd
/opt/redis/cluster2
[root@VM_0_6_centos cluster2]# tree
.
|-- dokcer-compose-master-slave.yml
|-- master-slave-down.sh
|-- master-slave-up.sh
`-- redis-cluster
    |-- master
    |   |-- conf
    |   |   `-- redis.conf
    |   `-- data
    |-- slave1
    |   |-- conf
    |   |   `-- redis.conf
    |   `-- data
    `-- slave2
        |-- conf
        |   `-- redis.conf
        `-- data

10 directories, 13 files

依据以上结构构建目录后编写行管配置文件。

dokcer-compose-master-slave.yml

version: '3.3'
services:
  master:
    image: 'redis'
    container_name: redis-master
    command:
      ["redis-server", "/usr/local/etc/redis/redis.conf"]
    volumes:
      - ./redis-cluster/master/conf/redis.conf:/usr/local/etc/redis/redis.conf
      - ./redis-cluster/master/data:/data
    ports:
      - "6379:6379"
    environment:
      - TZ=Asia/Shanghai
  slave1:
    image: 'redis'
    container_name: redis-slave-1
    command:
      ["redis-server", "/usr/local/etc/redis/redis.conf"]
    volumes:
      - ./redis-cluster/slave1/conf/redis.conf:/usr/local/etc/redis/redis.conf
      - ./redis-cluster/slave1/data:/data
    ports:
      - "6380:6380"
    environment:
      - TZ=Asia/Shanghai
  slave2:
    image: 'redis'
    container_name: redis-slave-2
    command:
      ["redis-server", "/usr/local/etc/redis/redis.conf"]
    volumes:
      - ./redis-cluster/slave2/conf/redis.conf:/usr/local/etc/redis/redis.conf
      - ./redis-cluster/slave2/data:/data
    ports:
      - "6381:6381"
    environment:
      - TZ=Asia/Shanghai

master-slave-up.sh

[root@VM_0_6_centos cluster2]# cat master-slave-up.sh
docker-compose -f dokcer-compose-master-slave.yml up -d

下面是redis的几个配置文件

master的配置文件redis.conf

# redis端口
port 6379
# 关闭保护模式
protected-mode no
# 开启集群
cluster-enabled yes
# 集群节点配置
cluster-config-file nodes.conf
# 超时
cluster-node-timeout 5000
# 集群节点IP host模式为宿主机IP
cluster-announce-ip 172.17.0.6
# 集群节点端口 7001 - 7006
cluster-announce-port 6379
cluster-announce-bus-port 16379
# RDB
# #表示900 秒内如果至少有 1 个 key 的值变化，则保存
save 900 1
# #表示300 秒内如果至少有 10 个 key 的值变化，则保存
save 300 10
# #表示60 秒内如果至少有 10000 个 key 的值变化，则保存
save 60 10000
# #启用AOF
# 开启 appendonly 备份模式
appendonly yes
# 每秒钟备份
appendfsync everysec
# 对aof文件进行压缩时，是否执行同步操作
no-appendfsync-on-rewrite no
# 当目前aof文件大小超过上一次重写时的aof文件大小的100%时会再次进行重写
auto-aof-rewrite-percentage 100
# 重写前AOF文件的大小最小值 默认 64mb
auto-aof-rewrite-min-size 64mb
#设置redis密码 各个节点请保持密码一致
requirepass 123456
masterauth 123456
#监听
bind 0.0.0.0

slave1的配置文件redis.conf

# redis端口
port 6380
# 关闭保护模式
protected-mode no
# RDB
# #表示900 秒内如果至少有 1 个 key 的值变化，则保存
save 900 1
# #表示300 秒内如果至少有 10 个 key 的值变化，则保存
save 300 10
# #表示60 秒内如果至少有 10000 个 key 的值变化，则保存
save 60 10000
# #启用AOF
# 开启 appendonly 备份模式
appendonly yes
# 每秒钟备份
appendfsync everysec
# 对aof文件进行压缩时，是否执行同步操作
no-appendfsync-on-rewrite no
# 当目前aof文件大小超过上一次重写时的aof文件大小的100%时会再次进行重写
auto-aof-rewrite-percentage 100
# 重写前AOF文件的大小最小值 默认 64mb
auto-aof-rewrite-min-size 64mb
#设置redis密码 各个节点请保持密码一致
requirepass 123456
masterauth 123456
slaveof 172.17.0.6 6379
#因为使用docker部署的redis服务默认上报的元数据信息是docker容器内部ip与port，这里显示声明物理机的ip与port
slave-announce-ip 172.17.0.6
slave-announce-port 6380
#监听
bind 0.0.0.0

slave2的配置文件redis.conf

# redis端口
port 6381
# 关闭保护模式
protected-mode no
# RDB
# #表示900 秒内如果至少有 1 个 key 的值变化，则保存
save 900 1
# #表示300 秒内如果至少有 10 个 key 的值变化，则保存
save 300 10
# #表示60 秒内如果至少有 10000 个 key 的值变化，则保存
save 60 10000
# #启用AOF
# 开启 appendonly 备份模式
appendonly yes
# 每秒钟备份
appendfsync everysec
# 对aof文件进行压缩时，是否执行同步操作
no-appendfsync-on-rewrite no
# 当目前aof文件大小超过上一次重写时的aof文件大小的100%时会再次进行重写
auto-aof-rewrite-percentage 100
# 重写前AOF文件的大小最小值 默认 64mb
auto-aof-rewrite-min-size 64mb
#设置redis密码 各个节点请保持密码一致
requirepass 123456
masterauth 123456
slaveof 172.17.0.6 6379
#因为使用docker部署的redis服务默认上报的元数据信息是docker容器内部ip与port，这里显示声明物理机的ip与port
slave-announce-ip 172.17.0.6
slave-announce-port 6381
#监听
bind 0.0.0.0

其次，执行docker-compose文件。

[root@VM_0_6_centos cluster2]# ./master-slave-up.sh
Creating network "cluster2_default" with the default driver
Creating redis-master ... done
Creating redis-slave-2 ...
Creating redis-master ...                                                                                                                           mongoMaster
[root@VM_0_6_centos cluster2]# docker ps
CONTAINER ID        IMAGE                 COMMAND                  CREATED             STATUS              PORTS                                                                                                                     NAMES
1d1f94513c02        redis                 "docker-entrypoint.s…"   16 seconds ago      Up 15 seconds       0.0.0.0:6379->6379/tcp                                                                                                    redis-master
cb643c9f9fe0        redis                 "docker-entrypoint.s…"   16 seconds ago      Up 15 seconds       0.0.0.0:6381->6379/tcp                                                                                                    redis-slave-2
eb79d0c58b76        redis                 "docker-entrypoint.s…"   16 seconds ago      Up 15 seconds                                                                                                       redisStandalone

最后，进入主节点进行测试，查看信息（以下信息为部分信息）。

[root@VM_0_6_centos cluster2]# docker exec -it 1d1f94513c02 /bin/bash
root@1d1f94513c02:/data# redis-cli
127.0.0.1:6379> auth 123456
OK
127.0.0.1:6379> cluster info
cluster_state:fail
cluster_slots_assigned:0
cluster_slots_ok:0
cluster_slots_pfail:0
cluster_slots_fail:0
cluster_known_nodes:1
cluster_size:0
cluster_current_epoch:0
cluster_my_epoch:0
cluster_stats_messages_sent:0
cluster_stats_messages_received:0
127.0.0.1:6379> info
# Server
redis_version:6.0.3
redis_git_sha1:00000000
redis_git_dirty:0
redis_build_id:839141755a9d16cb
redis_mode:cluster
os:Linux 3.10.0-862.el7.x86_64 x86_64
arch_bits:64
multiplexing_api:epoll
atomicvar_api:atomic-builtin
gcc_version:8.3.0
process_id:1
run_id:72c5cccaa67b7fc5b9d8681f8381b5b246df984e
tcp_port:6379
uptime_in_seconds:1056
uptime_in_days:0
hz:10
configured_hz:10
lru_clock:12179924
executable:/data/redis-server
config_file:/usr/local/etc/redis/redis.conf

# Clients
connected_clients:1
client_recent_max_input_buffer:2
client_recent_max_output_buffer:0
blocked_clients:0
tracking_clients:0
clients_in_timeout_table:0

# Replication
role:master
connected_slaves:2
slave0:ip=172.17.0.6,port=6380,state=online,offset=1470,lag=1
slave1:ip=172.17.0.6,port=6381,state=online,offset=1470,lag=1
master_replid:ede7cd9279ac44ec837b46ee025f164b297a06e2
master_replid2:0000000000000000000000000000000000000000
master_repl_offset:1470
master_repl_meaningful_offset:0
second_repl_offset:-1
repl_backlog_active:1
repl_backlog_size:1048576
repl_backlog_first_byte_offset:1
repl_backlog_histlen:1470


# Modules

# Cluster
cluster_enabled:1

# Keyspace
127.0.0.1:6379>

可以从Replication部分看到两个从节点的信息。

2、三哨兵模式构建

构建目录结构。

[root@VM_0_6_centos cluster2]# tree
.
|-- dokcer-compose-master-slave.yml
|-- dokcer-compose-sentinel.yml
|-- master-slave-down.sh
|-- master-slave-up.sh
|-- redis-cluster
|   |-- master
|   |   |-- conf
|   |   |   `-- redis.conf
|   |   `-- data
|   |       |-- appendonly.aof
|   |       |-- dump.rdb
|   |       `-- nodes.conf
|   |-- sentinel1
|   |   |-- conf
|   |   |   `-- sentinel.conf
|   |-- sentinel2
|   |   |-- conf
|   |   |   `-- sentinel.conf
|   |-- sentinel3
|   |   |-- conf
|   |   |   `-- sentinel.conf
|   |-- slave1
|   |   |-- conf
|   |   |   `-- redis.conf
|   |   `-- data
|   |       |-- appendonly.aof
|   |       `-- dump.rdb
|   `-- slave2
|       |-- conf
|       |   `-- redis.conf
|       `-- data
|           |-- appendonly.aof
|           `-- dump.rdb
|-- sentinel-down.sh
`-- sentinel-up.sh

19 directories, 28 files

依据以上结构构建目录后编写行管配置文件。

dokcer-compose-sentinel.yml

version: '3.3'
services:
  sentinel1:
    image: 'redis'
    container_name: redis-sentinel-1
    command:
      ["redis-sentinel", "/usr/local/etc/redis/sentinel.conf"]
    volumes:
      - ./redis-cluster/sentinel1/conf/sentinel.conf:/usr/local/etc/redis/sentinel.conf
    ports:
      - "26379:26379"
    environment:
      - TZ=Asia/Shanghai
  sentinel2:
    image: 'redis'
    container_name: redis-sentinel-2
    command:
      ["redis-sentinel", "/usr/local/etc/redis/sentinel.conf"]
    volumes:
      - ./redis-cluster/sentinel2/conf/sentinel.conf:/usr/local/etc/redis/sentinel.conf
    ports:
      - "26380:26380"
    environment:
      - TZ=Asia/Shanghai
  sentinel3:
    image: 'redis'
    container_name: redis-sentinel-3
    command:
      ["redis-sentinel", "/usr/local/etc/redis/sentinel.conf"]
    volumes:
      - ./redis-cluster/sentinel3/conf/sentinel.conf:/usr/local/etc/redis/sentinel.conf
    ports:
      - "26381:26381"
    environment:
      - TZ=Asia/Shanghai

sentinel-up.sh

[root@VM_0_6_centos cluster2]# cat sentinel-up.sh
docker-compose -f dokcer-compose-sentinel.yml up -d

下面则是三个sentinel配置文件sentinel.conf

其中Redis 5.0.5 哨兵（Sentinel）配置文件说明如下：

# Example sentinel.conf

# *** IMPORTANT ***
# 绑定IP地址
# bind 127.0.0.1 192.168.1.1
# 保护模式（是否禁止外部链接，除绑定的ip地址外）
# protected-mode no

# port <sentinel-port>
# 此Sentinel实例运行的端口
port 26379

# 默认情况下，Redis Sentinel不作为守护程序运行。 如果需要，可以设置为 yes。
daemonize no

# 启用守护进程运行后，Redis将在/var/run/redis-sentinel.pid中写入一个pid文件
pidfile /var/run/redis-sentinel.pid

# 指定日志文件名。 如果值为空，将强制Sentinel日志标准输出。守护进程下，如果使用标准输出进行日志记录，则日志将发送到/dev/null
logfile ""

# sentinel announce-ip <ip>
# sentinel announce-port <port>
#
# 上述两个配置指令在环境中非常有用，因为NAT可以通过非本地地址从外部访问Sentinel。
#
# 当提供announce-ip时，Sentinel将在通信中声明指定的IP地址，而不是像通常那样自动检测本地地址。
#
# 类似地，当提供announce-port 有效且非零时，Sentinel将宣布指定的TCP端口。
#
# 这两个选项不需要一起使用，如果只提供announce-ip，Sentinel将宣告指定的IP和“port”选项指定的服务器端口。
# 如果仅提供announce-port，Sentinel将通告自动检测到的本地IP和指定端口。
#
# Example:
#
# sentinel announce-ip 1.2.3.4

# dir <working-directory>
# 每个长时间运行的进程都应该有一个明确定义的工作目录。对于Redis Sentinel来说，/tmp就是自己的工作目录。
dir /tmp

# sentinel monitor <master-name> <ip> <redis-port> <quorum>
#
# 告诉Sentinel监听指定主节点，并且只有在至少<quorum>哨兵达成一致的情况下才会判断它 O_DOWN 状态。
#
#
# 副本是自动发现的，因此您无需指定副本。
# Sentinel本身将重写此配置文件，使用其他配置选项添加副本。另请注意，当副本升级为主副本时，将重写配置文件。
#
# 注意：主节点（master）名称不能包含特殊字符或空格。
# 有效字符可以是 A-z 0-9 和这三个字符 ".-_".
sentinel monitor mymaster 127.0.0.1 6379 2

# 如果redis配置了密码，那这里必须配置认证，否则不能自动切换
# Example:
#
# sentinel auth-pass mymaster MySUPER--secret-0123passw0rd

# sentinel down-after-milliseconds <master-name> <milliseconds>
#
# 主节点或副本在指定时间内没有回复PING，便认为该节点为主观下线 S_DOWN 状态。
#
# 默认是30秒
sentinel down-after-milliseconds mymaster 30000

# sentinel parallel-syncs <master-name> <numreplicas>
#
# 在故障转移期间，多少个副本节点进行数据同步
sentinel parallel-syncs mymaster 1

# sentinel failover-timeout <master-name> <milliseconds>
#
# 指定故障转移超时（以毫秒为单位）。 它以多种方式使用：
#
# - 在先前的故障转移之后重新启动故障转移所需的时间已由给定的Sentinel针对同一主服务器尝试，是故障转移超时的两倍。
#
# - 当一个slave从一个错误的master那里同步数据开始计算时间。直到slave被纠正为向正确的master那里同步数据时。
#
# - 取消已在进行但未生成任何配置更改的故障转移所需的时间
#
# - 当进行failover时，配置所有slaves指向新的master所需的最大时间。
#   即使过了这个超时，slaves依然会被正确配置为指向master。
#
# 默认3分钟
sentinel failover-timeout mymaster 180000

# 脚本执行
#
# sentinel notification-script和sentinel reconfig-script用于配置调用的脚本，以通知系统管理员或在故障转移后重新配置客户端。
# 脚本使用以下规则执行以进行错误处理：
#
# 如果脚本以“1”退出，则稍后重试执行（最多重试次数为当前设置的10次）。
#
# 如果脚本以“2”（或更高的值）退出，则不会重试执行。
#
# 如果脚本因为收到信号而终止，则行为与退出代码1相同。
#
# 脚本的最长运行时间为60秒。 达到此限制后，脚本将以SIGKILL终止，并重试执行。

# 通知脚本
#
# sentinel notification-script <master-name> <script-path>
#
# 为警告级别生成的任何Sentinel事件调用指定的通知脚本（例如-sdown，-odown等）。
# 此脚本应通过电子邮件，SMS或任何其他消息传递系统通知系统管理员 监控的Redis系统出了问题。
#
# 使用两个参数调用脚本：第一个是事件类型，第二个是事件描述。
#
# 该脚本必须存在且可执行，以便在提供此选项时启动sentinel。
#
# 举例:
#
# sentinel notification-script mymaster /var/redis/notify.sh

# 客户重新配置脚本
#
# sentinel client-reconfig-script <master-name> <script-path>
#
# 当主服务器因故障转移而变更时，可以调用脚本执行特定于应用程序的任务，以通知客户端，配置已更改且主服务器地址已经变更。
#
# 以下参数将传递给脚本：
#
# <master-name> <role> <state> <from-ip> <from-port> <to-ip> <to-port>
#
# <state> 目前始终是故障转移 "failover"
# <role> 是 "leader" 或 "observer"
#
# 参数 from-ip, from-port, to-ip, to-port 用于传递主服务器的旧地址和所选副本的新地址。
#
# 举例:
#
# sentinel client-reconfig-script mymaster /var/redis/reconfig.sh

# 安全
# 避免脚本重置，默认值yes
# 默认情况下，SENTINEL SET将无法在运行时更改notification-script和client-reconfig-script。
# 这避免了一个简单的安全问题，客户端可以将脚本设置为任何内容并触发故障转移以便执行程序。
sentinel deny-scripts-reconfig yes

# REDIS命令重命名
#
#
# 在这种情况下，可以告诉Sentinel使用不同的命令名称而不是正常的命令名称。
# 例如，如果主“mymaster”和相关副本的“CONFIG”全部重命名为“GUESSME”，我可以使用：
#
# SENTINEL rename-command mymaster CONFIG GUESSME
#
# 设置此类配置后，每次Sentinel使用CONFIG时，它将使用GUESSME。 请注意，实际上不需要尊重命令案例，因此在上面的示例中写“config guessme”是相同的。
#
# SENTINEL SET也可用于在运行时执行此配置。
#
# 为了将命令设置回其原始名称（撤消重命名），可以将命令重命名为它自身：
#
# SENTINEL rename-command mymaster CONFIG CONFIG

sentinel1.conf

port 26379
dir /tmp
#监听
bind 0.0.0.0 
# 自定义集群名，其中 172.17.0.6 为 redis-master 的 ip，6379 为 redis-master 的端口，2 为最小投票数（因为有 3 台 Sentinel 所以可以设置成 2）
sentinel monitor mymaster 172.17.0.6 6379 2
sentinel auth-pass mymaster 123456 
sentinel down-after-milliseconds mymaster 30000
sentinel parallel-syncs mymaster 1
sentinel failover-timeout mymaster 10000  
sentinel deny-scripts-reconfig yes
#同样需要声明物理机的ip与port
sentinel announce-ip 172.17.0.6
sentinel announce-port 26379

sentinel2.conf

port 26380
dir /tmp
#监听
bind 0.0.0.0 
# 自定义集群名，其中 172.17.0.6 为 redis-master 的 ip，6379 为 redis-master 的端口，2 为最小投票数（因为有 3 台 Sentinel 所以可以设置成 2）
sentinel monitor mymaster 172.17.0.6 6379 2
sentinel auth-pass mymaster 123456 
sentinel down-after-milliseconds mymaster 30000
sentinel parallel-syncs mymaster 1
sentinel failover-timeout mymaster 10000  
sentinel deny-scripts-reconfig yes
#同样需要声明物理机的ip与port
sentinel announce-ip 172.17.0.6
sentinel announce-port 26380

sentinel3.conf

port 26381
dir /tmp
#监听
bind 0.0.0.0 
# 自定义集群名，其中 172.17.0.6 为 redis-master 的 ip，6379 为 redis-master 的端口，2 为最小投票数（因为有 3 台 Sentinel 所以可以设置成 2）
sentinel monitor mymaster 172.17.0.6 6379 2
sentinel auth-pass mymaster 123456 
sentinel down-after-milliseconds mymaster 30000
sentinel parallel-syncs mymaster 1
sentinel failover-timeout mymaster 10000  
sentinel deny-scripts-reconfig yes
#同样需要声明物理机的ip与port
sentinel announce-ip 172.17.0.6
sentinel announce-port 26381

注意要加#监听 bind 0.0.0.0

配置文件编写好后，执行docker-compose-up.sh

[root@VM_0_6_centos cluster2]# ./sentinel-up.sh
WARNING: Found orphan containers (redis-master, redis-slave-2, redis-slave-1) for this project. If you removed or renamed this service in your compose file, you can run this command with the --remove-orphans flag to clean it up.
Creating redis-sentinel-1 ... done
Creating redis-sentinel-1 ...
Creating redis-sentinel-3 ...
[root@VM_0_6_centos cluster2]# docker ps
CONTAINER ID        IMAGE                 COMMAND                  CREATED             STATUS              PORTS                                                                                                                     NAMES
a25b66cd015d        redis                 "docker-entrypoint.s…"   8 seconds ago       Up 6 seconds        6379/tcp, 0.0.0.0:26379->26379/tcp                                                                                        redis-sentinel-1
eeda8cb157bd        redis                 "docker-entrypoint.s…"   8 seconds ago       Up 6 seconds        6379/tcp, 0.0.0.0:26381->26379/tcp                                                                                        redis-sentinel-3
356db0b2b118        redis                 "docker-entrypoint.s…"   8 seconds ago       Up 7 seconds        6379/tcp, 0.0.0.0:26380->26379/tcp                                                                                        redis-sentinel-2
1d1f94513c02        redis                 "docker-entrypoint.s…"   50 minutes ago      Up 50 minutes       0.0.0.0:6379->6379/tcp                                                                                                    redis-master
cb643c9f9fe0        redis                 "docker-entrypoint.s…"   50 minutes ago      Up 50 minutes       0.0.0.0:6381->6379/tcp                                                                                                    redis-slave-2
eb79d0c58b76        redis                 "docker-entrypoint.s…"   50 minutes ago      Up 50 minutes       0.0.0.0:6380->6379/tcp                                                                                                    redis-slave-1

启动后，查看其中一个sentinel的日志

[root@VM_0_6_centos cluster2]# docker logs -f 7b889021a667
1:X 05 Jun 2021 11:08:35.715 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
1:X 05 Jun 2021 11:08:35.715 # Redis version=6.0.3, bits=64, commit=00000000, modified=0, pid=1, just started
1:X 05 Jun 2021 11:08:35.715 # Configuration loaded
1:X 05 Jun 2021 11:15:21.811 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
1:X 05 Jun 2021 11:15:21.811 # Redis version=6.0.3, bits=64, commit=00000000, modified=0, pid=1, just started
1:X 05 Jun 2021 11:15:21.811 # Configuration loaded
1:X 05 Jun 2021 11:15:21.812 * Running mode=sentinel, port=26379.
1:X 05 Jun 2021 11:15:21.813 # WARNING: The TCP backlog setting of 511 cannot be enforced because /proc/sys/net/core/somaxconn is set to the lower value of 128.
1:X 05 Jun 2021 11:15:21.820 # Sentinel ID is 3a6f08199a6ea5b90f2b99598a10fd18f3a43d25
1:X 05 Jun 2021 11:15:21.820 # +monitor master mymaster 172.17.0.6 6379 quorum 2
1:X 05 Jun 2021 11:15:21.823 * +slave slave 172.17.0.6:6381 172.17.0.6 6381 @ mymaster 172.17.0.6 6379
1:X 05 Jun 2021 11:15:21.829 * +slave slave 172.17.0.6:6380 172.17.0.6 6380 @ mymaster 172.17.0.6 6379
1:X 05 Jun 2021 11:15:23.775 * +sentinel sentinel b5bc2cacad0def241e4151ea3c0c01e9731cd971 172.17.0.6 26380 @ mymaster 172.17.0.6 6379
1:X 05 Jun 2021 11:15:23.899 * +sentinel sentinel 6c7c9ad59679528368cb8b810265d913f9e54113 172.17.0.6 26381 @ mymaster 172.17.0.6 6379
1:X 05 Jun 2021 11:22:00.429 # +sdown master mymaster 172.17.0.6 6379
1:X 05 Jun 2021 11:22:00.482 # +odown master mymaster 172.17.0.6 6379 #quorum 2/2
1:X 05 Jun 2021 11:22:00.482 # +new-epoch 1
1:X 05 Jun 2021 11:22:00.482 # +try-failover master mymaster 172.17.0.6 6379
1:X 05 Jun 2021 11:22:00.486 # +vote-for-leader 3a6f08199a6ea5b90f2b99598a10fd18f3a43d25 1
1:X 05 Jun 2021 11:22:00.494 # 6c7c9ad59679528368cb8b810265d913f9e54113 voted for 3a6f08199a6ea5b90f2b99598a10fd18f3a43d25 1
1:X 05 Jun 2021 11:22:00.494 # b5bc2cacad0def241e4151ea3c0c01e9731cd971 voted for 3a6f08199a6ea5b90f2b99598a10fd18f3a43d25 1

进入其中一个sentinel容器内部，查看信息（部分信息）

[root@VM_0_6_centos cluster2]# docker exec -it a25b66cd015d /bin/bash
root@a25b66cd015d:/data# redis-cli -p 26379
127.0.0.1:26379> info


# Sentinel
sentinel_masters:1
sentinel_tilt:0
sentinel_running_scripts:0
sentinel_scripts_queue_length:0
sentinel_simulate_failure_flags:0
master0:name=mymaster,status=ok,address=172.17.0.6:6379,slaves=2,sentinels=3
127.0.0.1:26379>

3、验证故障转移

哨兵可以自动转移故障，也就是当主节点宕机的时候，它们能通过选举制度，在从节点中选出一个节点来代替主节点。

停掉redis-master主节点。

[root@VM_0_6_centos conf]# docker stop bd60c614e0f4
bd60c614e0f4

查看其中一个sentinel的日志。

[root@VM_0_6_centos cluster2]# docker ps
CONTAINER ID        IMAGE                 COMMAND                  CREATED             STATUS              PORTS                                                                                                                     NAMES
39dcda126d59        redis                 "docker-entrypoint.s…"   6 minutes ago       Up 2 seconds        6379/tcp, 0.0.0.0:26381->26381/tcp                                                                                        redis-sentinel-3
a5738cc48f83        redis                 "docker-entrypoint.s…"   6 minutes ago       Up 2 seconds        6379/tcp, 0.0.0.0:26380->26380/tcp                                                                                        redis-sentinel-2
7b889021a667        redis                 "docker-entrypoint.s…"   6 minutes ago       Up 2 seconds        6379/tcp, 0.0.0.0:26379->26379/tcp                                                                                        redis-sentinel-1
4c7b5339a723        redis                 "docker-entrypoint.s…"   23 minutes ago      Up 23 minutes       6379/tcp, 0.0.0.0:6380->6380/tcp                                                                                          redis-slave-1
56a491c73bbe        redis                 "docker-entrypoint.s…"   23 minutes ago      Up 23 minutes       6379/tcp, 0.0.0.0:6381->6381/tcp                                                                                          redis-slave-2
bd60c614e0f4        redis                 "docker-entrypoint.s…"   23 minutes ago      Up 23 minutes       0.0.0.0:6379->6379/tcp                                                                                                    redis-master

[root@VM_0_6_centos cluster2]# docker logs -f 7b889021a667
1:X 05 Jun 2021 11:08:35.715 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
1:X 05 Jun 2021 11:08:35.715 # Redis version=6.0.3, bits=64, commit=00000000, modified=0, pid=1, just started
1:X 05 Jun 2021 11:08:35.715 # Configuration loaded
1:X 05 Jun 2021 11:15:21.811 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
1:X 05 Jun 2021 11:15:21.811 # Redis version=6.0.3, bits=64, commit=00000000, modified=0, pid=1, just started
1:X 05 Jun 2021 11:15:21.811 # Configuration loaded
1:X 05 Jun 2021 11:15:21.812 * Running mode=sentinel, port=26379.
1:X 05 Jun 2021 11:15:21.813 # WARNING: The TCP backlog setting of 511 cannot be enforced because /proc/sys/net/core/somaxconn is set to the lower value of 128.
1:X 05 Jun 2021 11:15:21.820 # Sentinel ID is 3a6f08199a6ea5b90f2b99598a10fd18f3a43d25
1:X 05 Jun 2021 11:15:21.820 # +monitor master mymaster 172.17.0.6 6379 quorum 2
1:X 05 Jun 2021 11:15:21.823 * +slave slave 172.17.0.6:6381 172.17.0.6 6381 @ mymaster 172.17.0.6 6379
1:X 05 Jun 2021 11:15:21.829 * +slave slave 172.17.0.6:6380 172.17.0.6 6380 @ mymaster 172.17.0.6 6379
1:X 05 Jun 2021 11:15:23.775 * +sentinel sentinel b5bc2cacad0def241e4151ea3c0c01e9731cd971 172.17.0.6 26380 @ mymaster 172.17.0.6 6379
1:X 05 Jun 2021 11:15:23.899 * +sentinel sentinel 6c7c9ad59679528368cb8b810265d913f9e54113 172.17.0.6 26381 @ mymaster 172.17.0.6 6379
1:X 05 Jun 2021 11:22:00.429 # +sdown master mymaster 172.17.0.6 6379
1:X 05 Jun 2021 11:22:00.482 # +odown master mymaster 172.17.0.6 6379 #quorum 2/2
1:X 05 Jun 2021 11:22:00.482 # +new-epoch 1
1:X 05 Jun 2021 11:22:00.482 # +try-failover master mymaster 172.17.0.6 6379
1:X 05 Jun 2021 11:22:00.486 # +vote-for-leader 3a6f08199a6ea5b90f2b99598a10fd18f3a43d25 1
1:X 05 Jun 2021 11:22:00.494 # 6c7c9ad59679528368cb8b810265d913f9e54113 voted for 3a6f08199a6ea5b90f2b99598a10fd18f3a43d25 1
1:X 05 Jun 2021 11:22:00.494 # b5bc2cacad0def241e4151ea3c0c01e9731cd971 voted for 3a6f08199a6ea5b90f2b99598a10fd18f3a43d25 1
1:X 05 Jun 2021 11:22:00.545 # +elected-leader master mymaster 172.17.0.6 6379
1:X 05 Jun 2021 11:22:00.545 # +failover-state-select-slave master mymaster 172.17.0.6 6379
1:X 05 Jun 2021 11:22:00.612 # +selected-slave slave 172.17.0.6:6380 172.17.0.6 6380 @ mymaster 172.17.0.6 6379
1:X 05 Jun 2021 11:22:00.613 * +failover-state-send-slaveof-noone slave 172.17.0.6:6380 172.17.0.6 6380 @ mymaster 172.17.0.6 6379
1:X 05 Jun 2021 11:22:00.665 * +failover-state-wait-promotion slave 172.17.0.6:6380 172.17.0.6 6380 @ mymaster 172.17.0.6 6379
1:X 05 Jun 2021 11:22:01.379 # +promoted-slave slave 172.17.0.6:6380 172.17.0.6 6380 @ mymaster 172.17.0.6 6379
1:X 05 Jun 2021 11:22:01.379 # +failover-state-reconf-slaves master mymaster 172.17.0.6 6379
1:X 05 Jun 2021 11:22:01.449 * +slave-reconf-sent slave 172.17.0.6:6381 172.17.0.6 6381 @ mymaster 172.17.0.6 6379
1:X 05 Jun 2021 11:22:01.604 # -odown master mymaster 172.17.0.6 6379
1:X 05 Jun 2021 11:22:02.402 * +slave-reconf-inprog slave 172.17.0.6:6381 172.17.0.6 6381 @ mymaster 172.17.0.6 6379
1:X 05 Jun 2021 11:22:02.402 * +slave-reconf-done slave 172.17.0.6:6381 172.17.0.6 6381 @ mymaster 172.17.0.6 6379
1:X 05 Jun 2021 11:22:02.478 # +failover-end master mymaster 172.17.0.6 6379
1:X 05 Jun 2021 11:22:02.478 # +switch-master mymaster 172.17.0.6 6379 172.17.0.6 6380
1:X 05 Jun 2021 11:22:02.478 * +slave slave 172.17.0.6:6381 172.17.0.6 6381 @ mymaster 172.17.0.6 6380
1:X 05 Jun 2021 11:22:02.478 * +slave slave 172.17.0.6:6379 172.17.0.6 6379 @ mymaster 172.17.0.6 6380
1:X 05 Jun 2021 11:22:32.520 # +sdown slave 172.17.0.6:6379 172.17.0.6 6379 @ mymaster 172.17.0.6 6380

进入其中一个从节点slave1，查看主从信息

[root@VM_0_6_centos conf]# docker exec -it 56a491c73bbe /bin/bash
root@56a491c73bbe:/data# redis -p 6381
bash: redis: command not found
root@56a491c73bbe:/data# redis-cli -p 6381
127.0.0.1:6381> info repolication
NOAUTH Authentication required.
127.0.0.1:6381> auth 123456
OK
127.0.0.1:6381> info replication
# Replication
role:slave
master_host:172.17.0.6
master_port:6380
master_link_status:up
master_last_io_seconds_ago:1
master_sync_in_progress:0
slave_repl_offset:123187
slave_priority:100
slave_read_only:1
connected_slaves:0
master_replid:bb6a63c254b286377e26d9af4b5a0d25954f35ad
master_replid2:b864ef7b06cba67d32ba47e979c23075fbdc5582
master_repl_offset:123187
master_repl_meaningful_offset:123187
second_repl_offset:1
repl_backlog_active:1
repl_backlog_size:1048576
repl_backlog_first_byte_offset:1
repl_backlog_histlen:123187
127.0.0.1:6381>

可以看到主节点已经换成6380，实现故障转移。

问题

1、ERR AUTH called without any password configured for the default user. Are you sure your configuration is correct?

[root@VM_0_6_centos ~]# docker exec -it redis7001 redis-cli -p 7001 -a 123456 --cluster create 172.17.0.6:7001 172.17.0.6:7002 172.17.0.6:7003 172.17.0.6:7004 172.17.0.6:7005 172.17.0.6:7006 --cluster-replicas 1
Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
Node 172.17.0.6:7001 replied with error:
ERR AUTH <password> called without any password configured for the default user. Are you sure your configuration is correct?

排查：

选择单节点进行测试。

[root@VM_0_6_centos conf]# docker exec -it ba0728d45348 /bin/bash
root@ba0728d45348:/data# redis-cli -p 6379 -c
127.0.0.1:6379>
127.0.0.1:6379>
127.0.0.1:6379> exit
root@ba0728d45348:/data# exit
exit

如果如果能连接则正常。

解决：

配置文件redis-cluster.tmpl在该版本中应该改为

# redis端口
port ${PORT}
# 关闭保护模式
protected-mode no
# 开启集群
cluster-enabled yes
# 集群节点配置
cluster-config-file nodes.conf
# 超时
cluster-node-timeout 5000
# 集群节点IP host模式为宿主机IP
cluster-announce-ip 172.17.0.6
# 集群节点端口 7001 - 7006
cluster-announce-port ${PORT}
cluster-announce-bus-port 1${PORT}
# 开启 appendonly 备份模式
appendonly yes
# 每秒钟备份
appendfsync everysec
# 对aof文件进行压缩时，是否执行同步操作
no-appendfsync-on-rewrite no
# 当目前aof文件大小超过上一次重写时的aof文件大小的100%时会再次进行重写
auto-aof-rewrite-percentage 100
# 重写前AOF文件的大小最小值 默认 64mb
auto-aof-rewrite-min-size 64mb
# 此处绑定ip 可以是阿里内网ip 和 本地ip 也可以直接注释掉该项
#bind 172.17.0.6
#设置redis密码 各个节点请保持密码一致
requirepass 123456

注意：

增加requirepass YOURPASSWORD ，同时应该注释掉bind

具体原因：https://blog.csdn.net/Ha_hha/article/details/79993929

修改后正确集群配置：

[root@VM_0_6_centos cluster1]# docker exec -it redis7001 redis-cli -p 7001 -a 123456 --cluster create 172.17.0.6:7001 172.17.0.6:7002 172.17.0.6:7003 172.17.0.6:7004 172.17.0.6:7005 172.17.0.6:7006 --cluster-replicas 1
Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
>>> Performing hash slots allocation on 6 nodes...
Master[0] -> Slots 0 - 5460
Master[1] -> Slots 5461 - 10922
Master[2] -> Slots 10923 - 16383
Adding replica 172.17.0.6:7005 to 172.17.0.6:7001
Adding replica 172.17.0.6:7006 to 172.17.0.6:7002
Adding replica 172.17.0.6:7004 to 172.17.0.6:7003
>>> Trying to optimize slaves allocation for anti-affinity
[WARNING] Some slaves are in the same host as their master
M: 4931ee71903cf1e5f284fdcc7207687c1896043c 172.17.0.6:7001
   slots:[0-5460] (5461 slots) master
M: 992226de65c5eaa36fd6d2c8845903f9ff8fac33 172.17.0.6:7002
   slots:[5461-10922] (5462 slots) master
M: a957432b7333a28b51fe6475963841d2e420bf78 172.17.0.6:7003
   slots:[10923-16383] (5461 slots) master
S: 1a7c29136a0d3a42e79eab44d7cd435963a25325 172.17.0.6:7004
   replicates 992226de65c5eaa36fd6d2c8845903f9ff8fac33
S: dd4cebc1ccde3907b07234e868c617d07f9c4002 172.17.0.6:7005
   replicates a957432b7333a28b51fe6475963841d2e420bf78
S: e5adfdd62fa00d8aef4ec5361d7179dba1d12d66 172.17.0.6:7006
   replicates 4931ee71903cf1e5f284fdcc7207687c1896043c
Can I set the above configuration? (type 'yes' to accept): yes
>>> Nodes configuration updated
>>> Assign a different config epoch to each node
>>> Sending CLUSTER MEET messages to join the cluster
Waiting for the cluster to join
...
>>> Performing Cluster Check (using node 172.17.0.6:7001)
M: 4931ee71903cf1e5f284fdcc7207687c1896043c 172.17.0.6:7001
   slots:[0-5460] (5461 slots) master
   1 additional replica(s)
M: a957432b7333a28b51fe6475963841d2e420bf78 172.17.0.6:7003
   slots:[10923-16383] (5461 slots) master
   1 additional replica(s)
S: e5adfdd62fa00d8aef4ec5361d7179dba1d12d66 172.17.0.6:7006
   slots: (0 slots) slave
   replicates 4931ee71903cf1e5f284fdcc7207687c1896043c
M: 992226de65c5eaa36fd6d2c8845903f9ff8fac33 172.17.0.6:7002
   slots:[5461-10922] (5462 slots) master
   1 additional replica(s)
S: dd4cebc1ccde3907b07234e868c617d07f9c4002 172.17.0.6:7005
   slots: (0 slots) slave
   replicates a957432b7333a28b51fe6475963841d2e420bf78
S: 1a7c29136a0d3a42e79eab44d7cd435963a25325 172.17.0.6:7004
   slots: (0 slots) slave
   replicates 992226de65c5eaa36fd6d2c8845903f9ff8fac33
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
[root@VM_0_6_centos cluster1]#

2、Error: Connection reset by peer

问题：

root@a0c440beb1d3:/data# redis-cli -h 172.17.0.6 -p 7003 -a 123456
Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
172.17.0.6:7003> set name admin
Error: Connection reset by peer
root@a0c440beb1d3:/data# ^C

解决方案：

https://blog.csdn.net/u014182411/article/details/72637975/

3、replicaof directive not allowed in cluster mode

[root@VM_0_6_centos cluster2]# docker logs -f 2cbe34bd6a9a

*** FATAL CONFIG FILE ERROR (Redis 6.0.3) ***
Reading the configuration file, at line 32
>>> 'slaveof 172.17.0.6 6379'
replicaof directive not allowed in cluster mode

解决：将 cluster-enabled yes 设置为 no

4、sentinel监控无法发生故障时使master与slave完成主从切换。

Next failover delay: I will not start a failover before Wed Dec 28 17:52:19 2016

解决：

在sentinel.conf配置文件中增加bind 0.0.0.0即可。

5、redis-cluster 异常：MASTER aborted replication with an error: NOAUTH Authentication required.

1:S 16 Jun 2021 15:39:53.278 * Connecting to MASTER 172.17.0.6:7001
1:S 16 Jun 2021 15:39:53.278 * MASTER <-> REPLICA sync started
1:S 16 Jun 2021 15:39:53.278 * Non blocking connect for SYNC fired the event.
1:S 16 Jun 2021 15:39:53.278 * Master replied to PING, replication can continue...
1:S 16 Jun 2021 15:39:53.278 * (Non critical) Master does not understand REPLCONF listening-port: -NOAUTH Authentication required.
1:S 16 Jun 2021 15:39:53.279 * (Non critical) Master does not understand REPLCONF capa: -NOAUTH Authentication required.
1:S 16 Jun 2021 15:39:53.279 * Partial resynchronization not possible (no cached master)
1:S 16 Jun 2021 15:39:53.279 # Unexpected reply to PSYNC from master: -NOAUTH Authentication required.
1:S 16 Jun 2021 15:39:53.279 * Retrying with SYNC...
1:S 16 Jun 2021 15:39:53.279 # MASTER aborted replication with an error: NOAUTH Authentication required.

解决：

主库master要求密码验证，在从库的配置文件中添加master的密码配置即可，其中123456是你设置连接master的密码：

masterauth 123456

补充说明：

推荐使用AOF+RDB混合模式。编辑属性