armbian docker Chrome_利用docker轻松搭建RedisCluster集群环境

简介

Redis Cluster是Redis官方的一个高可用分布式解决方案，其优点是高可用，缺点是不能保证数据强一致。在这里使用docker容器来搭建一套6节点(3主，3从)Redis-Cluster集群环境。

环境准备

操作系统版本：CentOS Linux release 7.6.1810 (Core)
docker版本：19.03.13
服务器IP地址：192.168.112.136
redis版本：5.0.9
redis实例端口：8001--8006

docker安装

首先需要在CentOS系统上安装好docker软件，目前docker软件分为社区版ce和企业版ee，企业版是需要收费的，所以在这里使用社区版，要安装docker软件，CentOS系统的内核版本高于 3.10。

下面是docker安装步骤

yum install -y yum-utils device-mapper-persistent-data lvm2yum-config-manager --add-repo https://download.docker.com/linux/centos/docker-ce.repoyum install docker-ce

启动并加入开机启动

systemctl start dockersystemctl enable docker

由于docker默认是从国外拉取镜像，速度会比较慢，所以需要配置docker从国内的镜像网站拉取镜像，其配置步骤如下所示

将国内镜像网站添加到daemon.json配置文件

>/etc/docker/daemon.json{    "registry-mirrors": [        "https://registry.docker-cn.com",        "https://docker.mirrors.ustc.edu.cn",        "https://reg-mirror.qiniu.com",        "https://mirror.ccs.tencentyun.com"    ]}

添加完成之后，需要重启docker服务器，使配置生效

systemctl daemon-reloadsystemctl restart docker

docker服务器重启之后，使用docker info命令，检查配置是否生效

redis镜像拉取

在docker-hub官网，有很多版本的redis镜像可供选择，在这里，选择redis:5.0.9-buster进行拉取

[root@mysql ~]# docker pull redis:5.0.9-buster[root@mysql ~]# docker imagesREPOSITORY          TAG                 IMAGE ID            CREATED             SIZEredis               5.0.9-buster        c440306287f4        7 days ago          98.3MB

创建redis目录和配置文件

由于这里需要配置6个redis实例的配置文件，所以在这里，可以先创建一个模板配置文件，之后使用代码批量替换的方式来创建每个redis的配置文件。

创建模板文件

创建目录

mkdir -p /data/docker_redis/编写redis_cluster.conf.template文件vi redis_cluster.conf.templateport ${PORT}requirepass 1234masterauth 1234protected-mode nodaemonize noappendonly yescluster-enabled yescluster-config-file nodes.confcluster-node-timeout 5000cluster-announce-ip 192.168.112.136cluster-announce-port ${PORT}cluster-announce-bus-port 1${PORT}

在这里需要讲解2个参数
cluster-announce-port：此端口为redis提供服务端口，用于应用客户端连接
cluster-announce-bus-port：此端口用于redis集群进行故障检测、配置更新、故障转移授权和内部通讯使用，不用于应用客户端连接使用。

创建6个redis实例的数据目录，配置文件目录和配置文件

for port in `seq 8001 8006`; do \  mkdir -p /data/redis_data/${port}/conf \  && PORT=${port} envsubst < /data/docker_redis/redis_cluster.conf.template > /data/redis_data/${port}/conf/redis.conf \  && mkdir -p /data/redis_data/${port}/data;\done

使用tree命令检查创建的目录和文件

[root@mysql data]$ tree redis_data/redis_data/|-- 8001|   |-- conf|   |   `-- redis.conf|   `-- data|-- 8002|   |-- conf|   |   `-- redis.conf|   `-- data|-- 8003|   |-- conf|   |   `-- redis.conf|   `-- data|-- 8004|   |-- conf|   |   `-- redis.conf|   `-- data|-- 8005|   |-- conf|   |   `-- redis.conf|   `-- data`-- 8006    |-- conf    |   `-- redis.conf    `-- data18 directories, 6 files

批量创建容器

在这里使用for循环批量创建redis容器，代码如下所示

for port in $(seq 8001 8006); do \  docker run -d -it -p ${port}:${port} -p 1${port}:1${port} --restart always --name redis-${port} \  -v /data/redis_data/${port}/conf/redis.conf:/usr/local/etc/redis/redis.conf \  -v /data/redis_data/${port}/data:/data \  redis:5.0.9-buster redis-server /usr/local/etc/redis/redis.conf; \done

在这里使用-v参数，让容器的redis配置参数和数据全部使用宿主机的文件目录。这样做的好处是，保证redis的数据不丢失。

查看刚刚创建的redis容器

[root@mysql data]# docker psCONTAINER ID        IMAGE                COMMAND                  CREATED             STATUS              PORTS                                                        NAMESf4c971ce2d84        redis:5.0.9-buster   "docker-entrypoint.s鈥 6 seconds ago       Up 5 seconds        0.0.0.0:8006->8006/tcp, 6379/tcp, 0.0.0.0:18006->18006/tcp   redis-80061b5a8b0a1c93        redis:5.0.9-buster   "docker-entrypoint.s鈥 7 seconds ago       Up 6 seconds        0.0.0.0:8005->8005/tcp, 6379/tcp, 0.0.0.0:18005->18005/tcp   redis-800535d8fe01fc71        redis:5.0.9-buster   "docker-entrypoint.s鈥 7 seconds ago       Up 6 seconds        0.0.0.0:8004->8004/tcp, 6379/tcp, 0.0.0.0:18004->18004/tcp   redis-8004408e5302378a        redis:5.0.9-buster   "docker-entrypoint.s鈥 8 seconds ago       Up 7 seconds        0.0.0.0:8003->8003/tcp, 6379/tcp, 0.0.0.0:18003->18003/tcp   redis-8003c1c23c543233        redis:5.0.9-buster   "docker-entrypoint.s鈥 8 seconds ago       Up 7 seconds        0.0.0.0:8002->8002/tcp, 6379/tcp, 0.0.0.0:18002->18002/tcp   redis-8002c752fab6c6b8        redis:5.0.9-buster   "docker-entrypoint.s鈥 9 seconds ago       Up 8 seconds        0.0.0.0:8001->8001/tcp, 6379/tcp, 0.0.0.0:18001->18001/tcp   redis-8001

创建redis-cluster集群

6个redis容器创建好之后，选择其中的一个容器进入，进行redis-cluster集群创建

[root@mysql bin]# docker-enter f4c971ce2d84root@f4c971ce2d84:~# redis-cli -a 1234 --cluster create 192.168.112.136:8001 192.168.112.136:8002 192.168.112.136:8003 192.168.112.136:8004 192.168.112.136:8005 192.168.112.136:8006 --cluster-replicas 1Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.>>> Performing hash slots allocation on 6 nodes...Master[0] -> Slots 0 - 5460Master[1] -> Slots 5461 - 10922Master[2] -> Slots 10923 - 16383Adding replica 192.168.112.136:8005 to 192.168.112.136:8001Adding replica 192.168.112.136:8006 to 192.168.112.136:8002Adding replica 192.168.112.136:8004 to 192.168.112.136:8003>>> Trying to optimize slaves allocation for anti-affinity[WARNING] Some slaves are in the same host as their masterM: c9e549f4d04d4466d2550d1e027412304099b1f2 192.168.112.136:8001   slots:[0-5460] (5461 slots) masterM: 0956927ee74737d5ff91a1885e77f94d531dab76 192.168.112.136:8002   slots:[5461-10922] (5462 slots) masterM: d6321788b2717b4142390c158ba70ca758f70964 192.168.112.136:8003   slots:[10923-16383] (5461 slots) masterS: 96d2cb55f163ecc13a714ba01d90348c1c3ad02f 192.168.112.136:8004   replicates c9e549f4d04d4466d2550d1e027412304099b1f2S: 8a771c26a95e82d9a6818e372d7c0226937670ac 192.168.112.136:8005   replicates 0956927ee74737d5ff91a1885e77f94d531dab76S: aa7c2f6173904973f041b35efc5200359188eb0f 192.168.112.136:8006   replicates d6321788b2717b4142390c158ba70ca758f70964Can I set the above configuration? (type 'yes' to accept): yes>>> Nodes configuration updated>>> Assign a different config epoch to each node>>> Sending CLUSTER MEET messages to join the clusterWaiting for the cluster to join.>>> Performing Cluster Check (using node 192.168.112.136:8001)M: c9e549f4d04d4466d2550d1e027412304099b1f2 192.168.112.136:8001   slots:[0-5460] (5461 slots) master   1 additional replica(s)M: 0956927ee74737d5ff91a1885e77f94d531dab76 192.168.112.136:8002   slots:[5461-10922] (5462 slots) master   1 additional replica(s)S: 96d2cb55f163ecc13a714ba01d90348c1c3ad02f 192.168.112.136:8004   slots: (0 slots) slave   replicates c9e549f4d04d4466d2550d1e027412304099b1f2S: aa7c2f6173904973f041b35efc5200359188eb0f 192.168.112.136:8006   slots: (0 slots) slave   replicates d6321788b2717b4142390c158ba70ca758f70964S: 8a771c26a95e82d9a6818e372d7c0226937670ac 192.168.112.136:8005   slots: (0 slots) slave   replicates 0956927ee74737d5ff91a1885e77f94d531dab76M: d6321788b2717b4142390c158ba70ca758f70964 192.168.112.136:8003   slots:[10923-16383] (5461 slots) master   1 additional replica(s)[OK] All nodes agree about slots configuration.>>> Check for open slots...>>> Check slots coverage...[OK] All 16384 slots covered.

上述命令执行之后，redis-cluster集群就搭建好了，下面是整个redis-cluster集群的架构图

redis-cluster集群状态检查

检查redis-cluster集群状态

root@f4c971ce2d84:~# redis-cli -a 1234 --cluster check 192.168.112.136:8001Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.192.168.112.136:8001 (c9e549f4...) -> 0 keys | 5461 slots | 1 slaves.192.168.112.136:8002 (0956927e...) -> 0 keys | 5462 slots | 1 slaves.192.168.112.136:8003 (d6321788...) -> 0 keys | 5461 slots | 1 slaves.[OK] 0 keys in 3 masters.0.00 keys per slot on average.>>> Performing Cluster Check (using node 192.168.112.136:8001)M: c9e549f4d04d4466d2550d1e027412304099b1f2 192.168.112.136:8001   slots:[0-5460] (5461 slots) master   1 additional replica(s)M: 0956927ee74737d5ff91a1885e77f94d531dab76 192.168.112.136:8002   slots:[5461-10922] (5462 slots) master   1 additional replica(s)S: 96d2cb55f163ecc13a714ba01d90348c1c3ad02f 192.168.112.136:8004   slots: (0 slots) slave   replicates c9e549f4d04d4466d2550d1e027412304099b1f2S: aa7c2f6173904973f041b35efc5200359188eb0f 192.168.112.136:8006   slots: (0 slots) slave   replicates d6321788b2717b4142390c158ba70ca758f70964S: 8a771c26a95e82d9a6818e372d7c0226937670ac 192.168.112.136:8005   slots: (0 slots) slave   replicates 0956927ee74737d5ff91a1885e77f94d531dab76M: d6321788b2717b4142390c158ba70ca758f70964 192.168.112.136:8003   slots:[10923-16383] (5461 slots) master   1 additional replica(s)[OK] All nodes agree about slots configuration.>>> Check for open slots...>>> Check slots coverage...[OK] All 16384 slots covered.

检查redis集群节点信息

root@f4c971ce2d84:~# redis-cli -c -a 1234 -h 192.168.112.136 -p 8001192.168.112.136:8001> cluster nodes0956927ee74737d5ff91a1885e77f94d531dab76 192.168.112.136:8002@18002 master - 0 1603265855374 2 connected 5461-1092296d2cb55f163ecc13a714ba01d90348c1c3ad02f 192.168.112.136:8004@18004 slave c9e549f4d04d4466d2550d1e027412304099b1f2 0 1603265856080 4 connectedaa7c2f6173904973f041b35efc5200359188eb0f 192.168.112.136:8006@18006 slave d6321788b2717b4142390c158ba70ca758f70964 0 1603265855575 6 connected8a771c26a95e82d9a6818e372d7c0226937670ac 192.168.112.136:8005@18005 slave 0956927ee74737d5ff91a1885e77f94d531dab76 0 1603265855575 5 connectedd6321788b2717b4142390c158ba70ca758f70964 192.168.112.136:8003@18003 master - 0 1603265856382 3 connected 10923-16383c9e549f4d04d4466d2550d1e027412304099b1f2 192.168.112.136:8001@18001 myself,master - 0 1603265855000 1 connected 0-5460

在这里可以很清晰明了的看到有3个master节点和3个slave节点的IP地址和端口信息。

redis-cluster连接测试

redis-cluster搭建好之后，随意连接一个redis节点，无论是主节点还是从节点都可以，在这里选择连接一个从节点。

192.168.112.136:8001> cluster nodes0956927ee74737d5ff91a1885e77f94d531dab76 192.168.112.136:8002@18002 master - 0 1603265855374 2 connected 5461-1092296d2cb55f163ecc13a714ba01d90348c1c3ad02f 192.168.112.136:8004@18004 slave c9e549f4d04d4466d2550d1e027412304099b1f2 0 1603265856080 4 connectedaa7c2f6173904973f041b35efc5200359188eb0f 192.168.112.136:8006@18006 slave d6321788b2717b4142390c158ba70ca758f70964 0 1603265855575 6 connected8a771c26a95e82d9a6818e372d7c0226937670ac 192.168.112.136:8005@18005 slave 0956927ee74737d5ff91a1885e77f94d531dab76 0 1603265855575 5 connectedd6321788b2717b4142390c158ba70ca758f70964 192.168.112.136:8003@18003 master - 0 1603265856382 3 connected 10923-16383c9e549f4d04d4466d2550d1e027412304099b1f2 192.168.112.136:8001@18001 myself,master - 0 1603265855000 1 connected 0-5460root@f4c971ce2d84:~# redis-cli -c -a 1234 -h 192.168.112.136 -p 8004Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.192.168.112.136:8004> 192.168.112.136:8004> set 5 test5-> Redirected to slot [9974] located at 192.168.112.136:8002192.168.112.136:8002> 

仔细看一下，当在slave节点写入数据时，redis-cluster将你的写请求重定向到另一个master节点了，至于原因，在这里就不说了，下次在讲解redis-cluster集群原理篇再细讲。

测试查询

root@f4c971ce2d84:~# redis-cli -c -a 1234 -h 192.168.112.136 -p 8006Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.192.168.112.136:8006> 192.168.112.136:8006> get 5-> Redirected to slot [9974] located at 192.168.112.136:8002"test5"192.168.112.136:8002>

在这里查询也被路由重定向了。