RQ配置使用Redis Cluster

最新推荐文章于 2024-08-22 17:35:08 发布

llc的足迹

最新推荐文章于 2024-08-22 17:35:08 发布

阅读量1.8k

点赞数 1

分类专栏： python Redis 文章标签： python rq redis redis cluster redis集群

本文链接：https://blog.csdn.net/this_is_id/article/details/123788432

版权

python 同时被 2 个专栏收录

34 篇文章 0 订阅

订阅专栏

Redis

1 篇文章 0 订阅

订阅专栏

一、基于docker-compose搭建redis集群

1.1 集群配置

3主+3从
redis1-redis6

1.2 编写redis.conf

创建redis.conf，填入如下内容，将redis.conf拷贝至./conf/{redis1-redis6}.conf：

# 开启集群
cluster-enabled yes
# 集群配置文件
cluster-config-file nodes.conf
# 集群节点多少时间未响应视为该节点丢失
cluster-node-timeout 5000
# 开启 appendonly 备份模式
appendonly yes
# redis监听端口：端口范围1379-6379
port 1379
# 集群节点端口：端口范围1379-6379，总线端口范围11379-16379
cluster-announce-ip 192.168.1.2
cluster-announce-port 1379
cluster-announce-bus-port 11379

1.3 编写docker-compose文件

version: '3'
services:
  redis1:
    image: redis:6
    container_name: redis-1
    restart: always
    environment:
      # 设置时区为上海
      - TZ=Asia/Shanghai
    ports:
      - "1379:1379"
      - "11379:11379"
    volumes:
      - ./conf/redis1.conf:/conf/redis.conf
    command: [ "redis-server", "/conf/redis.conf" ]
  redis2:
    image: redis:6
    container_name: redis-2
    restart: always
    environment:
      # 设置时区为上海
      - TZ=Asia/Shanghai
    ports:
      - "2379:2379"
      - "12379:12379"
    volumes:
      - ./conf/redis2.conf:/conf/redis.conf
    command: [ "redis-server", "/conf/redis.conf" ]
  redis3:
    image: redis:6
    container_name: redis-3
    restart: always
    environment:
      # 设置时区为上海
      - TZ=Asia/Shanghai
    ports:
      - "3379:3379"
      - "13379:13379"
    volumes:
      - ./conf/redis3.conf:/conf/redis.conf
    command: [ "redis-server", "/conf/redis.conf" ]
  redis4:
    image: redis:6
    container_name: redis-4
    restart: always
    environment:
      # 设置时区为上海
      - TZ=Asia/Shanghai
    ports:
      - "4379:4379"
      - "14379:14379"
    volumes:
      - ./conf/redis4.conf:/conf/redis.conf
    command: [ "redis-server", "/conf/redis.conf" ]
  redis5:
    image: redis:6
    container_name: redis-5
    restart: always
    environment:
      # 设置时区为上海
      - TZ=Asia/Shanghai
    ports:
      - "5379:5379"
      - "15379:15379"
    volumes:
      - ./conf/redis5.conf:/conf/redis.conf
    command: [ "redis-server", "/conf/redis.conf" ]
  redis6:
    image: redis:6
    container_name: redis-6
    restart: always
    environment:
      # 设置时区为上海
      - TZ=Asia/Shanghai
    ports:
      - "6379:6379"
      - "16379:16379"
    volumes:
      - ./conf/redis6.conf:/conf/redis.conf
    command: [ "redis-server", "/conf/redis.conf" ]

networks:
  default:
    external:
      name: redis

1.4 启动容器

最终目录结构：

.
├── conf
│   ├── redis1.conf
│   ├── redis2.conf
│   ├── redis3.conf
│   ├── redis4.conf
│   ├── redis5.conf
│   └── redis6.conf
├── docker-compose.yml

启动容器前需要创建网络
docker network create redis
启动容器
docker-compose up -d

1.5 服务状态

$ docker-compose ps
Name                Command               State                                           Ports                                         
-----------------------------------------------------------------------------------------------------------------------------------------
redis-1   docker-entrypoint.sh redis ...   Up      0.0.0.0:11379->16379/tcp,:::11379->16379/tcp, 0.0.0.0:1379->6379/tcp,:::1379->6379/tcp
redis-2   docker-entrypoint.sh redis ...   Up      0.0.0.0:12379->16379/tcp,:::12379->16379/tcp, 0.0.0.0:2379->6379/tcp,:::2379->6379/tcp
redis-3   docker-entrypoint.sh redis ...   Up      0.0.0.0:13379->16379/tcp,:::13379->16379/tcp, 0.0.0.0:3379->6379/tcp,:::3379->6379/tcp
redis-4   docker-entrypoint.sh redis ...   Up      0.0.0.0:14379->16379/tcp,:::14379->16379/tcp, 0.0.0.0:4379->6379/tcp,:::4379->6379/tcp
redis-5   docker-entrypoint.sh redis ...   Up      0.0.0.0:15379->16379/tcp,:::15379->16379/tcp, 0.0.0.0:5379->6379/tcp,:::5379->6379/tcp
redis-6   docker-entrypoint.sh redis ...   Up      0.0.0.0:16379->16379/tcp,:::16379->16379/tcp, 0.0.0.0:6379->6379/tcp,:::6379->6379/tcp

二、创建集群

2.1 进入一个容器

docker exec -it redis-1 bash

2.2 创建集群

redis-cli --cluster create \
192.168.1.2:1379 \
192.168.1.2:2379 \
192.168.1.2:3379 \
192.168.1.2:4379 \
192.168.1.2:5379 \
192.168.1.2:6379 \
--cluster-replicas 1

输出信息如下：

root@288cf292dd98:/data# redis-cli --cluster create \
>   192.168.1.2:1379 \
>   192.168.1.2:2379 \
>   192.168.1.2:3379 \
>   192.168.1.2:4379 \
>   192.168.1.2:5379 \
>   192.168.1.2:6379 \
>   --cluster-replicas 1
>>> Performing hash slots allocation on 6 nodes...
Master[0] -> Slots 0 - 5460
Master[1] -> Slots 5461 - 10922
Master[2] -> Slots 10923 - 16383
Adding replica 192.168.1.2:5379 to 192.168.1.2:1379
Adding replica 192.168.1.2:6379 to 192.168.1.2:2379
Adding replica 192.168.1.2:4379 to 192.168.1.2:3379
>>> Trying to optimize slaves allocation for anti-affinity
[WARNING] Some slaves are in the same host as their master
M: f9f0cdd20b8e7474c82c5cc3ea4177a2a8fa4dbc 192.168.1.2:1379
   slots:[0-5460] (5461 slots) master
M: 7ce6b7d37f3dc04f8b77e8d9bee37bdaf47b903d 192.168.1.2:2379
   slots:[5461-10922] (5462 slots) master
M: f90ea317783606a08bf70443fdc8a0e7d5c9e483 192.168.1.2:3379
   slots:[10923-16383] (5461 slots) master
S: 975c47d4045e64dada1c2a377dd49435cce53ad1 192.168.1.2:4379
   replicates f90ea317783606a08bf70443fdc8a0e7d5c9e483
S: 24aaaf5b071f5bf089208c7525ed992fdc7a487a 192.168.1.2:5379
   replicates f9f0cdd20b8e7474c82c5cc3ea4177a2a8fa4dbc
S: 096106d92a0d55279898b696d9ea9f81a8f1bbad 192.168.1.2:6379
   replicates 7ce6b7d37f3dc04f8b77e8d9bee37bdaf47b903d
Can I set the above configuration? (type 'yes' to accept): yes
>>> Nodes configuration updated
>>> Assign a different config epoch to each node
>>> Sending CLUSTER MEET messages to join the cluster
Waiting for the cluster to join
.
>>> Performing Cluster Check (using node 192.168.1.2:1379)
M: f9f0cdd20b8e7474c82c5cc3ea4177a2a8fa4dbc 192.168.1.2:1379
   slots:[0-5460] (5461 slots) master
   1 additional replica(s)
S: 24aaaf5b071f5bf089208c7525ed992fdc7a487a 192.168.1.2:5379
   slots: (0 slots) slave
   replicates f9f0cdd20b8e7474c82c5cc3ea4177a2a8fa4dbc
M: f90ea317783606a08bf70443fdc8a0e7d5c9e483 192.168.1.2:3379
   slots:[10923-16383] (5461 slots) master
   1 additional replica(s)
S: 096106d92a0d55279898b696d9ea9f81a8f1bbad 192.168.1.2:6379
   slots: (0 slots) slave
   replicates 7ce6b7d37f3dc04f8b77e8d9bee37bdaf47b903d
M: 7ce6b7d37f3dc04f8b77e8d9bee37bdaf47b903d 192.168.1.2:2379
   slots:[5461-10922] (5462 slots) master
   1 additional replica(s)
S: 975c47d4045e64dada1c2a377dd49435cce53ad1 192.168.1.2:4379
   slots: (0 slots) slave
   replicates f90ea317783606a08bf70443fdc8a0e7d5c9e483
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.

2.3 查看集群状态

cluster info

cluster_state:ok
cluster_slots_assigned:16384
cluster_slots_ok:16384
cluster_slots_pfail:0
cluster_slots_fail:0
cluster_known_nodes:6
cluster_size:3
cluster_current_epoch:6
cluster_my_epoch:2
cluster_stats_messages_ping_sent:9758
cluster_stats_messages_pong_sent:9882
cluster_stats_messages_meet_sent:1
cluster_stats_messages_sent:19641
cluster_stats_messages_ping_received:9882
cluster_stats_messages_pong_received:9759
cluster_stats_messages_received:19641

2.4 创建数据

redis-cli -c -h 192.168.1.2 -p 1379

192.168.1.2:1379> set test test
-> Redirected to slot [6918] located at 192.168.1.2:2379
OK
192.168.192.1:2379> get test
"test"

三、宕机测试

cluster nodes

7ce6b7d37f3dc04f8b77e8d9bee37bdaf47b903d 192.168.1.2:2379@12379 myself,master - 0 1648456090000 2 connected 5461-10922
975c47d4045e64dada1c2a377dd49435cce53ad1 192.168.1.2:4379@14379 slave f90ea317783606a08bf70443fdc8a0e7d5c9e483 0 1648456092000 3 connected
096106d92a0d55279898b696d9ea9f81a8f1bbad 192.168.1.2:6379@16379 slave 7ce6b7d37f3dc04f8b77e8d9bee37bdaf47b903d 0 1648456091000 2 connected
24aaaf5b071f5bf089208c7525ed992fdc7a487a 192.168.1.2:5379@15379 slave f9f0cdd20b8e7474c82c5cc3ea4177a2a8fa4dbc 0 1648456092294 1 connected
f9f0cdd20b8e7474c82c5cc3ea4177a2a8fa4dbc 192.168.1.2:1379@11379 master - 0 1648456091000 1 connected 0-5460
f90ea317783606a08bf70443fdc8a0e7d5c9e483 192.168.1.2:3379@13379 master - 0 1648456091000 3 connected 10923-16383

docker stop redis-1
cluster nodes

7ce6b7d37f3dc04f8b77e8d9bee37bdaf47b903d 192.168.1.2:2379@12379 myself,master - 0 1648456262000 2 connected 5461-10922
975c47d4045e64dada1c2a377dd49435cce53ad1 192.168.1.2:4379@14379 slave f90ea317783606a08bf70443fdc8a0e7d5c9e483 0 1648456263000 3 connected
096106d92a0d55279898b696d9ea9f81a8f1bbad 192.168.1.2:6379@16379 slave 7ce6b7d37f3dc04f8b77e8d9bee37bdaf47b903d 0 1648456261396 2 connected
24aaaf5b071f5bf089208c7525ed992fdc7a487a 192.168.1.2:5379@15379 master - 0 1648456262000 7 connected 0-5460
f9f0cdd20b8e7474c82c5cc3ea4177a2a8fa4dbc 192.168.1.2:1379@11379 master,fail - 1648456214523 1648456212003 1 disconnected
f90ea317783606a08bf70443fdc8a0e7d5c9e483 192.168.1.2:3379@13379 master - 0 1648456263411 3 connected 10923-16383

可以看到redis-1(1379)宕机后redis-5(5379)升级为master
docker start redis-1
可以看到重启服务后redis-5(5379)变为redis-1(1379)的slave

7ce6b7d37f3dc04f8b77e8d9bee37bdaf47b903d 192.168.1.2:2379@12379 myself,master - 0 1648456453000 2 connected 5461-10922
975c47d4045e64dada1c2a377dd49435cce53ad1 192.168.1.2:4379@14379 slave f90ea317783606a08bf70443fdc8a0e7d5c9e483 0 1648456454898 3 connected
096106d92a0d55279898b696d9ea9f81a8f1bbad 192.168.1.2:6379@16379 slave 7ce6b7d37f3dc04f8b77e8d9bee37bdaf47b903d 0 1648456454596 2 connected
24aaaf5b071f5bf089208c7525ed992fdc7a487a 192.168.1.2:5379@15379 master - 0 1648456454596 7 connected 0-5460
f9f0cdd20b8e7474c82c5cc3ea4177a2a8fa4dbc 192.168.1.2:1379@11379 slave 24aaaf5b071f5bf089208c7525ed992fdc7a487a 0 1648456454596 7 connected
f90ea317783606a08bf70443fdc8a0e7d5c9e483 192.168.1.2:3379@13379 master - 0 1648456454000 3 connected 10923-16383

四、RQ连接测试

4.1 测试脚本

from redis import RedisCluster
from redis.cluster import ClusterNode
from rq import Queue

nodes = [
    ClusterNode('192.168.1.2', 1379),
    ClusterNode('192.168.1.2', 2378),
    ClusterNode('192.168.1.2', 3378),
    ClusterNode('192.168.1.2', 4378),
    ClusterNode('192.168.1.2', 5378),
    ClusterNode('192.168.1.2', 6378),
]
redis_connect = RedisCluster(startup_nodes=nodes)
redis_connect.set('test', 'test')
queue = Queue(name='test', connection=redis_connect)

job = queue.enqueue('test')
print(job.get_id())

4.2 测试结果

在这里插入图片描述

五、总结

使用容器部署集群需要注意6379和16379两个端口都需要开放
cluster-announce-ip需要指定为服务器的真实IP地址

实际集群部署将redis.conf中的配置修改为相应的真实IP和端口即可

# 开启集群
cluster-enabled yes
# 集群配置文件
cluster-config-file nodes.conf
# 集群节点多少时间未响应视为该节点丢失
cluster-node-timeout 5000
# 开启 appendonly 备份模式
appendonly yes
# redis监听端口
port 6379
# 集群节点ip和端口
cluster-announce-ip 10.0.10.10
cluster-announce-port 6379
cluster-announce-bus-port 16379