python-redis高可用方案-sentinel

       sentinel监控redis的主节点和备份节点,并监控sentinel伙伴节点,
当redis主节点发生故障后,sentinel节点一起投票选举出redis主节点,
并进行redis主从节点切换,以达到高可用目的。
redis节点有多个,sentinel节点有多个,sentinel节点是最清楚
当前redis主节点和从节点的状态的,python redis客户端通过sentinel创建连接,
不管redis节点如何变化,都能得到主节点和从节点,从而不需要改动python代码,
灵活操作redis。
   

    操作系统: CentOS Linux release 7.6.1810
    ip:       127.0.0.1
    服务端:  Redis server v=5.0.3
    客户端:  python 连接redis插件版本 redis-3.2.0

0、redis安装

    a、下载redis-5.0.3.tar.gz
    b、tar xvzf redis-5.0.3.tar.gz
    c、cd redis-5.0.3/
    d、make PREFIX=/user/local/redis install

1、配置redis节点

    a、端口1111 主节点配置文件 /etc/redis-conf/redis1111.conf 内容如下:
        bind 127.0.0.1
        port 1111
        daemonize no #通过supervisor管理,不用守护方式启动
        requirepass "m-SDFdfle90IUD&)+U" #当前节点密码
        logfile "/home/logs/redis/1111.log" #redis日志
        dbfilename "dump-1111.rdb" #redis存储的文件
        dir "/home/redis/db" #redis存储文件的目录
        
        replicaof 127.0.0.1 1111
        
        #配置主服务器的masterauth属性,
        #因为后续主节点由于下线会变成从节点,
        #所以这里也要配置,否则后续主节点恢复故障后,无法切换
        masterauth "m-SDFdfle90IUD&)+U"
        
        
    b、端口2222 从节点 配置文件 /etc/redis-conf/redis2222.conf 内容如下:
        bind 127.0.0.1
        port 2222
        daemonize no #通过supervisor管理,不用守护方式启动
        requirepass "m-SDFdfle90IUD&)+U" #当前节点密码
        logfile "/home/logs/redis/2222.log" #redis日志
        dbfilename "dump-2222.rdb" #redis存储的文件
        dir "/home/redis/db" #redis存储文件的目录
        
        replicaof 127.0.0.1 1111
        
        #配置主服务器的masterauth属性,
        masterauth "m-SDFdfle90IUD&)+U"
    
    
    c、端口3333 从节点 配置文件 /etc/redis-conf/redis3333.conf 内容如下:
        bind 127.0.0.1
        port 3333
        daemonize no #通过supervisor管理,不用守护方式启动
        requirepass "m-SDFdfle90IUD&)+U" #当前节点密码
        logfile "/home/logs/redis/3333.log" #redis日志
        dbfilename "dump-3333.rdb" #redis存储的文件
        dir "/home/redis/db" #redis存储文件的目录
        
        replicaof 127.0.0.1 1111

        #配置主服务器的masterauth属性,
        masterauth "m-SDFdfle90IUD&)+U"


    d、supervisor启动三个redis节点命令
        /usr/local/bin/redis-server /etc/redis-conf/redis1111.conf
        /usr/local/bin/redis-server /etc/redis-conf/redis2222.conf
        /usr/local/bin/redis-server /etc/redis-conf/redis3333.conf


        
2、sentinel节点配置
   

a、第一个sentinel节点配置文件 /etc/redis-conf/sentinel_1.conf
        port 9111
        daemonize no
        logfile "sentinel_1.log"
        dir "/home/logs/redis"
        sentinel monitor mymaster 127.0.0.1 1111 2
        #redis数据master节点设置了认证,则需要如下配置
        sentinel auth-pass mymaster m-SDFdfle90IUD&)+U
        sentinel down-after-milliseconds mymaster 5000
        sentinel parallel-syncs mymaster 1
        sentinel failover-timeout mymaster 180000
    
    b、第二个sentinel节点配置文件 /etc/redis-conf/sentinel_2.conf
        port 9222
        daemonize no
        logfile "sentinel_2.log"
        dir "/home/logs/redis"
        sentinel monitor mymaster 127.0.0.1 1111 2
        #redis数据master节点设置了认证,则需要如下配置
        sentinel auth-pass mymaster m-SDFdfle90IUD&)+U
        sentinel down-after-milliseconds mymaster 5000
        sentinel parallel-syncs mymaster 1
        sentinel failover-timeout mymaster 180000
    
    c、第三个sentinel节点配置文件 /etc/redis-conf/sentinel_3.conf
        port 9333
        daemonize no
        logfile "sentinel_3.log"
        dir "/home/logs/redis"
        sentinel monitor mymaster 127.0.0.1 1111 2
        #redis数据master节点设置了认证,则需要如下配置
        sentinel auth-pass mymaster m-SDFdfle90IUD&)+U
        sentinel down-after-milliseconds mymaster 5000
        sentinel parallel-syncs mymaster 1
        sentinel failover-timeout mymaster 180000

    
    d、通过supervisor启动3个sentinel节点的命令如下:
        /usr/local/bin/redis-sentinel /etc/redis-conf/sentinel_1.conf 
        /usr/local/bin/redis-sentinel /etc/redis-conf/sentinel_2.conf 
        /usr/local/bin/redis-sentinel /etc/redis-conf/sentinel_3.conf 
        
    
    f、检查3个节点的运行状态
        redis-cli -h 127.0.0.1 -p 9111 info sentinel
        # Sentinel
        sentinel_masters:1
        sentinel_tilt:0
        sentinel_running_scripts:0
        sentinel_scripts_queue_length:0
        sentinel_simulate_failure_flags:0
        master0:name=mymaster,status=ok,address=127.0.0.1:1111,slaves=2,sentinels=3

        
        redis-cli -h 127.0.0.1 -p 9222 info sentinel
        # Sentinel
        sentinel_masters:1
        sentinel_tilt:0
        sentinel_running_scripts:0
        sentinel_scripts_queue_length:0
        sentinel_simulate_failure_flags:0
        master0:name=mymaster,status=ok,address=127.0.0.1:1111,slaves=2,sentinels=3
        
        
        redis-cli -h 127.0.0.1 -p 9333 info sentinel
        # Sentinel
        sentinel_masters:1
        sentinel_tilt:0
        sentinel_running_scripts:0
        sentinel_scripts_queue_length:0
        sentinel_simulate_failure_flags:0
        master0:name=mymaster,status=ok,address=127.0.0.1:1111,slaves=2,sentinels=

3、python连接测试
   

from redis.sentinel import Sentinel
    sentinel = Sentinel([
            ('127.0.0.1', 9111),
            ('127.0.0.1', 9222),
            ('127.0.0.1', 9333),
            ],
            socket_timeout=0.5
        )
    master = sentinel.discover_master('mymaster') #得到redis master节点的信息
    ('127.0.0.1', 1111 )
    slave = sentinel.discover_slaves('mymaster') #得到redis 所有从节点的信息
    [ ('127.0.0.1', 2222),('127.0.0.1', 3333)]
    
    #得到主节点连接
    master = sentinel.master_for('mymaster', socket_timeout=0.5, password='m-SDFdfle90IUD&)+U')
    master.set("k","v")
    #得到从节点连接
    slave = sentinel.slave_for('mymaster', socket_timeout=0.5, password='m-SDFdfle90IUD&)+U')
    slave.get("k")


        >>> slave.set("k6","v6")
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
      File "build/bdist.linux-x86_64/egg/redis/client.py", line 1451, in set
      File "build/bdist.linux-x86_64/egg/redis/client.py", line 775, in execute_command
      File "build/bdist.linux-x86_64/egg/redis/client.py", line 789, in parse_response
      File "build/bdist.linux-x86_64/egg/redis/sentinel.py", line 56, in read_response
      File "build/bdist.linux-x86_64/egg/redis/connection.py", line 642, in read_response
    redis.exceptions.ReadOnlyError: You can't write against a read only replica.
    
    注意:
        当有从节点的时候,是不能执行更新操作的,如果只有主节点,没有从节点,sentinel.slave_for
    获取的节点是主节点,是可以执行更新操作的

4、关掉1111主节点
   

supervisorctl stop redis-server-m-1111
    
    查看日志tail -f /home/logs/redis/*.log:
    
    ==> /var/log/redis/sentinel_1111.log <==
    26890:X 31 Aug 2020 19:39:31.204 # +sdown master mymaster 127.0.0.1 1111
    
    ==> /var/log/redis/sentinel_2222.log <==
    26894:X 31 Aug 2020 19:39:31.219 # +sdown master mymaster 127.0.0.1 1111
    
    ==> /var/log/redis/sentinel_3333.log <==
    26897:X 31 Aug 2020 19:39:31.226 # +sdown master mymaster 127.0.0.1 1111
    
    ==> /var/log/redis/sentinel_2222.log <==
    26894:X 31 Aug 2020 19:39:31.290 # +odown master mymaster 127.0.0.1 1111 #quorum 2/2
    26894:X 31 Aug 2020 19:39:31.290 # +new-epoch 8
    26894:X 31 Aug 2020 19:39:31.290 # +try-failover master mymaster 127.0.0.1 1111
    26894:X 31 Aug 2020 19:39:31.294 # +vote-for-leader db29dc803706e24d4ba744fde7986b9855ae4cb5 8
    
    ==> /var/log/redis/sentinel_1111.log <==
    26890:X 31 Aug 2020 19:39:31.296 # +new-epoch 8
    
    ==> /var/log/redis/sentinel_3333.log <==
    26897:X 31 Aug 2020 19:39:31.296 # +new-epoch 8
    
    ==> /var/log/redis/sentinel_1111.log <==
    26890:X 31 Aug 2020 19:39:31.299 # +vote-for-leader db29dc803706e24d4ba744fde7986b9855ae4cb5 8
    
    ==> /var/log/redis/sentinel_3333.log <==
    26897:X 31 Aug 2020 19:39:31.299 # +vote-for-leader db29dc803706e24d4ba744fde7986b9855ae4cb5 8
    26897:X 31 Aug 2020 19:39:31.299 # +odown master mymaster 127.0.0.1 1111 #quorum 3/2
    
    ==> /var/log/redis/sentinel_2222.log <==
    26894:X 31 Aug 2020 19:39:31.299 # de0d06fdeb8f3ea146a519f1c7571584f3f422a5 voted for db29dc803706e24d4ba744fde7986b9855ae4cb5 8
    
    ==> /var/log/redis/sentinel_3333.log <==
    26897:X 31 Aug 2020 19:39:31.299 # Next failover delay: I will not start a failover before Mon Aug 31 19:45:31 2020
    
    ==> /var/log/redis/sentinel_2222.log <==
    26894:X 31 Aug 2020 19:39:31.299 # e16204d26a678a613b0c0e9d18a5f2f35efe523c voted for db29dc803706e24d4ba744fde7986b9855ae4cb5 8
    26894:X 31 Aug 2020 19:39:31.365 # +elected-leader master mymaster 127.0.0.1 1111
    26894:X 31 Aug 2020 19:39:31.365 # +failover-state-select-slave master mymaster 127.0.0.1 1111
    26894:X 31 Aug 2020 19:39:31.421 # +selected-slave slave 127.0.0.1:3333 127.0.0.1 3333 @ mymaster 127.0.0.1 1111
    26894:X 31 Aug 2020 19:39:31.421 * +failover-state-send-slaveof-noone slave 127.0.0.1:3333 127.0.0.1 3333 @ mymaster 127.0.0.1 1111
    26894:X 31 Aug 2020 19:39:31.497 * +failover-state-wait-promotion slave 127.0.0.1:3333 127.0.0.1 3333 @ mymaster 127.0.0.1 1111
    26894:X 31 Aug 2020 19:39:31.825 # +promoted-slave slave 127.0.0.1:3333 127.0.0.1 3333 @ mymaster 127.0.0.1 1111
    26894:X 31 Aug 2020 19:39:31.825 # +failover-state-reconf-slaves master mymaster 127.0.0.1 1111
    26894:X 31 Aug 2020 19:39:31.876 * +slave-reconf-sent slave 127.0.0.1:2222 127.0.0.1 2222 @ mymaster 127.0.0.1 1111
    
    ==> /var/log/redis/sentinel_3333.log <==
    26897:X 31 Aug 2020 19:39:31.877 # +config-update-from sentinel db29dc803706e24d4ba744fde7986b9855ae4cb5 127.0.0.1 9222 @ mymaster 127.0.0.1 1111
    26897:X 31 Aug 2020 19:39:31.877 # +switch-master mymaster 127.0.0.1 1111 127.0.0.1 3333
    26897:X 31 Aug 2020 19:39:31.877 * +slave slave 127.0.0.1:2222 127.0.0.1 2222 @ mymaster 127.0.0.1 3333
    26897:X 31 Aug 2020 19:39:31.877 * +slave slave 127.0.0.1:1111 127.0.0.1 1111 @ mymaster 127.0.0.1 3333
    
    ==> /var/log/redis/sentinel_1111.log <==
    26890:X 31 Aug 2020 19:39:31.878 # +config-update-from sentinel db29dc803706e24d4ba744fde7986b9855ae4cb5 127.0.0.1 9222 @ mymaster 127.0.0.1 1111
    26890:X 31 Aug 2020 19:39:31.878 # +switch-master mymaster 127.0.0.1 1111 127.0.0.1 3333
    26890:X 31 Aug 2020 19:39:31.878 * +slave slave 127.0.0.1:2222 127.0.0.1 2222 @ mymaster 127.0.0.1 3333
    26890:X 31 Aug 2020 19:39:31.878 * +slave slave 127.0.0.1:1111 127.0.0.1 1111 @ mymaster 127.0.0.1 3333
    
    ==> /var/log/redis/sentinel_2222.log <==
    26894:X 31 Aug 2020 19:39:32.438 # -odown master mymaster 127.0.0.1 1111
    26894:X 31 Aug 2020 19:39:32.863 * +slave-reconf-inprog slave 127.0.0.1:2222 127.0.0.1 2222 @ mymaster 127.0.0.1 1111
    26894:X 31 Aug 2020 19:39:32.863 * +slave-reconf-done slave 127.0.0.1:2222 127.0.0.1 2222 @ mymaster 127.0.0.1 1111
    26894:X 31 Aug 2020 19:39:32.953 # +failover-end master mymaster 127.0.0.1 1111
    26894:X 31 Aug 2020 19:39:32.953 # +switch-master mymaster 127.0.0.1 1111 127.0.0.1 3333
    26894:X 31 Aug 2020 19:39:32.953 * +slave slave 127.0.0.1:2222 127.0.0.1 2222 @ mymaster 127.0.0.1 3333
    26894:X 31 Aug 2020 19:39:32.954 * +slave slave 127.0.0.1:1111 127.0.0.1 1111 @ mymaster 127.0.0.1 3333
    
    ==> /var/log/redis/sentinel_3333.log <==
    26897:X 31 Aug 2020 19:40:01.890 # +sdown slave 127.0.0.1:1111 127.0.0.1 1111 @ mymaster 127.0.0.1 3333
    
    ==> /var/log/redis/sentinel_1111.log <==
    26890:X 31 Aug 2020 19:40:01.955 # +sdown slave 127.0.0.1:1111 127.0.0.1 1111 @ mymaster 127.0.0.1 3333
    
    ==> /var/log/redis/sentinel_2222.log <==
    26894:X 31 Aug 2020 19:40:03.039 # +sdown slave 127.0.0.1:1111 127.0.0.1 1111 @ mymaster 127.0.0.1 3333

    
5、python查看当前状态
    >>> sentinel.discover_master('mymaster')
    ('127.0.0.1', 3333)

    >>> sentinel.discover_slaves('mymaster')
    [('127.0.0.1', 2222)]
    发现主redis已经切换成3333节点了,1111故障节点也移除了,2222从节点也切换了主redis节点

6、继续关掉3333主节点
    supervisorctl stop redis-server-d-3333       

查看日志tail -f /home/logs/redis/*.log:
    ==> /var/log/redis/sentinel_1111.log <==
    26890:X 31 Aug 2020 19:47:21.842 # +sdown master mymaster 127.0.0.1 3333
    
    ==> /var/log/redis/sentinel_3333.log <==
    26897:X 31 Aug 2020 19:47:21.846 # +sdown master mymaster 127.0.0.1 3333
    
    ==> /var/log/redis/sentinel_2222.log <==
    26894:X 31 Aug 2020 19:47:21.860 # +sdown master mymaster 127.0.0.1 3333
    26894:X 31 Aug 2020 19:47:21.912 # +odown master mymaster 127.0.0.1 3333 #quorum 3/2
    26894:X 31 Aug 2020 19:47:21.912 # +new-epoch 9
    26894:X 31 Aug 2020 19:47:21.912 # +try-failover master mymaster 127.0.0.1 3333
    26894:X 31 Aug 2020 19:47:21.915 # +vote-for-leader db29dc803706e24d4ba744fde7986b9855ae4cb5 9
    
    ==> /var/log/redis/sentinel_3333.log <==
    26897:X 31 Aug 2020 19:47:21.918 # +new-epoch 9
    
    ==> /var/log/redis/sentinel_1111.log <==
    26890:X 31 Aug 2020 19:47:21.918 # +new-epoch 9
    
    ==> /var/log/redis/sentinel_3333.log <==
    26897:X 31 Aug 2020 19:47:21.921 # +vote-for-leader db29dc803706e24d4ba744fde7986b9855ae4cb5 9
    
    ==> /var/log/redis/sentinel_2222.log <==
    26894:X 31 Aug 2020 19:47:21.921 # e16204d26a678a613b0c0e9d18a5f2f35efe523c voted for db29dc803706e24d4ba744fde7986b9855ae4cb5 9
    
    ==> /var/log/redis/sentinel_1111.log <==
    26890:X 31 Aug 2020 19:47:21.921 # +vote-for-leader db29dc803706e24d4ba744fde7986b9855ae4cb5 9
    
    ==> /var/log/redis/sentinel_2222.log <==
    26894:X 31 Aug 2020 19:47:21.921 # de0d06fdeb8f3ea146a519f1c7571584f3f422a5 voted for db29dc803706e24d4ba744fde7986b9855ae4cb5 9
    
    ==> /var/log/redis/sentinel_3333.log <==
    26897:X 31 Aug 2020 19:47:21.946 # +odown master mymaster 127.0.0.1 3333 #quorum 2/2
    26897:X 31 Aug 2020 19:47:21.946 # Next failover delay: I will not start a failover before Mon Aug 31 19:53:21 2020
    
    ==> /var/log/redis/sentinel_2222.log <==
    26894:X 31 Aug 2020 19:47:21.977 # +elected-leader master mymaster 127.0.0.1 3333
    26894:X 31 Aug 2020 19:47:21.977 # +failover-state-select-slave master mymaster 127.0.0.1 3333
    26894:X 31 Aug 2020 19:47:22.068 # +selected-slave slave 127.0.0.1:2222 127.0.0.1 2222 @ mymaster 127.0.0.1 3333
    26894:X 31 Aug 2020 19:47:22.068 * +failover-state-send-slaveof-noone slave 127.0.0.1:2222 127.0.0.1 2222 @ mymaster 127.0.0.1 3333
    26894:X 31 Aug 2020 19:47:22.144 * +failover-state-wait-promotion slave 127.0.0.1:2222 127.0.0.1 2222 @ mymaster 127.0.0.1 3333
    26894:X 31 Aug 2020 19:47:22.391 # +promoted-slave slave 127.0.0.1:2222 127.0.0.1 2222 @ mymaster 127.0.0.1 3333
    26894:X 31 Aug 2020 19:47:22.391 # +failover-state-reconf-slaves master mymaster 127.0.0.1 3333
    26894:X 31 Aug 2020 19:47:22.487 # +failover-end master mymaster 127.0.0.1 3333
    26894:X 31 Aug 2020 19:47:22.487 # +switch-master mymaster 127.0.0.1 3333 127.0.0.1 2222
    26894:X 31 Aug 2020 19:47:22.487 * +slave slave 127.0.0.1:1111 127.0.0.1 1111 @ mymaster 127.0.0.1 2222
    26894:X 31 Aug 2020 19:47:22.487 * +slave slave 127.0.0.1:3333 127.0.0.1 3333 @ mymaster 127.0.0.1 2222
    
    ==> /var/log/redis/sentinel_3333.log <==
    26897:X 31 Aug 2020 19:47:22.490 # +config-update-from sentinel db29dc803706e24d4ba744fde7986b9855ae4cb5 127.0.0.1 9222 @ mymaster 127.0.0.1 3333
    26897:X 31 Aug 2020 19:47:22.490 # +switch-master mymaster 127.0.0.1 3333 127.0.0.1 2222
    26897:X 31 Aug 2020 19:47:22.490 * +slave slave 127.0.0.1:1111 127.0.0.1 1111 @ mymaster 127.0.0.1 2222
    26897:X 31 Aug 2020 19:47:22.490 * +slave slave 127.0.0.1:3333 127.0.0.1 3333 @ mymaster 127.0.0.1 2222
    
    ==> /var/log/redis/sentinel_1111.log <==
    26890:X 31 Aug 2020 19:47:22.490 # +config-update-from sentinel db29dc803706e24d4ba744fde7986b9855ae4cb5 127.0.0.1 9222 @ mymaster 127.0.0.1 3333
    26890:X 31 Aug 2020 19:47:22.490 # +switch-master mymaster 127.0.0.1 3333 127.0.0.1 2222
    26890:X 31 Aug 2020 19:47:22.490 * +slave slave 127.0.0.1:1111 127.0.0.1 1111 @ mymaster 127.0.0.1 2222
    26890:X 31 Aug 2020 19:47:22.490 * +slave slave 127.0.0.1:3333 127.0.0.1 3333 @ mymaster 127.0.0.1 2222
    
    ==> /var/log/redis/sentinel_3333.log <==
    
    ==> /var/log/redis/sentinel_1111.log <==
    
    ==> /var/log/redis/sentinel_3333.log <==
    26897:X 31 Aug 2020 19:47:52.496 # +sdown slave 127.0.0.1:3333 127.0.0.1 3333 @ mymaster 127.0.0.1 2222
    26897:X 31 Aug 2020 19:47:52.496 # +sdown slave 127.0.0.1:1111 127.0.0.1 1111 @ mymaster 127.0.0.1 2222
    
    ==> /var/log/redis/sentinel_1111.log <==
    26890:X 31 Aug 2020 19:47:52.526 # +sdown slave 127.0.0.1:3333 127.0.0.1 3333 @ mymaster 127.0.0.1 2222
    26890:X 31 Aug 2020 19:47:52.526 # +sdown slave 127.0.0.1:1111 127.0.0.1 1111 @ mymaster 127.0.0.1 2222
    
    ==> /var/log/redis/sentinel_2222.log <==
    26894:X 31 Aug 2020 19:47:52.575 # +sdown slave 127.0.0.1:1111 127.0.0.1 1111 @ mymaster 127.0.0.1 2222
    26894:X 31 Aug 2020 19:47:52.575 # +sdown slave 127.0.0.1:3333 127.0.0.1 3333 @ mymaster 127.0.0.1 2222


7、python查看当前状态
    >>> sentinel.discover_master('mymaster')
    ('127.0.0.1', 2222)
    >>> sentinel.discover_slaves('mymaster')
    []
    发现主节点切换为2222了,从节点已经没有了
    
    要注意一下,从节点为空的时候
    slave = sentinel.slave_for('mymaster', socket_timeout=0.5, password='m-SDFdfle90IUD&)+U')
    save对象操作的是主节点
    从节点不为空的时候,执行更新操作是不允许的,
    为空的时候由于是操作的是主节点,所以可以执行更新操作
    
    >>> slave.get("k4")
    >>> slave.set("k4","v4")
    True
    >>> slave.get("k4")
    'v4'

8、查看连接数

    lsof -i:1111|grep redis-ser|wc -l

    或者登入之后通过 info clients 查看

  • 0
    点赞
  • 1
    收藏
    觉得还不错? 一键收藏
  • 0
    评论

“相关推荐”对你有帮助么?

  • 非常没帮助
  • 没帮助
  • 一般
  • 有帮助
  • 非常有帮助
提交
评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值