高效、安全的部署Twemproxy集群

已于 2022-03-07 11:45:52 修改
阅读量271 收藏 2
点赞数
文章标签： redis 运维 安全 linux ubuntu
于 2021-05-13 22:41:30 首次发布
版权声明：本文为博主原创文章，遵循 CC 4.0 BY-SA 版权协议，转载请附上原文出处链接和本声明。
本文链接：https://blog.csdn.net/qq_25241721/article/details/116769633
版权

文章目录

1. 环境准备

1.1. 硬件配置

标准化项说明备注
IO设置写策略设置Write Back，禁用Write Through
IO子系统设置使用Raid1+0若采用PCIe-SSD，则可不用做Raid
关闭物理磁盘cache，防止丢数据
写策略设置Write Back，禁用Write Through
关闭预读，使得宝贵的cahce用来作为写缓存
BIOS标准设置开启最大性能模式，选择DAPC或者选择Performance模式不同的主板配置上有差异
Memory Frequency（内存频率）选择Maximum Performance模式不同的主板配置上有差异
关闭NUMA不同的主板配置上有差异
文件系统数据存储文件系统类型建议EXT4格式该项不做强制要求XFS类型也可以
网络配置使用mode=4模式的网卡绑定双网卡连接不同的交换机，交换机上需要做适配
数据挂载目录/dataRedis数据库实例的所有文件全部存放在/data目录下；

1.2. 操作系统软件包&第三方工具

软件包说明备注
gcc至少是C89兼容的
GNU make版本3.80及以上
tar解压源码包时用到的tar解压工具

1.3. 关闭SELINUX

临时禁用selinux：

# setenforce 0
# more /etc/selinux/config | grep -Ev "$|[#;]"

永久禁用selinux：

# vi /etc/selinux/config

将
SELINUX= enforcing
修改为
SELINUX=disabled 
:x保存退出

1.4. 配置防火墙

如需要禁用防火墙，可执行如下命令：

1）若是CentOS 7.x系统，可执行如下命令禁用防火墙：
关闭防火墙：
# systemctl stop firewalld.service
禁用防火墙开机自启：
# systemctl disable firewalld.service
查看防火墙状态：
# systemctl status firewalld.service

2）若是Ubuntu系统，可执行如下命令禁用防火墙：
# iptables -P INPUT ACCEPT

3）其他系统禁用防火墙命令，请自行查找。

如需配置防火墙规则，可参照如下示例：

# vim /etc/iptables.rules

*filter
:INPUT ACCEPT [3715:523951]
:FORWARD ACCEPT [0:0]
:OUTPUT ACCEPT [1925:209933]
-A INPUT -i lo -j ACCEPT
#允许ssh端口通过
-A INPUT -p tcp -m tcp --dport 22 -j ACCEPT
#允许icmp
-A INPUT -p icmp -j ACCEPT
#允许redis服务器间互通
-A INPUT -s 192.168.11.11/24 -p tcp -m state --state NEW -m tcp --dport 6801:6840 -j ACCEPT
-A INPUT -s 192.168.11.12/24 -p tcp -m state --state NEW -m tcp --dport 6801:6840 -j ACCEPT
-A INPUT -s 192.168.11.13/24 -p tcp -m state --state NEW -m tcp --dport 6801:6840 -j ACCEPT
-A INPUT -s 192.168.11.14/24 -p tcp -m state --state NEW -m tcp --dport 6801:6840 -j ACCEPT
#运行其他服务器访问Redis，如允许172.172.122.222服务器访问本地Redis端口为6801:6840的Redis实例
-A INPUT -s 172.172.122.222/32 -p tcp -m state --state NEW -m tcp --dport 6801:6840 -j ACCEPT
#暴露本地22201服务端口
-A INPUT -p tcp -m tcp --dport 22201 -j ACCEPT
-A INPUT -m state --state ESTABLISHED -j ACCEPT
COMMIT

防火墙规则配置完毕后，将配置文件导入：

# iptables-restore < /etc/iptables.rules

导入防火墙规则配置文件后，打开防火墙：

# iptables -P INPUT DROP

如需要开机自动恢复规则，把恢复命令添加到启动脚本：

# echo '/sbin/iptables-restore < /etc/iptables.rules' >>/etc/rc.local
# echo '/sbin/iptables -P INPUT DROP' >>/etc/rc.local

# cat /etc/rc.local

1.5. 优化系统参数

编辑系统配置文件sysctl.conf：

# vim /etc/sysctl.conf

vm.overcommit_memory=1
vm.swappiness = 0
net.core.somaxconn = 1024
net.ipv4.tcp_timestamps = 0
net.ipv4.tcp_tw_reuse = 1
net.ipv4.tcp_tw_recycle = 0
net.ipv4.tcp_fin_timeout = 30
net.nf_conntrack_max = 655350
net.ipv4.ip_local_port_range = 1024 65000
net.ipv4.ip_local_reserved_ports = 16001,16801,6801-6840,20001-20016

^: net.ipv4.tcp_tw_reuse设置为1表示开启重用，允许将TIME-WAIT sockets重新用于新的TCP连接，默认为0，表示关闭。

装载ip_conntrack模块：

# modprobe ip_conntrack

载入sysctl配置文件：

# sysctl -p

1.6. 修改hugepage配置

临时禁用hugepage：

# echo 'never' > /sys/kernel/mm/transparent_hugepage/enabled

为了使机器重启后THP配置依然生效，可以在/etc/rc.local中追加：

# echo '/bin/echo never > /sys/kernel/mm/transparent_hugepage/enabled' >> /etc/rc.local

1.7. 修改环境变量

vim /etc/profile

......
unset i
unset -f pathmunge
export LD_LIBRARY_PATH=./
ulimit -SHn 655350

source /etc/profile

2. 部署redis

2.1. 节点规划

主节点：
192.168.11.11:6801-6820
192.168.11.12:6801-6820
192.168.11.13:6801-6820
192.168.11.14:6801-6820

从节点：
192.168.11.13:6821-6840
192.168.11.14:6821-6840
192.168.11.11:6821-6840
192.168.11.12:6821-6840

2.2. 安装依赖包

# apt-get install make
# apt-get install gcc

2.3. 安装Redis

2.3.1. 解压安装包、创建安装目录

解压安装包：
# cd /opt/app/softwares/
# tar zxvf redis-4.0.14.tar.gz -C /var/lib

编译安装：
# cd /var/lib/redis-4.0.14
# make MALLOC=libc

创建Redis部署所需目录：
# mkdir -p /data/redis/16801/dump
# mkdir -p /data/redis/16801/etc
# mkdir -p /data/redis/16801/log
# mkdir -p /data/redis/16801/run

2.3.2. 编辑配置文件

用脚本批量生成 6801-6840 端口的配置文件（需提前准备一个示例配置文件）：
# vim /data/redis/16801/etc/cp_conf.sh 
#!/bin/bash

i=6801

for ((;$i <= 6840; i=$i+1))
{
       cp 6000.conf $i.conf 
       sed -i "s,6001,$i,g" $i.conf
       sleep 1
}

为脚本赋予执行权限：
# chmod +x /data/redis/16801/etc/cp_conf.sh

^: 从 ubuntu 6.10 开始，ubuntu 就将先前默认的bash shell 更换成了dash shell；其表现为 /bin/sh 链接倒了/bin/dash而不是传统的/bin/bash。需执行dpkg-reconfigure dash，选择NO，即将默认的shell改回bash。

执行脚本，批量生成 6801-6840 端口的配置文件：
# cd /data/redis/16801/etc/
# sh /data/redis/16801/etc/cp_conf.sh 

删除示例配置文件：
# rm -rf 6000.conf

2.3.3. 启动redis

配置环境变量：
# vim /etc/profile
export PATH=/var/lib/redis-4.0.14/src:$PATH

使配置生效：
# source /etc/profile

批量启动Redis：
# cd /var/lib/redis-4.0.14
# ./start-redis.sh -n 16801

查看进程：
# ps -ef| grep redis-server|grep -v grep

3. 配置Redis主从

按照 “2.1. 节点规划”的规划配置Redis主从复制：
# redis-cli -h 192.168.11.11 -p 6821 slaveof 192.168.11.12 6801
# redis-cli -h 192.168.11.11 -p 6822 slaveof 192.168.11.12 6802
# redis-cli -h 192.168.11.11 -p 6823 slaveof 192.168.11.12 6803
# redis-cli -h 192.168.11.11 -p 6824 slaveof 192.168.11.12 6804
# redis-cli -h 192.168.11.11 -p 6825 slaveof 192.168.11.12 6805
# redis-cli -h 192.168.11.11 -p 6826 slaveof 192.168.11.12 6806
# redis-cli -h 192.168.11.11 -p 6827 slaveof 192.168.11.12 6807
# redis-cli -h 192.168.11.11 -p 6828 slaveof 192.168.11.12 6808
# redis-cli -h 192.168.11.11 -p 6829 slaveof 192.168.11.12 6809
# redis-cli -h 192.168.11.11 -p 6830 slaveof 192.168.11.12 6810
# redis-cli -h 192.168.11.11 -p 6831 slaveof 192.168.11.12 6811
# redis-cli -h 192.168.11.11 -p 6832 slaveof 192.168.11.12 6812
# redis-cli -h 192.168.11.11 -p 6833 slaveof 192.168.11.12 6813
# redis-cli -h 192.168.11.11 -p 6834 slaveof 192.168.11.12 6814
# redis-cli -h 192.168.11.11 -p 6835 slaveof 192.168.11.12 6815
# redis-cli -h 192.168.11.11 -p 6836 slaveof 192.168.11.12 6816
# redis-cli -h 192.168.11.11 -p 6837 slaveof 192.168.11.12 6817
# redis-cli -h 192.168.11.11 -p 6838 slaveof 192.168.11.12 6818
# redis-cli -h 192.168.11.11 -p 6839 slaveof 192.168.11.12 6819
# redis-cli -h 192.168.11.11 -p 6840 slaveof 192.168.11.12 6820


# redis-cli -h 192.168.11.12 -p 6821 slaveof 192.168.11.13 6801
# redis-cli -h 192.168.11.12 -p 6822 slaveof 192.168.11.13 6802
# redis-cli -h 192.168.11.12 -p 6823 slaveof 192.168.11.13 6803
# redis-cli -h 192.168.11.12 -p 6824 slaveof 192.168.11.13 6804
# redis-cli -h 192.168.11.12 -p 6825 slaveof 192.168.11.13 6805
# redis-cli -h 192.168.11.12 -p 6826 slaveof 192.168.11.13 6806
# redis-cli -h 192.168.11.12 -p 6827 slaveof 192.168.11.13 6807
# redis-cli -h 192.168.11.12 -p 6828 slaveof 192.168.11.13 6808
# redis-cli -h 192.168.11.12 -p 6829 slaveof 192.168.11.13 6809
# redis-cli -h 192.168.11.12 -p 6830 slaveof 192.168.11.13 6810
# redis-cli -h 192.168.11.12 -p 6831 slaveof 192.168.11.13 6811
# redis-cli -h 192.168.11.12 -p 6832 slaveof 192.168.11.13 6812
# redis-cli -h 192.168.11.12 -p 6833 slaveof 192.168.11.13 6813
# redis-cli -h 192.168.11.12 -p 6834 slaveof 192.168.11.13 6814
# redis-cli -h 192.168.11.12 -p 6835 slaveof 192.168.11.13 6815
# redis-cli -h 192.168.11.12 -p 6836 slaveof 192.168.11.13 6816
# redis-cli -h 192.168.11.12 -p 6837 slaveof 192.168.11.13 6817
# redis-cli -h 192.168.11.12 -p 6838 slaveof 192.168.11.13 6818
# redis-cli -h 192.168.11.12 -p 6839 slaveof 192.168.11.13 6819
# redis-cli -h 192.168.11.12 -p 6840 slaveof 192.168.11.13 6820


# redis-cli -h 192.168.11.13 -p 6821 slaveof 192.168.11.14 6801
# redis-cli -h 192.168.11.13 -p 6822 slaveof 192.168.11.14 6802
# redis-cli -h 192.168.11.13 -p 6823 slaveof 192.168.11.14 6803
# redis-cli -h 192.168.11.13 -p 6824 slaveof 192.168.11.14 6804
# redis-cli -h 192.168.11.13 -p 6825 slaveof 192.168.11.14 6805
# redis-cli -h 192.168.11.13 -p 6826 slaveof 192.168.11.14 6806
# redis-cli -h 192.168.11.13 -p 6827 slaveof 192.168.11.14 6807
# redis-cli -h 192.168.11.13 -p 6828 slaveof 192.168.11.14 6808
# redis-cli -h 192.168.11.13 -p 6829 slaveof 192.168.11.14 6809
# redis-cli -h 192.168.11.13 -p 6830 slaveof 192.168.11.14 6810
# redis-cli -h 192.168.11.13 -p 6831 slaveof 192.168.11.14 6811
# redis-cli -h 192.168.11.13 -p 6832 slaveof 192.168.11.14 6812
# redis-cli -h 192.168.11.13 -p 6833 slaveof 192.168.11.14 6813
# redis-cli -h 192.168.11.13 -p 6834 slaveof 192.168.11.14 6814
# redis-cli -h 192.168.11.13 -p 6835 slaveof 192.168.11.14 6815
# redis-cli -h 192.168.11.13 -p 6836 slaveof 192.168.11.14 6816
# redis-cli -h 192.168.11.13 -p 6837 slaveof 192.168.11.14 6817
# redis-cli -h 192.168.11.13 -p 6838 slaveof 192.168.11.14 6818
# redis-cli -h 192.168.11.13 -p 6839 slaveof 192.168.11.14 6819
# redis-cli -h 192.168.11.13 -p 6840 slaveof 192.168.11.14 6820

# redis-cli -h 192.168.11.14 -p 6821 slaveof 192.168.11.11 6801
# redis-cli -h 192.168.11.14 -p 6822 slaveof 192.168.11.11 6802
# redis-cli -h 192.168.11.14 -p 6823 slaveof 192.168.11.11 6803
# redis-cli -h 192.168.11.14 -p 6824 slaveof 192.168.11.11 6804
# redis-cli -h 192.168.11.14 -p 6825 slaveof 192.168.11.11 6805
# redis-cli -h 192.168.11.14 -p 6826 slaveof 192.168.11.11 6806
# redis-cli -h 192.168.11.14 -p 6827 slaveof 192.168.11.11 6807
# redis-cli -h 192.168.11.14 -p 6828 slaveof 192.168.11.11 6808
# redis-cli -h 192.168.11.14 -p 6829 slaveof 192.168.11.11 6809
# redis-cli -h 192.168.11.14 -p 6830 slaveof 192.168.11.11 6810
# redis-cli -h 192.168.11.14 -p 6831 slaveof 192.168.11.11 6811
# redis-cli -h 192.168.11.14 -p 6832 slaveof 192.168.11.11 6812
# redis-cli -h 192.168.11.14 -p 6833 slaveof 192.168.11.11 6813
# redis-cli -h 192.168.11.14 -p 6834 slaveof 192.168.11.11 6814
# redis-cli -h 192.168.11.14 -p 6835 slaveof 192.168.11.11 6815
# redis-cli -h 192.168.11.14 -p 6836 slaveof 192.168.11.11 6816
# redis-cli -h 192.168.11.14 -p 6837 slaveof 192.168.11.11 6817
# redis-cli -h 192.168.11.14 -p 6838 slaveof 192.168.11.11 6818
# redis-cli -h 192.168.11.14 -p 6839 slaveof 192.168.11.11 6819
# redis-cli -h 192.168.11.14 -p 6840 slaveof 192.168.11.11 6820



主从配置完成后，查看各个节点的角色：
# for((i=6801;i<=6820;i++));do echo "==========192.168.11.11 : ${i}=========="; # redis-cli -h 192.168.11.11 -p ${i}  role; done
# for((i=6801;i<=6820;i++));do echo "==========192.168.11.12 : ${i}=========="; # redis-cli -h 192.168.11.12 -p ${i}  role; done
# for((i=6801;i<=6820;i++));do echo "==========192.168.11.13 : ${i}=========="; # redis-cli -h 192.168.11.13 -p ${i}  role; done
# for((i=6801;i<=6820;i++));do echo "==========192.168.11.14 : ${i}=========="; # redis-cli -h 192.168.11.14 -p ${i}  role; done

4. 部署twemproxy

4.1. 配置twemproxy

创建安装目录：
# mkdir -p /test/nutcracker/


编辑配置文件：
# vim /test/nutcracker/nutcracker/bin/conf/nutcracker.yml

alpha:
  listen: 0.0.0.0:16801
  hash: fnv1a_64
  hash_tag: "{}"
  distribution: ketama
  preconnect: true
  server_connections: 30
  auto_eject_hosts: false
  server_failure_limit: 2 
  server_retry_timeout: 30000
  timeout: 500
  redis: true
  servers:
    - 192.168.11.11:6801:1 server1
    - 192.168.11.11:6802:1 server2
    - 192.168.11.11:6803:1 server3
    - 192.168.11.11:6804:1 server4
    - 192.168.11.11:6805:1 server5
    - 192.168.11.11:6806:1 server6
    - 192.168.11.11:6807:1 server7
    - 192.168.11.11:6808:1 server8
    - 192.168.11.11:6809:1 server9
    - 192.168.11.11:6810:1 server10
    - 192.168.11.11:6811:1 server11
    - 192.168.11.11:6812:1 server12
    - 192.168.11.11:6813:1 server13
    - 192.168.11.11:6814:1 server14
    - 192.168.11.11:6815:1 server15
    - 192.168.11.11:6816:1 server16
    - 192.168.11.11:6817:1 server17
    - 192.168.11.11:6818:1 server18
    - 192.168.11.11:6819:1 server19
    - 192.168.11.11:6820:1 server20
    - 192.168.11.12:6801:1 server21
    - 192.168.11.12:6802:1 server22
    - 192.168.11.12:6803:1 server23
    - 192.168.11.12:6804:1 server24
    - 192.168.11.12:6805:1 server25
    - 192.168.11.12:6806:1 server26
    - 192.168.11.12:6807:1 server27
    - 192.168.11.12:6808:1 server28
    - 192.168.11.12:6809:1 server29
    - 192.168.11.12:6810:1 server30
    - 192.168.11.12:6811:1 server31
    - 192.168.11.12:6812:1 server32
    - 192.168.11.12:6813:1 server33
    - 192.168.11.12:6814:1 server34
    - 192.168.11.12:6815:1 server35
    - 192.168.11.12:6816:1 server36
    - 192.168.11.12:6817:1 server37
    - 192.168.11.12:6818:1 server38
    - 192.168.11.12:6819:1 server39
    - 192.168.11.12:6820:1 server40
    - 192.168.11.13:6801:1 server41
    - 192.168.11.13:6802:1 server42
    - 192.168.11.13:6803:1 server43
    - 192.168.11.13:6804:1 server44
    - 192.168.11.13:6805:1 server45
    - 192.168.11.13:6806:1 server46
    - 192.168.11.13:6807:1 server47
    - 192.168.11.13:6808:1 server48
    - 192.168.11.13:6809:1 server49
    - 192.168.11.13:6810:1 server50
    - 192.168.11.13:6811:1 server51
    - 192.168.11.13:6812:1 server52
    - 192.168.11.13:6813:1 server53
    - 192.168.11.13:6814:1 server54
    - 192.168.11.13:6815:1 server55
    - 192.168.11.13:6816:1 server56
    - 192.168.11.13:6817:1 server57
    - 192.168.11.13:6818:1 server58
    - 192.168.11.13:6819:1 server59
    - 192.168.11.13:6820:1 server60
    - 192.168.11.14:6801:1 server68
    - 192.168.11.14:6802:1 server62
    - 192.168.11.14:6803:1 server63
    - 192.168.11.14:6804:1 server64
    - 192.168.11.14:6805:1 server65
    - 192.168.11.14:6806:1 server66
    - 192.168.11.14:6807:1 server67
    - 192.168.11.14:6808:1 server68
    - 192.168.11.14:6809:1 server69
    - 192.168.11.14:6810:1 server70
    - 192.168.11.14:6811:1 server71
    - 192.168.11.14:6812:1 server72
    - 192.168.11.14:6813:1 server73
    - 192.168.11.14:6814:1 server74
    - 192.168.11.14:6815:1 server75
    - 192.168.11.14:6816:1 server76
    - 192.168.11.14:6817:1 server77
    - 192.168.11.14:6818:1 server78
    - 192.168.11.14:6819:1 server79
    - 192.168.11.14:6820:1 server80

4.2. 启动twemproxy

创建日志目录：
# mkdir -p /log/

启动twemproxy：
# cd /test/nutcracker/nutcracker/bin/
# sh start.sh

查看进程：
# ps -ef| grep nutcracker|grep -v grep
# ps -ef| grep nutcracker|grep -v grep|wc -l

5. 部署哨兵

创建哨兵日志目录：
# mkdir /etc/sentinel

5.1. 编辑哨兵文件

# vim /etc/sentinel/sentinel.conf

port 26880
daemonize yes
dir "/tmp"
pidfile "/var/run/sentinel.pid"
loglevel notice
logfile "/var/log/sentinel.log"
protected-mode no

sentinel monitor test_16801_g1 192.168.11.11 6801 2
sentinel down-after-milliseconds test_16801_g1 100000
sentinel failover-timeout test_16801_g1 60000
sentinel client-reconfig-script test_16801_g1 /etc/sentinel/client-reconfig.sh

sentinel monitor test_16801_g2 192.168.11.11 6802 2
sentinel down-after-milliseconds test_16801_g2 100000
sentinel failover-timeout test_16801_g2 60000
sentinel client-reconfig-script test_16801_g2 /etc/sentinel/client-reconfig.sh

sentinel monitor test_16801_g3 192.168.11.11 6803 2
sentinel down-after-milliseconds test_16801_g3 100000
sentinel failover-timeout test_16801_g3 60000
sentinel client-reconfig-script test_16801_g3 /etc/sentinel/client-reconfig.sh

sentinel monitor test_16801_g4 192.168.11.11 6804 2
sentinel down-after-milliseconds test_16801_g4 100000
sentinel failover-timeout test_16801_g4 60000
sentinel client-reconfig-script test_16801_g4 /etc/sentinel/client-reconfig.sh

sentinel monitor test_16801_g5 192.168.11.11 6805 2
sentinel down-after-milliseconds test_16801_g5 100000
sentinel failover-timeout test_16801_g5 60000
sentinel client-reconfig-script test_16801_g5 /etc/sentinel/client-reconfig.sh

sentinel monitor test_16801_g6 192.168.11.11 6806 2
sentinel down-after-milliseconds test_16801_g6 100000
sentinel failover-timeout test_16801_g6 60000
sentinel client-reconfig-script test_16801_g6 /etc/sentinel/client-reconfig.sh

sentinel monitor test_16801_g7 192.168.11.11 6807 2
sentinel down-after-milliseconds test_16801_g7 100000
sentinel failover-timeout test_16801_g7 60000
sentinel client-reconfig-script test_16801_g7 /etc/sentinel/client-reconfig.sh

sentinel monitor test_16801_g8 192.168.11.11 6808 2
sentinel down-after-milliseconds test_16801_g8 100000
sentinel failover-timeout test_16801_g8 60000
sentinel client-reconfig-script test_16801_g8 /etc/sentinel/client-reconfig.sh

sentinel monitor test_16801_g9 192.168.11.11 6809 2
sentinel down-after-milliseconds test_16801_g9 100000
sentinel failover-timeout test_16801_g9 60000
sentinel client-reconfig-script test_16801_g9 /etc/sentinel/client-reconfig.sh

sentinel monitor test_16801_g10 192.168.11.11 6810 2
sentinel down-after-milliseconds test_16801_g10 100000
sentinel failover-timeout test_16801_g10 60000
sentinel client-reconfig-script test_16801_g10 /etc/sentinel/client-reconfig.sh

sentinel monitor test_16801_g11 192.168.11.11 6811 2
sentinel down-after-milliseconds test_16801_g11 100000
sentinel failover-timeout test_16801_g11 60000
sentinel client-reconfig-script test_16801_g11 /etc/sentinel/client-reconfig.sh

sentinel monitor test_16801_g12 192.168.11.11 6812 2
sentinel down-after-milliseconds test_16801_g12 100000
sentinel failover-timeout test_16801_g12 60000
sentinel client-reconfig-script test_16801_g12 /etc/sentinel/client-reconfig.sh

sentinel monitor test_16801_g13 192.168.11.11 6813 2
sentinel down-after-milliseconds test_16801_g13 100000
sentinel failover-timeout test_16801_g13 60000
sentinel client-reconfig-script test_16801_g13 /etc/sentinel/client-reconfig.sh

sentinel monitor test_16801_g14 192.168.11.11 6814 2
sentinel down-after-milliseconds test_16801_g14 100000
sentinel failover-timeout test_16801_g14 60000
sentinel client-reconfig-script test_16801_g14 /etc/sentinel/client-reconfig.sh

sentinel monitor test_16801_g15 192.168.11.11 6815 2
sentinel down-after-milliseconds test_16801_g15 100000
sentinel failover-timeout test_16801_g15 60000
sentinel client-reconfig-script test_16801_g15 /etc/sentinel/client-reconfig.sh

sentinel monitor test_16801_g16 192.168.11.11 6816 2
sentinel down-after-milliseconds test_16801_g16 100000
sentinel failover-timeout test_16801_g16 60000
sentinel client-reconfig-script test_16801_g16 /etc/sentinel/client-reconfig.sh

sentinel monitor test_16801_g17 192.168.11.11 6817 2
sentinel down-after-milliseconds test_16801_g17 100000
sentinel failover-timeout test_16801_g17 60000
sentinel client-reconfig-script test_16801_g17 /etc/sentinel/client-reconfig.sh

sentinel monitor test_16801_g18 192.168.11.11 6818 2
sentinel down-after-milliseconds test_16801_g18 100000
sentinel failover-timeout test_16801_g18 60000
sentinel client-reconfig-script test_16801_g18 /etc/sentinel/client-reconfig.sh

sentinel monitor test_16801_g19 192.168.11.11 6819 2
sentinel down-after-milliseconds test_16801_g19 100000
sentinel failover-timeout test_16801_g19 60000
sentinel client-reconfig-script test_16801_g19 /etc/sentinel/client-reconfig.sh

sentinel monitor test_16801_g20 192.168.11.11 6820 2
sentinel down-after-milliseconds test_16801_g20 100000
sentinel failover-timeout test_16801_g20 60000
sentinel client-reconfig-script test_16801_g20 /etc/sentinel/client-reconfig.sh

sentinel monitor test_16801_g21 192.168.11.12 6801 2
sentinel down-after-milliseconds test_16801_g21 100000
sentinel failover-timeout test_16801_g21 60000
sentinel client-reconfig-script test_16801_g21 /etc/sentinel/client-reconfig.sh

sentinel monitor test_16801_g22 192.168.11.12 6802 2
sentinel down-after-milliseconds test_16801_g22 100000
sentinel failover-timeout test_16801_g22 60000
sentinel client-reconfig-script test_16801_g22 /etc/sentinel/client-reconfig.sh

sentinel monitor test_16801_g23 192.168.11.12 6803 2
sentinel down-after-milliseconds test_16801_g23 100000
sentinel failover-timeout test_16801_g23 60000
sentinel client-reconfig-script test_16801_g23 /etc/sentinel/client-reconfig.sh

sentinel monitor test_16801_g24 192.168.11.12 6804 2
sentinel down-after-milliseconds test_16801_g24 100000
sentinel failover-timeout test_16801_g24 60000
sentinel client-reconfig-script test_16801_g24 /etc/sentinel/client-reconfig.sh

sentinel monitor test_16801_g25 192.168.11.12 6805 2
sentinel down-after-milliseconds test_16801_g25 100000
sentinel failover-timeout test_16801_g25 60000
sentinel client-reconfig-script test_16801_g25 /etc/sentinel/client-reconfig.sh

sentinel monitor test_16801_g26 192.168.11.12 6806 2
sentinel down-after-milliseconds test_16801_g26 100000
sentinel failover-timeout test_16801_g26 60000
sentinel client-reconfig-script test_16801_g26 /etc/sentinel/client-reconfig.sh

sentinel monitor test_16801_g27 192.168.11.12 6807 2
sentinel down-after-milliseconds test_16801_g27 100000
sentinel failover-timeout test_16801_g27 60000
sentinel client-reconfig-script test_16801_g27 /etc/sentinel/client-reconfig.sh

sentinel monitor test_16801_g28 192.168.11.12 6808 2
sentinel down-after-milliseconds test_16801_g28 100000
sentinel failover-timeout test_16801_g28 60000
sentinel client-reconfig-script test_16801_g28 /etc/sentinel/client-reconfig.sh

sentinel monitor test_16801_g29 192.168.11.12 6809 2
sentinel down-after-milliseconds test_16801_g29 100000
sentinel failover-timeout test_16801_g29 60000
sentinel client-reconfig-script test_16801_g29 /etc/sentinel/client-reconfig.sh

sentinel monitor test_16801_g30 192.168.11.12 6810 2
sentinel down-after-milliseconds test_16801_g30 100000
sentinel failover-timeout test_16801_g30 60000
sentinel client-reconfig-script test_16801_g30 /etc/sentinel/client-reconfig.sh

sentinel monitor test_16801_g31 192.168.11.12 6811 2
sentinel down-after-milliseconds test_16801_g31 100000
sentinel failover-timeout test_16801_g31 60000
sentinel client-reconfig-script test_16801_g31 /etc/sentinel/client-reconfig.sh

sentinel monitor test_16801_g32 192.168.11.12 6812 2
sentinel down-after-milliseconds test_16801_g32 100000
sentinel failover-timeout test_16801_g32 60000
sentinel client-reconfig-script test_16801_g32 /etc/sentinel/client-reconfig.sh

sentinel monitor test_16801_g33 192.168.11.12 6813 2
sentinel down-after-milliseconds test_16801_g33 100000
sentinel failover-timeout test_16801_g33 60000
sentinel client-reconfig-script test_16801_g33 /etc/sentinel/client-reconfig.sh

sentinel monitor test_16801_g34 192.168.11.12 6814 2
sentinel down-after-milliseconds test_16801_g34 100000
sentinel failover-timeout test_16801_g34 60000
sentinel client-reconfig-script test_16801_g34 /etc/sentinel/client-reconfig.sh

sentinel monitor test_16801_g35 192.168.11.12 6815 2
sentinel down-after-milliseconds test_16801_g35 100000
sentinel failover-timeout test_16801_g35 60000
sentinel client-reconfig-script test_16801_g35 /etc/sentinel/client-reconfig.sh

sentinel monitor test_16801_g36 192.168.11.12 6816 2
sentinel down-after-milliseconds test_16801_g36 100000
sentinel failover-timeout test_16801_g36 60000
sentinel client-reconfig-script test_16801_g36 /etc/sentinel/client-reconfig.sh

sentinel monitor test_16801_g37 192.168.11.12 6817 2
sentinel down-after-milliseconds test_16801_g37 100000
sentinel failover-timeout test_16801_g37 60000
sentinel client-reconfig-script test_16801_g37 /etc/sentinel/client-reconfig.sh

sentinel monitor test_16801_g38 192.168.11.12 6818 2
sentinel down-after-milliseconds test_16801_g38 100000
sentinel failover-timeout test_16801_g38 60000
sentinel client-reconfig-script test_16801_g38 /etc/sentinel/client-reconfig.sh

sentinel monitor test_16801_g39 192.168.11.12 6819 2
sentinel down-after-milliseconds test_16801_g39 100000
sentinel failover-timeout test_16801_g39 60000
sentinel client-reconfig-script test_16801_g39 /etc/sentinel/client-reconfig.sh

sentinel monitor test_16801_g40 192.168.11.12 6820 2
sentinel down-after-milliseconds test_16801_g40 100000
sentinel failover-timeout test_16801_g40 60000
sentinel client-reconfig-script test_16801_g40 /etc/sentinel/client-reconfig.sh

sentinel monitor test_16801_g41 192.168.11.13 6801 2
sentinel down-after-milliseconds test_16801_g41 100000
sentinel failover-timeout test_16801_g41 60000
sentinel client-reconfig-script test_16801_g41 /etc/sentinel/client-reconfig.sh

sentinel monitor test_16801_g42 192.168.11.13 6802 2
sentinel down-after-milliseconds test_16801_g42 100000
sentinel failover-timeout test_16801_g42 60000
sentinel client-reconfig-script test_16801_g42 /etc/sentinel/client-reconfig.sh

sentinel monitor test_16801_g43 192.168.11.13 6803 2
sentinel down-after-milliseconds test_16801_g43 100000
sentinel failover-timeout test_16801_g43 60000
sentinel client-reconfig-script test_16801_g43 /etc/sentinel/client-reconfig.sh

sentinel monitor test_16801_g44 192.168.11.13 6804 2
sentinel down-after-milliseconds test_16801_g44 100000
sentinel failover-timeout test_16801_g44 60000
sentinel client-reconfig-script test_16801_g44 /etc/sentinel/client-reconfig.sh

sentinel monitor test_16801_g45 192.168.11.13 6805 2
sentinel down-after-milliseconds test_16801_g45 100000
sentinel failover-timeout test_16801_g45 60000
sentinel client-reconfig-script test_16801_g45 /etc/sentinel/client-reconfig.sh

sentinel monitor test_16801_g46 192.168.11.13 6806 2
sentinel down-after-milliseconds test_16801_g46 100000
sentinel failover-timeout test_16801_g46 60000
sentinel client-reconfig-script test_16801_g46 /etc/sentinel/client-reconfig.sh

sentinel monitor test_16801_g47 192.168.11.13 6807 2
sentinel down-after-milliseconds test_16801_g47 100000
sentinel failover-timeout test_16801_g47 60000
sentinel client-reconfig-script test_16801_g47 /etc/sentinel/client-reconfig.sh

sentinel monitor test_16801_g48 192.168.11.13 6808 2
sentinel down-after-milliseconds test_16801_g48 100000
sentinel failover-timeout test_16801_g48 60000
sentinel client-reconfig-script test_16801_g48 /etc/sentinel/client-reconfig.sh

sentinel monitor test_16801_g49 192.168.11.13 6809 2
sentinel down-after-milliseconds test_16801_g49 100000
sentinel failover-timeout test_16801_g49 60000
sentinel client-reconfig-script test_16801_g49 /etc/sentinel/client-reconfig.sh

sentinel monitor test_16801_g50 192.168.11.13 6810 2
sentinel down-after-milliseconds test_16801_g50 100000
sentinel failover-timeout test_16801_g50 60000
sentinel client-reconfig-script test_16801_g50 /etc/sentinel/client-reconfig.sh

sentinel monitor test_16801_g51 192.168.11.13 6811 2
sentinel down-after-milliseconds test_16801_g51 100000
sentinel failover-timeout test_16801_g51 60000
sentinel client-reconfig-script test_16801_g51 /etc/sentinel/client-reconfig.sh

sentinel monitor test_16801_g52 192.168.11.13 6812 2
sentinel down-after-milliseconds test_16801_g52 100000
sentinel failover-timeout test_16801_g52 60000
sentinel client-reconfig-script test_16801_g52 /etc/sentinel/client-reconfig.sh

sentinel monitor test_16801_g53 192.168.11.13 6813 2
sentinel down-after-milliseconds test_16801_g53 100000
sentinel failover-timeout test_16801_g53 60000
sentinel client-reconfig-script test_16801_g53 /etc/sentinel/client-reconfig.sh

sentinel monitor test_16801_g54 192.168.11.13 6814 2
sentinel down-after-milliseconds test_16801_g54 100000
sentinel failover-timeout test_16801_g54 60000
sentinel client-reconfig-script test_16801_g54 /etc/sentinel/client-reconfig.sh

sentinel monitor test_16801_g55 192.168.11.13 6815 2
sentinel down-after-milliseconds test_16801_g55 100000
sentinel failover-timeout test_16801_g55 60000
sentinel client-reconfig-script test_16801_g55 /etc/sentinel/client-reconfig.sh

sentinel monitor test_16801_g56 192.168.11.13 6816 2
sentinel down-after-milliseconds test_16801_g56 100000
sentinel failover-timeout test_16801_g56 60000
sentinel client-reconfig-script test_16801_g56 /etc/sentinel/client-reconfig.sh

sentinel monitor test_16801_g57 192.168.11.13 6817 2
sentinel down-after-milliseconds test_16801_g57 100000
sentinel failover-timeout test_16801_g57 60000
sentinel client-reconfig-script test_16801_g57 /etc/sentinel/client-reconfig.sh

sentinel monitor test_16801_g58 192.168.11.13 6818 2
sentinel down-after-milliseconds test_16801_g58 100000
sentinel failover-timeout test_16801_g58 60000
sentinel client-reconfig-script test_16801_g58 /etc/sentinel/client-reconfig.sh

sentinel monitor test_16801_g59 192.168.11.13 6819 2
sentinel down-after-milliseconds test_16801_g59 100000
sentinel failover-timeout test_16801_g59 60000
sentinel client-reconfig-script test_16801_g59 /etc/sentinel/client-reconfig.sh

sentinel monitor test_16801_g60 192.168.11.13 6820 2
sentinel down-after-milliseconds test_16801_g60 100000
sentinel failover-timeout test_16801_g60 60000
sentinel client-reconfig-script test_16801_g60 /etc/sentinel/client-reconfig.sh


sentinel monitor test_16801_g68 192.168.11.14 6801 2
sentinel down-after-milliseconds test_16801_g68 100000
sentinel failover-timeout test_16801_g68 60000
sentinel client-reconfig-script test_16801_g68 /etc/sentinel/client-reconfig.sh

sentinel monitor test_16801_g62 192.168.11.14 6802 2
sentinel down-after-milliseconds test_16801_g62 100000
sentinel failover-timeout test_16801_g62 60000
sentinel client-reconfig-script test_16801_g62 /etc/sentinel/client-reconfig.sh

sentinel monitor test_16801_g63 192.168.11.14 6803 2
sentinel down-after-milliseconds test_16801_g63 100000
sentinel failover-timeout test_16801_g63 60000
sentinel client-reconfig-script test_16801_g63 /etc/sentinel/client-reconfig.sh

sentinel monitor test_16801_g64 192.168.11.14 6804 2
sentinel down-after-milliseconds test_16801_g64 100000
sentinel failover-timeout test_16801_g64 60000
sentinel client-reconfig-script test_16801_g64 /etc/sentinel/client-reconfig.sh

sentinel monitor test_16801_g65 192.168.11.14 6805 2
sentinel down-after-milliseconds test_16801_g65 100000
sentinel failover-timeout test_16801_g65 60000
sentinel client-reconfig-script test_16801_g65 /etc/sentinel/client-reconfig.sh

sentinel monitor test_16801_g66 192.168.11.14 6806 2
sentinel down-after-milliseconds test_16801_g66 100000
sentinel failover-timeout test_16801_g66 60000
sentinel client-reconfig-script test_16801_g66 /etc/sentinel/client-reconfig.sh

sentinel monitor test_16801_g67 192.168.11.14 6807 2
sentinel down-after-milliseconds test_16801_g67 100000
sentinel failover-timeout test_16801_g67 60000
sentinel client-reconfig-script test_16801_g67 /etc/sentinel/client-reconfig.sh

sentinel monitor test_16801_g68 192.168.11.14 6808 2
sentinel down-after-milliseconds test_16801_g68 100000
sentinel failover-timeout test_16801_g68 60000
sentinel client-reconfig-script test_16801_g68 /etc/sentinel/client-reconfig.sh

sentinel monitor test_16801_g69 192.168.11.14 6809 2
sentinel down-after-milliseconds test_16801_g69 100000
sentinel failover-timeout test_16801_g69 60000
sentinel client-reconfig-script test_16801_g69 /etc/sentinel/client-reconfig.sh

sentinel monitor test_16801_g70 192.168.11.14 6810 2
sentinel down-after-milliseconds test_16801_g70 100000
sentinel failover-timeout test_16801_g70 60000
sentinel client-reconfig-script test_16801_g70 /etc/sentinel/client-reconfig.sh

sentinel monitor test_16801_g71 192.168.11.14 6811 2
sentinel down-after-milliseconds test_16801_g71 100000
sentinel failover-timeout test_16801_g71 60000
sentinel client-reconfig-script test_16801_g71 /etc/sentinel/client-reconfig.sh

sentinel monitor test_16801_g72 192.168.11.14 6812 2
sentinel down-after-milliseconds test_16801_g72 100000
sentinel failover-timeout test_16801_g72 60000
sentinel client-reconfig-script test_16801_g72 /etc/sentinel/client-reconfig.sh

sentinel monitor test_16801_g73 192.168.11.14 6813 2
sentinel down-after-milliseconds test_16801_g73 100000
sentinel failover-timeout test_16801_g73 60000
sentinel client-reconfig-script test_16801_g73 /etc/sentinel/client-reconfig.sh

sentinel monitor test_16801_g74 192.168.11.14 6814 2
sentinel down-after-milliseconds test_16801_g74 100000
sentinel failover-timeout test_16801_g74 60000
sentinel client-reconfig-script test_16801_g74 /etc/sentinel/client-reconfig.sh

sentinel monitor test_16801_g75 192.168.11.14 6815 2
sentinel down-after-milliseconds test_16801_g75 100000
sentinel failover-timeout test_16801_g75 60000
sentinel client-reconfig-script test_16801_g75 /etc/sentinel/client-reconfig.sh

sentinel monitor test_16801_g76 192.168.11.14 6816 2
sentinel down-after-milliseconds test_16801_g76 100000
sentinel failover-timeout test_16801_g76 60000
sentinel client-reconfig-script test_16801_g76 /etc/sentinel/client-reconfig.sh

sentinel monitor test_16801_g77 192.168.11.14 6817 2
sentinel down-after-milliseconds test_16801_g77 100000
sentinel failover-timeout test_16801_g77 60000
sentinel client-reconfig-script test_16801_g77 /etc/sentinel/client-reconfig.sh

sentinel monitor test_16801_g78 192.168.11.14 6818 2
sentinel down-after-milliseconds test_16801_g78 100000
sentinel failover-timeout test_16801_g78 60000
sentinel client-reconfig-script test_16801_g78 /etc/sentinel/client-reconfig.sh

sentinel monitor test_16801_g79 192.168.11.14 6819 2
sentinel down-after-milliseconds test_16801_g79 100000
sentinel failover-timeout test_16801_g79 60000
sentinel client-reconfig-script test_16801_g79 /etc/sentinel/client-reconfig.sh

sentinel monitor test_16801_g80 192.168.11.14 6820 2
sentinel down-after-milliseconds test_16801_g80 100000
sentinel failover-timeout test_16801_g80 60000
sentinel client-reconfig-script test_16801_g80 /etc/sentinel/client-reconfig.sh

5.2. 编辑Redis主从切换执行脚本

# vim /etc/sentinel/client-reconfig.sh

#!/bin/sh
###  <master-name> <role> <state> <from-ip> <from-port> <to-ip> <to-port>
#
monitor_name="$1"
master_old_ip="$4"
master_old_port="$5"
master_new_ip="$6"
master_new_port="$7"

twemproxy_base="/test/nutcracker/nutcracker/bin"
twemproxy_bin="${twemproxy_base}/nutcracker"
twemproxy_conf="${twemproxy_base}/conf/nutcracker.yml"

time=`date '+%Y-%m-%d %H:%M:%S'`
echo "[${time}] ${master_old_ip}:${master_old_port} start failover!" >> /var/log/redis_failover.log

# 将新的master 端口和ip 替换掉 twemproxy 配置文件中旧的master 信息
flock -e -w 10 ${twemproxy_conf} -c "sed -i \"s/${master_old_ip}:${master_old_port}/${master_new_ip}:${master_new_port}/\" ${twemproxy_conf}"

# 如果修改成功，重启twemproxy，否则记录失败日志
if [ $? -eq 0 ]; then
	killall spawn_twemproxy
    killall nutcracker
    sleep 1
    cd ${twemproxy_base}
    bash ./start.sh
    time=`date '+%Y-%m-%d %H:%M:%S'`
    echo "[${time}] ${master_old_ip}:${master_old_port} failover to ${master_new_ip}:${master_new_port} ok!" >> /var/log/redis_failover.log
else
    time=`date '+%Y-%m-%d %H:%M:%S'`
    echo "[${time}] ${master_old_ip}:${master_old_port} failover to ${master_new_ip}:${master_new_port} failed!" >> /var/log/redis_failover.log
fi



为Redis主从切换脚本赋予执行权限：
# chmod +x /etc/sentinel/client-reconfig.sh

5.3. 启动哨兵

启动哨兵：
# cd /etc/sentinel/
# redis-sentinel /etc/sentinel/sentinel.conf

查看进程：
# ps -ef|grep redis-sentinel|grep -v grep

5.4. 查看哨兵集群

# redis-cli -p 26880 info sentinel

# Sentinel
sentinel_masters:80
sentinel_tilt:0
sentinel_running_scripts:0
sentinel_scripts_queue_length:0
sentinel_simulate_failure_flags:0
master0:name=test_16801_g56,status=ok,address=192.168.11.13:6816,slaves=1,sentinels=4
master1:name=test_16801_g27,status=ok,address=192.168.11.12:6807,slaves=1,sentinels=4
master2:name=test_16801_g75,status=ok,address=192.168.11.14:6815,slaves=1,sentinels=4
master3:name=test_16801_g49,status=ok,address=192.168.11.13:6809,slaves=1,sentinels=4
master4:name=test_16801_g22,status=ok,address=192.168.11.12:6802,slaves=1,sentinels=4
master5:name=test_16801_g18,status=ok,address=192.168.11.11:6818,slaves=1,sentinels=4
master6:name=test_16801_g41,status=ok,address=192.168.11.13:6801,slaves=1,sentinels=4
master7:name=test_16801_g50,status=ok,address=192.168.11.13:6810,slaves=1,sentinels=4
master8:name=test_16801_g47,status=ok,address=192.168.11.13:6807,slaves=1,sentinels=4
master9:name=test_16801_g21,status=ok,address=192.168.11.12:6801,slaves=1,sentinels=4
master10:name=test_16801_g62,status=ok,address=192.168.11.14:6802,slaves=1,sentinels=4
master11:name=test_16801_g64,status=ok,address=192.168.11.14:6804,slaves=1,sentinels=4
master12:name=test_16801_g5,status=ok,address=192.168.11.11:6805,slaves=1,sentinels=4
master13:name=test_16801_g15,status=ok,address=192.168.11.11:6815,slaves=1,sentinels=4
master14:name=test_16801_g12,status=ok,address=192.168.11.11:6812,slaves=1,sentinels=4
master15:name=test_16801_g11,status=ok,address=192.168.11.11:6811,slaves=1,sentinels=4
master16:name=test_16801_g16,status=ok,address=192.168.11.11:6816,slaves=1,sentinels=4
master17:name=test_16801_g48,status=ok,address=192.168.11.13:6808,slaves=1,sentinels=4
master18:name=test_16801_g14,status=ok,address=192.168.11.11:6814,slaves=1,sentinels=4
master19:name=test_16801_g13,status=ok,address=192.168.11.11:6813,slaves=1,sentinels=4
master20:name=test_16801_g80,status=ok,address=192.168.11.14:6820,slaves=1,sentinels=4
master21:name=test_16801_g58,status=ok,address=192.168.11.13:6818,slaves=1,sentinels=4
master22:name=test_16801_g67,status=ok,address=192.168.11.14:6807,slaves=1,sentinels=4
master23:name=test_16801_g45,status=ok,address=192.168.11.13:6805,slaves=1,sentinels=4
master24:name=test_16801_g37,status=ok,address=192.168.11.12:6817,slaves=1,sentinels=4
master25:name=test_16801_g34,status=ok,address=192.168.11.12:6814,slaves=1,sentinels=4
master26:name=test_16801_g17,status=ok,address=192.168.11.11:6817,slaves=1,sentinels=4
master27:name=test_16801_g42,status=ok,address=192.168.11.13:6802,slaves=1,sentinels=4
master28:name=test_16801_g31,status=ok,address=192.168.11.12:6811,slaves=1,sentinels=4
master29:name=test_16801_g79,status=ok,address=192.168.11.14:6819,slaves=1,sentinels=4
master30:name=test_16801_g54,status=ok,address=192.168.11.13:6814,slaves=1,sentinels=4
master31:name=test_16801_g53,status=ok,address=192.168.11.13:6813,slaves=1,sentinels=4
master32:name=test_16801_g46,status=ok,address=192.168.11.13:6806,slaves=1,sentinels=4
master33:name=test_16801_g71,status=ok,address=192.168.11.14:6811,slaves=1,sentinels=4
master34:name=test_16801_g1,status=ok,address=192.168.11.11:6801,slaves=1,sentinels=4
master35:name=test_16801_g3,status=ok,address=192.168.11.11:6803,slaves=1,sentinels=4
master36:name=test_16801_g77,status=ok,address=192.168.11.14:6817,slaves=1,sentinels=4
master37:name=test_16801_g74,status=ok,address=192.168.11.14:6814,slaves=1,sentinels=4
master38:name=test_16801_g33,status=ok,address=192.168.11.12:6813,slaves=1,sentinels=4
master39:name=test_16801_g59,status=ok,address=192.168.11.13:6819,slaves=1,sentinels=4
master40:name=test_16801_g23,status=ok,address=192.168.11.12:6803,slaves=1,sentinels=4
master41:name=test_16801_g68,status=ok,address=192.168.11.14:6808,slaves=1,sentinels=4
master42:name=test_16801_g9,status=ok,address=192.168.11.11:6809,slaves=1,sentinels=4
master43:name=test_16801_g25,status=ok,address=192.168.11.12:6805,slaves=1,sentinels=4
master44:name=test_16801_g36,status=ok,address=192.168.11.12:6816,slaves=1,sentinels=4
master45:name=test_16801_g26,status=ok,address=192.168.11.12:6806,slaves=1,sentinels=4
master46:name=test_16801_g66,status=ok,address=192.168.11.14:6806,slaves=1,sentinels=4
master47:name=test_16801_g78,status=ok,address=192.168.11.14:6818,slaves=1,sentinels=4
master48:name=test_16801_g20,status=ok,address=192.168.11.11:6820,slaves=1,sentinels=4
master49:name=test_16801_g52,status=ok,address=192.168.11.13:6812,slaves=1,sentinels=4
master50:name=test_16801_g24,status=ok,address=192.168.11.12:6804,slaves=1,sentinels=4
master51:name=test_16801_g51,status=ok,address=192.168.11.13:6811,slaves=1,sentinels=4
master52:name=test_16801_g60,status=ok,address=192.168.11.13:6820,slaves=1,sentinels=4
master53:name=test_16801_g35,status=ok,address=192.168.11.12:6815,slaves=1,sentinels=4
master54:name=test_16801_g10,status=ok,address=192.168.11.11:6810,slaves=1,sentinels=4
master55:name=test_16801_g70,status=ok,address=192.168.11.14:6810,slaves=1,sentinels=4
master56:name=test_16801_g63,status=ok,address=192.168.11.14:6803,slaves=1,sentinels=4
master57:name=test_16801_g76,status=ok,address=192.168.11.14:6816,slaves=1,sentinels=4
master58:name=test_16801_g30,status=ok,address=192.168.11.12:6810,slaves=1,sentinels=4
master59:name=test_16801_g6,status=ok,address=192.168.11.11:6806,slaves=1,sentinels=4
master60:name=test_16801_g38,status=ok,address=192.168.11.12:6818,slaves=1,sentinels=4
master68:name=test_16801_g19,status=ok,address=192.168.11.11:6819,slaves=1,sentinels=4
master62:name=test_16801_g68,status=ok,address=192.168.11.14:6801,slaves=1,sentinels=4
master63:name=test_16801_g39,status=ok,address=192.168.11.12:6819,slaves=1,sentinels=4
master64:name=test_16801_g2,status=ok,address=192.168.11.11:6802,slaves=1,sentinels=4
master65:name=test_16801_g40,status=ok,address=192.168.11.12:6820,slaves=1,sentinels=4
master66:name=test_16801_g55,status=ok,address=192.168.11.13:6815,slaves=1,sentinels=4
master67:name=test_16801_g28,status=ok,address=192.168.11.12:6808,slaves=1,sentinels=4
master68:name=test_16801_g57,status=ok,address=192.168.11.13:6817,slaves=1,sentinels=4
master69:name=test_16801_g43,status=ok,address=192.168.11.13:6803,slaves=1,sentinels=4
master70:name=test_16801_g65,status=ok,address=192.168.11.14:6805,slaves=1,sentinels=4
master71:name=test_16801_g44,status=ok,address=192.168.11.13:6804,slaves=1,sentinels=4
master72:name=test_16801_g73,status=ok,address=192.168.11.14:6813,slaves=1,sentinels=4
master73:name=test_16801_g32,status=ok,address=192.168.11.12:6812,slaves=1,sentinels=4
master74:name=test_16801_g7,status=ok,address=192.168.11.11:6807,slaves=1,sentinels=4
master75:name=test_16801_g72,status=ok,address=192.168.11.14:6812,slaves=1,sentinels=4
master76:name=test_16801_g29,status=ok,address=192.168.11.12:6809,slaves=1,sentinels=4
master77:name=test_16801_g8,status=ok,address=192.168.11.11:6808,slaves=1,sentinels=4
master78:name=test_16801_g4,status=ok,address=192.168.11.11:6804,slaves=1,sentinels=4
master79:name=test_16801_g69,status=ok,address=192.168.11.14:6809,slaves=1,sentinels=4

6. 验证twemproxy集群

6.1. 验证twemproxy集群的可用性

# redis-cli -p 16801 set testTwemproxy Twemproxy
OK
# redis-cli -p 16801 get testTwemproxy
"Twemproxy"

验证成功，可正常读写。

6.2. 验证Redis主库宕机后twemproxy集群可正常使用

6.2.1. 查看 Redis 复制集群

如下，192.168.11.11:6801和192.168.11.14:6821 为一对Redis主从：
# redis-cli -h 192.168.11.11 -p 6801 role

1) "master"
2) (integer) 28343
3) 1) 1) "192.168.11.14"
      2) "6821"
      3) "27905"



# redis-cli -h 192.168.11.14 -p 6821 role

1) "slave"
2) "192.168.11.11"
3) (integer) 6801
4) "connected"
5) (integer) 31827

6.2.2. 停止复制集群的192.168.11.11:6801主库节点

/* 192.168.11.11:6801 */
停止复制集群的192.168.11.11:6801主库节点：
# ps -ef| grep redis-server|grep 6801| grep -v grep
# redis-cli -h 192.168.11.11 -p 6801 shutdown

确认节点已经停止：
# ps -ef| grep redis-server|grep 6801| grep -v grep

6.2.3. 查看监控日志

查看哨兵的监控日志：

# tail -f /var/log/sentinel.log

104017:X 06 Nov 12:45:18.118 * +sentinel sentinel 3cc080c86cd407b38ec8797e2aef12c19b771bc2 192.168.11.11 26880 @ test_16801_g29 192.168.11.12 6809
104017:X 06 Nov 12:45:18.121 * +sentinel sentinel 3cc080c86cd407b38ec8797e2aef12c19b771bc2 192.168.11.11 26880 @ test_16801_g55 192.168.11.13 6815
104017:X 06 Nov 12:47:43.820 # +sdown master test_16801_g1 192.168.11.11 6801
104017:X 06 Nov 12:47:43.868 # +new-epoch 1
104017:X 06 Nov 12:47:43.865 # +vote-for-leader ea5e6485ee7c4b6b0c52da29001c2fc868389712 1
104017:X 06 Nov 12:47:43.910 # +odown master test_16801_g1 192.168.11.11 6801 #quorum 4/2
104017:X 06 Nov 12:47:43.910 # Next failover delay: I will not start a failover before Fri Nov  6 12:49:43 2020
104017:X 06 Nov 12:47:44.586 # +config-update-from sentinel ea5e6485ee7c4b6b0c52da29001c2fc868389712 192.168.11.14 26880 @ test_16801_g1 192.168.11.11 6801
104017:X 06 Nov 12:47:44.586 # +switch-master test_16801_g1 192.168.11.11 6801 192.168.11.14 6821
104017:X 06 Nov 12:47:44.586 * +slave slave 192.168.11.11:6801 192.168.11.11 6801 @ test_16801_g1 192.168.11.14 6821
104017:X 06 Nov 12:47:54.660 # +sdown slave 192.168.11.11:6801 192.168.11.11 6801 @ test_16801_g1 192.168.11.14 6821

6.2.4. 查看master test_16801_g1 的信息

从哨兵配置文件中可以看出，监控的master test_16801_g1发生了主从切换：
# grep -E 'sentinel known-slave test_16801_g1|sentinel monitor test_16801_g1' /etc/sentinel/sentinel.conf 
sentinel monitor test_16801_g1 192.168.11.14 6821 2
sentinel known-slave test_16801_g1 192.168.11.11 6801


从哨兵查看mater test_16801_g1的信息：
# redis-cli -p 26880

127.0.0.1:26880> sentinel master test_16801_g1
 1) "name"
 2) "test_16801_g1"
 3) "ip"
 4) "192.168.11.14"
 5) "port"
 6) "6821"
 7) "runid"
 8) "0376a9459c562a9443d801023535ef436a1a4552"
 9) "flags"
10) "master"
11) "link-pending-commands"
12) "0"
13) "link-refcount"
14) "1"
15) "last-ping-sent"
16) "0"
17) "last-ok-ping-reply"
18) "479"
19) "last-ping-reply"
20) "479"
21) "down-after-milliseconds"
22) "10000"
23) "info-refresh"
24) "5767"
25) "role-reported"
26) "master"
27) "role-reported-time"
28) "66669"
29) "config-epoch"
30) "1"
31) "num-slaves"
32) "1"
33) "num-other-sentinels"
34) "3"
35) "quorum"
36) "2"
37) "failover-timeout"
38) "60000"
39) "parallel-syncs"
40) "1"
41) "client-reconfig-script"
42) "/etc/sentinel/client-reconfig.sh"

6.2.5. 查看twemproxy的配置信息，确认切换脚本成功切换

Redis主从切换，引发脚本/etc/sentinel/client-reconfig.sh更新了/nutcracker.yml配置文件并重启：
# cat /test/nutcracker/nutcracker/bin/conf/nutcracker.yml | grep 192.168.11.11 | grep 6801

# cat /test/nutcracker/nutcracker/bin/conf/nutcracker.yml | grep 192.168.11.14 | grep 6821
- 192.168.11.14:6821:1 server1

6.2.6. 启动宕机的Redis节点

/* 192.168.11.11:6801 */
启动宕机的Redis节点：
# cd /data/redis/16801/etc/
# redis-server /data/redis/16801/etc/6801.conf

查看进程：
# ps -ef| grep redis-server|grep 6801|grep -v grep


查看Redis节点角色：
# redis-cli -h 192.168.11.11 -p 6801 role

1) "slave"
2) "192.168.11.14"
3) (integer) 6821
4) "connected"
5) (integer) 86581

# redis-cli -h 192.168.11.14 -p 6821 role

1) "master"
2) (integer) 89668
3) 1) 1) "192.168.11.11"
      2) "6801"
      3) "89515"

6.2.7. 手动进行主从切换，恢复原主从复制关系

# redis-cli -p 26880
127.0.0.1:26880> sentinel failover test_16801_g1
OK
127.0.0.1:26880> sentinel master test_16801_g1
 1) "name"
 2) "test_16801_g1"
 3) "ip"
 4) "192.168.11.11"
 5) "port"
 6) "6801"
 7) "runid"
 8) ""
 9) "flags"
10) "master"
11) "link-pending-commands"
12) "0"
13) "link-refcount"
14) "1"
15) "last-ping-sent"
16) "0"
17) "last-ok-ping-reply"
18) "956"
19) "last-ping-reply"
20) "956"
21) "down-after-milliseconds"
22) "10000"
23) "info-refresh"
24) "5131"
25) "role-reported"
26) "master"
27) "role-reported-time"
28) "2102"
29) "config-epoch"
30) "2"
31) "num-slaves"
32) "1"
33) "num-other-sentinels"
34) "3"
35) "quorum"
36) "2"
37) "failover-timeout"
38) "60000"
39) "parallel-syncs"
40) "1"
41) "client-reconfig-script"
42) "/etc/sentinel/client-reconfig.sh"

查看哨兵配置文件，确认原复制关系已经恢复：
# grep -E 'sentinel known-slave test_16801_g1|sentinel monitor test_16801_g1' /etc/sentinel/sentinel.conf 
sentinel monitor test_16801_g1 192.168.11.11 6801 2
sentinel known-slave test_16801_g1 192.168.11.14 6821



# redis-cli -p 26880

 1) "name"
 2) "test_16801_g1"
 3) "ip"
 4) "192.168.11.11"
 5) "port"
 6) "6801"
 7) "runid"
 8) "645ff55a5590e5e280286790a5939c6b3da5f7bd"
 9) "flags"
10) "master"
11) "link-pending-commands"
12) "0"
13) "link-refcount"
14) "1"
15) "last-ping-sent"
16) "0"
17) "last-ok-ping-reply"
18) "919"
19) "last-ping-reply"
20) "919"
21) "down-after-milliseconds"
22) "10000"
23) "info-refresh"
24) "5867"
25) "role-reported"
26) "master"
27) "role-reported-time"
28) "42627"
29) "config-epoch"
30) "2"
31) "num-slaves"
32) "1"
33) "num-other-sentinels"
34) "3"
35) "quorum"
36) "2"
37) "failover-timeout"
38) "60000"
39) "parallel-syncs"
40) "1"
41) "client-reconfig-script"
42) "/etc/sentinel/client-reconfig.sh"

查看twemproxy，确认nutcracker已经恢复最初的配置并完成了重启：
# cat /test/nutcracker/nutcracker/bin/conf/nutcracker.yml | grep 192.168.11.11 | grep 6801
- 192.168.11.11:6801:1 server1
# cat /test/nutcracker/nutcracker/bin/conf/nutcracker.yml | grep 192.168.11.14 | grep 6821
Zhi@Li
关注
  • 0
    点赞
  • 踩
  • 2
    收藏
    觉得还不错? 一键收藏
  • 打赏
    打赏
  • 0
    评论
twemproxy集群部署
hanyueqi的专栏
03-30 636
1、twemproxy主要用于memcached集群与redis集群的部署，其中定义了不少相关的参数； 2、redis集群部署示例： redis1:   listen: 0.0.0.0:9999   redis: true   hash: fnv1a_64   distribution: ketama   auto_eject_hosts: true   timeout: 40
基于Twemproxy的群集部署方案
zhaokuner的专栏
04-10 7085
概述   本文描述的twemproxy基于nutcracker-0.2.4版本。   twemproxy是memcached 和redis的协议层面的代理，其提供的features如下：   注： twemproxy不会增加redis的性能指标数据，据业界测算，使用twemproxy相比直接使用redis会带来~10%的性能下降。   但是单个redis进程的内存管理能力有限
Twemproxy的部署和配置详解
mischen520的博客
06-29 4549
Twemproxy 概述 Twemproxy（又称为nutcracker）是一个轻量级的Redis和Memcached代理，主要用来减少对后端缓存服务器的连接数。Twemproxy是由Twitter开源出来的缓存服务器集群管理工具，主要用来弥补Redis/Memcached 对集群（cluster）管理的不足。 antirez（Redis作者）写过一篇对twemproxy的介绍，他...
Redis集群之twemproxy
风的脚步
03-28 2192
twemproxy
Redis代理twemproxy安装,配置,使用
07-28 4033
1.简介Twemproxy 也叫 nutcraker。是 Twtter 开源的一个 Redis 和 Memcache 代理服务器，主要用于管理 Redis 和 Memcached 集群，减少与Cache 服务器直接连接的数量。Twemproxy特性： 轻量级、快速 保持长连接 减少了直接与缓存服务器连接的连接数量 使用 pipelining 处理请求和响应 支持代理到多台服务器上 同时支持多个服务
redis数据库集群部署
最新发布
06-24
通过对Redis集群部署的相关知识点的详细讲解，我们可以看到，不同的分片方案各有优劣，选择合适的方案对于构建高效的Redis集群至关重要。在实际部署过程中，还需要考虑具体的应用场景和技术背景，以达到最佳的性能...
Redis集群方案
09-08
为了解决这些挑战，可以考虑采用社区成熟的Redis集群解决方案，如Twemproxy（Twitter的Redis代理）或Codis（阿里巴巴开源的Redis集群管理工具），它们提供了更完善的分布式和故障恢复机制，减轻了开发和运维的压力。...
redis集群在腾讯游戏的应用及演进
12-08
Redis在腾讯游戏的应用经历了一个逐步深化的过程，从最初的直接使用到后来的定制化部署，再到现在的集群管理和在线扩容缩容。这一过程不仅反映了Redis本身的强大功能和技术优势，也体现了腾讯游戏在面对海量用户和...
keepalive+twemproxy+ redis主从安装配置的例子(linux)
11-16
通过以上详述，我们可以看到`Keepalive`、`Twemproxy`和Redis主从配置在构建高效、稳定的Redis集群中的重要角色。每个组件都有其特定的设置和优化点，理解并熟练掌握这些知识对于构建高性能的分布式系统至关重要。
第三章 Redis/SSDB+Twemproxy安装与使用
08-09
Twemproxy支持TCP和UDP协议，具有低延迟、高性能的特点，且配置简单，适合快速部署。它并不处理数据，而是直接转发请求，因此对于复杂的业务逻辑，需要在客户端进行处理。 在安装Redis时，通常需要下载源码，编译并...
Redis集群详解：Twemproxy Codis 和 Redis Cluster，搭建cluster集群，slots（槽），Jedis连接cluster，SpringBoot整合cluster模式
qq120631157的博客
12-25 1186
Redis集群详解：Twemproxy Codis 和 Redis Cluster，搭建cluster集群，slots（槽），Jedis连接cluster，SpringBoot整合cluster模式
Redis（持久化、主从复制、主从切换、twemproxy、redis集群）
qq_42564122的博客
08-29 1279
文章目录Redis Redis redis是一个key-value存储系统。和Memcached类似，它支持存储的value类型相对更多，包括string(字符串)、list(链表)、set(集合)、zset(sorted set --有序集合)和hash（哈希类型）。这些数据类型都支持push/pop、add/remove及取交集并集和差集及更丰富的操作，而且这些操作都是原子性的。在此基础上，redis支持各种不同方式的排序。与memcached一样，为了保证效率，数据都是缓存在内存中。区别的是redis
twemproxy支持的redis命令
IT老兵
07-10 521
https://raw.github.com/twitter/twemproxy/master/notes/redis.md Keys Command Command Supported Format ...
简述redis的sentinel和Twemproxy
米二的博客
02-09 1648
redis, 哨兵 ,Twemproxy ,高并发
twemproxy安装与配置
沁舞
03-14 5137
说到twemproxy就不得不说要一下redis，这是因为twemproxy是为redis服务的，当然这里所说的服务是指代理服务。为什么是这么说得呢？如果你之前有接触过redis的话，你应该很熟悉redis集群。虽然redis集群带来了很多的优势，但还有一个问题需要解决，那就是独立的redis集群之间的关联。我们知道redis有16834个slot(槽)，当我们创建集群并赋予给定范围的槽后，如果因
twemproxy安装访问及同机部署多实例
wkNote
09-28 859
twemproxy是干啥的网上讲的很多，这里就不提了。 项目github地址：https://github.com/twitter/twemproxy/releases twemproxy不支持事务操作 twemproxy支持的命令情况：https://github.com/twitter/twemproxy/blob/master/notes/redis.md 首先安装配置redis，此处略过。...
twemproxy安装与简单应用
daiyudong2020的博客
08-01 1097
twemproxy可用于redis分片代理。 github:https://github.com/twitter/twemproxy 1、从github下载源码包，安装： cd twemproxy autoreconf -fvi ./configure make sudo make install 2、查看帮助手册： nutcracker -v
twemproxy与sentinel整合步骤
甘韦的博客
03-12 535
1.tw-redis-server-a将哨兵的配置文件拷贝到tw-proxy-server-a主机上； scp /usr/data/redis/back/sentinel.conf 192.168.68.170:/usr/local/redis/conf； 2.tw-proxy-server-a通过源代码文件拷贝出哨兵进程的启动项： cp /usr/local/src/redis-3.2.9/sr...
Twemproxy 介绍与使用
zxb_lxy的博客
07-30 518
Twemproxy 介绍与使用 Twemproxy是一种代理分片机制，由Twitter开源。Twemproxy作为代理，可接受来自多个程序的访问，按照路由规则，转发给后台的各个Redis服务器，再原路返回。该方案很好的解决了单个Redis实例承载能力的问题。当然，Twemproxy本身也是单点，需要用Keepalived做高可用方案。通过Twemproxy可以使用多台服务器来水平扩张redis服务，可以有效的避免单点故障问题。虽然使用Twemproxy需要更多的硬件资源和在redis性能有一定的损失（tw
评论
添加红包

请填写红包祝福语或标题

个

红包个数最小为10个

元

红包金额最低5元

当前余额3.43元 前往充值 >
需支付：10.00元
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包

打赏作者

Zhi@Li

你的鼓励将是我创作的最大动力

¥1 ¥2 ¥4 ¥6 ¥10 ¥20
扫码支付：¥1
获取中
扫码支付

您的余额不足，请更换扫码支付或充值

打赏作者

实付元
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明：

1.余额是钱包充值的虚拟货币，按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载，可以购买VIP、付费专栏及课程。

余额充值