Redis + Keepalived 部署手册
1 解压
请将上传或下载的安装包存放在/root/download目录下。
# tar -zxf keepalived-1.2.18.tar.gz
安装依赖
# yum install gcc openssl-devel pcre-devel zlib-devel -y
2 编译&安装
# cd keepalived-1.2.18
# ./configure --prefix=/usr/local/keepalived
# make && make install
# mkdir /etc/keepalived
# cp /usr/local/keepalived/etc/keepalived/keepalived.conf /etc/keepalived/
# cp /usr/local/keepalived/etc/rc.d/init.d/keepalived /etc/init.d/
# cp /usr/local/keepalived/etc/sysconfig/keepalived /etc/sysconfig/
# cp /usr/local/sbin/keepalived /usr/sbin/
如果上面最后的命令提示No such file or directory,改执行下面的命令:
# cp /usr/local/keepalived/sbin/keepalived /usr/local/sbin/
继续命令(如果文件已存在删除后再执行)
# ln -s /usr/local/keepalived/sbin/keepalived /sbin/
设置开机启动
# chkconfig keepalived on
3 配置
在主从两台上的redis配置中都补充上一条:
# vi /etc/redis/6379.conf
masterauth [另一个节点密码],设置后才能相互连通。
新开一个shell窗口,查看网卡地址,然后填在下面的配置文件中比如eth1
# ifconfig
编辑配置文件
# vi /etc/keepalived/keepalived.conf
3.1master
! Configuration File for keepalived
global_defs {
router_id test-04 #主机名
}
vrrp_script chk_redis {
script "/etc/keepalived/redis_check.sh" ## 检测 redis 状态的脚本路径
interval 2 ## 检测时间间隔
weight -10 ## 如果条件成立,权重-10
false 2 #检测连续2次失败才算确定是真失败。会用weight减少优先级(1-255之间)
rise 1 #检测1次成功就算成功。但不修改优先级
}
vrrp_instance VI_1 {
state MASTER
interface eth1
virtual_router_id 55
priority 100
advert_int 1
nopreempt
# authentication {
# auth_type PASS
# auth_pass 1111
# }
track_script {
chk_redis ## 执行 Nginx 监控的服务
}
virtual_ipaddress {
172.168.2.211/24 dev eth1 label eth1:2
}
notify_master "/etc/keepalived/redis_master.sh master"
notify_backup /etc/keepalived/redis_backup.sh
notify_fault /etc/keepalived/redis_fault.sh
notify_stop /etc/keepalived/redis_stop.sh
}
3.2slave
! Configuration File for keepalived
global_defs {
router_id test_05 ## 标识本节点的字条串,通常为 hostname
}
vrrp_script chk_redis {
script "/etc/keepalived/redis_check.sh" ## 检测 nginx 状态的脚本路径
interval 2 ## 检测时间间隔
weight -10 ## 如果条件成立,权重-10
false 2 #检测连续2次失败才算确定是真失败。会用weight减少优先级(1-255之间)
rise 1 #检测1次成功就算成功。但不修改优先级
}
vrrp_instance VI_1 {
state BACKUP
interface eth1
virtual_router_id 51
# 初始化备机值要小于主机的值,主机nginx挂掉后,权重值会-10,小于95,这样备会顶上来作为主
priority 95
advert_int 1
nopreempt
track_script {
chk_redis
}
# authentication {
# auth_type PASS
# auth_pass 1111
# }
track_script {
chk_redis
}
virtual_ipaddress {
172.168.1.177/24 dev eth1 label eth1:2
# 虚拟IP地址要与当前机器ip在同一网段内
}
notify_master "/etc/keepalived/redis_master.sh master"
notify_backup /etc/keepalived/redis_backup.sh
notify_fault /etc/keepalived/redis_fault.sh
notify_stop /etc/keepalived/redis_stop.sh
}
4 防火墙
# vi /etc/sysconfig/iptables
添加以下内容
#允许组播地址通信(172.168.1.0为服务器所在网段)
-A INPUT -s 172.168.1.0/24 -d 224.0.0.18 -j ACCEPT
#允许 VRRP(虚拟路由器冗余协议)通信
-A INPUT -s 172.168.1.0/24 -p vrrp -j ACCEPT
重启iptables(CenterOS6)
# service iptables restart
重启iptables(CenterOS7)
# systemctl restart iptables.service
5 编辑监听脚本
# vi /etc/keepalived/redis_check.sh
脚本内容:
#!/bin/bash
ALIVE=`/usr/local/redis/bin/redis-cli -a redis密码 PING`
if [ "$ALIVE" == "PONG" ];then
echo $ALIVE
exit 0
else
echo $ALIVE
service keepalived stop #可确保让出MASTER
exit 1
fi
6 执行脚本
为脚本创建一个日志目录
# mkdir /var/redis/keepalived_log
6.1 文件一:redis_fault.sh
# vi /etc/keepalived/redis_fault.sh
文件内容如下:
#!/bin/bash
LOGFILE=/var/redis/keepalived_log/keepalived-redis-state.log
echo "[fault]" >> $LOGFILE
date >> $LOGFILE
6.2 文件二:redis_stop.sh
# vi /etc/keepalived/redis_stop.sh
文件内容如下:
#!/bin/bash
LOGFILE=/var/redis/keepalived_log/keepalived-redis-state.log
echo "[stop]" >> $LOGFILE
date >> $LOGFILE
6.3 主节点脚本
6.3.1 redis_master.sh
脚本解释:
-
在该节点的keepalived第一次启动,指keepalived组中仅有这一台或者这一台的优先级别最高,自动成为主节点,然后就会调用该脚本;
-
脚本中先从备机上同步数据,如果备机没有运行或故障了,则过几秒执行命令使这个节点的Redis成为主节点(提供读写能力)。
创建脚本文件:
# vi /etc/keepalived/redis_master.sh
脚本内容:
#!/bin/bash
REDISCLI="/usr/local/redis/bin/redis-cli -a [备节点Redis密码]"
LOGFILE="/var/redis/keepalived_log/keepalived-redis-state.log"
echo "[master]" >> $LOGFILE
date >> $LOGFILE
echo "Being master...." >> $LOGFILE 2>&1
echo "Run SLAVEOF cmd ..." >> $LOGFILE
$REDISCLI SLAVEOF [备机ip] 6379 >> $LOGFILE 2>&1
sleep 10 #延迟10秒以后待数据同步完成后再取消同步状态
echo "Run REPLICAOF NO ONE cmd ..." >> $LOGFILE
$REDISCLI REPLICAOF NO ONE >> $LOGFILE 2>&1
6.3.2 redis_backup.sh
脚本解释:
- 当这个原来是主节点,但是中途挂了,又启动后,这个时候已经有一个备节点变成了主节点,则keepalived会执行该脚本;
- 该脚本首先等一段时间,给备机充足的时间切换为主节点,然后该脚本执行命令,让本节点的Redis成为新主节点Redis的备份服务。
创建脚本:
# vi /etc/keepalived/redis_backup.sh
脚本内容:
#!/bin/bash
REDISCLI="/usr/local/redis/bin/redis-cli -a [备节点Redis密码]"
LOGFILE="/var/redis/keepalived_log/keepalived-redis-state.log"
echo "[backup]" >> $LOGFILE
date >> $LOGFILE
echo "Being slave...." >> $LOGFILE 2>&1
sleep 15 #延迟15秒待数据被对方同步完成之后再切换主从角色
echo "Run SLAVEOF cmd ..." >> $LOGFILE
$REDISCLI SLAVEOF [备机ip] 6379 >> $LOGFILE 2>&1
6.4 备份节点脚本
第一次启动时,该节点的服务一定是在主节点服务都开启后才启动。
服务启动后,本机Redis只能提供读的功能(可配置为可写,但不安全)。
6.4.1 redis_master.sh
脚本解释:
- 当该节点在主节点挂掉之后会执行;
- 脚本会先从主节点同步数据,然后脚本执行命令让本机Redis独立为主服务(提供读写能力)。
创建脚本:
vi /etc/keepalived/redis_master.sh
脚本内容:
#!/bin/bash
REDISCLI="/usr/local/redis/bin/redis-cli -a [主节点Redis密码]"
LOGFILE="/var/redis/keepalived_log/keepalived-redis-state.log"
echo "[master]" >> $LOGFILE
date >> $LOGFILE
echo "Being master...." >> $LOGFILE 2>&1
echo "Run SLAVEOF cmd ..." >> $LOGFILE
$REDISCLI SLAVEOF [主节点ip] 6379 >> $LOGFILE 2>&1
sleep 10 #延迟10秒以后待数据同步完成后再取消同步状态
echo "Run SLAVEOF NO ONE cmd ..." >> $LOGFILE
$REDISCLI SLAVEOF NO ONE >> $LOGFILE 2>&1
6.4.2 redis_backup.sh
脚本解释:
- 该脚本是在主节点的Redis成为主服务时,或主节点服务已经开启,备机服务再开启后自动执行;
- 脚本先等待一段时间,让主节点切换为主服务,然后脚本执行命令使本节点的Redis成为主节点的备机。
创建脚本:
vi /etc/keepalived/redis_backup.sh
脚本内容:
#!/bin/bash
REDISCLI="/usr/local/redis/bin/redis-cli -a [主节点Redis密码]"
LOGFILE="/var/redis/keepalived_log/keepalived-redis-state.log"
echo "[backup]" >> $LOGFILE
date >> $LOGFILE
echo "Being slave...." >> $LOGFILE 2>&1
sleep 15 #延迟15秒待数据被对方同步完成之后再切换主从角色
echo "Run REPLICAOF cmd ..." >> $LOGFILE
$REDISCLI REPLICAOF [主节点ip] 6379 >> $LOGFILE 2>&1
6.3 授权
# chmod +x /etc/keepalived/redis_*
6 邮件通知
安装
yum install -y mailx sendmail
编辑文件
# vi /etc/mail.rc
文件内容
# 这里填入smtp地址,如果用的云服务器,安全组策略要开放465端口,入站和出站都要开放该端口
set smtp=smtps://smtp.xxx.com:465
# 认证方式
set smtp-auth=login
# 这里输入邮箱账号
set smtp-auth-user=user@xxx.com
# 这里填入密码,这里是授权码而不是邮箱密码
set smtp-auth-password=password
# 忽略证书警告
set ssl-verify=ignore
# 证书所在目录
set nss-config-dir=/etc/pki/nssdb
# 设置发信人邮箱和昵称
set from=user@xxx.com
开机启动 CenterOS 6.x
service sendmail start
chkconfig sendmail on
开机启动 CenterOS 7.x
systemctl start sendmail.service
systemctl enable sendmail.service
7 补充配置
需要在主备两台服务器上同时配置
# vi /etc/keepalived/redis_master.sh
在文件底部编辑并粘贴以下内容内容:
#接收者邮箱,多个以空格分隔
contact=(abc@qq.com)
#本机ip
HOST_IP=[这台机器的ip]
notify() {
#邮件主题
mailsubject="Redis发生切换($HOST_IP)"
#邮件正文
mailbody="$(date +'%F %T'): vrrp transition, $HOST_IP keepalived changed to be $1"
for receiver in ${contact[*]}
do
#发送邮件
echo "$mailbody" | mail -s "$mailsubject" $receiver
done
}
case $1 in
master)
notify master
;;
backup)
notify backup
;;
fault)
notify fault
;;
*)
echo "Usage:{master|backup|fault}"
exit 1
;;
esac
授权
chmod +x /etc/keepalived/notify.sh
重启keepalived,完成。
启动服务
-
主机:
# service redisn_6379 start # service keepalived start
-
备机:
# service redisn_6379 start # service keepalived start
测试验证
在主备中任意选择一台作测试,打开一个新连接。
连接到Redis虚拟ip客户端:
# redis-cli -h [设置的虚拟ip] -p 6379 -a [redis密码]
验证:
-
往redis中写入一个值
> set k 1
-
分别使用上面的客户端命令,连接到主备两个服务上,查看写入的值两台是否都存在,都存在则没问题
# redis-cli -h [ip] -p 6379 -a [redis密码] > get k
-
使用虚拟ip修改k的值,再验证两台redis上的值是否都已经更新
-
停掉主节点,通过虚拟ip连接redis服务修改k的值,重启停掉的节点,验证两台redis上的数据一致,便完成部署。