高可用集群KEEPALIVED详解
一、Keepalived
1.1 Keepalived 相关文件
软件包名：keepalived
主程序文件：/usr/sbin/keepalived
主配置文件：/etc/keepalived/keepalived.conf
配置文件示例：/usr/share/doc/keepalived/
Unit File：/lib/systemd/system/keepalived.service
Unit File的环境配置文件：/etc/sysconfig/keepalived
注意：RHEL7中可能会遇到一下bug
systemctl restart keepalived #新配置可能无法生效
systemctl stop keepalived;systemctl start keepalived #无法停止进程，需要 kill 停止
1.2 Keepalived 安装
安装 keepalived
[root@KA1 ~]# dnf install keepalived -y
[root@KA1 ~]# systemctl start keepalived
[root@KA1 ~]# ps axf | grep keepalived
2385 pts/0 S+ 0:00 \_ grep --color=auto keepalived
2326 ? Ss 0:00 /usr/sbin/keepalived -D
2327 ? S 0:00 \_ /usr/sbin/keepalived -D
1.3 KeepAlived 配置说明
1.3.1 配置文件组成部分
配置文件：/etc/keepalived/keepalived.conf
配置文件组成
GLOBAL CONFIGURATION
Global definitions： 定义邮件配置，route\_id，vrrp配置，多播地址等
VRRP CONFIGURATION
VRRP instance(s)： 定义每个vrrp虚拟路由器
LVS CONFIGURATION
Virtual server group(s)
Virtual server(s)： LVS集群的VS和RS
1.3.2 配置语法说明
帮助
man keepalived.conf
1.3.2.1 全局配置
! Configuration File for keepalived
global_defs {
notification_email {
594233887@qq.com #keepalived 发生故障切换时邮件发送的目标邮箱，可以按行区分写多个
timiniglee-zln@163.com }
notification_email_from keepalived@KA1.timinglee.org #发邮件的地址
smtp_server 127.0.0.1 #邮件服务器地址
smtp_connect_timeout 30 #邮件服务器连接timeout
router_id KA1.timinglee.org #每个keepalived主机唯一标识
#建议使用当前主机名，但多节点重名不影响
vrrp_skip_check_adv_addr #对所有通告报文都检查，会比较消耗性能
#启用此配置后，如果收到的通告报文和上一个报文是同一
#个路由器，则跳过检查，默认值为全检查
vrrp_strict #严格遵循vrrp协议
#启用此项后以下状况将无法启动服务:
#1.无VIP地址
#2.配置了单播邻居
#3.在VRRP版本2中有IPv6地址
#建议不加此项配置
vrrp_garp_interval 0 #报文发送延迟，0表示不延迟
vrrp_gna_interval 0 #消息发送延迟
vrrp_mcast_group4 224.0.0.18 #指定组播IP地址范围：
}
1.3.2.2 配置虚拟路由器
vrrp_instance VI_1 {
state MASTER
interface eth0 #绑定为当前虚拟路由器使用的物理接口，如：eth0,可以和VIP不在一个网卡
virtual_router_id 51 #每个虚拟路由器惟一标识,范围：0-255，每个虚拟路由器此值必须唯一
#否则服务无法启动
#同属一个虚拟路由器的多个keepalived节点必须相同
#务必要确认在同一网络中此值必须唯一
priority 100 #当前物理节点在此虚拟路由器的优先级，范围：1-254
#值越大优先级越高,每个keepalived主机节点此值不同
advert_int 1 #vrrp通告的时间间隔，默认1s
authentication { #认证机制
auth_type AH|PASS #AH为IPSEC认证(不推荐),PASS为简单密码(建议使用)
uth_pass 1111 #预共享密钥，仅前8位有效
#同一个虚拟路由器的多个keepalived节点必须一样
}
virtual_ipaddress { #虚拟IP,生产环境可能指定上百个IP地址
<IPADDR>/<MASK> brd <IPADDR> dev <STRING> scope <SCOPE> label <LABEL>
172.25.254.100 #指定VIP，不指定网卡，默认为eth0,注意：不指定/prefix,默认32
172.25.254.101/24 dev eth1
172.25.254.102/24 dev eth2 label eth2:1
}
}
示例：
#配置master端
[root@KA1 ~]# vim /etc/keepalived/keepalived.conf
! Configuration File for keepalived
global_defs {
notification_email {
8888888@qq.com
}
notification_email_from keepalived@KA1.timinglee.org
smtp_server 127.0.0.1
smtp_connect_timeout 30
router_id KA1.timinglee.org
vrrp_skip_check_adv_addr
#vrrp_strict #nft list ruleset
vrrp_garp_interval 0
vrrp_gna_interval 0
vrrp_mcast_group4 224.0.0.18
}
vrrp_instance VI_1 {
state MASTER
interface eth0
virtual_router_id 20
priority 100
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
172.25.254.100/24 dev eth0 label eth0:0
}
}
配置slave端
[root@KA2 ~]# vim /etc/keepalived/keepalived.conf
! Configuration File for keepalived
global_defs {
notification_email {
88888888@qq.com
}
notification_email_from keepalived@timinglee.org
smtp_server 127.0.0.1
smtp_connect_timeout 30
router_id KA2.timinglee.org
vrrp_skip_check_adv_addr
#vrrp_strict
vrrp_garp_interval 0
vrrp_gna_interval 0
vrrp_mcast_group4 224.0.0.18
}
vrrp_instance VI_1 {
state BACKUP
interface eth0
virtual_router_id 20 #相同id管理同一个虚拟路由
priority 80 #低优先级
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
172.25.254.100/24 dev eth0 label eth0:0
}
}
测试
[root@KA2 ~]# tcpdump -i eth0 -nn host 224.0.0.18
dropped privs to tcpdump
tcpdump: verbose output suppressed, use -v[v]... for full protocol decode
listening on eth0, link-type EN10MB (Ethernet), snapshot length 262144 bytes
22:48:23.294894 IP 172.25.254.20 > 224.0.0.18: VRRPv2, Advertisement, vrid 20,prio 100, authtype none, intvl 1s, length 20
22:48:24.084793 IP 172.25.254.30 > 224.0.0.18: VRRPv2, Advertisement, vrid 30,prio 80, authtype none, intvl 1s, length 20
22:48:24.295075 IP 172.25.254.20 > 224.0.0.18: VRRPv2, Advertisement, vrid 20,prio 100, authtype none, intvl 1s, length 20
22:48:25.085256 IP 172.25.254.30 > 224.0.0.18: VRRPv2, Advertisement, vrid 30,prio 80, authtype none, intvl 1s, length 20
22:48:25.296296 IP 172.25.254.20 > 224.0.0.18: VRRPv2, Advertisement, vrid 20,prio 100, authtype none, intvl 1s, length 20
22:48:26.085843 IP 172.25.254.30 > 224.0.0.18: VRRPv2, Advertisement, vrid 30,prio 80, authtype none, intvl 1s, length 20
关闭KA1后再看组播信息
1.3.2.3 启用keepalived日志功能
示例：
[root@KA1 ~]# vim /etc/sysconfig/keepalived
KEEPALIVED_OPTIONS="-D -S 6"
[root@ka1 ~]#vim /etc/rsyslog.conf
local6.* /var/log/keepalived.log
[root@ka1 ~]#systemctl restart keepalived.service rsyslog.service
1.3.2.4 实现独立子配置文件
当生产环境复杂时， /etc/keepalived/keepalived.conf 文件中内容过多，不易管理
将不同集群的配置，比如：不同集群的VIP配置放在独立的子配置文件中利用include 指令可以实现包含子配置文件
格式：
include /path/file
示例：
[root@KA1 ~]# mkdir /etc/keepalived/conf.d
[root@KA1 ~]# vim /etc/keepalived/keepalived.conf
! Configuration File for keepalived
global_defs {
notification_email {
88888888@qq.com
}
notification_email_from keepalived@KA1.timinglee.org
smtp_server 127.0.0.1
smtp_connect_timeout 30
router_id KA1.timinglee.org
vrrp_skip_check_adv_addr
vrrp_strict
vrrp_garp_interval 0
vrrp_gna_interval 0
vrrp_ipsets keepalived
vrrp_iptables
}
include /etc/keepalived/conf.d/*.conf #相关子配置文件
[root@KA1 ~]# vim /etc/keepalived/conf.d/router.conf
vrrp_instance VI_1 {
state MASTER
interface eth0
virtual_router_id 20
priority 100
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
172.25.254.100/24 dev eth0 label eth0:0
}
}
二、Keepalived 企业应用示例
2.1 实现master/slave的 Keepalived 单主架构
2.1.1 MASTER配置
[root@KA1 ~]# vim /etc/keepalived/keepalived.conf
! Configuration File for keepalived
global_defs {
notification_email {
88888888@qq.com
}
notification_email_from keepalived@KA1.timinglee.org
smtp_server 127.0.0.1
smtp_connect_timeout 30
router_id KA1.timinglee.org
vrrp_skip_check_adv_addr
#vrrp_strict #添加此选项无法访问vip，可以用nft list ruleset查看
vrrp_garp_interval 0
vrrp_gna_interval 0
vrrp_mcast_group4 224.0.0.18
}
vrrp_instance VI_1 {
state MASTER
interface eth0
virtual_router_id 20
priority 100
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
172.25.254.100/24 dev eth0 label eth0:0
}
}
2.1.2 BACKUP配置
#配置文件和master基本一致，只需修改三行
[root@KA2 ~]# vim /etc/keepalived/keepalived.conf
! Configuration File for keepalived
global_defs {
notification_email {
88888888@qq.com
}
notification_email_from keepalived@timinglee.org
smtp_server 127.0.0.1
smtp_connect_timeout 30
router_id KA2.timinglee.org
vrrp_skip_check_adv_addr
#vrrp_strict
vrrp_garp_interval 0
vrrp_gna_interval 0
vrrp_mcast_group4 224.0.0.18
}
vrrp_instance VI_1 {
state BACKUP
interface eth0
virtual_router_id 20 #相同id管理同一个虚拟路由
priority 80 #低优先级
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
172.25.254.100/24 dev eth0 label eth0:0
}
}
抓包观察
tcpdump -i eth0 -nn host 224.0.0.18
2.2 抢占模式和非抢占模式
默认为抢占模式preempt，即当高优先级的主机恢复在线后，会抢占低先级的主机的master角色， 这样会使vip在KA主机中来回漂移，造成网络抖动
建议设置为非抢占模式 nopreempt ，即高优先级主机恢复后，并不会抢占低优先级主机的master角色非抢占模块下,如果原主机down机, VIP迁移至的新主机, 后续也发生down时,仍会将VIP迁移回原主机
注意：要关闭 VIP 抢占，必须将各 keepalived 服务器 state 配置为 BACKUP
#ka1主机配置
vrrp_instance VI_1 {
state BACKUP
interface eth0
virtual_router_id 20
priority 100 #优先级高
nopreempt #非抢占模式
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
172.25.254.100/24 dev eth0 label eth0:0
}
}
#KA2主机配置
vrrp_instance VI_1 {
state BACKUP
interface eth0
virtual_router_id 20
priority 80 #优先级低
advert_int 1
nopreempt #非抢占模式
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
172.25.254.100/24 dev eth0 label eth0:0
}
}
2.2.2 抢占延迟模式 preempt_delay
抢占延迟模式，即优先级高的主机恢复后，不会立即抢回 VIP ，而是延迟一段时间（默认 300s ）再抢回VIP
preempt_delay # #指定抢占延迟时间为#s，默认延迟300s
注意：需要各keepalived服务器state为BACKUP,并且不要启用 vrrp_strict
示例：
#ka1主机配置
vrrp_instance VI_1 {
state BACKUP
interface eth0
virtual_router_id 20
priority 100 #优先级高
preempt_delay 10s #抢占延迟10s
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
172.25.254.100/24 dev eth0 label eth0:0
}
}
#KA2主机配置
vrrp_instance VI_1 {
state BACKUP
interface eth0
virtual_router_id 20
priority 80 #优先级低
advert_int 1
preempt_delay 10s #抢占延迟10S
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
172.25.254.100/24 dev eth0 label eth0:0
}
}
2.3 VIP单播配置
默认keepalived主机之间利用多播相互通告消息，会造成网络拥塞，可以替换成单播，减少网络流量
注意：启用 vrrp_strict 时，不能启用单播
#在所有节点vrrp_instance语句块中设置对方主机的IP，建议设置为专用于对应心跳线网络的地址，而非使用业务网络
unicast_src_ip <IPADDR> #指定发送单播的源IP
unicast_peer {
<IPADDR> #指定接收单播的对方目标主机IP
......
}
示例：
#master 主机配置
[root@KA1 ~]# vim /etc/keepalived/keepalived.conf
! Configuration File for keepalived
global_defs {
notification_email {
88888888@qq.com
}
notification_email_from keepalived@KA1.timinglee.org
smtp_server 127.0.0.1
smtp_connect_timeout 30
router_id KA1.timinglee.org
vrrp_skip_check_adv_addr
#vrrp_strict #注释此参数，与vip单播模式冲突
vrrp_garp_interval 0
vrrp_gna_interval 0
vrrp_ipsets keepalived
}
vrrp_instance VI_1 {
state MASTER
interface eth0
virtual_router_id 20
priority 100
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
172.25.254.100/24 dev eth0 label eth0:0
}
unicast_src_ip 172.25.254.20 #本机IP
unicast_peer {
172.25.254.30 #指向对方主机IP
#如果有多个keepalived,再加其它节点的IP
}
}
##在slave主机中
[root@KA2 ~]# vim /etc/keepalived/keepalived.conf
! Configuration File for keepalived
global_defs {
notification_email {
88888888@qq.com
}
notification_email_from keepalived@KA1.timinglee.org
smtp_server 127.0.0.1
smtp_connect_timeout 30
router_id KA1.timinglee.org
vrrp_skip_check_adv_addr
#vrrp_strict #注释此参数，与vip单播模式冲突
vrrp_garp_interval 0
vrrp_gna_interval 0
vrrp_ipsets keepalived
}
vrrp_instance VI_1 {
state BACKUP
interface eth0
virtual_router_id 20
priority 80
advert_int 1
preempt_delay 60
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
172.25.254.100/24 dev eth0 label eth0:0
}
unicast_src_ip 172.25.254.30 #本机ip
unicast_peer {
172.25.254.20 #对端主机IP
}
}
抓包查看单播效果
[root@KA1 ~]# tcpdump -i eth0 -nn src host 172.25.254.20 and dst 172.25.254.30
2.4 Keepalived 通知脚本配置
当keepalived的状态变化时，可以自动触发脚本的执行，比如：发邮件通知用户
默认以用户keepalived_script身份执行脚本
如果此用户不存在，以root执行脚本可以用下面指令指定脚本执行用户的身份
global_defs {
......
script_user <USER>
......
}
2.4.1 通知脚本类型
1.当前节点成为主节点时触发的脚本
notify_master <STRING>|<QUOTED-STRING>
2.当前节点转为备节点时触发的脚本
notify_backup <STRING>|<QUOTED-STRING>
3.当前节点转为“失败”状态时触发的脚本
notify_fault <STRING>|<QUOTED-STRING>
4.通用格式的通知触发机制，一个脚本可完成以上三种状态的转换时的通知
notify <STRING>|<QUOTED-STRING>
5.当停止VRRP时触发的脚本
notify_stop <STRING>|<QUOTED-STRING>
2.4.2 脚本的调用方法
在 vrrp_instance VI_1 语句块的末尾加下面行
notify_master "/etc/keepalived/notify.sh master"
notify_backup "/etc/keepalived/notify.sh backup"
notify_fault "/etc/keepalived/notify.sh fault"
2.4.3 创建通知脚本
[root@KA1 ~]# vim /etc/keepalived/mail.sh
#!/bin/bash
mail_dest='594233887@qq.com'
mail_send()
{
mail_subj="$HOSTNAME to be $1 vip 转移"
mail_mess="`date +%F\ %T`: vrrp 转移,$HOSTNAME 变为 $1"
echo "$mail_mess" | mail -s "$mail_subj" $mail_dest
}
case $1 in
master)
mail_send master
;;
backup)
mail_send backup
;;
fault)
mail_send fault
;;
*)
exit 1
;;
esac
2.4.4 邮件配置
安装邮件发送工具
[root@KA2 ~]# dnf install mailx -y
QQ邮箱配置
[root@KA1 ~]# vim /etc/mail.rc
#######mail set##########
set from=88888888@qq.com
set smtp=smtp.qq.com
set smtp-auth-user=88888888@qq.com
set smtp-auth-password=isjatjwmcxtxbefj
set smtp-auth=login
set ssl-verify=ignore
发送测试邮件
[root@KA1 ~]# echo test message |mail -s test 594233887@qq.com
2.4.5 实战案例：实现 Keepalived 状态切换的通知脚本
#在所有 keepalived节点配置如下
[root@KA1 + KA2 ~]# vim /etc/keepalived/mail.sh
#!/bin/bash
mail_dest='594233887@qq.com'
mail_send()
{
mail_subj="$HOSTNAME to be $1 vip 转移"
mail_mess="`date +%F\ %T`: vrrp 转移,$HOSTNAME 变为 $1"
echo "$mail_mess" | mail -s "$mail_subj" $mail_dest
}
case $1 in
master)
mail_send master
;;
backup)
mail_send backup
;;
fault)
mail_send fault
;;
*)
exit 1
;;
esac
[root@KA1 +KA2 ~]# chmod +x /etc/keepalived/mail.sh
[root@KA1 +K2 ~]#vim /etc/keepalived/keepalived.conf
vrrp_instance VI_1 {
state MASTER
interface eth0
virtual_router_id 20
priority 100
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
172.25.254.100/24 dev eth0 label eth0:0
}
unicast_src_ip 172.25.254.20
unicast_peer {
172.25.254.30
}
notify_master "/etc/keepalived/mail.sh master"
notify_backup "/etc/keepalived/mail.sh backup"
notify_fault "/etc/keepalived/mail.sh fault"
}
#模拟master故障
[root@ka1-centos8 ~]#killall keepalived
2.5 实现 master/master 的 Keepalived 双主架构
master/slave的单主架构，同一时间只有一个Keepalived对外提供服务，此主机繁忙，而另一台主机却很空闲，利用率低下，可以使用master/master的双主架构，解决此问题。
master/master 的双主架构：
即将两个或以上VIP分别运行在不同的keepalived服务器，以实现服务器并行提供web访问的目的，提高服务器资源利用率
#ha1主机配置
[root@rhel7-ka1 ~]# vim /etc/keepalived/keepalived.conf
@@@@ 内容省略 @@@@
vrrp_instance VI_1 {
state MASTER #主
interface ens33
virtual_router_id 50
priority 100
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
172.25.254.50 dev ens33 label ens33:0
}
}
vrrp_instance VI_60 {
state BACKUP #备
interface ens33
virtual_router_id 60
priority 80
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
172.25.254.60 dev ens33 label ens33:1
}
}
#ka2主机配置,和ka1配置只需五行不同
[root@rhel7-ka2 ~]# vim /etc/keepalived/keepalived.conf
@@@@ 内容省略 @@@@
vrrp_instance VI_1 {
state BACKUP #备
interface ens33
virtual_router_id 50
priority 80
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
172.25.254.50 dev ens33 label ens33:0
}
}
vrrp_instance VI_60 {
state MASTER #主
interface ens33
virtual_router_id 60
priority 100
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
172.25.254.60 dev eth0 label eth0:1
}
}
2.6 实现IPVS的高可用性
2.6.1 IPVS相关配置
2.6.1.1 虚拟服务器配置结构
virtual_server IP port {
...
real_server {
...
}
real_server {
...
}
...
}
2.6.1.2 virtual server （虚拟服务器）的定义格式
virtual_server IP port #定义虚拟主机IP地址及其端口
virtual_server fwmark int #ipvs的防火墙打标，实现基于防火墙的负载均衡集群
virtual_server group string #使用虚拟服务器组
2.6.1.3 虚拟服务器配置
virtual_server IP port { #VIP和PORT
delay_loop <INT> #检查后端服务器的时间间隔
lb_algo rr|wrr|lc|wlc|lblc|sh|dh #定义调度方法
lb_kind NAT|DR|TUN #集群的类型,注意要大写
persistence_timeout <INT> #持久连接时长
protocol TCP|UDP|SCTP #指定服务协议,一般为TCP
sorry_server <IPADDR> <PORT> #所有RS故障时，备用服务器地址
real_server <IPADDR> <PORT> { #RS的IP和PORT
weight <INT> #RS权重
notify_up <STRING>|<QUOTED-STRING> #RS上线通知脚本
notify_down <STRING>|<QUOTED-STRING> #RS下线通知脚本
HTTP_GET|SSL_GET|TCP_CHECK|SMTP_CHECK|MISC_CHECK { ... } #定义当前主机健康状态检测方法
}
}
#注意:括号必须分行写,两个括号写在同一行,如: }} 会出错
2.6.1.4 应用层监测
应用层检测：HTTP_GET|SSL_GET
HTTP_GET|SSL_GET {
url {
path <URL_PATH> #定义要监控的URL
status_code <INT> #判断上述检测机制为健康状态的响应码，一般为 200
}
connect_timeout <INTEGER> #客户端请求的超时时长, 相当于haproxy的timeout server
nb_get_retry <INT> #重试次数
delay_before_retry <INT> #重试之前的延迟时长
connect_ip <IP ADDRESS> #向当前RS哪个IP地址发起健康状态检测请求
connect_port <PORT> #向当前RS的哪个PORT发起健康状态检测请求
bindto <IP ADDRESS> #向当前RS发出健康状态检测请求时使用的源地址
bind_port <PORT> #向当前RS发出健康状态检测请求时使用的源端口}
2.6.1.5 TCP监测
传输层检测：TCP_CHECK
TCP_CHECK {
connect_ip <IP ADDRESS> #向当前RS的哪个IP地址发起健康状态检测请求
connect_port <PORT> #向当前RS的哪个PORT发起健康状态检测请求
bindto <IP ADDRESS> #发出健康状态检测请求时使用的源地址
bind_port <PORT> #发出健康状态检测请求时使用的源端口
connect_timeout <INTEGER> #客户端请求的超时时长
#等于haproxy的timeout server}
2.6.2 实战案例
2.6.2.1 实战案例1：实现单主的 LVS-DR 模式
准备web服务器并使用脚本绑定VIP至web服务器lo网卡
#准备两台后端RS主机
[root@rs1 ~]# yum install httpd -y
[root@rs1 ~]# echo RS1 - 172.25.254.101 > /var/www/html/index.html
[root@rs1 ~]# ip addr add 172.25.254.100/32 dev lo
[root@rs1 ~]# echo 1 > /proc/sys/net/ipv4/conf/all/arp_ignore
[root@rs1 ~]# echo 2 > /proc/sys/net/ipv4/conf/all/arp_announce
[root@rs1 ~]# echo 1 > /proc/sys/net/ipv4/conf/lo/arp_ignore
[root@rs1 ~]# echo 2 > /proc/sys/net/ipv4/conf/lo/arp_announce
[root@rs2 ~]# yum install httpd -y
[root@rs1 ~]# echo RS1 - 172.25.254.101 > /var/www/html/index.html
[root@rs2 ~]# ip addr add 172.25.254.100/32 dev lo
[root@rs2 ~]# echo 1 > /proc/sys/net/ipv4/conf/all/arp_ignore
[root@rs2 ~]# echo 2 > /proc/sys/net/ipv4/conf/all/arp_announce
[root@rs2 ~]# echo 1 > /proc/sys/net/ipv4/conf/lo/arp_ignore
[root@rs2 ~]# echo 2 > /proc/sys/net/ipv4/conf/lo/arp_announce
[root@node30 ~]# yum install httpd -y
[root@node30 ~]# echo RS1 - 172.25.254.101 > /var/www/html/index.html
[root@node30 ~]# ip addr add 172.25.254.100/32 dev lo
[root@node30 ~]# echo 1 > /proc/sys/net/ipv4/conf/all/arp_ignore
[root@node30 ~]# echo 2 > /proc/sys/net/ipv4/conf/all/arp_announce
[root@node30 ~]# echo 1 > /proc/sys/net/ipv4/conf/lo/arp_ignore
[root@node30 ~]# echo 2 > /proc/sys/net/ipv4/conf/lo/arp_announce
配置keepalived
#ka1节点的配置
[root@rhel7-ka1 ~]# vim /etc/keepalived/keepalived.conf
@@@@ 省略内容 @@@@
virtual_server 172.25.254.100 80 {
delay_loop 6
lb_algo wrr
lb_kind DR
protocol TCP
sorry_server 172.25.254.30
real_server 172.25.254.101 80 {
weight 1
TCP_CHECK {
connect_timeout 5
nb_get_retry 3
delay_before_retry 3
connect_port 80
}
}
real_server 172.25.254.102 80 {
weight 1
HTTP_GET {
url {
path /
status_code 200
}
connect_timeout 1
nb_get_retry 3
delay_before_retry 1
}
}
}
#ka2节点的配置，配置和ka1基本相同，只需修改三行
[root@rhel7-ka2 ~]# vim /etc/keepalived/keepalived.conf
@@@@ 省略内容 @@@@
virtual_server 172.25.254.100 80 {
delay_loop 6
lb_algo wrr
lb_kind DR
protocol TCP
sorry_server 172.25.254.30
real_server 172.25.254.101 80 {
weight 1
TCP_CHECK {
connect_timeout 5
nb_get_retry 3
delay_before_retry 3
connect_port 80
}
}
real_server 172.25.254.102 80 {
weight 1
HTTP_GET {
url {
path /
status_code 200
}
connect_timeout 1
nb_get_retry 3
delay_before_retry 1
}
}
}
访问测试结果
[root@client ~]# for i in {1..6}; do curl 172.25.254.100; done
RS1 - 172.25.254.101
RS2 - 172.25.254.102
RS1 - 172.25.254.101
RS2 - 172.25.254.102
RS1 - 172.25.254.101
RS2 - 172.25.254.102
[root@rhel7-ka1 ~]# ipvsadm -Ln
IP Virtual Server version 1.2.1 (size=4096
)Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
TCP 172.25.254.100:80 wrr
-> 172.25.254.101:80 Route 1 0 6
-> 172.25.254.102:80 Route 1 0 6
模拟故障
#第一台RS1故障，自动切换至RS2
[root@rs1 ~]# systemctl stop httpd #当RS1故障
[root@client ~]# for i in {1..6}; do curl 172.25.254.100; done
#全部流浪被定向到RS2中
RS2 - 172.25.254.102
RS2 - 172.25.254.102
RS2 - 172.25.254.102
RS2 - 172.25.254.102
RS2 - 172.25.254.102
RS2 - 172.25.254.102
[root@rhel7-ka1 ~]# ipvsadm -Ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
TCP 172.25.254.100:80 wrr
-> 172.25.254.102:80 Route 1 0 12 #RS1被踢出保留RS2
#后端RS服务器都故障，启动Sorry Server
[root@rs2 ~]#systemctl stop httpd
[root@client ~]# for i in {1..6}; do curl 172.25.254.100; done
sorry server
sorry server
sorry server
sorry server
sorry server
sorry server
[root@rhel7-ka1 ~]# ipvsadm -Ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
TCP 172.25.254.100:80 wrr
-> 172.25.254.30:80 Route 1 0 3
#陆续启动RS1 RS2
[root@rhel7-ka1 ~]# ipvsadm -Ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
TCP 172.25.254.100:80 wrr
-> 172.25.254.101:80 Route 1 0 3
-> 172.25.254.102:80 Route 1 0 9
#ka1故障，自动切换至ka2
[root@rhel7-ka1 ~]# systemctl stop keepalived.service
[root@rhel7-ka2 ~]# ipvsadm -LnIP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
TCP 172.25.254.100:80 wrr
-> 172.25.254.101:80 Route 1 0 0
-> 172.25.254.102:80 Route 1 0 0
2.6.2.2 实战案例2：实现双主的 LVS-DR 模式
[root@ka1-centos8 ~]#vim /etc/keepalived/keepalived.conf
! Configuration File for keepalived
global_defs {
notification_email {
root@localhost
}
notification_email_from keepalived@localhost
smtp_server 127.0.0.1
smtp_connect_timeout 30
router_id ka1.magedu.org #另一个节点为ka2.magedu.org
vrrp_mcast_group4 224.0.100.10
}
vrrp_instance VI_1 {
state MASTER #在另一个结点上为BACKUP
interface eth0
virtual_router_id 66
priority 100 #在另一个结点上为80
advert_int 1
authentication {
auth_type PASS
auth_pass 123456
}
virtual_ipaddress {
10.0.0.10/24 dev eth0 label eth0:1 #指定VIP
}
}
vrrp_instance VI_2 {
state BACKUP #在另一个结点上为MASTER
interface eth0
virtual_router_id 88
priority 80
#在另一个结点上为100
advert_int 1
authentication {
auth_type PASS
auth_pass 123456
}
virtual_ipaddress {
10.0.0.20/24 dev eth0 label eth0:2 #指定VIP2
}
}
virtual_server 10.0.0.10 80 {
delay_loop 6
lb_algo rr
lb_kind DR
protocol TCP
sorry_server 127.0.0.1 80
real_server 10.0.0.7 80 { #指定RS1地址
weight 1
HTTP_GET {
url {
path /
status_code 200
}
connect_timeout 3
nb_get_retry 3
delay_before_retry 3
}
}
real_server 10.0.0.17 80 { #指定RS2地址
weight 1
HTTP_GET {
url {
path /
status_code 200
}
connect_timeout 3
nb_get_retry 3
delay_before_retry 3
}
}
}virtual_server 10.0.0.20 80 { #指定VIP2
delay_loop 6
lb_algo rr
lb_kind DR
protocol TCP
sorry_server 127.0.0.1 80
real_server 10.0.0.27 80 { #指定RS3地址
weight 1
HTTP_GET {
url {
path /
status_code 200
}
connect_timeout 3
nb_get_retry 3
delay_before_retry 3
}
}
real_server 10.0.0.37 80 { #指定RS4地址
weight 1
HTTP_GET {
url {
path /
status_code 200
}
connect_timeout 3
nb_get_retry 3
delay_before_retry 3
}
}
}
2.7 实现其它应用的高可用性 VRRP Script
keepalived利用 VRRP Script 技术，可以调用外部的辅助脚本进行资源监控，并根据监控的结果实现优先动态调整，从而实现其它应用的高可用性功能
参考配置文件：/usr/share/doc/keepalived/keepalived.conf.vrrp.localcheck
2.7.1 VRRP Script 配置
分两步实现：
定义脚本
vrrp\_script：自定义资源监控脚本，vrrp实例根据脚本返回值，公共定义，可被多个实例调用，定义在vrrp实例之外的独立配置块，一般放在global\_defs设置块之后。
通常此脚本用于监控指定应用的状态。一旦发现应用的状态异常，则触发对MASTER节点的权重减至低于SLAVE节点，从而实现 VIP 切换到 SLAVE 节点
vrrp_script <SCRIPT_NAME> {
script <STRING>|<QUOTED-STRING> #此脚本返回值为非0时，会触发下面OPTIONS执行
OPTIONS
}
调用脚本
track\_script：调用vrrp\_script定义的脚本去监控资源，定义在VRRP实例之内，调用事先定义的vrrp\_scrip
track_script {
SCRIPT_NAME_1
SCRIPT_NAME_2
}
2.7.1.1 定义 VRRP script
vrrp_script <SCRIPT_NAME> { #定义一个检测脚本，在global_defs 之外配置
script <STRING>|<QUOTED-STRING> #shell命令或脚本路径
interval <INTEGER> #间隔时间，单位为秒，默认1秒
timeout <INTEGER> #超时时间
weight <INTEGER:-254..254> #默认为0,如果设置此值为负数，
#当上面脚本返回值为非0时
#会将此值与本节点权重相加可以降低本节点权重，
#即表示fall.
#如果是正数，当脚本返回值为0，
#会将此值与本节点权重相加可以提高本节点权重
#即表示 rise.通常使用负值
fall <INTEGER> #执行脚本连续几次都失败,则转换为失败，建议设为2以上
rise <INTEGER> #执行脚本连续几次都成功，把服务器从失败标记为成功
user USERNAME [GROUPNAME] #执行监测脚本的用户或组
init_fail #设置默认标记为失败状态，监测成功之后再转换为成功状态
}
2.7.1.2 调用 VRRP script
vrrp_instance test {
... ...
track_script {
check_down
}
}
2.7.2 实战案例：利用脚本实现主从角色切换
[root@rhel7-ka1 ~]# vim /mnt/check_long.sh
#!/bin/bash
[ ！ -f "/mnt/long" ]
[root@rhel7-ka1 ~]# chmod +x /mnt/check_long.sh
[root@rhel7-ka1 ~]# vim /etc/keepalived/keepalived.conf
@@@@ 省略内容 @@@@
vrrp_script check_lee {
script "/mnt/check_long.sh"
interval 1
weight -30
fall 2
rise 2
timeout 2
}
vrrp_instance web {
state MASTER
interface ens33
virtual_router_id 50
priority 100
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
172.25.254.100 dev ens33 label ens33:0
}
track_script {
check_long
}
}
[root@rhel7-ka1 ~]# touch /mnt/long
[root@rhel7-ka1 ~]# tail -f /var/log/messages
2.7.3 实战案例：实现HAProxy高可用
#在两个ka1和ka2先实现haproxy的配置
[root@rhel7-ka1 & ka2 ~]# vim /etc/haproxy/haproxy.cfg
listen webserver
bind 172.25.254.100:80
server web1 172.25.254.101:80 check
server web2 172.25.254.102:80 check
#在两个ka1和ka2两个节点启用内核参数
[root@rhel7-ka1 & ka2 ~]# vim /etc/sysctl.conf
net.ipv4.ip_nonlocal_bind = 1
[root@rhel7-ka1 & ka2 ~]# sysctl -p
#在ka1中编写检测脚本
[root@rhel7-ka1 ~]# vim /etc/keepalived/scripts/haproxy.sh
#!/bin/bash/
usr/bin/killall -0 haproxy
[root@rhel7-ka1 ~]# chmod +X /etc/keepalived/scripts/haproxy.sh
#在ka1中配置keepalived
[root@ka1-centos8 ~]#cat /etc/keepalived/keepalived.conf
vrrp_script check_haproxy {
script "/etc/keepalived/scripts/haproxy.sh"
interval 1
weight -30
fall 2
rise 2
timeout 2
}
vrrp_instance web {
state MASTER
interface ens33
virtual_router_id 50
priority 100
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
172.25.254.100 dev ens33 label ens33:0
}
track_script {
check_haproxy
}
}
#测试
[root@rhel7-ka1 ~]# systemctl stop haproxy.service
683
被折叠的 条评论
为什么被折叠?
到【灌水乐园】发言
