介绍:
keepalived 是一款基于 VRRP(Virtual Router Redundancy Protocol,虚拟路由冗余协议)协议来实现高可用(High Availability, HA)的轻量级软件。它主要用于防止单点故障,特别是在 Linux 环境下,为 LVS(Linux Virtual Server,Linux 虚拟服务器)集群和其他服务提供高可用性和故障转移功能。虽然它最初是为 LVS 设计的,但现在也被广泛用于其他需要高可用性的场景,比如 MySQL 数据库集群、Nginx 负载均衡器等。
Keepalived 的关键特性有:
1. 高可用性(HA):Keepalived 通过 VRRP 协议实现主备模式的故障转移。当主节点发生故障时,备份节点会自动接管虚拟 IP(VIP),确保服务的连续性。
2. 负载均衡:Keepalived 可以与 LVS 结合使用,通过配置不同的负载均衡算法(如轮询、最少连接等)来分发流量到后端服务器。
3. 健康检查:Keepalived 可以定期检查后端服务器的健康状态,如果发现某个服务器不可用,会将其从负载均衡池中移除
4. 灵活的配置:Keepalived 的配置文件(通常是 keepalived.conf)非常灵活,允许用户根据需要定义复杂的规则和策略。
5. 多层检查:Keepalived 不仅可以在 IP 层进行检查,还可以在应用层进行检查,确保服务的可用性和性能。
6. 脚本支持:Keepalived 允许用户编写自定义脚本,用于执行特定的检查或操作,增加了系统的灵活性和可扩展性。
环境准备:
realserver1:
[root@realserver1 ~]# yum install httpd -y
[root@realserver1 ~]# echo realserver1 - 172.25.254.110 > /var/www/html/index.html
[root@realserver1 ~]# systemctl stop firewalld
[root@realserver1 ~]# systemctl enable --now httpd
realserver2:
[root@realserver2 ~]# yum install httpd -y
[root@realserver2 ~]# echo realserver2 - 172.25.254.120 > /var/www/html/index.html
[root@realserver2 ~]# systemctl stop firewalld
[root@realserver2 ~]# systemctl enable --now httpd
KA1: 172.25.254.10
KA2: 172.25.254.20
realserver1: 172.25.254.110
realserver2: 172.25.254.120
测试:
[root@ka1 ~]# curl 172.25.254.110
realserver1 - 172.25.254.110
[root@ka1 ~]# curl 172.25.254.120
realserver2 - 172.25.254.120
[root@ka2 ~]# curl 172.25.254.110
realserver1 - 172.25.254.110
[root@ka2 ~]# curl 172.25.254.120
realserver2 - 172.25.254.120
1.keepalived虚拟路由管理
[root@ka1 ~]# yum install keepalived -y #KA1 KA2
[root@ka1 ~]# vim /etc/keepalived/keepalived.conf
global_defs {
notification_email {
2138852636@qq.com
}
notification_email_from keepalived@byh.org
smtp_server 127.0.0.1
smtp_connect_timeout 30
router_id ka1.byh.org
vrrp_skip_check_adv_addr
vrrp_strict
vrrp_garp_interval 0
vrrp_gna_interval 0
vrrp_mcast_group4 224.0.0.18
}
vrrp_instance VI_1 {
state MASTER
interface eth0
virtual_router_id 100
priority 100
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
172.25.254.100/24 dev eth0 label eth0:1
}
[root@ka1 ~]# systemctl restart keepalived
[root@ka1 ~]# scp /etc/keepalived/keepalived.conf root@172.25.254.20: /etc/keepalived/keepalived.conf
KA2:
vim /etc/keepalived/keepalived.conf
global_defs {
notification_email {
2138852636@qq.com
}
notification_email_from keepalived@byh.org
smtp_server 127.0.0.1
smtp_connect_timeout 30
router_id ka2.byh.org
vrrp_skip_check_adv_addr
vrrp_strict
vrrp_garp_interval 0
vrrp_gna_interval 0
vrrp_mcast_group4 224.0.0.18
}
vrrp_instance VI_1 {
state BACKUP
interface eth0
virtual_router_id 100
priority 80
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
172.25.254.100/24 dev eth0 label eth0:1
测试:
2.启用keepalived日志功能:
[root@ka1 ~]# vim /etc/sysconfig/keepalived
KEEPALIVED_OPTIONS="-D -S 6"
[root@ka1 ~]# vim /etc/rsyslog.conf
local6.* /var/log/keepalived.log
[root@ka1 ~]# systemctl restart keepalived.service
[root@ka1 ~]# systemctl restart keepalived.service rsyslog.service
[root@ka1 ~]# ll /var/log/keepalived.log
-rw-------. 1 root root 4569 Aug 17 11:24 /var/log/keepalived.log
3.实现独立子配置文件:
[root@ka1 ~]# vim /etc/keepalived/keepalived.conf
include "/etc/keepalived/conf.d/*.conf" #将上面写的vrrp全部注释掉
[root@ka1 ~]# mkdir -p /etc/keepalived/conf.d
[root@ka1 ~]# vim /etc/keepalived/conf.d/172.25.254.100.conf
vrrp_instance web {
state MASTER
interface eth0
virtual_router_id 100
priority 100
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
172.25.254.100 dev eth0 label eth0:1
}
}
[root@ka1 ~]# systemctl restart keepalived.servive
4.抢占模式和非抢占模式:
1. 抢占模式:
抢占模式(preempt 系统默认是抢占)不需要配置,即当高优先级的主机恢复在线后,会抢占低先级的主机的master角色, 这样会使vip在KA主机中来回漂移,造成网络抖动,建议设置为非抢占模式 nopreempt ,即高优先级主机恢复后,并不会抢占低优先级主机的master角色 非抢占模块下,如果原主机down机, VIP迁移至的新主机, 后续也发生down时,仍会将VIP迁移回原主机。。
2、非抢占模式 nopreempt
环境要求:
关闭 VIP抢占,必须将各 keepalived 服务器state配置为BACKUP
priority 100 #优先级高
nopreempt #非抢占模式
## KA1 priority 100 ##KA2 priority 80 其余配置相同
3. 抢占延迟模式preempt_delay
抢占延迟模式,即优先级高的主机恢复后,不会立即抢回VIP,而是延迟一段时间(默认300s)再抢回VIP。
环境:需要各keepalived服务器state为BACKUP,并且不要启用 vrrp_strict
用法:
preempt_delay # # 指定抢占延迟时间为 #s
VIP的单播配置:
注意:启用 vrrp_strict 时,不能启用单播
KA1中的配置:
virtual_ipaddress {
172.25.254.100/24 dev ens33 label eth0:1
}
unicast_src_ip 172.25.254.10 #本机IP
unicast_peer {
172.25.254.20 #指向对端主机IP
#如果有多个keepalived,再加其它节点的IP
}
KA2中的配置:
virtual_ipaddress {
172.25.254.100/24 dev ens33 label eth0:1
}
unicast_src_ip 172.25.254.20
unicast_peer {
172.25.254.10
}
测试:
Keepalived 通知脚本配置
1、 邮件配置
[root@ka1 ~]# yum install mailx -y
[root@ka1 ~]# vim /etc/mail.rc
set from=2138825636@qq.com
set smtp=smtp.qq.com
set smtp-auth-user=2138852636@qq.com
set smtp-auth-password=hbjdgetvshllincy # 授权码
set smtp-auth=login
set ssl-verify=ignore
邮件发送:
[root@ka1 ~]# echo test message | mail -s test 2138852636@qq.com
2.脚本配置:
##KA1 ##KA2 配置相同
[root@ka1 ~]# vim /etc/keepalived/mail.sh
#!/bin/bash
mail_dest='2138852636@qq.com'
mail_send()
{
mail_subj="$HOSTNAME to be $1 vip 转移"
mail_mess="`date +%F\ %T`: vrrp 转移,$HOSTNAME 变为 $1"
echo "$mail_mess" | mail -s "$mail_subj" $mail_dest
}
case $1 in
master)
mail_send master
;;
backup)
mail_send backup
;;
fault)
mail_send fault
;;
*)
exit 1
;;
esac
[root@ka1 ~]# chmod +x /etc/keepaliced/mail.sh
[root@ka1 ~]# vim /etc/keepaliced/keepalived.conf
将文件中:
notify_master "/etc/keepalived/mail.sh master"
notify_backup "/etc/keepalived/mail.sh backup"
notify_fault "/etc/keepalived/mail.sh fault"
调用。
实现 master/master 的 Keepalived 双主架构:
KA1:
vrrp_instance VI_1 {
state MASTER
interface eth0
virtual_router_id 100
priority 100
# preempt_delay 5s
advert_int 1
# nopreempt
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
172.25.254.100/24 dev eth0 label eth0:1
}
unicast_src_ip 172.25.254.10
unicast_peer {
172.25.254.20
}
track_script {
check_haproxy
}
}
vrrp_instance VI_2 {
state BACKUP
interface eth0
virtual_router_id 200
priority 80
# preempt_delay 5s
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
172.25.254.200/24 dev eth0 label eth0:2
}
unicast_src_ip 172.25.254.10
unicast_peer {
172.25.254.20
}
}
KA2:
vrrp_instance VI_1 {
state BACKUP
interface eth0
virtual_router_id 100
priority 80
advert_int 1
# nopreempt
preempt_delay 5s
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
172.25.254.100/24 dev eth0 label eth0:1
}
unicast_src_ip 172.25.254.20
unicast_peer {
172.25.254.10
}
track_script {
check_haproxy
}
}
vrrp_instance VI_2 {
state MASTER
interface eth0
virtual_router_id 200
priority 100
advert_int 1
# nopreempt
preempt_delay 5s
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
172.25.254.200/24 dev eth0 label eth0:2
}
unicast_src_ip 172.25.254.20
unicast_peer {
172.25.254.10
}
}
测试:
[root@Ka2 ~]# systemctl stop keepalived.service
六、实现IPVS的高可用性:
1. 虚拟服务器配置结构
2.virtual server (虚拟服务器)的定义格式
virtual_server IP port # 定义虚拟主机 IP 地址及其端口
virtual_server fwmark int #ipvs 的防火墙打标,实现基于防火墙的负载均衡集群
virtual_server group string # 使用虚拟服务器组
rs配置:
[root@realserver1 ~]# ip a a172.25.254.100 dev lo
[root@realserver1 ~]# vim /etc/sysctl.d/arp.conf
net.ipv4.conf.all.arp_ignore=1
net.ipv4.conf.lo.arp_ignore=1
net.ipv4.conf.all.arp_announce=2
net.ipv4.conf.lo.arp_announce=2
[root@realserver1 ~]# sysctl --system
[root@realserver2 ~]# ip a a172.25.254.100 dev lo
[root@realserver2 ~]# vim /etc/sysctl.d/arp.conf
net.ipv4.conf.all.arp_ignore=1
net.ipv4.conf.lo.arp_ignore=1
net.ipv4.conf.all.arp_announce=2
net.ipv4.conf.lo.arp_announce=2
[root@realserver2 ~]# sysctl --system
3.ka1主机和ka2主机配置keepalived服务:
[root@ka1 ~]# vim /etc/keepalived/keepalived.conf
virtual_server 172.25.254.100 80 {
delay_loop 6
lb_algo wrr
lb_kind DR
protocol TCP
real_server 172.25.254.110 80 {
weight 1
HTTP_GET {
url {
path /
status_code 200
}
connect_timeout 3
nb_get_retry 2
delay_before_retry 2
}
}
real_server 172.25.254.120 80 {
weight 1
HTTP_GET {
url {
path /
status_code 200
}
connect_timeout 3
nb_get_retry 2
delay_before_retry 2
}
}
}
[root@ka1 ~]# systemctl restart keepalived.service
测试:
[root@ka1 ~]# ipvsadm -Ln
[root@ka2 ~] for i in {1..6}; do curl 172.25.254.100; done
realserver2 - 172.25.254.120
realserver1 - 172.25.254.110
realserver2 - 172.25.254.120
realserver1 - 172.25.254.110
realserver2 - 172.25.254.120
realserver1 - 172.25.254.110
实现其它应用的高可用性:
keepalived 利用 VRRP Script 技术,可以调用外部的辅助脚本进行资源监控,并根据监控的结果实现优先动态调整,从而实现其它应用的高可用性功能
定义脚本
vrrp_script :自定义资源监控脚本, vrrp 实例根据脚本返回值,公共定义,可被多个实例调用,定
义在 vrrp 实例之外的独立配置块,一般放在 global_defs 设置块之后。
通常此脚本用于监控指定应用的状态。一旦发现应用的状态异常,则触发对 MASTER 节点的权重减至低于SLAVE 节点,从而实现 VIP 切换到 SLAVE 节点
1.利用脚本实现主从角色切换
[root@ka1 ~]# vim /etc/keepalived/haproxy.sh
[root@ka1 ~]# chmod +x /etc/keepalived/haproxy.sh
[root@ka1 ~]# cat /etc/keepalived/haproxy.sh
#!/bin/bash
[ ! -f /mnt/byh ]
[root@ka1 ~]# vim /etc/keepalived/keepalived.conf
[root@ka1 ~]# vim /etc/keepalived/keepalived.conf
vrrp_script check_haproxy {
script "/etc/keepalived/haproxy.sh"
interval 1
weight -30
fall 2
rise 2
timeout 2
}
vrrp_instance VI_1 {
state MASTER
interface eth0
virtual_router_id 100
priority 100
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
172.25.254.100/24 dev eth0 label eth0:1
}
unicast_src_ip 172.25.254.10
unicast_peer {
172.25.254.20
}
track_script {
check_haproxy
}
}
[root@ka1 ~]# systemctl restart keepalived.service
重启keepalived服务,如果存在 /mnt/byh 文件的话,ka1的172.25.254.100虚拟IP将被ka2抢占。
测试:
[root@ka1 ~]# touch /mnt/byh
2.实现HAProxy高可用:
[root@ka1 ~]# yum install haproxy -y
[root@ka1 ~]# systemctl enable --now haproxy
[root@ka1 ~]# vim /etc/sysctl.conf
net.ipv4.ip_nonlocal_bind = 1 # 启用内核参数ks1 ka2
[root@ka1 ~]# sysctl -p
[root@ka1 ~]# vim /etc/haproxy/haproxy.cfg
listen webserver
mode http
balance roundrobin
bind 172.25.254.100:80
server web1 172.25.254.110:80 check
server web2 172.25.254.120:80 check
[root@ka1 ~]# systemctl restart haproxy.service
[root@ka1 ~]# vim /etc/keepalived/haproxy.sh # 在ka1和ka2中编写检测脚本
[root@ka1 ~]# cat /etc/keepalived/haproxy.sh
#!/bin/bash
killall -0 haproxy
[root@ka1 ~]# chmod +x /etc/keepalived/haproxy.sh # 并且给这个文件赋予权限
[root@ka1 ~]# sh /etc/keepalived/haproxy.sh
[root@ka1 ~]#echo $?
在KA1和kA2中keepalived文件
将ARP响应规则全部改为0
[root@realserver1 ~]# echo 0 > /proc/sys/net/ipv4/conf/all/arp_ignore
[root@realserver1 ~]# echo 0 > /proc/sys/net/ipv4/conf/lo/arp_ignore
[root@realserver1 ~]# echo 0 > /proc/sys/net/ipv4/conf/lo/arp_announce
[root@realserver1 ~]# echo 0 > /proc/sys/net/ipv4/conf/all/arp_announce
[root@ka1 ~]# sysctl -p
[root@ka1 ~]#systemctl stop haproxy.service