目录
1.5.VRRP:Virtual Router Redundancy Protocol
3.7延迟抢占模式(需要停用全局配置中的vrrp_stric)
3.8组播变单播(需要停用全局配置中的vrrp_stric)
3.10实现 master/master 的 Keepalived 双主架构
高可用集群KEEPALIVED
解决高可用的问题,同样也支持后端的健康检测,加强版的haproxy
一.高可用keepalived介绍
1.1 集群类型
LB:
Load Balance
负载均衡
LVS/HAProxy/nginx(
http/upstream, stream/upstream
)
HA:
High Availability
高可用集群 数据库、Redis
SPoF: Single Point of Failure,解决单点故障
HPC:High Performance Computing
高性能集群
1.2 系统可用性
SLA:
Service-Level Agreement
服务等级协议(提供服务的企业与客户之间就服务的品质、 水准、性能等方面所达成的双方共同认可的协议或契约)
A = MTBF / (MTBF+MTTR
)
指标 :
99.9%, 99.99%, 99.999%,99.9999%
1.3 系统故障
硬件故障:设计缺陷、wear out
(损耗)、非人为不可抗拒因素 、软件故障:设计缺陷 bug
1.4 实现高可用
提升系统高用性的解决方案:降低MTTR- Mean Time To Repair(
平均故障时间
)
解决方案:建立冗余机制
active/passive 主
/
备
active/active 双主
active --> HEARTBEAT --> passive
active <--> HEARTBEAT <--> active
1.5.VRRP:Virtual Router Redundancy Protocol
虚拟路由冗余协议,
解决静态网关单点风险
物理层:
路由器、三层交换机
软件层:keepalived
1.5.1 VRRP 相关术语
虚拟路由器:Virtual Router
虚拟路由器标识:VRID(0-255)
,唯一标识虚拟路由器
VIP:
Virtual IP
VMAC:
Virutal MAC (00-00-5e-00-01-VRID)
物理路由器:
master:主设备
backup:备用设备
priority:优先级
1.5.2 VRRP 相关技术
通告:心跳,优先级等;周期性
工作方式:抢占式,非抢占式
安全认证:
无认证
简单字符认证:预共享密钥
MD5
工作模式:
主/备:单虚拟路由器
主/主:主/
备(虚拟路由器
1
),备
/
主(虚拟路由器
2
)
二.keepalived安装和配置文件
2.1 Keepalived 安装
[root@KA1 ~]# dnf install keepalived -y
[root@KA1 ~]# systemctl start keepalived
[root@KA1 ~]# ps axf | grep keepalived
2385 pts/0 S+ 0:00 \_ grep --color=auto keepalived
2326 ? Ss 0:00 /usr/sbin/keepalived -D
2327 ? S 0:00 \_ /usr/sbin/keepalived -D
配置文件:/etc/keepalived/keepalived.conf
配置文件组成
GLOBAL CONFIGURATION
Global definitions: 定义邮件配置,route_id,vrrp配置,多播地址等
VRRP CONFIGURATION
VRRP instance(s): 定义每个vrrp虚拟路由器
LVS CONFIGURATION
Virtual server group(s)
Virtual server(s): LVS集群的VS和RS
用户空间核心组件:
vrrp stack:VIP消息通告
checkers:监测real server
system call:实现 vrrp 协议状态转换时调用脚本的功能
SMTP:邮件组件
IPVS wrapper:生成IPVS规则
Netlink Reflector:网络接口
WatchDog:监控进程
控制组件:提供keepalived.conf 的解析器,完成Keepalived配置
IO复用器:针对网络目的而优化的自己的线程抽象
内存管理组件:为某些通用的内存管理功能(例如分配,重新分配,发布等)提供访问权限
三.keepalived实验
3.1环境配置
keep1 :172.25.254.10
keep2 :172.25.254.20
rserver1:172.25.254.110
rserver2:172.25.254.120
#rserver1
[root@rserver1 ~]# yum install httpd -y
[root@rserver1 ~]# echo 172.25.254.110 > /var/www/html/index.html
[root@rserver1 ~]# systemctl enable --now httpd
#rserver2
[root@rserver2 ~]# yum install httpd -y
[root@rserver2 ~]# echo 172.25.254.120 > /var/www/html/index.html
[root@rserver2 ~]# systemctl enable --now httpd
3.2keepalived虚拟路由器
[root@keep1 ~]# yum install keepalived -y
[root@keep1 ~]# vim /etc/keepalived/keepalived.conf
global_defs {
notification_email {
acassen@firewall.loc
failover@firewall.loc
sysadmin@firewall.loc
}
notification_email_from Alexandre.Cassen@firewall.loc
smtp_server 192.168.200.1
#邮件服务器地址
smtp_connect_timeout 30
router_id keep1.timinglee.org
#每个keepalived主机唯一标识#建议使用当前主机名,但多节点重名不影响
vrrp_skip_check_adv_addr
#启用此配置后,如果收到的通告报文和上一个报文是同一 个路由器,则跳过检查,默认值为全检查
vrrp_strict
vrrp_garp_interval 0
#报文发送延迟,0表示不延迟
vrrp_gna_interval 0
#消息发送延迟
vrrp_mcast_group4 224.0.0.18
# #指定组播IP地址范围
}
vrrp_instance VI_1 {
state MASTER
interface eth0
#绑定为当前虚拟路由器使用的物理接口,如:eth0,可以和VIP不在一个网卡
virtual_router_id 100
#每个虚拟路由器惟一标识,范围:0-255,每个虚拟路由器此值必须唯一
#否则服务无法启动
#同属一个虚拟路由器的多个keepalived节点必须相同
#务必要确认在同一网络中此值必须唯一
priority 100
#值越大优先级越高,每个keepalived主机节点此值不同
advert_int 1
#vrrp通告的时间间隔,默认1s
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
172.25.254.100/24 dev eth0 label eth0:1
#用于定义虚拟IP地址(VIP),这是高可用性(HA)解决方案中常见的做法,以确保在服务器故障时,服务能够通过另一个服务器上的相同IP地址继续提供。
}
}
[root@keep1 ~]# enable --now keepalived.service
[root@keep1 ~]# restart keepalived.service
[root@keep2 ~]# yum install keepalived -y
[root@keep2 ~]# vim /etc/keepalived/keepalived.conf
global_defs {
notification_email {
acassen@firewall.loc
failover@firewall.loc
sysadmin@firewall.loc
}
notification_email_from Alexandre.Cassen@firewall.loc
smtp_server 192.168.200.1
smtp_connect_timeout 30
router_id keep1.timinglee.org
vrrp_skip_check_adv_addr
vrrp_strict
vrrp_garp_interval 0
vrrp_gna_interval 0
vrrp_mcast_group4 224.0.0.18
}
vrrp_instance VI_1 {
state MASTER
interface eth0
virtual_router_id 100
priority 80
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
172.25.254.100/24 dev eth0 label eth0:1
}
}
[root@keep2 ~]# tcpdump -i eth0 -nn host 224.0.0.18
11:02:40.120434 IP 172.25.254.10 > 224.0.0.18: VRRPv2, Advertisement, vrid 100, prio 100, authtype simple, intvl 1s, length 20
########
[root@keep1 ~]# systemctl stop keepalived.service
[root@keep2 ~]# tcpdump -i eth0 -nn host 224.0.0.18
11:01:58.059105 IP 172.25.254.20 > 224.0.0.18: VRRPv2, Advertisement, vrid 100, prio 80, authtype simple, intvl 1s, length 20
3.3如何ping通虚拟vip
在keep1和keep2主机里面的global全局参数中添加 vrrp_iptables,重启服务可以ping通vip了;也可以#vrrp_strict
vim /etc/keepalived/keepalived.conf
global_defs {
vrrp_strict
vrrp_iptables
}
3.4独立子配置文件
[root@keep1 ~]# vim /etc/keepalived/keepalived.conf
include "/etc/keepalived/conf.d/*.conf"
[root@keep1 ~]# mkdir -p /etc/keepalived/conf.d/
[root@keep1 ~]# vim /etc/keepalived/conf.d/keep1.conf
vrrp_instance VI_1 {
state MASTER
interface eth0
virtual_router_id 100
priority 100
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
172.25.254.100/24 dev eth0 label eth0:1
}
}
[root@keep1 ~]# systemctl restart keepalived.service
3.5独立日志
[root@keep1 ~]# vim /etc/sysconfig/keepalived
KEEPALIVED_OPTIONS="-D -S 6"
[root@keep1 ~]# vim /etc/rsyslog.conf
local6.* /var/log/keepalived.log
[root@keep1 ~]# systemctl restart keepalived.service
[root@keep1 ~]# systemctl restart rsyslog.service
[root@keep1 ~]# ll /var/log/keepalived.log
-rw------- 1 root root 724 8月 12 14:02 /var/log/keepalived.log
3.6非抢占式优先级
默认为抢占模式preempt,即当高优先级的主机恢复在线后,会抢占低先级的主机的master角色,
这样会使vip在KA主机中来回漂移,造成网络抖动,
建议设置为非抢占模式 nopreempt ,即高优先级主机恢复后,并不会抢占低优先级主机的master角色
非抢占模块下,如果原主机down机, VIP迁移至的新主机, 后续也发生down时,仍会将VIP迁移回原主机
注意:要关闭 VIP抢占,必须将各 keepalived 服务器state配置为BACKUP
[root@keep1 ~]# vim /etc/sysconfig/keepalived
vrrp_instance VI_1 {
state BACKUP
#两台都要修改为backup模式
nopreempt
#非抢占优先级
virtual_router_id 100
priority 100
}
[root@keep1 ~]# systemctl restart keepalived.service
[root@keep2 ~]# vim /etc/sysconfig/keepalived
vrrp_instance VI_1 {
state BACKUP
#两台都要修改为backup模式
nopreempt
#非抢占优先级
interface eth0
virtual_router_id 100
priority 80
}
[root@keep2 ~]# systemctl restart keepalived.service
3.7延迟抢占模式(需要停用全局配置中的vrrp_stric)
抢占延迟模式,即优先级高的主机恢复后,不会立即抢回VIP,而是延迟一段时间(默认300s)再抢回
VIP
注意:需要各keepalived服务器state为BACKUP,并且不要启用 vrrp_strict
[root@keep1 ~]# vim /etc/sysconfig/keepalived
vrrp_instance VI_1 {
state BACKUP
#修改为backup
preempt_delay 5s
#恢复后,延迟5s
interface eth0
virtual_router_id 100
priority 100
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
172.25.254.100/24 dev eth0 label eth0:1
}
}
[root@keep1 ~]# systemctl restart keepalived.service
[root@keep2 ~]# vim /etc/sysconfig/keepalived
vrrp_instance VI_1 {
state BACKUP
#修改为backup
preempt_delay 5s
#恢复后,延迟5s
interface eth0
virtual_router_id 100
priority 80
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
172.25.254.100/24 dev eth0 label eth0:1
}
}
[root@keep2 ~]# systemctl restart keepalived.service
3.8组播变单播(需要停用全局配置中的vrrp_stric)
[root@keep1 ~]# vim /etc/sysconfig/keepalived
vrrp_instance VI_1 {
state MASTER
interface eth0
virtual_router_id 100
priority 100
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
172.25.254.100/24 dev eth0 label eth0:1
}
unicast_src_ip 172.25.254.10
#广播源地址ip
unicast_peer {
#接受广播端ip
172.25.254.20
}
}
[root@keep1 ~]# systemctl restart keepalived.service
[root@keep1 ~]# tcpdump -i eth0 -nn src host 172.25.254.20 and dst 172.25.254.10
#在vip在本机的时候
[root@keep2 ~]# vim /etc/sysconfig/keepalived
vrrp_instance VI_1 {
state BACKUP
#preempt_delay 5s
interface eth0
virtual_router_id 100
priority 80
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
172.25.254.100/24 dev eth0 label eth0:1
}
unicast_src_ip 172.25.254.20
#广播源地址ip
unicast_peer {
#接受广播端ip
172.25.254.10
}
}
[root@keep2 ~]# systemctl restart keepalived.service
[root@keep1 ~]# tcpdump -i eth0 -nn src host 172.25.254.20 and dst 172.25.254.10
3.9实现 Keepalived 状态切换的通知脚本
[root@keep1 ~]#dnf install mailx -y
[root@keep1 ~]#vim /etc/mail.rc
set from=2784117361@qq.com
set smtp=smtp.qq.com
set smtp-auth-user=2784117361@qq.com
set smtp-auth-password=gjtqeiajudzldfdd
set smtp-auth=login
set ssl-verify=ignore
~
[root@keep1 ~]# vim /etc/keepalived/mail.sh
#!/bin/bash
mail_who=2784117361@qq.com
hostname='keep1'
date1=$(date +'%F %T')
mail_send()
{
mail_subj="$hostname to be $1 vip 转移"
mail_mess="$date1发生了vrrp 转移,$hostname 变为$1"
echo "$mail_mess" | mail -s "$mail_subj" $mail_who
}
case $1 in
master)
mail_send master
;;
backup)
mail_send backup
;;
fault)
mail_send fault
;;
*)
;;
esac
[root@keep1 ~]# vim /etc/keepalived/keepalived.conf
vrrp_instance VI_1 {
state MASTER
interface eth0
virtual_router_id 100
priority 100
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
172.25.254.100/24 dev eth0 label eth0:1
}
notify_master "/etc/keepalived/mail.sh master"
notify_backup "/etc/keepalived/mail.sh backup"
notify_fault "/etc/keepalived/mail.sh fault"
unicast_src_ip 172.25.254.10
unicast_peer {
172.25.254.20
}
}
[root@keep1 ~]# systemctl restart keepalived.service
3.10实现 master/master 的 Keepalived 双主架构
[root@keep1 ~]# vim /etc/keepalived/keepalived.conf
vrrp_instance VI_1 {
state MASTER
interface eth0
virtual_router_id 100
priority 100
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
172.25.254.100/24 dev eth0 label eth0:1
}
unicast_src_ip 172.25.254.10
unicast_peer {
172.25.254.20
}
}
vrrp_instance VI_2 {
state BACKUP
interface eth0
virtual_router_id 200
priority 80
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
172.25.254.200/24 dev eth0 label eth0:2
}
unicast_src_ip 172.25.254.10
unicast_peer {
172.25.254.20
}
}
[root@keep1 ~]# systemctl restart keepalived.service
[root@keep2 ~]# vim /etc/keepalived/keepalived.conf
vrrp_instance VI_1 {
state BACKUP
interface eth0
virtual_router_id 100
priority 80
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
172.25.254.100/24 dev eth0 label eth0:1
}
unicast_src_ip 172.25.254.20
unicast_peer {
172.25.254.10
}
}
vrrp_instance VI_2 {
state MASTER
interface eth0
virtual_router_id 200
priority 100
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
172.25.254.200/24 dev eth0 label eth0:2
}
unicast_src_ip 172.25.254.10
unicast_peer {
172.25.254.20
}
}
[root@keep2 ~]# systemctl restart keepalived.service
3.11实现单主的 LVS-DR 模式(keep+lvs)
keepalived服务器1
########################################################
[root@keep1 ~]# vim /etc/keepalived/keepalived.conf
vrrp_instance VI_1 {
state MASTER
interface eth0
virtual_router_id 100
priority 100
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
172.25.254.100/24 dev eth0 label eth0:1
}
unicast_src_ip 172.25.254.10
unicast_peer {
172.25.254.20
}
}
virtual_server 172.25.254.100 80 {
delay_loop 6
lb_algo wrr
lb_kind DR
#persistence_timeout 50
protocol TCP
real_server 172.25.254.110 80 {
weight 1
HTTP_GET {
url {
path /
status_code 200
}
connect_timeout 3
nb_get_retry 3
delay_before_retry 3
}
}
real_server 172.25.254.120 80 {
weight 1
HTTP_GET {
url {
path /
status_code 200
}
connect_timeout 3
nb_get_retry 3
delay_before_retry 3
}
}
}
[root@keep1 ~]# systemctl restart keepalived.service
############################################################
keepalived服务器
[root@keep2 ~]# vim /etc/keepalived/keepalived.conf
vrrp_instance VI_1 {
state BACKUP
interface eth0
virtual_router_id 100
priority 80
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
172.25.254.100/24 dev eth0 label eth0:1
}
unicast_src_ip 172.25.254.20
unicast_peer {
172.25.254.10
}
}
virtual_server 172.25.254.100 80 {
delay_loop 6
lb_algo wrr
lb_kind DR
#persistence_timeout 50
protocol TCP
real_server 172.25.254.110 80 {
weight 1
HTTP_GET {
url {
path /
status_code 200
}
connect_timeout 3
nb_get_retry 3
delay_before_retry 3
}
}
real_server 172.25.254.120 80 {
weight 1
HTTP_GET {
url {
path /
status_code 200
}
connect_timeout 3
nb_get_retry 3
delay_before_retry 3
}
}
}
[root@keep2 ~]# systemctl restart keepalived.service
########################################################
[root@rserver1 ~]# ip a a 172.25.254.100/32 dev lo
[root@rserver1 ~]# vim /etc/sysctl.d/arp.conf
net.ipv4.conf.all.arp_ignore=1
net.ipv4.conf.all.arp_announce=2
net.ipv4.conf.lo.arp_ignore=1
net.ipv4.conf.lo.arp_announce=2
[root@rserver1 ~]# sysctl --system
[root@rserver1 ~]# sysctl -p
###########################################################
[root@rserver2 ~]# ip a a 172.25.254.100/32 dev lo
[root@rserver2 ~]# vim /etc/sysctl.d/arp.conf
net.ipv4.conf.all.arp_ignore=1
net.ipv4.conf.all.arp_announce=2
net.ipv4.conf.lo.arp_ignore=1
net.ipv4.conf.lo.arp_announce=2
[root@rserver2 ~]# sysctl --system
[root@rserver2 ~]# sysctl -p
[root@rserver2 ~]# sysctl --system
3.12利用keepalived实现HAProxy高可用
HAProxy:HAProxy是一个高性能的TCP/HTTP反向代理服务器和负载均衡器。在这个实验中,HAProxy被配置为监听所有发往172.25.254.100(虚拟IP,VIP)的HTTP请求,并将这些请求以轮询(roundrobin)的方式分发到后端的两台Web服务器(172.25.254.110和172.25.254.120)上。
Keepalived:Keepalived用于实现服务器的健康检查和故障转移。它通过VRRP(Virtual Router Redundancy Protocol)协议来确保服务的高可用性。在这个配置中,Keepalived监控HAProxy的健康状态,如果HAProxy出现故障,则可以将VIP转移到另一台健康的服务器上。
VIP(虚拟IP):VIP是浮动的,不直接绑定在任何物理服务器上。它根据Keepalived的配置和状态,动态地绑定到当前的MASTER服务器上。这样,无论MASTER服务器是否发生故障,客户端都可以通过VIP访问到后端的Web服务。
健康检查:HAProxy和Keepalived都配置了健康检查。HAProxy通过check指令定期检查后端服务器的状态;Keepalived则通过执行自定义脚本(如/etc/keepalived/haproxy.sh)来检查HAProxy的运行状态。
#rserver1
[root@rserver1 ~]# yum install httpd -y
[root@rserver1 ~]# echo 172.25.254.110 > /var/www/html/index.html
[root@rserver1 ~]# systemctl enable --now httpd
#rserver2
[root@rserver2 ~]# yum install httpd -y
[root@rserver2 ~]# echo 172.25.254.120 > /var/www/html/index.html
[root@rserver2 ~]# systemctl enable --now httpd
#########################################################################################
#keep1
[root@keep1 ~]# vim /etc/sysctl.conf
net.ipv4.ip_nonlocal_bind=1
[root@keep1 ~]# sysctl -p
[root@keep1 ~]# yum install haproxy -y
[root@keep1 ~]# vim /etc/haproxy/haproxy.cfg
listen webserver
bind 172.25.254.100:80
mode http
balance roundrobin
server web1 172.25.254.110:80 check inter 2 fall 3 rise 5 weight 1
server web2 172.25.254.120:80 check inter 2 fall 3 rise 5 weight 1
[root@keep1 ~]# systemctl restart haproxy.service
vrrp_script check_haproxy {
script "/etc/keepalived/haproxy.sh"
interval 1
weight -30
fall 2
rise 2
timeout 2
}
vrrp_instance VI_1 {
state MASTER
interface eth0
virtual_router_id 100
priority 100
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
172.25.254.100/24 dev eth0 label eth0:1
}
track_script {
check_haproxy
}
unicast_src_ip 172.25.254.10
unicast_peer {
172.25.254.20
}
}
[root@keep1 ~]# systemctl restart keepalived.service
############################################################################################
#keep2
[root@keep2 ~]# vim /etc/sysctl.conf
net.ipv4.ip_nonlocal_bind=1
#因为我们使用keepalived,vip会到两台keep服务器,指定这个参数,可以实现没有vip也可以调用haproxy
[root@keep2 ~]# sysctl -p
[root@keep2 ~]# yum install haproxy -y
[root@keep2 ~]# vim /etc/haproxy/haproxy.cfg
listen webserver
bind 172.25.254.100:80
mode http
balance roundrobin
server web1 172.25.254.110:80 check inter 2 fall 3 rise 5 weight 1
server web2 172.25.254.120:80 check inter 2 fall 3 rise 5 weight 1
[root@keep2 ~]# systemctl restart haproxy.service
[root@keep2 ~]# vim /etc/keepalived/haproxy.sh
#!/bin/bash
killall -0 haproxy
[root@keep2 ~]# chmod +x /etc/keepalived/haproxy.sh
[root@keep2 ~]# vim /etc/keepalived/keepalived.conf
vrrp_instance VI_1 {
state BACKUP
interface eth0
virtual_router_id 100
priority 80
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
172.25.254.100/24 dev eth0 label eth0:1
}
track_script {
check_haproxy
}
unicast_src_ip 172.25.254.20
unicast_peer {
172.25.254.10
}
}
[root@keep2 ~]# systemctl restart keepalived.service