基本介绍
Keepalived 是一个基于 VRRP(虚拟路由器冗余协议)协议来实现高可用性的服务软件,主要用于负载均衡和故障转移。它最初是为 LVS(Linux 虚拟服务器)设计的,用于管理并监控 LVS 集群系统中各个服务节点的状态,并后来加入了可以实现高可用的 VRRP 功能。
Keepalived 的主要功能包括:
- 防止单点故障:通过在两个或多个服务器上运行 Keepalived 实现双机热备(High Availability, HA),当主服务器出现故障时,备份服务器会接管其任务,从而保证系统的连续运行。
- 负载均衡:通过将请求分发到多个服务器上,避免单个服务器过载,提高系统的整体性能和可靠性。
- 故障隔离与切换:Keepalived 可以检测到主服务器的异常状态(如连接失败、服务停止等),并触发故障转移机制,使备份服务器接替主服务器的工作。
一个典型的 Keepalived 配置包括一台主服务器(MASTER)和一台备份服务器(BACKUP),它们共同对外表现为一个虚拟 IP 地址。主服务器会定期发送特定的消息给备份服务器,以确认其在线状态。如果主服务器未能接收到这些消息,备份服务器将自动接管主服务器的任务。
Keepalived 还支持多种其他服务的高可用解决方案,例如 Nginx、Haproxy 和 MySQL 等。它的配置文件包含全局配置、VRRP 配置以及 LVS 配置,所有配置都在一个闭合的大括号内进行定义。
Keepalived 是一种强大的高可用性工具,能够有效防止单点故障,确保系统的稳定性和可靠性。通过其简单的配置和强大的功能,Keepalived 已成为许多企业级应用中的首选高可用解决方案.
配置案例
环境准备
主机网络配置
主机名 | ip | vip |
KA1 | 172.25.254.10 | 172.25.254.100 |
KA2 | 172.25.254.20 | |
webserver1 | 172.25.254.110 | \ |
webserver2 | 172.25.254.120 | \ |
- 各节点时间必须同步:ntp, chrony
- 关闭防火墙及SELinux
- 各节点之间可通过主机名互相通信:非必须
- 建议使用/etc/hosts文件实现:非必须
- 各节点之间的root用户可以基于密钥认证的ssh服务完成互相通信:非必须
安装keepalived
[root@localhost ~]# yum install keepalived -y
启用keepaliced日志功能
配置独立的子配置文件
单主架构
master配置
[root@localhost ~]# mkdir /etc/keepalived/conf.d
backup配置
测试检查
抢占模式和非抢占模式
两边主机配置相同
vip单播配置
KA1配置
KA2配置
测试效果
把KA1服务停了后测
Email通知配置
通知脚本的编写
vim /etc/keepalived/mail.sh
#!/bin/bash
mail_dest='yourQQ@qq.com'
mail_send()
{
mail_subj="$HOSTNAME to be $1 vip 转移"
mail_mess="`date +%F\ %T`: vrrp 转移,$HOSTNAME 变为 $1"
echo "$mail_mess" | mail -s "$mail_subj" $mail_dest
}
case $1 in
master)
mail_send master
;;
backup)
mail_send backup
;;
fault)
mail_send fault
;;
*)
exit 1
;;
esac
安装邮件发送工具
dnf install mailx -y
QQ邮箱配置
vim /etc/mail.rc
#######mail set##########
set from=yourQQ@qq.com
set smtp=smtp.qq.com
set smtp-auth-user=594233887@qq.com
set smtp-auth-password=isjatjwmcxtxbefj
set smtp-auth=login
set ssl-verify=ignore
注意,qq邮箱要打开smtp服务
发送邮件测试
echo test message |mail -s test yourQQ@qq.com
#模拟master故障
killall keepalived
Keepalived 双主架构
KA1
KA2
测试
实现单主的 LVS-DR 模式
#准备两台后端RS主机
[root@rs1 ~]# yum install httpd -y
[root@rs1 ~]# echo RS1 - 172.25.254.101 > /var/www/html/index.html
[root@rs1 ~]# ip addr add 172.25.254.100/32 dev lo
[root@rs1 ~]# echo 1 > /proc/sys/net/ipv4/conf/all/arp_ignore
[root@rs1 ~]# echo 1 > /proc/sys/net/ipv4/conf/lo/arp_ignore
[root@rs1 ~]# echo 2 > /proc/sys/net/ipv4/conf/lo/arp_announce
[root@rs2 ~]# yum install httpd -y
[root@rs1 ~]# echo RS1 - 172.25.254.101 > /var/www/html/index.html
[root@rs2 ~]# ip addr add 172.25.254.100/32 dev lo
[root@rs2 ~]# echo 1 > /proc/sys/net/ipv4/conf/all/arp_ignore
[root@rs2 ~]# echo 1 > /proc/sys/net/ipv4/conf/lo/arp_ignore
[root@rs2 ~]# echo 2 > /proc/sys/net/ipv4/conf/lo/arp_announce
[root@node30 ~]# yum install httpd -y
[root@node30 ~]# echo RS1 - 172.25.254.101 > /var/www/html/index.html
[root@node30 ~]# ip addr add 172.25.254.100/32 dev lo
[root@node30 ~]# echo 1 > /proc/sys/net/ipv4/conf/all/arp_ignore
[root@node30 ~]# echo 1 > /proc/sys/net/ipv4/conf/lo/arp_ignore
[root@node30~]# echo 2 > /proc/sys/net/ipv4/conf/lo/arp_announce
#ka1节点的配置
[root@rhel7-ka1 ~]# vim /etc/keepalived/keepalived.conf
virtual_server 172.25.254.100 80 {
delay_loop 6
lb_algo wrr
lb_kind DR
protocol TCP
sorry_server 172.25.254.30
real_server 172.25.254.101 80 {
weight 1
TCP_CHECK {
connect_timeout 5
nb_get_retry 3
delay_before_retry 3
connect_port 80
}
}
real_server 172.25.254.102 80 {
weight 1
HTTP_GET {
url {
path /
status_code 200
}
connect_timeout 1
nb_get_retry 3
delay_before_retry 1
}
}
}
#ka2节点的配置,配置和ka1基本相同,只需修改三行
[root@rhel7-ka2 ~]# vim /etc/keepalived/keepalived.conf
virtual_server 172.25.254.100 80 {
delay_loop 6
lb_algo wrr
lb_kind DR
protocol TCP
sorry_server 172.25.254.30
real_server 172.25.254.101 80 {
weight 1
TCP_CHECK {
connect_timeout 5
nb_get_retry 3
delay_before_retry 3
connect_port 80
}
}
real_server 172.25.254.102 80 {
weight 1
HTTP_GET {
url {
path /
status_code 200
}
connect_timeout 1
nb_get_retry 3
delay_before_retry 1
}
}
}
#访问测试结果
[Administrator.WIN-20240602BIS] ➤ for i in {1..6}; do curl 172.25.254.100; done
RS1 - 172.25.254.101
RS2 - 172.25.254.102
RS1 - 172.25.254.101
RS2 - 172.25.254.102
RS1 - 172.25.254.101
RS2 - 172.25.254.102
[root@rhel7-ka1 ~]# ipvsadm -Ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
TCP 172.25.254.100:80 wrr
-> 172.25.254.101:80 Route 1 0 6
-> 172.25.254.102:80 Route 1 0 6
#第一台RS1故障,自动切换至RS2
[root@rs1 ~]# systemctl stop httpd #当RS1故障
[Administrator.WIN-20240602BIS] ➤ for i in {1..6}; do curl 172.25.254.100; done
#全部流浪被定向到RS2中
RS2 - 172.25.254.102
RS2 - 172.25.254.102
RS2 - 172.25.254.102
RS2 - 172.25.254.102
3.6.2.2 实战案例2:实现双主的 LVS-DR 模式
RS2 - 172.25.254.102
RS2 - 172.25.254.102
[root@rhel7-ka1 ~]# ipvsadm -Ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
TCP 172.25.254.100:80 wrr
-> 172.25.254.102:80 Route 1 0 12 #RS1被踢出保留
RS2
#后端RS服务器都故障,启动Sorry Server
[root@rs2 ~]#systemctl stop httpd
[Administrator.WIN-20240602BIS] ➤ for i in {1..6}; do curl 172.25.254.100; done
sorry server
sorry server
sorry server
sorry server
sorry server
sorry server
[root@rhel7-ka1 ~]# ipvsadm -Ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
TCP 172.25.254.100:80 wrr
-> 172.25.254.30:80 Route 1 0 3
#陆续启动RS1 RS2
[root@rhel7-ka1 ~]# ipvsadm -Ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
TCP 172.25.254.100:80 wrr
-> 172.25.254.101:80 Route 1 0 3
-> 172.25.254.102:80 Route 1 0 9
#ka1故障,自动切换至ka2
[root@rhel7-ka1 ~]# systemctl stop keepalived.service
[root@rhel7-ka2 ~]# ipvsadm -Ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
TCP 172.25.254.100:80 wrr
-> 172.25.254.101:80 Route 1 0 0
-> 172.25.254.102:80 Route 1 0 0
实现双主的 LVS-DR 模式
[root@ka1-centos8 ~]#vim /etc/keepalived/keepalived.conf
! Configuration File for keepalived
global_defs {
notification_email {
root@localhost
}
notification_email_from keepalived@localhost
smtp_server 127.0.0.1
smtp_connect_timeout 30
router_id ka1.magedu.org #另一个节点为ka2.magedu.org
vrrp_mcast_group4 224.0.100.10
}
vrrp_instance VI_1 {
state MASTER #在另一个结点上为BACKUP
interface eth0
virtual_router_id 66
priority 100
#在另一个结点上为80
advert_int 1
authentication {
auth_type PASS
auth_pass 123456
}
virtual_ipaddress {
10.0.0.10/24 dev eth0 label eth0:1 #指定VIP
}
}
vrrp_instance VI_2 {
state BACKUP #在另一个结点上为MASTER
interface eth0
virtual_router_id 88
priority 80
#在另一个结点上为100
advert_int 1
authentication {
auth_type PASS
auth_pass 123456
}
virtual_ipaddress {
10.0.0.20/24 dev eth0 label eth0:2 #指定VIP2
}
}
virtual_server 10.0.0.10 80 {
delay_loop 6
lb_algo rr
lb_kind DR
protocol TCP
sorry_server 127.0.0.1 80
real_server 10.0.0.7 80 { #指定RS1地址
weight 1
HTTP_GET {
url {
path /
status_code 200
}
connect_timeout 3
nb_get_retry 3
delay_before_retry 3
}
}
real_server 10.0.0.17 80 { #指定RS2地址
weight 1
HTTP_GET {
url {
path /
status_code 200
}
connect_timeout 3
nb_get_retry 3
delay_before_retry 3
}
}
}
virtual_server 10.0.0.20 80 { #指定VIP2
delay_loop 6
lb_algo rr
lb_kind DR
protocol TCP
sorry_server 127.0.0.1 80
real_server 10.0.0.27 80 { #指定RS3地址
weight 1
HTTP_GET {
url {
path /
status_code 200
}
connect_timeout 3
nb_get_retry 3
delay_before_retry 3
}
}
real_server 10.0.0.37 80 { #指定RS4地址
weight 1
HTTP_GET {
url {
path /
status_code 200
}
connect_timeout 3
nb_get_retry 3
delay_before_retry 3
}
}
}
双主分别实现httpd和MySQL服务调度
准备实验素材
在RS1和RS2中准备httpd和mysql环境
在RS1中:
[root@rs1 ~]# ip addr add 172.25.254.200/32 dev lo
[root@rs1 ~]# yum install mariadb-server -y
root@rs1 ~]# vim /etc/my.cnf
server-id=1
[root@rs1 ~]# systemctl enable --now mariadb
[root@rs1 ~]# mysql -e "grant ALL on *.* to lee@'%' identified by 'lee'"
[root@rs1 ~]# mysql -ulee -plee -h172.25.254.101 -e 'select @@server_id'
+-------------+
| @@server_id |
+-------------+
| 1 |
+-------------+
在RS2中:
[root@rs2 ~]# ip addr add 172.25.254.200/32 dev lo
[root@rs2 ~]# yum install mariadb-server -y
[root@rs2 ~]# vim /etc/my.cnf
server-id=2
[root@rs2 ~]# systemctl enable --now mariadb
[root@rs2 ~]# mysql -e "grant ALL on *.* to lee@'%' identified by 'lee'"
[root@rs2 ~]# mysql -ulee -plee -h172.25.254.102 -e 'select @@server_id'
+-------------+
| @@server_id |
+-------------+
| 2 |
+-------------+
#配置双主模式
[root@rhel7-ka1 ~]# vim /etc/keepalived/keepalived.conf
vrrp_instance web {
state MASTER
interface ens33
virtual_router_id 100
priority 100
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
172.25.254.100 dev ens33 label ens33:0
}
}
vrrp_instance sql {
state BACKUP
interface ens33
virtual_router_id 200
priority 80
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
172.25.254.200 dev ens33 label ens33:1
}
}
include "/etc/keepalived/conf.d/web.conf"
include "/etc/keepalived/conf.d/sql.conf"
[root@rhel7-ka1 ~]# vim /etc/keepalived/conf.d/web.conf
virtual_server 172.25.254.100 80 {
delay_loop 6
lb_algo wrr
lb_kind DR
protocol TCP
sorry_server 172.25.254.30 80
real_server 172.25.254.101 80 {
weight 1
TCP_CHECK {
connect_timeout 5
nb_get_retry 3
delay_before_retry 3
connect_port 80
}
}
real_server 172.25.254.102 80 {
weight 1
HTTP_GET {
url {
path /
status_code 200
}
connect_timeout 1
nb_get_retry 3
delay_before_retry 1
}
}
}
[root@rhel7-ka1 ~]# vim /etc/keepalived/conf.d/sql.conf
virtual_server 172.25.254.200 3306 {
delay_loop 3
lb_algo rr
lb_kind DR
protocol TCP
real_server 172.25.254.101 3306 {
weight 1
TCP_CHECK {
connect_timeout 5
nb_get_retry 3
delay_before_retry 3
connect_port 3306
}
}
real_server 172.25.254.102 3306 {
weight 1
TCP_CHECK {
connect_timeout 5
nb_get_retry 3
delay_before_retry 3
connect_port 3306
}
}
}
##在ka2中:
[root@rhel7-ka2 ~]# vim /etc/keepalived/keepalived.conf
vrrp_instance web {
state BACKUP #备机
interface eth0
virtual_router_id 50
priority 80
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
172.25.254.100 dev eth0 label eth0:0
}
}
vrrp_instance sql {
state MASTER #主机
interface eth0
virtual_router_id 60
priority 100
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
172.25.254.200 dev eth0 label eth0:1
}
}
include "/etc/keepalived/conf.d/web.conf"
include "/etc/keepalived/conf.d/sql.conf"
[root@rhel7-ka2 ~]# vim /etc/keepalived/conf.d/web.conf
virtual_server 172.25.254.100 80 {
delay_loop 6
lb_algo wrr
lb_kind DR
protocol TCP
sorry_server 172.25.254.30 80
real_server 172.25.254.101 80 {
weight 1
TCP_CHECK {
connect_timeout 5
nb_get_retry 3
delay_before_retry 3
connect_port 80
}
}
real_server 172.25.254.102 80 {
weight 1
HTTP_GET {
url {
path /
status_code 200
}
connect_timeout 1
nb_get_retry 3
delay_before_retry 1
}
}
}
[root@rhel7-ka2 ~]# vim /etc/keepalived/conf.d/web.conf
virtual_server 172.25.254.200 3306 {
delay_loop 3
lb_algo rr
lb_kind DR
protocol TCP
real_server 172.25.254.101 3306 {
weight 1
TCP_CHECK {
connect_timeout 5
nb_get_retry 3
delay_before_retry 3
#####
3.7 实现其它应用的高可用性 VRRP Script
keepalived利用 VRRP Script 技术,可以调用外部的辅助脚本进行资源监控,并根据监控的结果实现优先
动态调整,从而实现其它应用的高可用性功能
参考配置文件:/usr/share/doc/keepalived/keepalived.conf.vrrp.localcheck
connect_port 3306
}
}
real_server 172.25.254.102 3306 {
weight 1
TCP_CHECK {
connect_timeout 5
nb_get_retry 3
delay_before_retry 3
connect_port 3306
}
}
}
[root@rhel7-ka1 ~]# ipvsadm -Ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
TCP 172.25.254.100:80 wrr
-> 172.25.254.101:80 Route 1 0 0
-> 172.25.254.102:80 Route 1 0 0
TCP 172.25.254.200:3306 rr
-> 172.25.254.101:3306 Route 1 0 0
-> 172.25.254.102:3306 Route 1 0 0
#测试
[root@node30 html]# mysql -ulee -plee -h172.25.254.200 -e 'select @@server_id'
+-------------+
| @@server_id |
+-------------+
| 2 |
+-------------+
[root@node30 html]# mysql -ulee -plee -h172.25.254.200 -e 'select @@server_id'
+-------------+
| @@server_id |
+-------------+
| 1 |
+-------------+
[root@node30 html]# mysql -ulee -plee -h172.25.254.200 -e 'select @@server_id'
VRRP Script 配置
脚本配置
keepalived配置
vim /etc/keepalived/keepalived.conf
测试
关闭haproxy