实现 master/master 的 Keepalived 双主架构
一、keepalived简介
Keepalived 是一款用于实现高可用(High Availability)的软件工具。
它主要的作用是检测服务器的状态,并在主服务器出现故障时,自动将服务切换到备用服务器上,以确保服务的连续性和稳定性。Keepalived 基于虚拟路由冗余协议(VRRP)来工作。VRRP 协议会在多台服务器中选举出一台主服务器和多台备用服务器。主服务器负责处理业务,备用服务器处于监听状态。Keepalived 可以对服务器的多种状态进行监测,比如网络连接、服务进程等。一旦检测到主服务器故障,备用服务器会迅速接管服务,对外提供相同的服务,用户几乎感觉不到服务的中断。
二、环境部署
角色 | IP | vip |
KA1 | 192.168.35.160 | 192.168.35.100 |
KA2 | 192.168.35.170 | 192.168.35.100 |
realserver1 | 192.168.35.110 | |
realserver2 | 192.168.35.120 |
安装keepalived
[root@KA1 ~]# dnf install keepalived -y
[root@KA1 ~]# systemctl start keepalived
[root@KA1 ~]# ps axf | grep keepalived
2385 pts/0 S+ 0:00 \_ grep --color=auto keepalived
2326 ? Ss 0:00 /usr/sbin/keepalived -D
2327 ? S 0:00 \_ /usr/sbin/keepalived -D
ka1的网络
ka2的网络
realserver1的网络
realserver2的网络
三、基础设置
主配置文件修改
ka1和ka2都要配置,首先敲入命令:vim /etc/keepalived/keepalived.conf
! Configuration File for keepalived
global_defs {
notification_email {
1584454586@qq.com #keepalived 发生故障切换时邮件发送的目标邮箱,可以按行区
分写多个
timiniglee-zln@163.com
}
notification_email_from keepalived@KA1.timinglee.org #发邮件的地址
smtp_server 127.0.0.1 #邮件服务器地址
smtp_connect_timeout 30 #邮件服务器连接timeout
router_id KA1.timinglee.org #每个keepalived主机唯一标识
#建议使用当前主机名,但多节点
重名不影响
vrrp_skip_check_adv_addr #对所有通告报文都检查,会比较消耗性能
#启用此配置后,如果收到的通告报文和上一
个报文是同一 #个路由器,则跳过检查,默认
值为全检查
vrrp_strict #严格遵循vrrp协议
#启用此项后以下状况将无法启动服务:
#1.无VIP地址
#2.配置了单播邻居
#3.在VRRP版本2中有IPv6地址
#建议不加此项配置
vrrp_garp_interval 0 #报文发送延迟,0表示不延迟
vrrp_gna_interval 0 #消息发送延迟
vrrp_mcast_group4 224.0.0.18 #指定组播IP地址范围:
}
vrrp_instance VI_1 {
state MASTER
interface eth0
virtual_router_id 100
priority 100
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
192.168.35.100/24 dev eth0 label eth0:1
}
unicast_src_ip 192.168.35.160
unicast_peer{
192.168.35.170
}
}
}
测试结果
独立子配置文件实现
当生产环境复杂时,
/etc/keepalived/keepalived.conf
文件中内容过多,不易管理 ,将不同集群的配置,比如:不同集群的VIP
配置放在独立的子配置文件中利用
include
指令可以实现包含 子配置文件。
[root@KA1 ~]# mkdir /etc/keepalived/conf.d
[root@KA1 ~]# vim /etc/keepalived/keepalived.conf
! Configuration File for keepalived
global_defs {
notification_email {
1584454586@qq.com
}
notification_email_from keepalived@KA1.timinglee.org
smtp_server 127.0.0.1
smtp_connect_timeout 30
router_id KA1.timinglee.org
vrrp_skip_check_adv_addr
vrrp_strict
vrrp_garp_interval 0
vrrp_gna_interval 0
vrrp_ipsets keepalived
vrrp_iptables
include /etc/keepalived/conf.d/*.conf #相关子配置文件
在文件中加入一下
[root@KA1 ~]# vim /etc/keepalived/conf.d/router.conf
vrrp_instance VI_1 {
state MASTER
interface eth0
virtual_router_id 20
priority 100
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
192.168.35.100/24 dev eth0 label eth0:0
}
}
实现日志分离
[root@localhost ~]# vim /etc/sysconfig/keepalived
KEEPALIVED_OPTIONS="-D -S 6"
[root@localhost ~]# vim /etc/rsyslog.conf
local6.* /var/log/keepalived.log
[root@localhost ~]# systemctl restart keepalived.service
[root@localhost ~]# systemctl restart rsyslog.service
[root@localhost ~]# ls -l /var/log/keepalived.log
-rw-------. 1 root root 1437 Aug 17 23:43 /var/log/keepalived.log
四、keepalived企业应用示例
非抢占模式
默认为抢占模式
preempt
,即当高优先级的主机恢复在线后,会抢占低先级的主机的
master
角色, 这样会使vip
在
KA
主机中来回漂移,造成网络抖动, 建议设置为非抢占模式 nopreempt
,即高优先级主机恢复后,并不会抢占低优先级主机的
maste
角色 非抢占模块下,
如果原主机
down
机
, VIP
迁移至的新主机
,
后续也发生
down
时
,
仍会将
VIP
迁移回原主机。
注意:要关闭 VIP
抢占,必须将各
keepalived
服务器
state
配置为
BACKUP
#ka1主机配置
vrrp_instance VI_1 {
state BACKUP
interface eth0
virtual_router_id 20
priority 100 #优先级高
nopreempt #非抢占模式
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
192.168.35.100/24 dev eth0 label eth0:1
}
}
#KA2主机配置
vrrp_instance VI_1 {
state BACKUP
interface eth0
virtual_router_id 20
priority 80 #优先级低
advert_int 1
nopreempt #非抢占模式
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
192.168.35.100/24 dev eth0 label eth0:1
}
}
抢占延迟模式
抢占延迟模式,即优先级高的主机恢复后,不会立即抢回
VIP
,而是延迟一段时间(默认
300s
)再抢回 VIP
#ka1主机配置
vrrp_instance VI_1 {
state BACKUP
interface eth0
virtual_router_id 20
priority 100 #优先级高
preempt_delay 10s #抢占延迟10s
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
192.168.35.100/24 dev eth0 label eth0:1
}
}
#KA2主机配置
vrrp_instance VI_1 {
state BACKUP
interface eth0
virtual_router_id 20
priority 80 #优先级低
advert_int 1
preempt_delay 10s #抢占延迟10S
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
192.168.35.100/24 dev eth0 label eth0:1
}
}
vip单播设置
KA1的配置文件
vrrp_instance VI_1 {
state MASTER
interface eth0
virtual_router_id 100
priority 100
advert_int 1
# nopreempt
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
192.168.35.100/24 dev eth0 label eth0:1
}
unicast_src_ip 192.168.35.160
unicast_peer{
192.168.35.170
}
}
KA2的配置文件
vrrp_instance VI_1 {
state MASTER
interface eth0
virtual_router_id 100
priority 100
advert_int 1
# nopreempt
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
192.168.35.100/24 dev eth0 label eth0:1
}
unicast_src_ip 192.168.35.170
unicast_peer{
192.168.35.160
}
}
用抓包查看单播效果,测试代码:
tcpdump -i eth0 -nn src host 192.168.35.160 and dst 192.168.35.170
消息通知脚本设置
1.编写脚本
[root@KA1 ~]# vim /etc/keepalived/mail.sh
#!/bin/bash
mail_dest='1584454586@qq.com'
mail_send()
{
mail_subj="$HOSTNAME to be $1 vip 转移"
mail_mess="`date +%F\ %T`: vrrp 转移,$HOSTNAME 变为 $1"
echo "$mail_mess" | mail -s "$mail_subj" $mail_dest
}
case $1 in
master)
mail_send master
;;
backup)
mail_send backup
;;
fault)
mail_send fault
;;
*)
exit 1
;;
esac
2.给文件权限
chmod +x /etc/keepalived/mail.sh
3.在主配置文件中加入
notify_master "/etc/keepalived/mail.sh master"
notify_backup "/etc/keepalived/mail.sh backup"
notify_fault "/etc/keepalived/mail.sh fault"
4.qq邮件设置
[root@localhost ~]# yum install mail -y
[root@localhost ~]# vim /etc/mail.rc
set from=XXXXXXXX@qq.com
set smtp=smtp.qq.com
set smtp-auth-user=1584454586@qq.com
set smtp-auth-password=QQ的smtp授权码
set smtp-auth=login
set ssl-verify=ignore
五、keepalived其他高可用方式实现
实现 master/master 的 Keepalived 双主架构
master/slave
的单主架构,同一时间只有一个
Keepalived
对外提供服务,此主机繁忙,而另一台主机却 很空闲,利用率低下,可以使用master/master
的双主架构,解决此问题。 master/master 的双主架构: 即将两个或以上VIP
分别运行在不同的
keepalived
服务器,以实现服务器并行提供
web
访问的目的,提高 服务器资源利用率
首先编写ka1的主配置文件
vim /etc/keepalived/keepalived.conf
vrrp_instance VI_1 {
state MASTER
interface eth0
virtual_router_id 100
priority 100
advert_int 1
# nopreempt
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
192.168.35.100/24 dev eth0 label eth0:1
}
unicast_src_ip 192.168.35.160
unicast_peer{
192.168.35.170
}
vrrp_instance VI_2 {
state BACKUP
interface eth0
virtual_router_id 200
priority 80
advert_int 1
# nopreempt
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
192.168.35.200/24 dev eth0 label eth0:2
}
unicast_src_ip 192.168.35.160
unicast_peer{
192.168.35.170
}
}
其次编写ka2的主配置文件
vrrp_instance VI_1 {
state BACKUP
interface eth0
virtual_router_id 100
priority 80
advert_int 1
# nopreempt
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
192.168.35.100/24 dev eth0 label eth0:1
}
unicast_src_ip 192.168.35.170
unicast_peer{
192.168.35.160
}
vrrp_instance VI_2 {
state MASTER
interface eth0
virtual_router_id 200
priority 100
advert_int 1
# nopreempt
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
192.168.35.200/24 dev eth0 label eth0:2
}
unicast_src_ip 192.168.35.170
unicast_peer{
192.168.35.160
}
}
效果如下图所示:
ka1拥有vip192.168.35.100
ka2拥有vip192.168.35.200
keepalived+lvs实现
ka1的主配置文件
vim /etc/keepalived/keepalived.conf
virtual_server 192.168.35.100 80 {
delay_loop 6
lb_algo wrr
lb_kind DR
# persistence_timeout 50
protocol TCP
real_server 192.168.35.110 80 {
weight 1
HTTP_GET {
url {
path /
status_code 200
# digest ff20ad2481f97b1754ef3e12ecd3a9cc
}
# url {
# path /mrtg/
# digest 9b3a0c85a887a256d6939da88aabd8cd
# }
connect_timeout 3
nb_get_retry 2
delay_before_retry 2
}
}
real_server 192.168.35.120 80 {
weight 1
HTTP_GET {
url {
path /
status_code 200
# digest ff20ad2481f97b1754ef3e12ecd3a9cc
}
# url {
# path /mrtg/
# digest 9b3a0c85a887a256d6939da88aabd8cd
# }
connect_timeout 3
nb_get_retry 2
delay_before_retry 2
}
}
}
ka2的主配置文件
vim /etc/keepalived/keepalived
virtual_server 192.168.35.100 80 {
delay_loop 6
lb_algo wrr
lb_kind DR
# persistence_timeout 50
protocol TCP
real_server 192.168.35.110 80 {
weight 1
HTTP_GET {
url {
path /
status_code 200
# digest ff20ad2481f97b1754ef3e12ecd3a9cc
}
# url {
# path /mrtg/
# digest 9b3a0c85a887a256d6939da88aabd8cd
# }
connect_timeout 3
nb_get_retry 2
delay_before_retry 2
}
}
real_server 192.168.35.120 80 {
weight 1
HTTP_GET {
url {
path /
status_code 200
# digest ff20ad2481f97b1754ef3e12ecd3a9cc
}
# url {
# path /mrtg/
# digest 9b3a0c85a887a256d6939da88aabd8cd
# }
connect_timeout 3
nb_get_retry 2
delay_before_retry 2
}
}
}
实现HAProxy高可用
#在两个ka1和ka2先实现haproxy的配置
[root@rhel7-ka1 & ka2 ~]# vim /etc/haproxy/haproxy.cfg
listen webserver
bind 192.168.35.100:80
server web1 192.168.35.101:80 check
server web2 192.168.35.102:80 check
#在两个ka1和ka2两个节点启用内核参数
[root@rhel7-ka1 & ka2 ~]# vim /etc/sysctl.conf
net.ipv4.ip_nonlocal_bind = 1
[root@rhel7-ka1 & ka2 ~]# sysctl -p
#在ka1中编写检测脚本
[root@rhel7-ka1 ~]# vim /etc/keepalived/scripts/haproxy.sh
#!/bin/bash
/usr/bin/killall -0 haproxy
[root@rhel7-ka1 ~]# chmod +X /etc/keepalived/scripts/haproxy.sh
#在ka1中配置keepalived
[root@ka1-centos8 ~]#cat /etc/keepalived/keepalived.conf
vrrp_script check_haproxy {
script "/etc/keepalived/scripts/haproxy.sh"
interval 1
weight -30
fall 2
rise 2
timeout 2
}
vrrp_instance web {
state MASTER
interface eth0
virtual_router_id 50
priority 100
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
192.168.35.100 dev eth0 label eth0:1
}
track_script {
check_haproxy
}
}
#测试
root@rhel7-ka1 ~]# systemctl stop haproxy.service