1.keepalived简介
keepalived是什么?
Keepalived 软件起初是专为LVS负载均衡软件设计的,用来管理并监控LVS集群系统中各个服务节点的状态,后来又加入了可以实现高可用的VRRP功能。因此,Keepalived除了能够管理LVS软件外,还可以作为其他服务(例如:Nginx、Haproxy、MySQL等)的高可用解决方案软件。
Keepalived软件主要是通过VRRP协议实现高可用功能的。VRRP是Virtual Router RedundancyProtocol(虚拟路由器冗余协议)的缩写,VRRP出现的目的就是为了解决静态路由单点故障问题的,它能够保证当个别节点宕机时,整个网络可以不间断地运行。
所以,Keepalived 一方面具有配置管理LVS的功能,同时还具有对LVS下面节点进行健康检查的功能,另一方面也可实现系统网络服务的高可用功能。
keepalived的工作原理:
-
Keepalived高可用是通过 VRRP 进行通信的, VRRP是通过竞选机制来确定主备的,主的优先级高于备,因此,工作时主会优先获得所有的资源,备节点处于等待状态,当主挂了的时候,备节点就会接管主节点的资源,然后顶替主节点对外提供服务。
-
在 Keepalived 服务之间,只有作为主的服务器会一直发送 VRRP 广播包,告诉备它还活着,此时备不会枪占主,当主不可用时,即备监听不到主发送的广播包时,就会启动相关服务接管资源,保证业务的连续性.接管速度最快可以小于1秒。
1.2 keepalived的重要功能
keepalived 有三个重要的功能,分别是:
- 管理LVS负载均衡软件
- 实现LVS集群节点的健康检查
- 作为系统网络服务的高可用性(failover)
2. keepalived配置文件讲解
[root@master ~]# cat /etc/keepalived/keepalived.conf
! Configuration File for keepalived
global_defs { //全局配置
notification_email { //定义报警收件人邮件地址
acassen@firewall.loc
failover@firewall.loc
sysadmin@firewall.loc
}
notification_email_from Alexandre.Cassen@firewall.loc //定义报警发件人邮箱
smtp_server 192.168.200.1 //邮箱服务器地址
smtp_connect_timeout 30 //定义邮箱超时时间
router_id LVS_DEVEL //定义路由标识信息,同局域网内唯一
vrrp_skip_check_adv_addr
vrrp_strict
vrrp_garp_interval 0
vrrp_gna_interval 0
}
vrrp_instance VI_1 { //定义实例
state MASTER //指定keepalived节点的初始状态,可选值为MASTER|BACKUP
interface eth0 //VRRP实例绑定的网卡接口,用户发送VRRP包
virtual_router_id 51 //虚拟路由的ID,同一集群要一致
priority 100 //定义优先级,按优先级来决定主备角色,优先级越大越优先
nopreempt //设置不抢占
advert_int 1 //主备通讯时间间隔
authentication { //配置认证
auth_type PASS //认证方式,此处为密码
auth_pass 1111 //同一集群中的keepalived配置里的此处必须一致,推荐使用8位随机数
}
virtual_ipaddress { //配置要使用的VIP地址
192.168.200.16
}
}
virtual_server 192.168.200.16 1358 { //配置虚拟服务器
delay_loop 6 //健康检查的时间间隔
lb_algo rr //lvs调度算法
lb_kind NAT //lvs模式
persistence_timeout 50 //持久化超时时间,单位是秒
protocol TCP //4层协议
sorry_server 192.168.200.200 1358 //定义备用服务器,当所有RS都故障时用sorry_server来响应客户端
real_server 192.168.200.2 1358 { //定义真实处理请求的服务器
weight 1 //给服务器指定权重,默认为1
HTTP_GET {
url {
path /testurl/test.jsp //指定要检查的URL路径
digest 640205b7b0fc66c1ea91c463fac6334d //摘要信息
}
url {
path /testurl2/test.jsp
digest 640205b7b0fc66c1ea91c463fac6334d
}
url {
path /testurl3/test.jsp
digest 640205b7b0fc66c1ea91c463fac6334d
}
connect_timeout 3 //连接超时时间
nb_get_retry 3 //get尝试次数
delay_before_retry 3 //在尝试之前延迟多长时间
}
}
real_server 192.168.200.3 1358 {
weight 1
HTTP_GET {
url {
path /testurl/test.jsp
digest 640205b7b0fc66c1ea91c463fac6334c
}
url {
path /testurl2/test.jsp
digest 640205b7b0fc66c1ea91c463fac6334c
}
connect_timeout 3
nb_get_retry 3
delay_before_retry 3
}
}
}
3. keepalived实现nginx负载均衡机高可用
环境说明
系统信息 | 主机名 | IP |
---|---|---|
rhel7.4 | master | 192.168.20.99 |
rhel7.4 | slave | 192.168.20.120 |
本次高可用虚拟IP(VIP)地址暂定为 192.168.20.250
3.1 keepalived安装
配置主keepalived
//关闭防火墙与SELINUX
[root@master ~]# systemctl disable --now firewalld
[root@master ~]# vim /etc/selinux/config
SELINUX=disabled
//安装keepalived
[root@master ~]# yum -y install keepalived
//查看安装生成的文件
[root@master ~]# rpm -ql keepalived
/etc/keepalived //配置目录
/etc/keepalived/keepalived.conf //此为主配置文件
/etc/sysconfig/keepalived
/usr/bin/genhash
/usr/lib/systemd/system/keepalived.service //此为服务控制文件
/usr/libexec/keepalived
/usr/sbin/keepalived
.....此处省略N行
用同样的方法在备服务器上安装keepalived
[root@slave ~]# systemctl disable --now firewalld
Removed symlink /etc/systemd/system/multi-user.target.wants/firewalld.service.
Removed symlink /etc/systemd/system/dbus-org.fedoraproject.FirewallD1.service.
[root@slave ~]# vim /etc/selinux/config
SELINUX=disabled
//安装keepalived
[root@slave ~]# yum -y install keepalived
3.2 在主备机上分别安装httpd
在master上安装httpd
[root@master ~]# yum -y install httpd
[root@master ~]# cd /var/www/html/
[root@master html]# echo master > index.html
[root@master html]# systemctl enable --now httpd
Created symlink from /etc/systemd/system/multi-user.target.wants/httpd.service to /usr/lib/systemd/system/httpd.service.
[root@master html]# ss -antl
State Recv-Q Send-Q Local Address:Port Peer Address:Port
LISTEN 0 128 *:22 *:*
LISTEN 0 128 :::80 :::*
LISTEN 0 128 :::22 :::*
[root@master html]#
在slave上安装httpd
[root@slave ~]# yum -y install httpd
[root@slave ~]# cd /var/www/html/
[root@slave html]# echo slave > index.html
[root@slave html]# systemctl restart httpd
3.3 keepalived配置
3.3.1 配置主keepalived
[root@master ~]# vim /etc/keepalived/keepalived.conf
! Configuration File for keepalived
global_defs {
router_id lb01
}
vrrp_instance VI_1 {
state MASTER
interface ens33
virtual_router_id 28
priority 100
advert_int 1
authentication {
auth_type PASS
auth_pass wangqing
}
virtual_ipaddress {
192.168.20.250
}
}
virtual_server 192.168.20.250 80 {
delay_loop 6
lb_algo rr
lb_kind DR
persistence_timeout 50
protocol TCP
real_server 192.168.20.99 80 {
weight 1
TCP_CHECK {
connect_port 80
connect_timeout 3
nb_get_retry 3
delay_before_retry 3
}
}
real_server 192.168.20.120 80 {
weight 1
TCP_CHECK {
connect_port 80
connect_timeout 3
nb_get_retry 3
delay_before_retry 3
}
}
}
[root@master ~]# systemctl enable --now keepalived
Created symlink from /etc/systemd/system/multi-user.target.wants/keepalived.service to /usr/lib/systemd/system/keepalived.service.
3.3.2 配置备keepalived
[root@slave ~]# vim /etc/keepalived/keepalived.conf
! Configuration File for keepalived
global_defs {
router_id lb02
}
vrrp_instance VI_1 {
state BACKUP
interface ens33
virtual_router_id 28
priority 80
advert_int 1
nopreempt
authentication {
auth_type PASS
auth_pass wangqing
}
virtual_ipaddress {
192.168.20.250
}
}
virtual_server 192.168.20.250 80 {
delay_loop 6
lb_algo rr
lb_kind DR
persistence_timeout 50
protocol TCP
real_server 192.168.20.99 80 {
weight 1
TCP_CHECK {
connect_port 80
connect_timeout 3
nb_get_retry 3
delay_before_retry 3
}
}
real_server 192.168.20.120 80 {
weight 1
TCP_CHECK {
connect_port 80
connect_timeout 3
nb_get_retry 3
delay_before_retry 3
}
}
}
[root@slave ~]# systemctl enable --now keepalived
Created symlink from /etc/systemd/system/multi-user.target.wants/keepalived.service to /usr/lib/systemd/system/keepalived.service.
3.3.3 查看VIP在哪里
在MASTER上查看
[root@master ~]# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN qlen 1
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: ens33: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
link/ether 00:0c:29:29:6d:75 brd ff:ff:ff:ff:ff:ff
inet 192.168.20.99/24 brd 192.168.20.255 scope global ens33
valid_lft forever preferred_lft forever
inet 192.168.20.250/32 scope global ens33
valid_lft forever preferred_lft forever
inet6 fe80::ce17:cb1:75db:563d/64 scope link
valid_lft forever preferred_lft forever
在SLAVE上查看
[root@slave ~]# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN qlen 1
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: ens33: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
link/ether 00:0c:29:90:27:d3 brd ff:ff:ff:ff:ff:ff
inet 192.168.20.120/24 brd 192.168.20.255 scope global ens33
valid_lft forever preferred_lft forever
inet6 fe80::cc4:6cc0:d6df:6335/64 scope link
valid_lft forever preferred_lft forever
3.4 让keepalived监控nginx负载均衡机
keepalived通过脚本来监控httpd负载均衡机的状态
在master上编写脚本
[root@master ~]# mkdir /scripts
[root@master ~]# cd /scripts/
[root@master scripts]# vim check_httpd.sh
#!/bin/bash
httpd_status=$(ps -ef|grep -Ev "grep|$0"|grep '\bhttpd\b'|wc -l)
if [ $httpd_status -lt 1 ];then
systemctl stop keepalived
fi
[root@master scripts]# chmod +x check_httpd.sh
[root@master scripts]# vim notify.sh
[root@master scripts]# cat notify.sh
#!/bin/bash
VIP=$2
case "$1" in
master)
httpd_status=$(ps -ef|grep -Ev "grep|$0"|grep '\bhttpd\b'|wc -l)
if [ $httpd_status -lt 1 ];then
systemctl start httpd
fi
sendmail
;;
backup)
httpd_status=$(ps -ef|grep -Ev "grep|$0"|grep '\bhttpd\b'|wc -l)
if [ $httpd_status -gt 0 ];then
systemctl stop httpd
fi
;;
*)
echo "Usage:$0 master|backup VIP"
;;
esac
[root@master scripts]# chmod +x notify.sh
在slave上编写脚本
[root@slave ~]# mkdir /scripts
[root@slave ~]# cd /scripts/
[root@slave scripts]# vim notify.sh
#!/bin/bash
VIP=$2
case "$1" in
master)
httpd_status=$(ps -ef|grep -Ev "grep|$0"|grep '\bhttpd\b'|wc -l)
if [ $httpd_status -lt 1 ];then
systemctl start httpd
fi
sendmail
;;
backup)
httpd_status=$(ps -ef|grep -Ev "grep|$0"|grep '\bhttpd\b'|wc -l)
if [ $httpd_status -gt 0 ];then
systemctl stop httpd
fi
;;
*)
echo "Usage:$0 master|backup VIP"
;;
esac
[root@slave scripts]# chmod +x notify.sh
3.5 配置keepalived加入监控脚本的配置
[root@master scripts]# vim /etc/keepalived/keepalived.conf
[root@master scripts]# cat /etc/keepalived/keepalived.conf
! Configuration File for keepalived
global_defs {
router_id lb01
}
vrrp_script httpd_check {
script "/scripts/check_httpd.sh"
interval 1
weight -40
vrrp_instance VI_1 {
state MASTER
interface ens33
virtual_router_id 28
priority 100
advert_int 1
authentication {
auth_type PASS
auth_pass wangqing
}
virtual_ipaddress {
192.168.20.250
}
track_script { //添加
httpd_check //添加
} //添加
notify_master "/scripts/notify.sh master 192.168.20.250" //添加
notify_backup "/scripts/notify.sh backup 192.168.20.250" //添加
}
virtual_server 192.168.20.250 80 {
delay_loop 6
lb_algo rr
lb_kind DR
persistence_timeout 50
protocol TCP
real_server 192.168.20.99 80 {
weight 1
TCP_CHECK {
connect_port 80
connect_timeout 3
nb_get_retry 3
delay_before_retry 3
}
}
real_server 192.168.20.120 80 {
weight 1
TCP_CHECK {
connect_port 80
connect_timeout 3
nb_get_retry 3
delay_before_retry 3
}
}
}
[root@master ~]# systemctl start keepalived
配置备keepalived
[root@slave ~]# vim /etc/keepalived/keepalived.conf
[root@slave ~]# cat /etc/keepalived/keepalived.conf
! Configuration File for keepalived
global_defs {
router_id lb02
}
vrrp_instance VI_1 {
state BACKUP
interface ens33
virtual_router_id 28
priority 80
advert_int 1
authentication {
auth_type PASS
auth_pass wangqing
}
virtual_ipaddress {
192.168.20.250
}
notify_master "/scripts/notify.sh master 192.168.20.250"
notify_backup "/scripts/notify.sh backup 192.168.20.250"
}
virtual_server 192.168.20.250 80 {
delay_loop 6
lb_algo rr
lb_kind DR
persistence_timeout 50
protocol TCP
real_server 192.168.20.99 80 {
weight 1
TCP_CHECK {
connect_port 80
connect_timeout 3
nb_get_retry 3
delay_before_retry 3
}
}
real_server 192.168.20.120 80 {
weight 1
TCP_CHECK {
connect_port 80
connect_timeout 3
nb_get_retry 3
delay_before_retry 3
}
}
}
[root@slave ~]# systemctl start keepalived