~Keepalived高可用集群~

一、Keepalived简介

是一个用于实现高可用性的解决方案,它主要应用于云主机的主备切换,以达到高可用性(HA)的目的。当主服务器发生故障无法对外提供服务时,动态将虚拟IP切换到备服务器,继续对外提供服务,从而增强系统的容灾性能。Keepalived通过监控主服务器的状态,实现主备服务器的自动切换,确保服务的持续可用性。

集群类型

LB:Load Balance 负载均衡

LVS/HAProxy/nginx(http/upstream, stream/upstream)

HA:High Availability 高可用集群

数据库、Redis

SPoF: Single Point of Failure,解决单点故障

HPC:High Performance Computing 高性能集群

系统可用性

SLA:Service-Level Agreement 服务等级协议(提供服务的企业与客户之间就服务的品质、水准、性能 等方面所达成的双方共同认可的协议或契约)

A = MTBF / (MTBF+MTTR)

开始实验

准备工作
KA1:172.25.254.20
KA2:172.25.254.30
realserver1:172.25.254.110
realserver2:172.25.254.120

realserver1

[root@realserver1 ~]# yum install httpd  -y
[root@realserver1 ~]# systemctl stop firewalld
[root@realserver1 ~]# echo realserver1 - 172.25.254.110 > /var/www/html/index.html
[root@realserver1 ~]# systemctl start httpd

realserver2

[root@realserver2 ~]# yum install httpd  -y
[root@realserver2 ~]# systemctl stop firewalld
[root@realserver2 ~]# echo realserver2 - 172.25.254.120 > /var/www/html/index.html
[root@realserver2 ~]# systemctl start httpd

 安装keepalived

[root@KA1 ~]# dnf install keepalived -y
[root@KA1 ~]# systemctl start keepalived
[root@KA2 ~]# dnf install keepalived -y
[root@KA2 ~]# systemctl start keepalived

测试一下 

[root@kA1 ~]# curl 172.25.254.110
realserver1 - 172.25.254.110
[root@kA1 ~]# curl 172.25.254.120
realserver2 - 172.25.254.120

 二、配置keepalived文件(配置虚拟路由)

! Configuration File for keepalived
global_defs {
notification_email {
3595566522@qq.com 
timiniglee-zln@163.com
}
notification_email_from keepalived@KA1.timinglee.org 
smtp_server 127.0.0.1 
smtp_connect_timeout 30 
router_id KA1.timinglee.org 
vrrp_skip_check_adv_addr 
vrrp_garp_interval 0
vrrp_gna_interval 0 
vrrp_mcast_group4 224.0.0.18 
}

进行测试

[root@KA2 ~]# tcpdump -i eth0 -nn host 224.0.0.18
dropped privs to tcpdump
tcpdump: verbose output suppressed, use -v[v]... for full protocol decode
listening on eth0, link-type EN10MB (Ethernet), snapshot length 262144 bytes
22:48:23.294894 IP 172.25.254.20 > 224.0.0.18: VRRPv2, Advertisement, vrid 20, 
prio 100, authtype none, intvl 1s, length 20
22:48:24.084793 IP 172.25.254.30 > 224.0.0.18: VRRPv2, Advertisement, vrid 30, 
prio 80, authtype none, intvl 1s, length 20
22:48:24.295075 IP 172.25.254.20 > 224.0.0.18: VRRPv2, Advertisement, vrid 20

 进行抓包

[root@KA1 ~]# yum install tcpdump
[root@KA1 ~]# tcpdump -i ens33 -nn host 224.0.0.18
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on ens33, link-type EN10MB (Ethernet), capture size 262144 bytes
20:58:45.323341 IP 172.25.254.20 > 224.0.0.18: VRRPv2, Advertisement, vrid 100, prio 100, authtype simple, intvl 1s, length 20

三、启用keepalived日志功能

[root@ka1 ~]#systemctl restart keepalived.service rsyslog.service 
[root@ka1 ~]#tail -f /var/log/keepalived.log 
Apr 14 09:25:51 ka1 Keepalived_vrrp[1263]: Sending gratuitous ARP on eth0 for
10.0.0.10
Apr 14 09:25:51 ka1 Keepalived_vrrp[1263]: Sending gratuitous ARP on eth0 for
10.0.0.10
Apr 14 09:25:51 ka1 Keepalived_vrrp[1263]: Sending gratuitous ARP on eth0 for
10.0.0.10

四、抢占模式和非抢占模式

非抢占模式 nopreempt

默认为抢占模式preempt,即当高优先级的主机恢复在线后,会抢占低先级的主机的master角色, 这样会使vip在KA主机中来回漂移,造成网络抖动, 建议设置为非抢占模式 nopreempt ,即高优先级主机恢复后,并不会抢占低优先级主机的master角色 非抢占模块下,如果原主机down机, VIP迁移至的新主机, 后续也发生down时,仍会将VIP迁移回原主机。

ka1主机配置
vrrp_instance VI_1 {
   state BACKUP
   interface eth0
   virtual_router_id 20
   priority 100 #优先级高
   nopreempt #非抢占模式
   advert_int 1
   authentication {
       auth_type PASS
       auth_pass 1111
   }
   virtual_ipaddress {
       172.25.254.110/24 dev eth0 label eth0:0
   }
}
ka2主机配置
vrrp_instance VI_1 {
   state BACKUP
   interface eth0
   virtual_router_id 20
   priority 80 #优先级低
   advert_int 1
   nopreempt #非抢占模式
   authentication {
       auth_type PASS
       auth_pass 1111
   }
   virtual_ipaddress {
     172.25.254.120/24 dev eth0 label eth0:0
   }
}

抢占延迟模式 preempt_delay

抢占延迟模式,即优先级高的主机恢复后,不会立即抢回VIP,而是延迟一段时间(默认300s)再抢回

#ka1主机配置
vrrp_instance VI_1 {
   state BACKUP
   interface eth0
   virtual_router_id 20
   priority 100 #优先级高
   preempt_delay 10s #抢占延迟10s
   advert_int 1
   authentication {
       auth_type PASS
       auth_pass 1111
   }
   virtual_ipaddress {
       172.25.254.110/24 dev eth0 label eth0:0
   }
}
   
#KA2主机配置
vrrp_instance VI_1 {
   state BACKUP
   interface eth0
   virtual_router_id 20
   priority 80 #优先级低
   advert_int 1
   preempt_delay 10s #抢占延迟10S
   authentication {
       auth_type PASS
       auth_pass 1111
   }
   virtual_ipaddress {
     172.25.254.120/24 dev eth0 label eth0:0
   }
}

五、VIP单播配置

默认keepalived主机之间利用多播相互通告消息,会造成网络拥塞,可以替换成单播,减少网络流

配置文件且启用 vrrp_strict 

[root@KA1 ~]# vim /etc/keepalived/keepalived.conf
[root@KA1 ~]# systemctl restart keepalived
[root@KA1 ~]# tcpdump -i eth0 -nn src host 172.25.254.20 and dst 172.25.254.30
[root@KA2 ~]#  vim /etc/keepalived/keepalived.conf
[root@KA2 ~]# systemctl restart keepalived
[root@KA2 ~]# tcpdump -i eth0 -nn src host 172.25.254.30 and dst 172.25.254.20
KA1主机配置
[root@KA1 ~]# vim /etc/keepalived/keepalived.conf
! Configuration File for keepalived
global_defs {
   notification_email {
  3595566522@qq.com
   }
   notification_email_from keepalived@KA1.timinglee.org
   smtp_server 127.0.0.1
   smtp_connect_timeout 30
   router_id KA1.timinglee.org
   vrrp_skip_check_adv_addr
   #vrrp_strict #注释此参数,与vip单播模式冲突
   vrrp_garp_interval 0
   vrrp_gna_interval 0
   vrrp_ipsets keepalived
}
vrrp_instance VI_1 {
   state MASTER
   interface eth0
   virtual_router_id 20
   priority 100
   advert_int 1
   authentication {
       auth_type PASS
       auth_pass 1111
   }
   virtual_ipaddress {
   172.25.254.110/24 dev eth0 label eth0:0
   }
   unicast_src_ip 172.25.254.20 #本机IP
   unicast_peer {
       172.25.254.30 #指向对方主机IP
       #如果有多个keepalived,再加其它节点的IP
   }
}
KA2配置
[root@KA2 ~]# vim /etc/keepalived/keepalived.conf
 ! Configuration File for keepalived抓包查看单播效果
global_defs {
   notification_email {
   3595566522@qq.com
   }
   notification_email_from keepalived@KA1.timinglee.org
   smtp_server 127.0.0.1
   smtp_connect_timeout 30
   router_id KA1.timinglee.org
   vrrp_skip_check_adv_addr
   #vrrp_strict #注释此参数,与vip单播模式冲突
   vrrp_garp_interval 0
   vrrp_gna_interval 0
   vrrp_ipsets keepalived
}
vrrp_instance VI_1 {
   state BACKUP
   interface eth0
   virtual_router_id 20
   priority 80
   advert_int 1
   preempt_delay 60
   authentication {
       auth_type PASS
       auth_pass 1111
   }
   virtual_ipaddress {
     172.25.254.120/24 dev eth0 label eth0:0
   }
   unicast_src_ip 172.25.254.30 #本机ip
   unicast_peer {
     172.25.254.20 #对端主机IP
   }
}

抓包查看单播效果

[root@KA1 ~]# tcpdump -i ens33 -nn src host 172.25.254.20 and dst 172.25.254.30

 Keepalived 通知脚本配置

当keepalived的状态变化时,可以自动触发脚本的执行,比如:发邮件通知用户

默认以用户keepalived_script身份执行脚本

如果此用户不存在,以root执行脚本可以用下面指令指定脚本执行用户的身份

global_defs {
    ......
    script_user <USER>
    ......
}

 脚本类型

当前节点成为主节点时触发的脚本

notify_master <STRING>|<QUOTED-STRING>

当前节点转为备节点时触发的脚本

notify_backup <STRING>|<QUOTED-STRING>

通用格式的通知触发机制,一个脚本可完成以上三种状态的转换时的通知

notify <STRING>|<QUOTED-STRING>

当停止VRRP时触发的脚本

notify_stop <STRING>|<QUOTED-STRING>

 调用脚本

notify_master "/etc/keepalived/notify.sh master"
notify_backup "/etc/keepalived/notify.sh backup"
notify_fault "/etc/keepalived/notify.sh fault"
[root@KA1 ~]# vim /etc/keepalived/mail.sh
#!/bin/bash
mail_dest='3595566522@qq.com'
 
mail_send()
{
    mail_subj="$HOSTNAME to be $1 vip 转移"
    mail_mess="`date +%F\ %T`: vrrp 转移,$HOSTNAME 变为 $1"
    echo "$mail_mess" | mail -s "$mail_subj" $mail_dest
}
case $1 in
    master)
    mail_send master
    ;;
    backup)
    mail_send backup
    ;;
    fault)
    mail_send fault
    ;;
    *)
    exit 1
    ;;
esac

六、邮件通知

配置文件

[root@KA1 ~]# vim /etc/mail.rc
[root@KA1 ~]# cat /etc/mail.rc
set from=3595566522@qq.com
set smtp=smtp.qq.com
set smtp-auth-user=3595566522@qq.com
set smtp-auth-password=fdvoyibvazmecfbd
set smtp-auth=login
set ssl-verify=ignore

进行测试

[root@KA1 ~]# echo test message |mail -s test 3595566522@qq.com  

 实现 master/master 的 Keepalived 双主架构

master/slave的单主架构,同一时间只有一个Keepalived对外提供服务,此主机繁忙,而另一台主机却 很空闲,利用率低下,可以使用master/master的双主架构,解决此问题。

master/master 的双主架构: 即将两个或以上VIP分别运行在不同的keepalived服务器,以实现服务器并行提供web访问的目的,提高 服务器资源利用率

示例
#ha1主机配置
[root@rhel7-ka1 ~]# vim /etc/keepalived/keepalived.conf
@@@@ 内容省略 @@@@
vrrp_instance VI_1 {
   state MASTER #主
   interface ens33
   virtual_router_id 50
   priority 100
   advert_int 1
   authentication {
       auth_type PASS
       auth_pass 1111
   }
   virtual_ipaddress {
       172.25.254.50 dev ens33 label ens33:0
   }
}
vrrp_instance VI_60 {
   state BACKUP #备
   interface ens33
   virtual_router_id 60
   priority 80
   advert_int 1
   authentication {
       auth_type PASS
       auth_pass 1111
   }
   virtual_ipaddress {
       172.25.254.60 dev ens33 label ens33:1
   }
}

七、实现IPVS的高可用性

IPVS相关配置

[root@KA1 ~]# yum install ipvsadm -y
[root@KA1 ~]# vim /etc/keepalived/keepalived.conf
[root@KA1 ~]# systemctl restart keepalived.service
[root@KA1 ~]# ipvsadm -Ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
  -> RemoteAddress:Port           Forward Weight ActiveConn InActConn
TCP  172.25.254.100:80 wrr
  -> 172.25.254.110:80            Route   1      0          0
  -> 172.25.254.120:80            Route   1      0          0
[root@KA2 ~]# systemctl stop firewalld
[root@KA2 ~]# systemctl restart keepalived
[root@KA2 ~]# ipvsadm -Ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
  -> RemoteAddress:Port           Forward Weight ActiveConn InActConn
TCP  172.25.254.100:80 wrr
  -> 172.25.254.110:80            Route   1      0          0
  -> 172.25.254.120:80            Route   1      0          0

示例

#准备两台后端RS主机
[root@rs1 ~]# yum install httpd -y
[root@rs1 ~]# echo RS1 - 172.25.254.101 > /var/www/html/index.html
[root@rs1 ~]# ip addr add 172.25.254.100/32 dev lo
[root@rs1 ~]# echo 1 > /proc/sys/net/ipv4/conf/all/arp_ignore
[root@rs1 ~]# echo 1 > /proc/sys/net/ipv4/conf/lo/arp_ignore
[root@rs1 ~]# echo 2 > /proc/sys/net/ipv4/conf/lo/arp_announce
[root@rs2 ~]# yum install httpd -y
[root@rs1 ~]# echo RS1 - 172.25.254.101 > /var/www/html/index.html
[root@rs2 ~]# ip addr add 172.25.254.100/32 dev lo
[root@rs2 ~]# echo 1 > /proc/sys/net/ipv4/conf/all/arp_ignore
[root@rs2 ~]# echo 1 > /proc/sys/net/ipv4/conf/lo/arp_ignore
[root@rs2 ~]# echo 2 > /proc/sys/net/ipv4/conf/lo/arp_announce
[root@node30 ~]# yum install httpd -y配置keepalived
[root@node30 ~]# echo RS1 - 172.25.254.101 > /var/www/html/index.html
[root@node30 ~]# ip addr add 172.25.254.100/32 dev lo
[root@node30 ~]# echo 1 > /proc/sys/net/ipv4/conf/all/arp_ignore
[root@node30 ~]# echo 1 > /proc/sys/net/ipv4/conf/lo/arp_ignore
[root@node30
~]# echo 2 > /proc/sys/net/ipv4/conf/lo/arp_announce

配置keepalived

#ka1节点的配置
[root@rhel7-ka1 ~]# vim /etc/keepalived/keepalived.conf
@@@@ 省略内容 @@@@
virtual_server 172.25.254.100 80 {
   delay_loop 6
   lb_algo wrr
   lb_kind DR
   protocol TCP
   sorry_server 172.25.254.30
   real_server 172.25.254.101 80 {
       weight 1
       TCP_CHECK {
           connect_timeout 5
           nb_get_retry 3
           delay_before_retry 3
           connect_port 80
       }
   }
   real_server 172.25.254.102 80 {
       weight 1
       HTTP_GET {
           url {
               path /
               status_code 200
           }
           connect_timeout 1
           nb_get_retry 3
           delay_before_retry 1
       }
   }
}
#ka2节点的配置,配置和ka1基本相同,只需修改三行
[root@rhel7-ka2 ~]# vim /etc/keepalived/keepalived.conf
@@@@ 省略内容 @@@@
virtual_server 172.25.254.100 80 {
   delay_loop 6
   lb_algo wrr
   lb_kind DR
   protocol TCP
   sorry_server 172.25.254.30访问测试结果
模拟故障
   real_server 172.25.254.101 80 {
       weight 1
       TCP_CHECK {
           connect_timeout 5
           nb_get_retry 3
           delay_before_retry 3
           connect_port 80
       }
   }
   real_server 172.25.254.102 80 {
       weight 1
       HTTP_GET {
           url {
               path /
               status_code 200
           }
           connect_timeout 1
           nb_get_retry 3
           delay_before_retry 1
       }
   }
}

 测试结果

八、keepalived+lvs

[root@KA1 ~]# touch /etc/keepalived/test.sh
[root@KA1 ~]# vim  /etc/keepalived/test.sh
[root@KA1 ~]# cat  /etc/keepalived/test.sh
[root@KA1 ~]# vim /etc/keepalived/keepalived.conf
[root@KA1 ~]# systemctl restart keepalived.service
[root@KA1 ~]# ifconfig
eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 172.25.254.10  netmask 255.255.0.0  broadcast 172.25.255.255
        inet6 fe80::20c:29ff:fe0c:6c2d  prefixlen 64  scopeid 0x20<link>
        ether 00:0c:29:0c:6c:2d  txqueuelen 1000  (Ethernet)
        RX packets 75496  bytes 6118881 (5.8 MiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 128025  bytes 13978589 (13.3 MiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0
 
eth0:1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 172.25.254.100  netmask 255.255.255.0  broadcast 0.0.0.0
        ether 00:0c:29:0c:6c:2d  txqueuelen 1000  (Ethernet)

 

九、keepalived+haproxy

KA1和KA2都需要安装haproxy

需要在两个ka1和ka2两个节点启用内核参数,目的是为了即使另一台主机没有100的VIP,也能进行远程解析,在原100的VIP下线的时候,保证服务能正常进行

[root@KA1~] yum install haproxy -y

[root@KA2~] yum install haproxy -y
 设置参数
[root@KA1 ~]# vim /etc/sysctl.conf 
[root@KA1 ~]# sysctl -p
net.ipv4.ip_nonlocal_bind = 1

[root@KA2 ~]# vim /etc/sysctl.conf 
[root@KA2 ~]# sysctl -p
net.ipv4.ip_nonlocal_bind = 1
进行配置文件 
[root@KA1 ~]# vim /etc/haproxy/haproxy.cfg
[root@KA1 ~]# systemctl enable haproxy.service
Created symlink from /etc/systemd/system/multi-user.target.wants/haproxy.service to /usr/lib/systemd/system/haproxy.service.
[root@KA1 ~]# systemctl restart haproxy.service

 删除两台服务器的环回

[root@realserver1 ~]# vim /etc/sysconfig/network-scripts/ifcfg-lo
[root@realserver1 ~]# systemctl restart network
[root@realserver1 ~]# ifconfig

[root@realserver2 ~]# vim /etc/sysconfig/network-scripts/ifcfg-lo
[root@realserver2 ~]# systemctl restart network
[root@realserver2 ~]# ifconfig
 进行测试 

  • 22
    点赞
  • 17
    收藏
    觉得还不错? 一键收藏
  • 0
    评论
评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值