【KEEPALIVED】全网超详细,包括实验和理论

理论部分

高可用集群KEEPALIVED是一种用于实现集群高可用性的服务软件,主要用于防止单点故障,确保集群中的服务能够持续、稳定地对外提供服务。以下是对KEEPALIVED的详细解析:

一、基本概念

  • 定义:KEEPALIVED基于VRRP(Virtual Router Redundancy Protocol,虚拟路由冗余协议)实现,是集群管理中保证集群高可用性的关键组件。

  • 作用:KEEPALIVED通过检测集群中各节点的状态,并在必要时进行故障转移,以确保集群服务的连续性。

二、工作原理

  • VRRP协议:KEEPALIVED以VRRP协议为实现基础,该协议允许多台路由器组成一个虚拟路由器组,通过选举产生一个主路由器(Master)和多个备份路由器(Backup)。主路由器负责对外提供服务,并周期性地发送VRRP报文以维持其状态。

  • 故障检测与转移:当BACKUP主机收不到MASTER主机的VRRP报文时,会认为MASTER主机宕机,此时会根据VRRP的优先级选举一个新的MASTER主机接替服务。

三、主要模块

  • core模块:KEEPALIVED的核心模块,负责主进程的启动、维护以及全局配置文件的加载和解析。

  • check模块:负责健康检查,包括各种常见的检查方式,如ICMP、TCP端口状态、HTTP GET等,以确保集群中各节点的正常运行。

  • vrrp模块:实现VRRP协议的关键模块,负责虚拟路由器的选举、状态维护和报文发送等工作。

四、应用场景

  • LVS高可用架构:KEEPALIVED常用于LVS(Linux Virtual Server,Linux虚拟服务器)集群中,作为调度器的高可用解决方案。通过设置主调度器和备调度器,确保在主调度器故障时能够迅速切换至备调度器,从而保证服务的连续性。

  • 其他高可用场景:除了LVS集群外,KEEPALIVED还可用于其他需要高可用性的场景,如数据库集群、Web服务器集群等。

五、注意事项

  • 脑裂问题:在高可用架构中,可能会出现脑裂问题,即主和备节点都认为自己是主节点,并同时提供服务。这通常是由于网络问题、防火墙策略等原因导致的。为了避免脑裂问题,可以采取多种措施,如添加更多的检测手段、使用串行电缆和以太网电缆连接等。

  • 配置与监控:在使用KEEPALIVED时,需要合理配置其参数和监控其状态,以确保其能够正常工作并及时发现潜在问题。

实验部分

一、Keepalived基本部署

1、基本配置

#环境准备
​
四台都用rh7
​
kA1: 172.25.254.10
KA2: 172.25.254.20
realserver1: 172.25.254.110
realserver2: 172.25.254.120
​
#realserver1、realserver2
yum install httpd -y
​
[root@realserver1 ~]# echo realserver1 - 172.25.254.110 > /var/www/html/index.html
[root@realserver1 ~]# systemctl enable --now httpd
[root@realserver2 ~]# echo realserver2 - 172.25.254.120 > /var/www/html/index.html
[root@realserver2 ~]# systemctl enable --now httpd
​
​
#ka1、ka2访问
curl 172.25.254.110
curl 172.25.254.120
2、虚拟路由管理
#ka1、ka2
[root@ka1 ~]# yum install keepalived -y
​
#全局配置
[root@ka1 ~]# vim /etc/keepalived/keepalived.conf
global_defs {
   notification_email {
     2727584440@qq.com
   }
   notification_email_from keepalived@timinglee.org
   smtp_server 127.0.0.1
   smtp_connect_timeout 30
   router_id ka1.timinglee.org
   vrrp_skip_check_adv_addr
   vrrp_strict
   vrrp_garp_interval 0
   vrrp_gna_interval 0
   vrrp_mcast_group4 224.0.0.18
}
​
#虚拟路由器配置
vrrp_instance VI_1 {
    state MASTER
    interface eth0
    virtual_router_id 100
    priority 100
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass 1111
    }
    virtual_ipaddress {
        172.25.254.100/24 dev eth0 label eth0:1
    }
}
​
[root@ka1 ~]# systemctl enable --now keepalived.service
[root@ka1 ~]# ifconfig
[root@ka1 ~]# scp /etc/keepalived/keepalived.conf root@172.25.254.20:/etc/keepalived/keepalived.conf 
# 在ka2上修改这两条,先不重新启动,查看是那个被用
vrrp_instance VI_1 {
    state BACKUP
    priority 80
​
​
#测试
[root@ka1 ~]# tcpdump -i eth0 -nn host 224.0.0.18
23:04:18.883958 IP 172.25.254.10 > 224.0.0.18: VRRPv2, Advertisement, vrid 100, prio 100, authtype simple, intvl 1s, length 20
#关掉ka1的keepalived,会出现另一台的
23:04:05.872730 IP 172.25.254.20 > 224.0.0.18: VRRPv2, Advertisement, vrid 100, prio 80, authtype simple, intvl 1s, length 20
3、ping172.25.254.10能通

同时关闭则不能ping通,同时开启则可以

# ka1、2
[root@ka1 ~]# vim /etc/keepalived/keepalived.conf
global_defs {
   notification_email {
     2727584440@qq.com
   }
   notification_email_from keepalived@timinglee.org
   smtp_server 127.0.0.1
   smtp_connect_timeout 30
   router_id ka1.timinglee.org
   vrrp_skip_check_adv_addr
   vrrp_strict    #和vrrp_iptables 同时开启同时关闭
   vrrp_garp_interval 0
   vrrp_gna_interval 0
   vrrp_mcast_group4 224.0.0.18
   vrrp_iptables  #
}
4、启用keepalived日志功能
[root@ka1 ~]# vim /etc/sysconfig/keepalived 
KEEPALIVED_OPTIONS="-D -S 6"
[root@ka1 ~]# vim /etc/rsyslog.conf 
local6.*                                                /var/log/keepa
lived.log
[root@ka1 ~]# systemctl restart keepalived.service 
[root@ka1 ~]# systemctl restart rsyslog.service 
[root@ka1 ~]# ll /var/log/keepalived.log 
5、独立子配置文件
[root@ka1 ~]# vim /etc/keepalived/keepalived.conf 
#vrrp_instance VI_1 {
#    state MASTER
#    interface eth0
#    virtual_router_id 100
#    priority 100
#    advert_int 1
#    authentication {
#        auth_type PASS
#        auth_pass 1111
#    }
#    virtual_ipaddress {
#        172.25.254.100/24 dev eth0 label eth0:1
#    }
#}
include "/etc/keepalived/conf.d/*.conf"
​
[root@ka1 ~]# mkdir -p /etc/keepalived/conf.d
[root@ka1 ~]# vim /etc/keepalived/conf.d/172.25.254.100.conf
vrrp_instance VI_1 {
    state MASTER
    interface eth0
    virtual_router_id 100
    priority 100
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass 1111
    }
    virtual_ipaddress {
        172.25.254.100/24 dev eth0 label eth0:1
    }
}
​
[root@ka1 ~]# systemctl restart keepalived.service 
[root@ka1 ~]# ifconfig

二、抢占模式和非抢占模式

默认为抢占模式preempt,即当高优先级的主机恢复在线后,会抢占低先级的主机的master角色,

这样会使vip在KA主机中来回漂移,造成网络抖动,

建议设置为非抢占模式 nopreempt ,即高优先级主机恢复后,并不会抢占低优先级主机的master角色

非抢占模块下,如果原主机down机, VIP迁移至的新主机, 后续也发生down时,仍会将VIP迁移回原主机

1、非抢占模式
#注意:要关闭 VIP抢占,必须将各 keepalived 服务器state配置为BACKUP
​
#ka1、ka2主机配置
vrrp_instance VI_1 {
    state BACKUP
    interface eth0
    virtual_router_id 100
    priority 100   #优先级高
    advert_int 1
    nopreempt      #非抢占模式
    authentication {
        auth_type PASS
        auth_pass 1111
    }
    virtual_ipaddress {
        172.25.254.100/24 dev eth0 label eth0:1
    }
}
​
​
# 测试
关掉ka1,100在ka2,开启ka1,100还在ka2上
2、抢占延迟模式preempt_delay

抢占延迟模式,即优先级高的主机恢复后,不会立即抢回VIP,而是延迟一段时间(默认300s)再抢回VIP

# 注意:需要各keepalived服务器state为BACKUP,并且不要启用 vrrp_strict
​
#ka1、ka2主机配置
vrrp_instance VI_1 {
    state BACKUP
    interface eth0
    virtual_router_id 100
    priority 100
    advert_int 1
    preempt_delay 5s  #抢占延迟5S
    authentication {
        auth_type PASS
        auth_pass 1111
    }
    virtual_ipaddress {
        172.25.254.100/24 dev eth0 label eth0:1
    }
}

三、VIP的单播配置

默认keepalived主机之间利用多播相互通告消息,会造成网络拥塞,可以替换成单播,减少网络流量

# 注意:启用 vrrp_strict 时,不能启用单播
​
#ka1主机配置
global_defs {
   notification_email {
     2727584440@qq.com
   }
   notification_email_from keepalived@timinglee.org
   smtp_server 127.0.0.1
   smtp_connect_timeout 30
   router_id ka1.timinglee.org
   vrrp_skip_check_adv_addr
   #vrrp_strict    #注释此参数,与vip单播模式冲突
   vrrp_garp_interval 0
   vrrp_gna_interval 0
   vrrp_mcast_group4 224.0.0.18
}
​
​
vrrp_instance VI_1 {
    state BACKUP
    interface eth0
    virtual_router_id 100
    priority 100
    advert_int 1
    preempt_delay 5s
    authentication {
        auth_type PASS
        auth_pass 1111
    }
    virtual_ipaddress {
        172.25.254.100/24 dev eth0 label eth0:1
    }
    unicast_src_ip 172.25.254.10   #本机IP
    unicast_peer {
        172.25.254.20    #对端IP
    }
}
​
#ka2主机配置
unicast_src_ip 172.25.254.20
    unicast_peer {
        172.25.254.10
    }
​
​
#抓包查看单播效果
[root@ka1 ~]# tcpdump -i eth0 -nn src host 172.25.254.10 and dst 172.25.254.20
[root@ka2 ~]# tcpdump -i eth0 -nn src host 172.25.254.20 and dst 172.25.254.10
# 只有VIP在哪,哪才能去发,如果把ka1停掉,那么ka2才能发,如果把ka1又开启,那么ka2不能发单播,只有ka1能发

四、Keepalived 通知脚本配置

1、 邮件配置
# 安装邮件发送工具
[root@KA1 ~]# yum install mailx -y
​
​
[root@KA1 ~]# vim /etc/mail.rc
#######mail set##########
set from=2727584440@qq.com
set smtp=smtp.qq.com
set smtp-auth-user=2727584440@qq.com
set smtp-auth-password=oyulniiqkxridgag   #自己邮箱的授权码
set smtp-auth=login
set ssl-verify=ignore
​
#发送邮件
echo hello word | mail -s test 2727584440@qq.com
2、实现Keepalived状态切换的通知脚本
#在所有 keepalived节点配置如下
[root@KA1 + KA2 ~]# vim /etc/keepalived/mail.sh
#!/bin/bash
mail_dst="2727584440@qq.com"
send_message()
{
  mail_sub="$HOSTNMAE to be $1 vip move"
  mail_msg="`date +%F\ %T`: vrrp move $HOSTNAME change $1"
  echo $mail_msg | mail -s "$mail_sub" $mail_dst
}
case $1 in
  master)
  send_message master
  ;;
  backup)
  send_message backup
  ;;
  fault)
  send_message fault
  ;;
  *)
  ;;
esac
[root@KA1 +KA2 ~]# chmod +x /etc/keepalived/mail.sh
[root@KA1 +K2 ~]#vim /etc/keepalived/keepalived.conf
vrrp_instance VI_1 {
...
    unicast_src_ip 172.25.254.10   #本机IP
    unicast_peer {
        172.25.254.20    #对端IP
    }
    notify_master "/etc/keepalived/mail.sh master"
    notify_backup "/etc/keepalived/mail.sh backup"
    notify_fault "/etc/keepalived/mail.sh fault"
}
​

五、实现 master/master Keepalived 双主架构

# ka1备
vrrp_instance VI_1 {
    state MASTER
    interface eth0
    virtual_router_id 100
    priority 100
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass 1111
    }
    virtual_ipaddress {
        172.25.254.100/24 dev eth0 label eth0:1
    }
    unicast_src_ip 172.25.254.10
    unicast_peer {
        172.25.254.20
    }
    track_script {
        check_haproxy
    }
}
vrrp_instance VI_2 {
    state BACKUP
    interface eth0
    virtual_router_id 200
    priority 80
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass 1111
    }
    virtual_ipaddress {
        172.25.254.200/24 dev eth0 label eth0:2
    }
    unicast_src_ip 172.25.254.10
    unicast_peer {
        172.25.254.20
    }
}
​
# ka2
vrrp_instance VI_1 {
    state BACKUP 
    interface eth0
    virtual_router_id 100
    priority 80
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass 1111
    }
    virtual_ipaddress {
        172.25.254.100/24 dev eth0 label eth0:1
    }
    unicast_src_ip 172.25.254.20
    unicast_peer {
        172.25.254.10
    }
}
vrrp_instance VI_2 {
    state MASTER
    interface eth0
    virtual_router_id 200
    priority 100
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass 1111
    }
    virtual_ipaddress {
        172.25.254.200/24 dev eth0 label eth0:2
    }
    unicast_src_ip 172.25.254.20
    unicast_peer {
        172.25.254.10
    }
}
​

六、实现IPVS的高可用性

# 在realserver1\2
[root@realserver2 ~]# ip a a 172.25.254.100/32 dev lo
[root@realserver2 ~]# vim /etc/sysctl.d/arp.conf
net.ipv4.conf.all.arp_ignore=1
net.ipv4.conf.all.arp_announce=2
net.ipv4.conf.lo.arp_ignore=1
net.ipv4.conf.lo.arp_announce=2
​
[root@realserver2 ~]# sysctl --system
​
# ka1\2
[root@ka1 ~]# yum install ipvsadm -y
[root@ka1 ~]# vim /etc/keepalived/keepalived.conf 
virtual_server 172.25.254.100 80 {
    delay_loop 6
    lb_algo wrr
    lb_kind DR
    #persistence_timeout 50
    protocol TCP
​
    real_server 172.25.254.110 80 {
        weight 1
        HTTP_GET {
            url {
              path /
              status_code 200
            }
            connect_timeout 3
            nb_get_retry 2
            delay_before_retry 2
        }
    }
    real_server 172.25.254.120 80 {
        weight 1
        HTTP_GET {
            url {
              path /
              status_code 200
            }
            connect_timeout 3
            nb_get_retry 2
            delay_before_retry 2
        }
     }
}
​
[root@ka1 ~]# systemctl restart keepalived.service  
[root@ka1 ~]# ipvsadm -Ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
  -> RemoteAddress:Port           Forward Weight ActiveConn InActConn
​
​
[root@ka2 ~]# ipvsadm -A -t 172.255.254.100:80 -s wrr
[root@ka2 ~]# ipvsadm -Ln (有)
[root@ka1 ~]# ipvsadm -Ln (有)
​
# 测试
[root@ka2 ~]# for i in {1..6}; do curl 172.25.254.100; done
realserver2 - 172.25.254.120
realserver1 - 172.25.254.110
realserver2 - 172.25.254.120
realserver1 - 172.25.254.110
realserver2 - 172.25.254.120
realserver1 - 172.25.254.110
​

七、实现其它应用的高可用性VRRP Script

keepalived利用 VRRP Script 技术,可以调用外部的辅助脚本进行资源监控,并根据监控的结果实现优先

动态调整,从而实现其它应用的高可用性功能

分两步实现:

1、定义脚本

vrrp_script:自定义资源监控脚本,vrrp实例根据脚本返回值,公共定义,可被多个实例调用,定义在vrrp实例之外的独立配置块,一般放在global_defs设置块之后。

通常此脚本用于监控指定应用的状态。一旦发现应用的状态异常,则触发对MASTER节点的权重减至低于SLAVE节点,从而实现 VIP 切换到 SLAVE 节点

vrrp_script <SCRIPT_NAME> {
script <STRING>|<QUOTED-STRING> #此脚本返回值为非0时,会触发下面OPTIONS执行
OPTIONS
}
2、调用脚本

track_script:调用vrrp_script定义的脚本去监控资源,定义在VRRP实例之内,调用事先定义的

vrrp_script

track_script {
SCRIPT_NAME_1
SCRIPT_NAME_2
}

3、实战案例:利用脚本实现主从角色切换
# 在ka1上
[root@ka1 ~]# vim /etc/keepalived/test.sh
#!/bin/bash
[ ! -f /mnt/lee ]
[root@ka1 ~]# sh /etc/keepalived/test.sh 
[root@ka1 ~]# echo $?
0
[root@ka1 ~]# chmod +x /etc/keepalived/test.sh 
[root@ka1 ~]# vim /etc/keepalived/keepalived.conf 
vrrp_script check_haproxy {
    script "/etc/keepalived/test.sh"
    interval 1
    weight -30
    fall 2
    rise 2
    timeout 2
}
vrrp_instance VI_1 {
    state MASTER
    interface eth0
    virtual_router_id 100
    priority 100
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass 1111
    }
    virtual_ipaddress {
        172.25.254.100/24 dev eth0 label eth0:1
    }
    unicast_src_ip 172.25.254.10
    unicast_peer {
        172.25.254.20
    }
    track_script {
        check_haproxy
    }
}
[root@ka1 ~]# systemctl restart keepalived.service 
​
# 测试
[root@ka1 ~]# touch /mnt/lee
[root@ka1 ~]# ifconfig
[root@ka2 ~]# ifconfig
eth0:1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 172.25.254.100  netmask 255.255.255.0  broadcast 0.0.0.0
        ether 00:0c:29:44:a9:9f  txqueuelen 1000  (Ethernet)
​
[root@ka1 ~]# rm -rf /mnt/lee
[root@ka1 ~]# systemctl restart keepalived.service 
[root@ka1 ~]# ifconfig
eth0:1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 172.25.254.100  netmask 255.255.255.0  broadcast 0.0.0.0
        ether 00:0c:29:42:39:a7  txqueuelen 1000  (Ethernet)
4、实战案例:实现HAProxy高可用
#ka1、ka2
[root@ka1 ~]# yum install haproxy -y
[root@ka1 ~]# vim /etc/sysctl.conf
net.ipv4.ip_nonlocal_bind = 1
[root@ka1 ~]# sysctl -p
[root@ka1 ~]# vim /etc/haproxy/haproxy.cfg
listen webcluster
    bind 172.25.254.100:80
    mode http
    balance roundrobin
    server web1 172.25.254.110:80 check inter 3 fall 2 rise 5
    server web2 172.25.254.120:80 check inter 3 fall 2 rise 5
[root@ka1 ~]# sysetemctl enable ---now haproxy
[root@ka1 ~]# netstat -antlupe | grep haproxy
[root@ka1 ~]# curl 172.25.254.100 (不能,和lvs冲突)
[root@ka1 ~]# vim /etc/keepalived/keepalived.conf 
注释掉
#virtual_server 172.25.254.100 80 {
[root@ka1 ~]# systemctl restart keepalived.service  
​
#realwebser1、2
[root@realserver2 ~]# systemctl restart network (删掉lo)
[root@realserver2 ~]# vim /etc/sysctl.d/arp.conf
net.ipv4.conf.all.arp_ignore=0
net.ipv4.conf.all.arp_announce=0
net.ipv4.conf.lo.arp_ignore=0
net.ipv4.conf.lo.arp_announce=0
[root@realserver2 ~]# sysctl --system
​
[root@ka1 ~]# curl 172.25.254.100
[root@ka1 ~]# curl 172.25.254.110
[root@ka1 ~]# curl 172.25.254.120
​
​
#现在
在ka1关掉haproxy,则在外面访问不了172.25.254.100,我们需要解决这个问题
# 在ka1
[root@ka1 ~]# vim /etc/keepalived/test.sh
#!/bin/bash
killall -0 haproxy
[root@ka1 ~]# sh /etc/keepalived/test.sh 
[root@ka1 ~]# echo $?
[root@ka1 ~]# vim /etc/keepalived/keepalived.conf
vrrp_script check_haproxy {
    script "/etc/keepalived/test.sh"
    interval 1
    weight -30
    fall 2
    rise 2
    timeout 2
}
vrrp_instance VI_1 {
    state MASTER
    interface eth0
    virtual_router_id 100
    priority 100
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass 1111
    }
    virtual_ipaddress {
        172.25.254.100/24 dev eth0 label eth0:1
    }
    unicast_src_ip 172.25.254.10
    unicast_peer {
        172.25.254.20
    }
    track_script {
        check_haproxy
    }
}
[root@ka1 ~]# systemctl restart keepalived.service 
​
# 测试 关掉ka1的haproxy,也能一直访问
[root@ka1 ~]# systemctl stop haproxy.service 
[root@ka1 ~]# systemctl start haproxy.service 
[root@ka2 ~]# while true; do curl 172.25.254.100; done
  • 21
    点赞
  • 18
    收藏
    觉得还不错? 一键收藏
  • 0
    评论
评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值