keepalived IP漂移

南客先生

已于 2022-08-24 16:07:24 修改

阅读量1.2k

点赞数

分类专栏： keepalived 文章标签： tcp/ip 网络运维

于 2022-08-22 16:48:38 首次发布

本文链接：https://blog.csdn.net/qq_38129621/article/details/126469215

版权

keepalived 专栏收录该内容

1 篇文章 0 订阅

订阅专栏

[mysql主备切换[高可用IP漂移]]

到这一步的时候, 是主备部署已经处理好, 请关注:mysql主备部署[高可用]

这次使用的是keepalived-2.2.7.tar.gz版, 官网地址:keeplived官网

第一步:安装keepalived-2.2.7

解压
tar -zxvf keepalived-2.2.7.tar.gz
 
编译
cd keepalived-2.2.7
./configure --prefix=/usr/local/keepalived/(新建文件夹喔)
安装
make && make install

 
将解压/编译好的文件进行复制

1、服务脚本
cp /usr/local/etc/rc.d/init.d/keepalived  /etc/init.d/keepalived
2、配置文件
cp /usr/local/keepalived/etc/keepalived.conf /etc/keepalived/keepalived.conf
3、服务
cp /usr/local/keepalived/etc/sysconfig/keepalived /etc/sysconfig/keepalived


加入服务
chkconfig --add keepalived
chkconfig keepalived on
 
service keepalived start   #启动服务
service keepalived stop    #停止服务
service keepalived restart #重启服务

上面的操作主备机器都需要安装

第二步:配置文件

master:keepalived.conf

! Configuration File for keepalived
 
global_defs {
   router_id HA_MySQL
}
 
vrrp_instance VI_1 {
    state BACKUP
    interface eth0
    virtual_router_id 51
    priority 100
    advert_int 1
    nopreempt
    authentication {
        auth_type PASS
        auth_pass 1111
    }
    virtual_ipaddress {
        192.168.88.88
    }
}
 
virtual_server 192.168.88.88 3306 {
    delay_loop 2
    lb_algo wrr
    lb_kind DR
    persistence_timeout 60
    protocol TCP
    real_server 192.168.75.61 3306 {
        weight 3
        notify_down /etc/keepalived/down.sh
        TCP_CHECK {
            connect_timeout 10
            nb_get_retry 3
            delay_before_retry 3
            connect_port 3306
        }
    }
}

slave:keepalived.conf

! Configuration File for keepalived
 
global_defs {
   router_id HA_MySQL
}
 
vrrp_instance VI_1 {
    state BACKUP
    interface eth0
    virtual_router_id 51
    priority 90
    advert_int 1
    nopreempt
    authentication {
        auth_type PASS
        auth_pass 1111
    }
    virtual_ipaddress {
        192.168.88.88
    }
}
 
virtual_server 192.168.88.88 3306 {
    delay_loop 2
    lb_algo wrr
    lb_kind DR
    persistence_timeout 60
    protocol TCP
    real_server 192.168.75.62 3306 {
        weight 3
#        echo"62....1"
        notify_down /etc/keepalived/down.sh
#        notify_up /etc/keepalived/up.sh
#        echo"62....2"
        TCP_CHECK {
            connect_timeout 10
            nb_get_retry 3
            delay_before_retry 3
            connect_port 3306
        }
    }
}

down:down.sh

#!/bin/bash``
pkill keepalived

第三步:测试

主:192.168.75.61
备:192.168.75.62
 
主:
[root@qwzs08 log]#service mysqld start
[root@qwzs08 log]#service keeplived start
 
保证已经启动成功!
[root@qwzs08 log]#ip addr
 
会显示出下面的vip(我上面配置文件把master放在61)
 
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue state UNKNOWN
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UNKNOWN qlen 1000
    link/ether 00:50:56:89:a3:28 brd ff:ff:ff:ff:ff:ff
    inet 192.168.75.61/24 brd 192.168.75.255 scope global eth0
    inet 192.168.88.88/32 scope global eth0
    inet6 fe80::250:56ff:fe89:a328/64 scope link
       valid_lft forever preferred_lft forever
 
表示vip正常
 
此时在备机
[root@qwzs07 log]#ip addr
 
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue state UNKNOWN
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UNKNOWN qlen 1000
    link/ether 00:50:56:89:cb:a6 brd ff:ff:ff:ff:ff:ff
    inet 192.168.75.62/24 brd 192.168.75.255 scope global eth0
    inet6 fe80::250:56ff:fe89:cba6/64 scope link
       valid_lft forever preferred_lft forever
 
 
是没有vip的, 恭喜您!配置成功!

第四步:漂移

模拟主机宕机
ps aux | grep mysqld
 
root      1176  0.0  0.0  11336  1408 ?        S    15:38   0:00 /bin/sh /usr/local/mysql/bin/mysqld_safe --datadir=/usr/local/mysql/data --pid-file=/usr/local/mysql/data/mysql3306.pid
mysql     1591  7.1  1.8 2736136 597708 ?      Sl   15:38   3:19 /usr/local/mysql/bin/mysqld --basedir=/usr/local/mysql --datadir=/usr/local/mysql/data --plugin-dir=/usr/local/mysql/lib/plugin --user=mysql --log-error=/usr/local/mysql/data/mysql3306.err --pid-file=/usr/local/mysql/data/mysql3306.pid
root      1978  0.0  0.0 103248   844 pts/0    S+   16:24   0:00 grep mysql
 
kill -9 1176 1591
 
此时,主机的mysql挂掉了,您怎么确定是有没有漂移呢?
查一下vip
ip addr
 
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue state UNKNOWN
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UNKNOWN qlen 1000
    link/ether 00:50:56:89:cb:a6 brd ff:ff:ff:ff:ff:ff
    inet 192.168.75.61/24 brd 192.168.75.255 scope global eth0
    inet6 fe80::250:56ff:fe89:cba6/64 scope link
       valid_lft forever preferred_lft forever
 
已经没有了
再去备机查一下vip
ip addr
 
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue state UNKNOWN
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UNKNOWN qlen 1000
    link/ether 00:50:56:89:a3:28 brd ff:ff:ff:ff:ff:ff
    inet 192.168.75.62/24 brd 192.168.75.255 scope global eth0
    inet 192.168.88.88/32 scope global eth0
    inet6 fe80::250:56ff:fe89:a328/64 scope link
       valid_lft forever preferred_lft forever
 
look, 漂过来了
 
现在切换到主机把的mysql和keeplived开启
 
此时主机的服务已经开起来了, 但是vip并没有漂过去, 这就是nopreempt的作用, 防止脑裂
 
这时候加入备机slave挂掉, vip就会漂移到主机master, 实现高可用主从切换

此时如果能实现漂移就是成功了, 下面是我遇到的问题

1.keepalived执行后日志狂刷, 2.脚本并没有执行(down.sh),具体情况就是mysql挂掉后,keeplived并没有自杀,导致vip漂移失败, 但是手动关闭keeplived后会实现漂移, 原因就在于down脚本没有得到执行。

日志报错:

Nov 21 16:05:27 qwzs08 abrt[13378]: Not saving repeating crash in '/usr/sbin/keepalived'
Nov 21 16:05:27 qwzs08 Keepalived_healthcheckers: IPVS: Can't initialize ipvs: Protocol not available
Nov 21 16:05:27 qwzs08 Keepalived_healthcheckers: Netlink reflector reports IP 192.168.75.62 added
Nov 21 16:05:27 qwzs08 Keepalived_healthcheckers: Netlink reflector reports IP 192.168.75.62 added
Nov 21 16:05:27 qwzs08 Keepalived_healthcheckers: Registering Kernel netlink reflector
Nov 21 16:05:27 qwzs08 Keepalived_healthcheckers: Registering Kernel netlink command channel
Nov 21 16:05:27 qwzs08 Keepalived_healthcheckers: Opening file '/etc/keepalived/keepalived.conf'.
Nov 21 16:05:27 qwzs08 Keepalived_healthcheckers: Configuration is using : 9635 Bytes

很纠结,经过排查,原因是ip_vs模块系统默认没有自动加载

执行命令:

查看
lsmod | grep ip_vs
(如果没有任何输出则表示ip_vs模块并没有被内核加载)
 
手动加载
modprobe ip_vs
 
modprobe ip_vs_wrr

然后再看日志已经恢复正常

此时再查看ip_vs

lsmod | grep ip_vs

ip_vs_wrr               2179  1
ip_vs                 115643  3 ip_vs_wrr
libcrc32c               1246  1 ip_vs
ipv6                  321422  68 ip_vs,ip6t_REJECT,nf_conntrack_ipv6,nf_defrag_ipv6

ok!

内网无编译环境

如遇内网没有编译环境，则可本地编译然后压缩上传keepalived-2.2.7.zip，内部文件含有etc/、keepalived/ 目录、keepalived.service文件三个。

如是内网上传的包，则将解压出来的文件进行复制
1、服务脚本（使用下面的keepalived服务脚本）
将配置文件复制到 /etc/init.d/ 目录下
2、配置文件 （将解压的目录etc/ 复制到 /usr/local/ 目录下）
cp -r /data/commchina/vcc/etc /usr/local/
3、服务（将解压的目录keepalived/ 复制到 /usr/local/ 目录下）
cp -r /data/commchina/vcc/keepalived /usr/local/

keepalived服务脚本文件名为keepalived（该文件也可以不使用）

#!/bin/sh
#
# Startup script for the Keepalived daemon
#
# processname: keepalived
# pidfile: /var/run/keepalived.pid
# config: /etc/keepalived/keepalived.conf
# chkconfig: - 21 79
# description: Start and stop Keepalived

# Source function library
. /etc/rc.d/init.d/functions

# Source configuration file (we set KEEPALIVED_OPTIONS there)
. /etc/sysconfig/keepalived

RETVAL=0

prog="keepalived"

start() {
    echo -n $"Starting $prog: "
    daemon keepalived ${KEEPALIVED_OPTIONS}
    RETVAL=$?
    echo
    [ $RETVAL -eq 0 ] && touch /var/lock/subsys/$prog
}

stop() {
    echo -n $"Stopping $prog: "
    killproc keepalived
    RETVAL=$?
    echo
    [ $RETVAL -eq 0 ] && rm -f /var/lock/subsys/$prog
}

reload() {
    echo -n $"Reloading $prog: "
    killproc keepalived -1
    RETVAL=$?
    echo
}

# See how we were called.
case "$1" in
    start)
        start
        ;;
    stop)
        stop
        ;;
    reload)
        reload
        ;;
    restart)
        stop
        start
        ;;
    condrestart)
        if [ -f /var/lock/subsys/$prog ]; then
            stop
            start
        fi
        ;;
    status)
        status keepalived
        RETVAL=$?
        ;;
    *)
        echo "Usage: $0 {start|stop|reload|restart|condrestart|status}"
        RETVAL=1
esac

exit $RETVAL

修改/usr/local/etc/keepalived/keepalived.conf（master）

! Configuration File for keepalived

global_defs {
   router_id LVS_DEVEL
   vrrp_skip_check_adv_addr
   vrrp_strict
   vrrp_garp_interval 0
   vrrp_gna_interval 0
}


! mysqld:为mysql的进程
vrrp_script chk_nginx {
    script "killall -0 mysqld"
    interval 1
    weight -30
    fall 3
    rise 5
    timeout 2
}
 
! ens192: 当前服务器的真实IP网络（可通过ip a命令查看） priority：优先级 virtual_ipaddress：虚拟IP和网络
vrrp_instance VI_1 {
    state MASTER
    interface ens192
    virtual_router_id 51
    priority 100
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass 1111
    }
    virtual_ipaddress {
        192.168.188.121/24 dev ens192 label ens192:0
    }

    track_script {
        chk_nginx
    }

}

修改/usr/local/etc/keepalived/keepalived.conf（slave）

! Configuration File for keepalived

global_defs {
   router_id LVS_DEVEL
   vrrp_skip_check_adv_addr
   vrrp_strict
   vrrp_garp_interval 0
   vrrp_gna_interval 0
}


! mysqld:为mysql的守护进程
vrrp_script chk_nginx {
    script "killall -0 mysqld"
    interval 1
    weight -30
    fall 3
    rise 5
    timeout 2
}
 
! ens192: 当前服务器的真实IP网络（可通过ip a命令查看） priority：优先级(需小于master) virtual_ipaddress：虚拟IP和网络
vrrp_instance VI_1 {
    state SALVE
    interface ens192
    virtual_router_id 51
    priority 90
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass 1111
    }
    virtual_ipaddress {
        192.168.188.121/24 dev ens192 label ens192:0
    }

    track_script {
        chk_nginx
    }

}

启动服务（sbin目录下面的二进制执行文件）

./keepalived -f /usr/local/etc/keepalived/keepalived.conf

测试

[root@VM-AICS-IF02 local]# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: ens192: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000
    link/ether 00:50:56:bc:57:10 brd ff:ff:ff:ff:ff:ff
    inet 172.19.87.223/24 brd 172.19.87.255 scope global noprefixroute ens192
       valid_lft forever preferred_lft forever
    inet 172.19.87.121/24 scope global ens192:0
       valid_lft forever preferred_lft forever
    inet6 fe80::250:56ff:febc:5710/64 scope link 
       valid_lft forever preferred_lft forever
3: docker0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN group default 
    link/ether 02:42:3a:75:91:59 brd ff:ff:ff:ff:ff:ff
    inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0
       valid_lft forever preferred_lft forever
4: br-e022c9ce2792: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN group default 
    link/ether 02:42:30:5b:45:7a brd ff:ff:ff:ff:ff:ff
    inet 172.18.0.1/16 brd 172.18.255.255 scope global br-e022c9ce2792
       valid_lft forever preferred_lft forever
    inet6 fe80::42:30ff:fe5b:457a/64 scope link 
       valid_lft forever preferred_lft forever

服务注册

在/usr/lib/systemd/system目录下创建keepalived.service文件，内容为下面：（如有报错，请改变下面的文件路径）

[Unit]
Description=LVS and VRRP High Availability Monitor
After=network-online.target syslog.target
Wants=network-online.target
Documentation=man:keepalived(8)
Documentation=man:keepalived.conf(5)
Documentation=man:genhash(1)
Documentation=https://keepalived.org

[Service]
Type=notify
NotifyAccess=all
PIDFile=/run/keepalived.pid
KillMode=process
EnvironmentFile=-/usr/local/etc/sysconfig/keepalived
ExecStart=/usr/local/keepalived/sbin/keepalived --dont-fork $KEEPALIVED_OPTIONS
ExecReload=/bin/kill -HUP $MAINPID

[Install]
WantedBy=multi-user.target