LVS+keepalived搭建负载均衡学习手札（二）

最新推荐文章于 2020-08-31 09:19:08 发布

朱智文

最新推荐文章于 2020-08-31 09:19:08 发布

阅读量604

点赞数

分类专栏： lvs + keepalived

lvs + keepalived 专栏收录该内容

5 篇文章 0 订阅

订阅专栏

上次做了LVS+keepalived的负载均衡，效果还不错，但是没有做主备，这次补上：

安装环境：环境 centos4.4

一共准备四台机器：

负载均衡机器：

主：192.168.2.158 #安装lvs+keepalived

备：192.168.2.159 #安装lvs+keepalived

VIP:192.168.2.188

real-server1：192.168.2.187 #仅执行脚本

real-server2：192.168.2.189 #仅执行脚本

安装过程：

在192.168.2.159上安装ipvsadm和keepalived，方法不再赘述，主要看keepalived的配置：

! Configuration File for keepalived

global_defs {

   notification_email {
   admin@xx.com                        #邮件地址，需要本机开启SMTP服务
   }
   notification_email_from root@localhost
   smtp_server 127.0.0.1
   smtp_connect_timeout 30
   router_id LVS_DEVEL2                  #负载均衡器标示：同一局域网内，应该是唯一的。
}

vrrp_sync_group VGM {
   group {
      VI_1
   }
}

vrrp_instance VI_1 {
    state BACKUP                       #设置为BACKUP，通过priority控制那台提升为主。
    interface eth0
    virtual_router_id 51
    priority 70
    advert_int 5
    authentication {
        auth_type PASS
        auth_pass 1111
    }
    virtual_ipaddress {
        192.168.2.188
    }
}

virtual_server 192.168.2.188 80 {
    delay_loop 6
    lb_algo rr
    lb_kind DR
    persistence_timeout 50
    protocol TCP

real_server 192.168.2.187 80 {
        weight 3
        TCP_CHECK {
        nb_get_retry 3
        delay_before_retry 3
        connect_port 80
        connect_timeout 3
        }
    }
    real_server 192.168.2.189 80 {
        weight 1
        TCP_CHECK {
        nb_get_retry 3
        delay_before_retry 3
        connect_port 80
        connect_timeout 3

}
}
}
红色标记为需要修改的地方。

执行/etc/init.d/keepalived start

附上备机keepalived的启动日志：

Oct 22 23:40:35 test1 Keepalived: Starting Keepalived v1.1.17 (10/22,2009)
Oct 22 23:40:35 test1 Keepalived_healthcheckers: Using MII-BMSR NIC polling thread...
Oct 22 23:40:35 test1 Keepalived_healthcheckers: Netlink reflector reports IP 192.168.2.187 added
Oct 22 23:40:35 test1 Keepalived_healthcheckers: Registering Kernel netlink reflector
Oct 22 23:40:35 test1 Keepalived_healthcheckers: Registering Kernel netlink command channel
Oct 22 23:40:35 test1 keepalived: keepalived startup succeeded
Oct 22 23:40:35 test1 Keepalived_healthcheckers: Opening file '/etc/keepalived/keepalived.conf'.
Oct 22 23:40:35 test1 Keepalived_healthcheckers: Configuration is using : 10675 Bytes
Oct 22 23:40:35 test1 Keepalived_healthcheckers: Activating healtchecker for service [192.168.2.187:80]
Oct 22 23:40:35 test1 Keepalived_healthcheckers: Activating healtchecker for service [192.168.2.189:80]
Oct 22 23:40:35 test1 Keepalived: Starting Healthcheck child process, pid=5387
Oct 22 23:40:35 test1 Keepalived_vrrp: Using MII-BMSR NIC polling thread...
Oct 22 23:40:35 test1 Keepalived_vrrp: Netlink reflector reports IP 192.168.2.187 added
Oct 22 23:40:35 test1 Keepalived_vrrp: Registering Kernel netlink reflector
Oct 22 23:40:35 test1 Keepalived: Starting VRRP child process, pid=5389
Oct 22 23:40:35 test1 Keepalived_vrrp: Registering Kernel netlink command channel
Oct 22 23:40:35 test1 Keepalived_vrrp: Registering gratutious ARP shared channel
Oct 22 23:40:35 test1 Keepalived_vrrp: Opening file '/etc/keepalived/keepalived.conf'.
Oct 22 23:40:35 test1 Keepalived_vrrp: Configuration is using : 37155 Bytes
Oct 22 23:40:35 test1 Keepalived_vrrp: VRRP_Instance(VI_1) Entering BACKUP STATE
Oct 22 23:40:35 test1 Keepalived_vrrp: VRRP sockpool: [ifindex(2), proto(112), fd(10,11)]

当停掉主负载均衡时，备机负载均衡的接管日志：

Oct 22 23:42:18 test1 Keepalived_vrrp: VRRP_Instance(VI_1) Transition to MASTER STATE
Oct 22 23:42:18 test1 Keepalived_vrrp: VRRP_Group(VGB) Syncing instances to MASTER state
Oct 22 23:42:23 test1 Keepalived_vrrp: VRRP_Instance(VI_1) Entering MASTER STATE
Oct 22 23:42:23 test1 Keepalived_vrrp: VRRP_Instance(VI_1) setting protocol VIPs.
Oct 22 23:42:23 test1 Keepalived_vrrp: VRRP_Instance(VI_1) Sending gratuitous ARPs on eth0 for 192.168.2.188
Oct 22 23:42:23 test1 Keepalived_vrrp: Netlink reflector reports IP 192.168.2.188 added
Oct 22 23:42:23 test1 Keepalived_healthcheckers: Netlink reflector reports IP 192.168.2.188 added
Oct 22 23:42:28 test1 Keepalived_vrrp: VRRP_Instance(VI_1) Sending gratuitous ARPs on eth0 for 192.168.2.188

当重启主负载均衡时,备机负载均衡的日志：

Oct 22 23:43:18 test1 Keepalived_vrrp: VRRP_Instance(VI_1) Received higher prio advert
Oct 22 23:43:18 test1 Keepalived_vrrp: VRRP_Instance(VI_1) Entering BACKUP STATE
Oct 22 23:43:18 test1 Keepalived_vrrp: VRRP_Instance(VI_1) removing protocol VIPs.
Oct 22 23:43:18 test1 Keepalived_vrrp: VRRP_Group(VGB) Syncing instances to BACKUP state
Oct 22 23:43:18 test1 Keepalived_vrrp: Netlink reflector reports IP 192.168.2.188 removed
Oct 22 23:43:18 test1 Keepalived_healthcheckers: Netlink reflector reports IP 192.168.2.188 removed

以上就完成双机热备的Lvs+keepalived的负载均衡。

relserver上需要执行的脚本：

[root@test1 ~]# more /usr/local/bin/lvs_real
#!/bin/sh
VIP=192.168.2.188              （直接路由模式的vip必须跟服务器对外提供服务的ip地址在同一个网段）
/etc/rc.d/init.d/functions
case "$1" in
start)
    echo " start tunl port"
    ifconfig lo:0 $VIP netmask 255.255.255.255 broadcast $VIP up

   （如果有多VIP，依次往下写）
    echo "2">/proc/sys/net/ipv4/conf/all/arp_announce
    echo "1">/proc/sys/net/ipv4/conf/all/arp_ignore
    echo "2">/proc/sys/net/ipv4/conf/lo/arp_announce
    echo "1">/proc/sys/net/ipv4/conf/lo/arp_ignore
    ;;
stop)
    echo " stop tunl port"
    ifconfig lo:0 down
    echo "0">/proc/sys/net/ipv4/conf/all/arp_announce
    echo "0">/proc/sys/net/ipv4/conf/all/arp_ignore
    echo "0">/proc/sys/net/ipv4/conf/lo/arp_announce
    echo "0">/proc/sys/net/ipv4/conf/lo/arp_ignore
    ;;

*)
echo "Usage: $0 {start|stop}"
exit 1
esac

总结：只需在负载均衡机器上安装ipvsadm和keepalived，并执行keepalived启动。在real server机器上仅需开启本地回环地址（脚本）即可，不用安装任何软件。

经验：千万不要将负载均衡机器和web服务器搭建在一起，虽然端口等都不冲突，但是会出现莫名其妙的问题..

1.real server上未执行此脚本时：
[root@linux keepalived]# ifconfig lo:0
lo:0 Link encap:Local Loopback
UP LOOPBACK RUNNING MTU:16436 Metric:1

2.real server上执行此脚本后：

[root@linux keepalived]# /usr/local/bin/lvs_real start
start tunl port
[root@YuHao-linux keepalived]# ifconfig lo:0
lo:0      Link encap:Local Loopback
          inet addr:192.168.2.188  Mask:255.255.255.255
          UP LOOPBACK RUNNING  MTU:16436  Metric:1
所以：

keepalive机器上不用执行此脚本，因为在keepalived.conf中已经定义了vip地址，

而real server机器上必须执行此脚本。将虚拟IP绑定在本机回环网卡上。

这里还需注意一下MAC地址的问题（netseek）：
假如两台VS之间使用的互备关系，那么当一台VS接管LVS服务时，可能会网络不通，这时因为路由器的MAC缓存表里无法及时刷新MAC.关于vip这个地址的MAC地址还是替换的VS的MAC，有两种解决方法，一种是修改新VS的MAC地
址，另一种是使用 send_arp /arpiing 命令.
以arping 命令为例.
/sbin/arping -I eth0 -c 3 -s ${vip}${gateway_ip} > /dev/null 2>&1
Eg:
/sbin/arping -I eth0 -c 3 -s 192.168.1.6192.168.1.1

如果采用Piranha/keealived方案切换的时候会内置自动发送 send_arp命令.UltraMonkey方案经测试也会自动发送此命令.如用 heartbeat方案,需要写一个send_arp 或者arping 相关的脚本当作heartbeat 一个资源切换服务的时候自动发送相关命令脚本.