LVS+Keepalived群集（群集）

一. Keepalived的概述

1.1 Keepalived的功能
keepalived 主要是用来提供故障切换和健康检查功能-----判断LVS负载调度器，节点服务器的可用性，及时隔离并替换为新的服务器，当故障主机恢复后将其重新加入集群。

1.2 Keepalived 的热备方式
keepalivefd 采用VRRP（虚拟路由冗余协议）热备份协议，以软件的方式实现linux服务器的多机热备功能。
注意:使用Keepalived时，漂移IP地址不需要手动建立虚接口配置文件如（ens33:0），而是由Keepalived 根据配置文件自动管理。

二. LVS + Keepalived 高可用集群配置

2.1 LVS + Keepalived 高可用集群的意义
Keepalived 的设计目的是构建高可用的LVS负载均衡群集，可以调用ipvsadm工具来创建虚拟服务器，管理服务器池，而不仅仅用作双机热备。使用Keepalived 构建LVS群集更加简便易用，主要优势体现在：对LVS负载均衡调度器实现热备切换，提高可用性；对服务器池中的节点进行健康检查，自动移除失效节点，恢复后再重新加入。
注意：使用Keepalived 构建LVS集群时，也需要用到ipvsadm管理工具，但大部分工作会由Keepalived 自动完成，不需要手动执行ipvsadm(除了查看和监控集群以外)

keepalived概述及安装

keepalived的安装与服务控制

[root@localhost ~]# systemctl stop NetworkManager

[root@localhost ~]# setenforce 0

[root@localhost ~]# systemctl stop firewalld

[root@localhost ~]# yum -y install keepalived ipvsadm

[root@localhost ~]# systemctl enable keepalived

1.2：使用keepalived实现双机热备

1：主服务器的配置

[root@localhost ~]# cd /etc/keepalived/

[root@localhost keepalived]# cp keepalived.conf keepalived.conf.bak

[root@localhost keepalived]# vi keepalived.conf

global_defs {

   notification_email {

     acassen@firewall.loc

     failover@firewall.loc

     sysadmin@firewall.loc

   }

   notification_email_from Alexandre.Cassen@firewall.loc

   smtp_server 192.168.200.1

   smtp_connect_timeout 30

   router_id LVS_01

   vrrp_skip_check_adv_addr

#  vrrp_strict

   vrrp_garp_interval 0

   vrrp_gna_interval 0

}

vrrp_instance VI_1 {

    state MASTER

    interface ens33

    virtual_router_id 1

    priority 100

    advert_int 1

    authentication {

        auth_type PASS

        auth_pass 1111

    }

    virtual_ipaddress {

        192.168.10.172

    }

}

注释：

vrrp_strict #严格执行VRRP协议规范，此模式不支持节点单播，如果配置了此参数，vip可以漂移到这台服务器，但是ping vip不通，因此需要将此参数去掉

[root@localhost keepalived]# systemctl start keepalived

[root@localhost keepalived]# ip add show dev ens33

2: ens33: mtu 1500 qdisc pfifo_fast state UP qlen 1000

    link/ether 00:0c:29:ae:7f:64 brd ff:ff:ff:ff:ff:ff

    inet 192.168.10.101/24 brd 172.16.16.255 scope global ens33

       valid_lft forever preferred_lft forever

    inet 192.168.10.172/32 scope global ens33

       valid_lft forever preferred_lft forever

2：备用服务器的配置

[root@localhost ~]# systemctl stop NetworkManager

[root@localhost ~]# setenforce 0

[root@localhost ~]# systemctl stop firewalld

[root@localhost ~]# cd /etc/keepalived/

[root@localhost keepalived]# cp keepalived.conf keepalived.conf.bak

[root@localhost keepalived]# vi keepalived.conf

global_defs {

   notification_email {

     acassen@firewall.loc

     failover@firewall.loc

     sysadmin@firewall.loc

   }

   notification_email_from Alexandre.Cassen@firewall.loc

   #vrrp_strict

   smtp_server 192.168.200.1

   smtp_connect_timeout 30

   router_id LVS_02

}

vrrp_instance VI_1 {

    state BACKUP

    interface ens33

    virtual_router_id 1

    priority 99

    advert_int 1

    authentication {

        auth_type PASS

        auth_pass 1111

    }

    virtual_ipaddress {

        192.168.10.172

    }

}

3：使用ping命令测试虚拟ip的连通性

在测试过程中down掉master服务器的网络，观察ping的结果，如果keepalive运行正常，ping的结果不会中断。

LVS+Keepalived高可用性

配置主调度器

主服务器keepalived的安装

[root@localhost ~]# systemctl stop NetworkManager

[root@localhost ~]# setenforce 0

[root@localhost ~]# systemctl stop firewalld

[root@localhost ~]# yum -y install keepalived ipvsadm

[root@localhost ~]# systemctl enable keepalived

2：主服务器keepalived的配置

[root@localhost ~]# cd /etc/keepalived/

[root@localhost keepalived]# cp keepalived.conf keepalived.conf.bak

[root@localhost keepalived]# vi keepalived.conf

global_defs {

   notification_email {

     acassen@firewall.loc

     failover@firewall.loc

     sysadmin@firewall.loc

   }

   notification_email_from Alexandre.Cassen@firewall.loc

   #vrrp_strict

   smtp_server 192.168.200.1

   smtp_connect_timeout 30

   router_id LVS_01

}

vrrp_instance VI_1 {

    state MASTER

    interface ens33

    virtual_router_id 51

    priority 100

    advert_int 1

    authentication {

        auth_type PASS

        auth_pass 1111

    }

    virtual_ipaddress {

        192.168.10.172

    }

}

virtual_server 192.168.10.172 80 {

    delay_loop 6

    lb_algo rr

    lb_kind DR

    nat_mask 255.255.255.0

!    persistence_timeout 50

    protocol TCP

    real_server 192.168.10.103 80 {

        weight 1

        TCP_CHECK {

            connect_port 80

            connect_timeout 3

            nb_get_retry 3

            delay_before_retry 3

        }

    }

    real_server 192.168.10.104 80 {

        weight 1

        TCP_CHECK {

            connect_port 80

            connect_timeout 3

            nb_get_retry 3

            delay_before_retry 3

        }

    }

}

注释：persistence_timeout 50

这个参数的意义是保持客户端的请求在这个时间段内全部发到同一个真实服务器。

3：主服务器内核参数的配置

[root@localhost ~]# vi /etc/sysctl.conf

在末尾添加：

net.ipv4.conf.all.send_redirects = 0

net.ipv4.conf.default.send_redirects = 0

net.ipv4.conf.ens33.send_redirects = 0

[root@localhost ~]# sysctl -p

4：开启主服务器的keepalived服务

[root@localhost keepalived]# systemctl start keepalived

[root@localhost keepalived]# ip add show dev ens33

2: ens33: mtu 1500 qdisc pfifo_fast state UP qlen 1000

    link/ether 00:0c:29:ae:7f:64 brd ff:ff:ff:ff:ff:ff

    inet 192.168.10.101/24 brd 172.16.16.255 scope global ens33

       valid_lft forever preferred_lft forever

    inet 192.168.10.172/32 scope global ens33

       valid_lft forever preferred_lft forever

配置从调度器

1：从调度器keepalived安装

[root@localhost ~]# systemctl stop NetworkManager

[root@localhost ~]# setenforce 0

[root@localhost ~]# systemctl stop firewalld

[root@localhost ~]# yum -y install keepalived ipvsadm

[root@localhost ~]# systemctl enable keepalived

2：从调度器keepalived的配置

[root@localhost ~]# cd /etc/keepalived/

[root@localhost keepalived]# cp keepalived.conf keepalived.conf.bak

[root@localhost keepalived]# vi keepalived.conf

global_defs {

   notification_email {

     acassen@firewall.loc

     failover@firewall.loc

     sysadmin@firewall.loc

   }

   notification_email_from Alexandre.Cassen@firewall.loc

   #vrrp_strict

   smtp_server 192.168.200.1

   smtp_connect_timeout 30

   router_id LVS_02

}

vrrp_instance VI_1 {

    state BACKUP

    interface ens33

    virtual_router_id 51

    priority 99

    advert_int 1

    authentication {

        auth_type PASS

        auth_pass 1111

    }

    virtual_ipaddress {

        192.168.10.172

    }

}

virtual_server 192.168.10.172 80 {

    delay_loop 6

    lb_algo rr

lb_kind DR

    nat_mask 255.255.255.0

!    persistence_timeout 50

    protocol TCP

    real_server 192.168.10.103 80 {

        weight 1

        TCP_CHECK {

            connect_port 80

            connect_timeout 3

            nb_get_retry 3

            delay_before_retry 3

        }

    }

    real_server 192.168.10.104 80 {

        weight 1

        TCP_CHECK {

            connect_port 80

            connect_timeout 3

            nb_get_retry 3

            delay_before_retry 3

        }

    }

}

注释：persistence_timeout 50

这个参数的意义是保持客户端的请求在这个时间段内全部发到同一个真实服务器。

3：主服务器内核参数的配置

[root@localhost ~]# vi /etc/sysctl.conf

在末尾添加：

net.ipv4.conf.all.send_redirects = 0

net.ipv4.conf.default.send_redirects = 0

net.ipv4.conf.ens33.send_redirects = 0

[root@localhost ~]# sysctl -p

4：开启主服务器的keepalived服务

[root@localhost keepalived]# systemctl start keepalived

[root@localhost keepalived]# ip add show dev ens33

2: ens33: mtu 1500 qdisc pfifo_fast state UP qlen 1000

    link/ether 00:0c:29:ae:7f:64 brd ff:ff:ff:ff:ff:ff

    inet 192.168.10.101/24 brd 172.16.16.255 scope global ens33

       valid_lft forever preferred_lft forever

    inet 192.168.10.172/32 scope global ens33

       valid_lft forever preferred_lft forever

配置从调度器

1：从调度器keepalived安装

[root@localhost ~]# systemctl stop NetworkManager

[root@localhost ~]# setenforce 0

[root@localhost ~]# systemctl stop firewalld

[root@localhost ~]# yum -y install keepalived ipvsadm

[root@localhost ~]# systemctl enable keepalived

2：从调度器keepalived的配置

[root@localhost ~]# cd /etc/keepalived/

[root@localhost keepalived]# cp keepalived.conf keepalived.conf.bak

[root@localhost keepalived]# vi keepalived.conf

global_defs {

   notification_email {

     acassen@firewall.loc

     failover@firewall.loc

     sysadmin@firewall.loc

   }

   notification_email_from Alexandre.Cassen@firewall.loc

   #vrrp_strict

   smtp_server 192.168.200.1

   smtp_connect_timeout 30

   router_id LVS_02

}

vrrp_instance VI_1 {

    state BACKUP

    interface ens33

    virtual_router_id 51

    priority 99

    advert_int 1

    authentication {

        auth_type PASS

        auth_pass 1111

    }

    virtual_ipaddress {

        192.168.10.172

    }

}

virtual_server 192.168.10.172 80 {

    delay_loop 6

    lb_algo rr

    lb_kind DR

    nat_mask 255.255.255.0

!    persistence_timeout 50

    protocol TCP

    real_server 192.168.10.101 80 {

        weight 1

        TCP_CHECK {

            connect_port 80

            connect_timeout 3

            nb_get_retry 3

            delay_before_retry 3

        }

    }

    real_server 192.168.10.102 80 {

        weight 1

        TCP_CHECK {

            connect_port 80

            connect_timeout 3

            nb_get_retry 3

            delay_before_retry 3

        }

    }

}

从服务器内核参数的配置

[root@localhost ~]# vi /etc/sysctl.conf

在末尾添加：

net.ipv4.conf.all.send_redirects = 0

net.ipv4.conf.default.send_redirects = 0

net.ipv4.conf.ens33.send_redirects = 0

[root@localhost ~]# sysctl -p

4：开启从服务器的keepalived服务

[root@localhost keepalived]# systemctl start keepalived

2.3：服务器池配置

1：web1服务器配置

（1）web1网络的配置

[root@localhost ~]# systemctl stop firewalld

[root@localhost ~]# cd /etc/sysconfig/network-scripts/

[root@localhost network-scripts]# cp ifcfg-lo ifcfg-lo:0

[root@localhost network-scripts]# vi ifcfg-lo:0

DEVICE=lo:0

IPADDR=192.168.10.172

NETMASK=255.255.255.255

ONBOOT=yes

[root@localhost network-scripts]# systemctl restrt network

[root@localhost network-scripts]# vi /etc/rc.local

/sbin/route add -host 192.168.10.172 dev lo:0

[root@localhost network-scripts]# route add -host 192.168.10.172 dev lo:0

（2）httpd服务的安装

[root@localhost ~]# yum -y install httpd

[root@localhost ~]# vi /var/www/html/index.html

test web01

（3）内核参数的设置

[root@localhost ~]# vi /etc/sysctl.conf

net.ipv4.conf.all.arp_ignore = 1

net.ipv4.conf.all.arp_announce = 2

net.ipv4.conf.default.arp_ignore = 1

net.ipv4.conf.default.arp_announce = 2

net.ipv4.conf.lo.arp_ignore = 1

net.ipv4.conf.lo.arp_announce = 2

（4）开启httpd服务

[root@localhost ~]# systemctl start httpd

2：web2服务器配置

（1）web2网络的配置

[root@localhost ~]# systemctl stop firewalld

[root@localhost ~]# cd /etc/sysconfig/network-scripts/

[root@localhost network-scripts]# cp ifcfg-lo ifcfg-lo:0

[root@localhost network-scripts]# vi ifcfg-lo:0

DEVICE=lo:0

IPADDR=192.168.10.172

NETMASK=255.255.255.255

ONBOOT=yes

[root@localhost network-scripts]# systemctl restrt network

[root@localhost network-scripts]# vi /etc/rc.local

/sbin/route add -host 192.168.10.172 dev lo:0

[root@localhost network-scripts]# route add -host 192.168.10.172 dev lo:0

（2）httpd服务的安装

[root@localhost ~]# yum -y install httpd

[root@localhost ~]# vi /var/www/html/index.html

test web02

（3）内核参数的设置

[root@localhost ~]# vi /etc/sysctl.conf

net.ipv4.conf.all.arp_ignore = 1

net.ipv4.conf.all.arp_announce = 2

net.ipv4.conf.default.arp_ignore = 1

net.ipv4.conf.default.arp_announce = 2

net.ipv4.conf.lo.arp_ignore = 1

net.ipv4.conf.lo.arp_announce = 2

（4）开启httpd服务

[root@localhost ~]# systemctl start httpd

2.4：测试LVS+Keepalived高可用性

1：用客户端访问网站

http://192.168.10.172

刷新页面并观察网页的变化

2：在客户端使用脚本测试

[root@localhost ~]# for i in $(seq 10); do curl http://192.168.10.172  ;done

3：注意事项

（1）生产环境中可以使用NFS服务器保证网站代码的一致性，在测试环境中为了观察效果，web服务器池中的网站代码可以不一样，更加便于观察实验效果。

（2）测试计算机不要使用master调度器，在master调度器上访问VIP时，调度器不会将访问的请求调度到web服务器，而是自己尝试解析；在web服务器上测试时只能访问到自己的网页，无法实现调度。所以客户端一定要使用独立的测试计算机，或者使用处于BACKUP状态的调度器。