LVS+Keepalied高可用架构
LVS是Linux Virtual Server的简写，意即Linux虚拟服务器，是一个虚拟的服务器集群系统。LVS的负载能力强，因为其工作方式逻辑非常简单，仅进行请求分发，而且工作在网络的第4层，没有流量，所以其效率不需要有过多的忧虑。所以在很多企业是应用很广泛的。LVS基本能支持所有应用，因为工作在第4层，所以LVS可以对几乎所有应用进行负载均衡，包括Web、数据库等。LVS可以提供负载均衡，keepalived可以提供健康检查，故障转移，两者结合很好地提高系统的可用性。
LVS并不能完全判别节点故障，比如在WLC规则下，如果集群里有一个节点没有配置VIP，将会导致整个集群不能使用。
接下来我们依据lvs的优点结合keepalived部署高可用架构，实现更高的可用性。
环境主机centos7,环境需求如下：
node2 测试主机 ens33：172.25.0.32/24
node3 Master ens33： 172.25.0.33/24
node4 backup ens33：172.25.0.34/24
vip : 172.25.0.200/32
为了简便我这里直接用172.25.0.33/24 172.25.0.34/24 直接做revel server
搭建拓扑图如下：
一、环境安装
1、开启路由转发并 、安装lvs。
[root@node3 ~]# echo "1" >/proc/sys/net/ipv4/ip_forward [root@node4 ~]# echo "1" >/proc/sys/net/ipv4/ip_forward
在node3和node4使用yum进行lvs安装
[root@node3 ~]# yum install -y ipvsadm [root@node4 ~]# yum install -y ipvsadm
2、keepalived安装。
我们可以采取源码在node3和node4上安装keepalived就可以了,我这里在node3演示做，另一台也要同样的操作。
先安装keepalived依赖组件
[root@node3 ~]#yum -y install libnl libnl-devel libnfnetlink-devel popt-devel gcc make [root@node3 ~]# cd /usr/local/src/
#下载keepalived压缩包
[root@node3 src]# wget http://www.keepalived.org/software/keepalived-1.2.7.tar.gz [root@node3 src]#tar zxvf keepalived-1.2.7.tar.gz -C /usr/local [root@node3 src]#cd ../keepalived-1.2.7 [root@node3 keepalived-1.2.7]#./configure
#编译成功的结果如下:
Keepalived configuration ------------------------ Keepalived version : 1.2.7 Compiler : gcc Compiler flags : -g -O2 Extra Lib : -lpopt -lssl -lcrypto -lnl Use IPVS Framework : Yes IPVS sync daemon support : Yes IPVS use libnl : Yes Use VRRP Framework : Yes Use VRRP VMAC : Yes SNMP support : No Use Debug flags : No
#继续以下步骤：
[root@node3 keepalived-1.2.7]# make && make install [root@node3 keepalived-1.2.7]#cp /usr/local/etc/rc.d/init.d/keepalived /etc/rc.d/init.d/ [root@node3 keepalived-1.2.7]#cp /usr/local/etc/sysconfig/keepalived /etc/sysconfig/ [root@node3 keepalived-1.2.7]#mkdir /etc/keepalived [root@node3 keepalived-1.2.7]#cp /usr/local/etc/keepalived/keepalived.conf /etc/keepalived/ [root@node3 keepalived-1.2.7]#cp /usr/local/sbin/keepalived /usr/sbin/
3、web服务安装与测试。
node3和node4上安装web服务,用作测试。
[root@node3 ~]# yum install -y httpd [root@node4 ~]# yum install -y httpd
#配置完httpd后我们要可以访问以下本地的web服务.
[root@node3 ~]# curl 172.25.0.33 node3 [root@node4 ~]# curl 172.25.0.34 node4
二、lvs、keepalived配置
1、环境需求。
我们可以先理解好整个架构的配置先
Master: 172.25.0.33
backup: 172.25.0.34
VIP ：172.25.0.200
测试直接用本地web服务测试
2、Real server 的配置
在lvs的DR和TUN模式下，用户的访问请求到达真实服务器后，是直接返回给用户的，而不再经过前端的Director Server，因此，就需要在每个Real server节点上增加虚拟的VIP地址，这样数据才能直接返回给用户，增加VIP地址的操作可以通过下面的命令来实现：
在各自主机绑定vip地址(两台上都执行)。
[root@node3 ~]#ifconfig lo:0 172.25.0.200 broadcast 172.25.0.200 netmask 255.255.255.255 up [root@node4 ~]#ifconfig lo:0 172.25.0.200 broadcast 172.25.0.200 netmask 255.255.255.255 up
#子网掩码255.255.255.255表示这个整个网段只有这一个地址
[root@node3 ~]# route add -host 172.25.0.200 dev lo:0 [root@node4 ~]# route add -host 172.25.0.200 dev lo:0
#上面的命令表示网络请求地址是172.25.0.200的请求，返回的时候的源地址是lo:0网卡上配置的地址，这样出口src地址就是绑定这个假VIP地址，就不会引起丢弃这个包。
3、在Real server上抑制ARP请求
node3上：
[root@node3 ~]# echo "1" >/proc/sys/net/ipv4/conf/lo/arp_ignore [root@node3 ~]# echo "2" >/proc/sys/net/ipv4/conf/lo/arp_announce [root@node3 ~]# echo "1" >/proc/sys/net/ipv4/conf/all/arp_ignore [root@node3 ~]# echo "2" >/proc/sys/net/ipv4/conf/all/arp_announce [root@node3 ~]# sysctl -p
node4上：
[root@node4 ~]# echo "1" >/proc/sys/net/ipv4/conf/lo/arp_ignore [root@node4 ~]# echo "2" >/proc/sys/net/ipv4/conf/lo/arp_announce [root@node4 ~]# echo "1" >/proc/sys/net/ipv4/conf/all/arp_ignore [root@node4 ~]# echo "2" >/proc/sys/net/ipv4/conf/all/arp_announce [root@node4 ~]# sysctl -p
#当然上面的过程可以直接写个脚本执行，就可以省了很多时间
4、查看到vip状态
node3上查看vip状态
[root@node3 ~]# ip addr show lo
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN qlen 1
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet 172.25.0.200/32 brd 172.25.0.200 scope global lo:0
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
node4上查看vip状态
[root@node4 ~]# ip addr show lo
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN qlen 1
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet 172.25.0.200/32 brd 172.25.0.200 scope global lo:0
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
#可以发现两个vip都已经起来了
5、因为keepalived是为了lvs而生的，所以我们可以直接用keepalived直接配置lvs的DR模型。
注意：在企业模式下使用keepalived时，建议是两端都为backup，因为为了防止业务切换繁忙，而使得整个业务系统出问题。
master配置：
[root@node3 ~]# cat /etc/keepalived/keepalived.conf
! Configuration File for keepalived
global_defs {
notification_email {
root@localhost
}
notification_email_from root@localhost
smtp_server localhost
smtp_connect_timeout 30
router_id lvs
vrrp_mcast_group4 224.0.100.19
}
vrrp_instance VI_1 {
state MASTER
interface ens33
virtual_router_id 51
priority 100
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
172.25.0.200
}
}
virtual_server 172.25.0.200 80 {
delay_loop 6
lb_algo rr
lb_kind DR
persistence_timeout 0
protocol TCP
sorry_server 127.0.0.1 80
real_server 172.25.0.33 80 { ##后台的真实服务器IP
weight 1
HTTP_GET {
url {
path /index.html
}
connect_timeout 3
nb_get_retry 3
delay_before_retry 3
}
}
real_server 172.25.0.34 80 { ##后台的真实服务器IP
weight 1
HTTP_GET {
url {
path /index.html
}
connect_timeout 3
nb_get_retry 3
delay_before_retry 3
}
}
}
backup配置：
[root@node4 ~]# cat /etc/keepalived/keepalived.conf
! Configuration File for keepalived
global_defs {
notification_email {
root@localhost
}
notification_email_from root@localhost
smtp_server localhost
smtp_connect_timeout 30
router_id lvs
vrrp_mcast_group4 224.0.100.19
}
vrrp_instance VI_1 {
state BACKUP
interface ens33
virtual_router_id 51
priority 100
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
172.25.0.200
}
}
virtual_server 172.25.0.200 80 {
delay_loop 6
lb_algo rr
lb_kind DR
persistence_timeout 0
protocol TCP
sorry_server 127.0.0.1 80
real_server 172.25.0.33 80 { ##后台的真实服务器IP
weight 1
HTTP_GET {
url {
path /index.html
}
connect_timeout 3
nb_get_retry 3
delay_before_retry 3
}
}
real_server 172.25.0.34 80 { ##后台的真实服务器IP
weight 1
HTTP_GET {
url {
path /index.html
}
connect_timeout 3
nb_get_retry 3
delay_before_retry 3
}
}
}
5、启动keepalived服务
[root@node3 ~]# service keepalived start Unit keepalived.service could not be found. Reloading systemd: [ OK ] Starting keepalived (via systemctl): [ OK ] [root@node4 ~]# service keepalived start Unit keepalived.service could not be found. Reloading systemd: [ OK ] Starting keepalived (via systemctl): [ OK ]
三、高可用测试
1、我们可以查看一下两端的keepalived状态
master上查看keepalived状态：
[root@node3 ~]# service keepalived status -l
● keepalived.service - SYSV: Start and stop Keepalived
Loaded: loaded (/etc/rc.d/init.d/keepalived; bad; vendor preset: disabled)
Active: active (running) since Sat 2018-01-06 17:30:53 CST; 3min 25s ago
Docs: man:systemd-sysv-generator(8)
Process: 73275 ExecStart=/etc/rc.d/init.d/keepalived start (code=exited, status=0/SUCCESS)
Main PID: 73278 (keepalived)
CGroup: /system.slice/keepalived.service
├─73278 keepalived -D
├─73280 keepalived -D
└─73281 keepalived -D
Jan 06 17:30:53 node3 Keepalived_vrrp[73281]: VRRP sockpool: [ifindex(2), proto(112), fd(10,11)]
Jan 06 17:30:53 node3 Keepalived_healthcheckers[73280]: Using LinkWatch kernel netlink reflector...
Jan 06 17:30:53 node3 Keepalived_healthcheckers[73280]: Activating healthchecker for service [172.25.0.33]:80
Jan 06 17:30:53 node3 Keepalived_healthcheckers[73280]: Activating healthchecker for service [172.25.0.34]:80
Jan 06 17:30:54 node3 Keepalived_vrrp[73281]: VRRP_Instance(VI_1) Transition to MASTER STATE
Jan 06 17:30:55 node3 Keepalived_vrrp[73281]: VRRP_Instance(VI_1) Entering MASTER STATE
Jan 06 17:30:55 node3 Keepalived_vrrp[73281]: VRRP_Instance(VI_1) setting protocol VIPs.
Jan 06 17:30:55 node3 Keepalived_vrrp[73281]: VRRP_Instance(VI_1) Sending gratuitous ARPs on ens33 for 172.25.0.200
Jan 06 17:30:55 node3 Keepalived_healthcheckers[73280]: Netlink reflector reports IP 172.25.0.200 added
Jan 06 17:31:00 node3 Keepalived_vrrp[73281]: VRRP_Instance(VI_1) Sending gratuitous ARPs on ens33 for 172.25.0.200
Hint: Some lines were ellipsized, use -l to show in full.
backup上查看keepalived状态：
[root@node4 ~]# service keepalived status -l
● keepalived.service - SYSV: Start and stop Keepalived
Loaded: loaded (/etc/rc.d/init.d/keepalived; bad; vendor preset: disabled)
Active: active (running) since Sat 2018-01-06 17:31:01 CST; 7min ago
Docs: man:systemd-sysv-generator(8)
Process: 18947 ExecStart=/etc/rc.d/init.d/keepalived start (code=exited, status=0/SUCCESS)
Main PID: 18950 (keepalived)
CGroup: /system.slice/keepalived.service
├─18950 keepalived -D
├─18952 keepalived -D
└─18953 keepalived -D
Jan 06 17:31:01 node4 Keepalived_healthcheckers[18952]: Configuration is using : 16719 Bytes
Jan 06 17:31:01 node4 Keepalived_vrrp[18953]: Configuration is using : 63020 Bytes
Jan 06 17:31:01 node4 Keepalived_vrrp[18953]: Using LinkWatch kernel netlink reflector...
Jan 06 17:31:01 node4 Keepalived_vrrp[18953]: VRRP_Instance(VI_1) Entering BACKUP STATE
Jan 06 17:31:01 node4 Keepalived_vrrp[18953]: VRRP sockpool: [ifindex(2), proto(112), fd(10,11)]
Jan 06 17:31:01 node4 Keepalived_healthcheckers[18952]: Using LinkWatch kernel netlink reflector...
Jan 06 17:31:01 node4 Keepalived_healthcheckers[18952]: Activating healthchecker for service [172.25.0.33]:80
Jan 06 17:31:01 node4 Keepalived_healthcheckers[18952]: Activating healthchecker for service [172.25.0.34]:80
Jan 06 17:35:38 node4 Keepalived_vrrp[18953]: Netlink reflector reports IP 192.168.65.135 added
Jan 06 17:35:38 node4 Keepalived_healthcheckers[18952]: Netlink reflector reports IP 192.168.65.135 added
#我们可以发现在我标注的地方可以看到vip已经在master即node3起来了
2、通过vip对web服务测试
[root@node2 ~]# curl 172.25.0.200 node3 [root@node2 ~]# curl 172.25.0.200 node4
发现访问vip可以访问成功
3、把master的keepalived停掉
[root@node3 ~]# service keepalived stop Stopping keepalived (via systemctl): [ OK ]
查看node4的keepalived状态
[root@node4 ~]# service keepalived status -l
● keepalived.service - SYSV: Start and stop Keepalived
Loaded: loaded (/etc/rc.d/init.d/keepalived; bad; vendor preset: disabled)
Active: active (running) since Sat 2018-01-06 17:31:01 CST; 25min ago
Docs: man:systemd-sysv-generator(8)
Process: 18947 ExecStart=/etc/rc.d/init.d/keepalived start (code=exited, status=0/SUCCESS)
Main PID: 18950 (keepalived)
CGroup: /system.slice/keepalived.service
├─18950 keepalived -D
├─18952 keepalived -D
└─18953 keepalived -D
Jan 06 17:35:38 node4 Keepalived_vrrp[18953]: Netlink reflector reports IP 192.168.65.135 added
Jan 06 17:35:38 node4 Keepalived_healthcheckers[18952]: Netlink reflector reports IP 192.168.65.135 added
Jan 06 17:47:09 node4 Keepalived_healthcheckers[18952]: Netlink reflector reports IP 192.168.65.135 added
Jan 06 17:47:09 node4 Keepalived_vrrp[18953]: Netlink reflector reports IP 192.168.65.135 added
Jan 06 17:55:37 node4 Keepalived_vrrp[18953]: VRRP_Instance(VI_1) Transition to MASTER STATE
Jan 06 17:55:38 node4 Keepalived_vrrp[18953]: VRRP_Instance(VI_1) Entering MASTER STATE
Jan 06 17:55:38 node4 Keepalived_vrrp[18953]: VRRP_Instance(VI_1) setting protocol VIPs.
Jan 06 17:55:38 node4 Keepalived_vrrp[18953]: VRRP_Instance(VI_1) Sending gratuitous ARPs on ens33 for 172.25.0.200
Jan 06 17:55:38 node4 Keepalived_healthcheckers[18952]: Netlink reflector reports IP 172.25.0.200 added
Jan 06 17:55:43 node4 Keepalived_vrrp[18953]: VRRP_Instance(VI_1) Sending gratuitous ARPs on ens33 for 172.25.0.200
Hint: Some lines were ellipsized, use -l to show in full.
可以发现已经切换过来了，继续访问vip
[root@node2 ~]# curl 172.25.0.200 node4 [root@node2 ~]# curl 172.25.0.200 node3 [root@node2 ~]# curl 172.25.0.200 node4 [root@node2 ~]# curl 172.25.0.200 node3
也是可以访问的
到这里为止，我们就已经完成了整个lvs+keepalived的高可用实现了。
总结：我们可以发现，整个过程主要是让vip跑在lvsDR模式上，然后通过keepalived来调度后台的真实服务器。
！！以上就是我的实现过程，希望能帮到大家！！！
转载于:https://blog.51cto.com/xiaozhagn/2058179
1356
被折叠的 条评论
为什么被折叠?
到【灌水乐园】发言
