18.11 LVS DR模式搭建
1、准备工作
三台机器:
dir:192.168.37.200
rs1:192.168.37.203
rs2:192.168.37.204
vip:192.168.37.100
2、在dir上编写脚本:vim /usr/local/sbin/lvs_dr.sh
#! /bin/bash #开启端口转发 echo 1 > /proc/sys/net/ipv4/ip_forward ipv=/usr/sbin/ipvsadm vip=192.168.37.100 rs1=192.168.37.203 rs2=192.168.37.204 #注意这里的网卡名字 ifdown ens37 ifup ens37 #命令行设置ip ifconfig ens37:2 $vip broadcast $vip netmask 255.255.255.255 up route add -host $vip dev ens37:2 $ipv -C $ipv -A -t $vip:80 -s wrr #-g 表示dr模式 $ipv -a -t $vip:80 -r $rs1:80 -g -w 1 $ipv -a -t $vip:80 -r $rs2:80 -g -w 1
配置完成后执行:
[root@liang-00 ~]# sh /usr/local/sbin/lvs_dr.sh Determining if ip address 192.168.37.199 is already in use for device ens33... [root@liang-00 ~]#
3、两台rs上的脚本配置vim /usr/local/sbin/lvs_rs.sh:
#/bin/bash vip=192.168.37.100 #把vip绑定在lo上,是为了实现rs直接把结果返回给客户端 ifdown lo ifup lo ifconfig lo:0 $vip broadcast $vip netmask 255.255.255.255 up route add -host $vip lo:0 #以下操作为更改arp内核参数,目的是为了让rs顺利发送mac地址给客户端 #参考文档www.cnblogs.com/lgfeng/archive/2012/10/16/2726308.html echo "1" >/proc/sys/net/ipv4/conf/lo/arp_ignore echo "2" >/proc/sys/net/ipv4/conf/lo/arp_announce echo "1" >/proc/sys/net/ipv4/conf/all/arp_ignore echo "2" >/proc/sys/net/ipv4/conf/all/arp_announce
配置完后执行:
[root@liang-03 ~]# vim /usr/local/sbin/lvs_rs.sh [root@liang-03 ~]# sh !$ sh /usr/local/sbin/lvs_rs.sh [root@liang-03 ~]#
用ip addr查看网口ip(dir上):
[root@liang-00 ~]# ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: ens33: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
link/ether 00:0c:29:f5:8a:7f brd ff:ff:ff:ff:ff:ff
inet 192.168.37.200/24 brd 192.168.37.255 scope global ens33
valid_lft forever preferred_lft forever
inet 192.168.37.100/32 brd 192.168.37.100 scope global ens33:2
valid_lft forever preferred_lft forever
inet 192.168.37.199/24 brd 192.168.37.255 scope global secondary ens33:0
valid_lft forever preferred_lft forever
inet6 fe80::20c:29ff:fef5:8a7f/64 scope link
valid_lft forever preferred_lft forever
3: ens37: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
link/ether 00:0c:29:f5:8a:89 brd ff:ff:ff:ff:ff:ff
[root@liang-00 ~]#
用ip addr查看网口ip(rs上):
[root@liang-03 ~]# ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet 192.168.37.100/32 brd 192.168.37.100 scope global lo:0
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: ens33: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
link/ether 00:0c:29:ff:51:9d brd ff:ff:ff:ff:ff:ff
inet 192.168.37.203/24 brd 192.168.37.255 scope global noprefixroute ens33
valid_lft forever preferred_lft forever
inet6 fe80::424d:1d7a:7483:4c50/64 scope link noprefixroute
valid_lft forever preferred_lft forever
[root@liang-03 ~]#
4、测试:
两台rs上开启nginx服务。
systemctl start nginx.service
在浏览器中测试效果不佳,可以在开一台服务器虚拟机测试。
可以用:ipvsadm -ln 查看连接数:
[root@liang-00 ~]# ipvsadm -ln IP Virtual Server version 1.2.1 (size=4096) Prot LocalAddress:Port Scheduler Flags -> RemoteAddress:Port Forward Weight ActiveConn InActConn TCP 192.168.37.100:80 wrr -> 192.168.37.203:80 Route 1 0 0 -> 192.168.37.204:80 Route 1 0 0 [root@liang-00 ~]#
18.12 keepalived+lvs(DR)
keepalived内置了lvs的功能。
把keepalived加入到lvs中原因:
- lvs中分发器(dir)有可能出现宕机,这时我们就可以用keepalived做高可用。
- 在使用lvs时,如果其中某一个rs宕机,这时dir也会把请求转发送给它,导致用户无法访问。用lvs做高可用可以解决。
1、keepalived+lvs搭建准备。
完整架构需要两台服务器(角色为dir)分别安装keepalived软件,目的是实现高可用,但keepalived本身也有负载均衡的功能,所以本次实验可以只安装一台keepalived。
keepalived内置了ipvsadm的功能,所以不需要再安装ipvsadm包,也不用编写和执行那个lvs_dir的脚本。
三台机器分别为:
dir(安装keepalived)192.168.37.200
rs1 192.168.37.203
rs2 192.168.37.204
vip 192.168.37.100
2、搭建。
编辑keepalived配置文件:vim /etc/keepalived/keepalived.conf
内容在:https://coding.net/u/aminglinux/p/aminglinux-book/git/blob/master/D21Z/lvs_keepalived.conf
vrrp_instance VI_1 { #备用服务器上为 BACKUP state MASTER #绑定vip的网卡为ens33,你的网卡和阿铭的可能不一样,这里需要你改一下 interface ens33 virtual_router_id 51 #备用服务器上为90 priority 100 advert_int 1 authentication { auth_type PASS auth_pass aminglinux } virtual_ipaddress { 192.168.188.110 } } virtual_server 192.168.37.100 80 { #(每隔10秒查询realserver状态) delay_loop 10 #(lvs 算法) lb_algo wlc #(DR模式) lb_kind DR #(同一IP的连接60秒内被分配到同一台realserver) persistence_timeout 60 #(用TCP协议检查realserver状态) protocol TCP real_server 192.168.37.203 80 { #(权重) weight 100 TCP_CHECK { #(10秒无响应超时) connect_timeout 10 nb_get_retry 3 delay_before_retry 3 connect_port 80 } } real_server 192.168.37.204 80 { weight 100 TCP_CHECK { connect_timeout 10 nb_get_retry 3 delay_before_retry 3 connect_port 80 } } }
- 清空之前的ipvsadm规则:ipvsadm -C
- 停掉之前的启动的dr网卡配置(dir上):systemctl restart network
- 打开dir上的端口转发:echo 1 > /proc/sys/net/ipv4/ip_forward
- 重新加载两台rs的lvs_rs.sh脚本。
- 在dir上启动keepalived:systemctl start keepalived.service
查看dir上的ipvsadm规则:
[root@liang-00 ~]# ipvsadm -ln IP Virtual Server version 1.2.1 (size=4096) Prot LocalAddress:Port Scheduler Flags -> RemoteAddress:Port Forward Weight ActiveConn InActConn TCP 192.168.37.100:80 wlc -> 192.168.37.203:80 Route 100 0 0 -> 192.168.37.204:80 Route 100 0 0 [root@liang-00 ~]#
3、测试。
停掉一台rs,可以看到ipvsadm中自动把宕机的设备剔除了。
[root@liang-00 ~]# ipvsadm -ln IP Virtual Server version 1.2.1 (size=4096) Prot LocalAddress:Port Scheduler Flags -> RemoteAddress:Port Forward Weight ActiveConn InActConn TCP 192.168.37.100:80 wlc -> 192.168.37.204:80 Route 100 0 0 [root@liang-00 ~]#
重新启动宕机的rs,ipvsadm重新加载回来rs。
4、完整的keepalived+lvs应该是:2台keepalived(主备)+2台rs
扩展:
heartbeat和keepalived比较:http://blog.csdn.net/yunhua_lee/article/details/9788433
DRBD工作原理和配置:http://502245466.blog.51cto.com/7559397/1298945
mysql+keepalived:http://lizhenliang.blog.51cto.com/7876557/1362313
lvs 三种模式详解:http://www.it165.net/admin/html/201401/2248.html
lvs几种算法:http://www.aminglinux.com/bbs/thread-7407-1-1.html
关于arp_ignore和 arp_announce :http://www.cnblogs.com/lgfeng/archive/2012/10/16/2726308.html
lvs原理相关:http://blog.csdn.net/pi9nc/article/details/23380589
haproxy+keepalived:http://blog.csdn.net/xrt95050/article/details/40926255
nginx、lvs、haproxy比较:http://www.csdn.net/article/2014-07-24/2820837
keepalived中自定义脚本:vrrp_script http://my.oschina.net/hncscwc/blog/158746
lvs dr模式只使用一个公网ip的实现方法:http://storysky.blog.51cto.com/628458/338726