目录
一. lvs简介
- LVS:Linux Virtual Server,负载调度器,内核集成,章文嵩,阿里的四层SLB(Server LoadBalance)是基 于LVS+keepalived实现
- LVS 官网: http://www.linuxvirtualserver.org/
二.lvs的集群和分布式
2.1.系统性能扩展方式
- Scale UP:向上扩展,增强
- Scale Out:向外扩展,增加设备,调度分配问题,Cluster
2.2.集群Cluster
Cluster: 集群是为了解决某个特定问题将堕胎计算机组合起来形成的单个系统
Cluster常见的三种类型:
- LB:LoadBalancing(负载均衡)由多个主机组成,每个主机只承担一部分访问
- HA:High Availiablity(高可用)SPOF(single Point Of failure)
三.lvs集群实验
3.1 lvs概念
- VS:Virtual Server
- RS:Real Server
- CIP:Client IP
- VIP: Virtual serve IP VS外网的IP
- DIP: Director IP VS内网的IP
- RIP: Real server IP
访问流程:CIP VIP == DIP RIP
3.2 lvs集群的类型
- vs-nat: 修改请求报文的目标IP,多目标IP的DNAT
- lvs-dr: 操纵封装新的MAC地址
- lvs-tun: 在原请求IP报文之外新加一个IP首部
- lvs-fullnat: 修改请求报文的源和目标IP
四.具体实验讲解
4.1nat模式
- 本质是多目标IP的DNAT,通过将请求报文中的目标地址和目标端口修改为某挑出的RS的RIP和 PORT实现转发
- RIP和DIP应在同一个IP网络,且应使用私网地址;RS的网关要指向DIP
- 请求报文和响应报文都必须经由Director转发,Director易于成为系统瓶颈
- 支持端口映射,可修改请求报文的目标PORT
- VS必须是Linux系统,RS可以是任意OS系统
4.1.1nat实验操作
4.1.1.1环境配置
主机名称 | 主机IP | 主机网关 |
lvs | 172.25.254.100 192.168.0.100 | 172.25.254.2 |
webserver1 | 192.168.0.10 | 192.168.0.100 |
webserver2 | 192.168.0.20 | 192.168.0.100 |
lvs:
webserver1:
webserver2:
4.1.1.2HTTP服务
为了方便观察饰演的效果,需要在两台webserver主机上安装HTTP服务
[root@webserver1 ~]# dnf install httpd -y
[root@webserver1 ~]# echo webserver1 - 192.168.0.10 > /var/www/html/index.html
[root@webserver1 ~]# systemctl restart httpd
[root@webserver2 ~]# dnf install httpd -y
[root@webserver2 ~]# echo webserve2 - 192.168.0.20 > /var/www/html/index.html
[root@webserver2 ~]# systemctl restart httpd
在lvs主机上进行测试
[root@lvs ~]# curl 192.168.0.10
webserver1 - 192.168.0.10
[root@lvs ~]# curl 192.168.0.20
webserve2 - 192.168.0.20
[root@lvs ~]#
4.1.1.3在lvs主机上进行lvs操作
首先下载lvs服务
[root@lvs ~]# dnf install ipvsadm -y
4.1.1.3.1放行ip_forward
[root@lvs ~]# vim /etc/sysctl.conf
[root@lvs ~]# cat /etc/sysctl.conf
# sysctl settings are defined through files in
# /usr/lib/sysctl.d/, /run/sysctl.d/, and /etc/sysctl.d/.
#
# Vendors settings live in /usr/lib/sysctl.d/.
# To override a whole file, create a new file with the same in
# /etc/sysctl.d/ and put new settings there. To override
# only specific settings, add a file with a lexically later
# name in /etc/sysctl.d/ and put new settings there.
#
# For more information, see sysctl.conf(5) and sysctl.d(5).
net.ipv4.ip_forward=1
[root@lvs ~]# sysctl -p
net.ipv4.ip_forward = 1
[root@lvs ~]#
4.1.1.3.2配置lvs进行配置以实现负载均衡操作
[root@lvs ~]# ipvsadm -A -t 172.25.254.100:80 -s rr
[root@lvs ~]# ipvsadm -a -t 172.25.254.100:80 -r 192.168.0.10:80 -m
[root@lvs ~]# ipvsadm -a -t 172.25.254.100:80 -r 192.168.0.20:80 -m
[root@lvs ~]# ipvsadm -Ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
TCP 172.25.254.100:80 rr
-> 192.168.0.10:80 Masq 1 0 0
-> 192.168.0.20:80 Masq 1 0 0
[root@lvs ~]#
实现效果如下:
[root@lvs ~]# curl 172.25.254.100
webserver1 - 192.168.0.10
[root@lvs ~]# curl 172.25.254.100
webserve2 - 192.168.0.20
[root@lvs ~]#
4.2 DR模式
DR:Direct Routing,直接路由,LVS默认模式,应用最广泛,通过为请求报文重新封装一个MAC首部进行 转发,源MAC是DIP所在的接口的MAC,目标MAC是某挑选出的RS的RIP所在接口的MAC地址;源 IP/PORT,以及目标IP/PORT均保持不变、
4.2.1主机环境配置
主机名称 | 主机IP | 网关配置 |
client | 172.25.254.200/24 | 172.25.254.100 |
router | nat:172.25.254.100 仅主机:192.168.0.100 | 172.25.254.2 |
lvs | 仅主机:192.168.0.50 VIP:192.168.0.200 | 192.168.0.100 |
webserver1 | 仅主机:192.168.0.10 vip:192.168.0.200 | 192.168.0.100 |
webserver2 | 仅主机:192.168.0.20 vip:192.168.0.200 | 192.168.0.100 |
4.2.2 在router中配置ip_forward
[root@client ~]# vim /etc/sysctl.conf
[root@client ~]# sysctl -p
net.ipv4.ip_forward = 1
[root@client ~]# cat /etc/sysctl.conf
# sysctl settings are defined through files in
# /usr/lib/sysctl.d/, /run/sysctl.d/, and /etc/sysctl.d/.
#
# Vendors settings live in /usr/lib/sysctl.d/.
# To override a whole file, create a new file with the same in
# /etc/sysctl.d/ and put new settings there. To override
# only specific settings, add a file with a lexically later
# name in /etc/sysctl.d/ and put new settings there.
#
# For more information, see sysctl.conf(5) and sysctl.d(5).
net.ipv4.ip_forward=1
4.2.3使rs主机中使vip不对外响应
webserver1:
[root@webserver1 ~]# echo 1 > /proc/sys/net/ipv4/conf/all/arp_ignore
[root@webserver1 ~]# echo 1 > /proc/sys/net/ipv4/conf/lo/arp_ignore
[root@webserver1 ~]# echo 2 > /proc/sys/net/ipv4/conf/lo/arp_announce
[root@webserver1 ~]# echo 2 > /proc/sys/net/ipv4/conf/all/arp_announce
[root@webserver1 ~]#
webserver2:
[root@webserver1 ~]# echo 1 > /proc/sys/net/ipv4/conf/all/arp_ignore
[root@webserver1 ~]# echo 1 > /proc/sys/net/ipv4/conf/lo/arp_ignore
[root@webserver1 ~]# echo 2 > /proc/sys/net/ipv4/conf/lo/arp_announce
[root@webserver1 ~]# echo 2 > /proc/sys/net/ipv4/conf/all/arp_announce
[root@webserver1 ~]#
4.2.4.配置lvs规则
[root@lvs ~]# ipvsadm -ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
FWM 66 rr
-> 192.168.0.10:0 Route 1 0 0
-> 192.168.0.20:0 Route 1 0 0
[root@lvs ~]#
4.2.5实现效果
[root@client ~]# for i in {1..10}
> do
> curl 192.168.0.200
> done
webserver1 - 192.168.0.10
webserver2 - 192.168.0.20
webserver1 - 192.168.0.10
webserver2 - 192.168.0.20
webserver1 - 192.168.0.10
webserver2 - 192.168.0.20
webserver1 - 192.168.0.10
webserver2 - 192.168.0.20
webserver1 - 192.168.0.10
webserver2 - 192.168.0.20
[root@client ~]#
五.lvs中的防火墙标签规则
5.1rs中安装mod_ssl模块 让rs支持https
[root@webserver1 ~]# dnf install mod_ssl -y
[root@webserver1 ~]# systemctl restart httpd
5.2 在lvs主机中为端口做标记
[root@lvs ~]# iptables -t mangle -A PREROUTING -d 192.168.0.200 -p tcp -m multiport --dports 80,443 -j MARK --set-mark 66
[root@lvs ~]# iptables -t mangle -nL
Chain PREROUTING (policy ACCEPT)
target prot opt source destination
MARK 6 -- 0.0.0.0/0 192.168.0.200 multiport dports 80,443 MARK set 0x42
Chain INPUT (policy ACCEPT)
target prot opt source destination
Chain FORWARD (policy ACCEPT)
target prot opt source destination
Chain OUTPUT (policy ACCEPT)
target prot opt source destination
Chain POSTROUTING (policy ACCEPT)
target prot opt source destination
5.3 配置lvs规则
[root@lvs ~]# ipvsadm -C
[root@lvs ~]# ipvsadm -ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
[root@lvs ~]# ipvsadm -A -f 66 -s rr
[root@lvs ~]# ipvsadm -a -f 66 -r 192.168.0.10 -g
[root@lvs ~]# ipvsadm -a -f 66 -r 192.168.0.20 -g
[root@lvs ~]# ipvsadm -Ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
FWM 66 rr
-> 192.168.0.10:0 Route 1 0 0
-> 192.168.0.20:0 Route 1 0 0
[root@lvs ~]#
5.4.实现效果
[root@client ~]# for i in {1..6}; do curl 192.168.0.200;curl -k https://192.168.0.200; done
webserver2 - 192.168.0.20
webserver1 - 192.168.0.10
webserver2 - 192.168.0.20
webserver1 - 192.168.0.10
webserver2 - 192.168.0.20
webserver1 - 192.168.0.10
webserver2 - 192.168.0.20
webserver1 - 192.168.0.10
webserver2 - 192.168.0.20
webserver1 - 192.168.0.10
webserver2 - 192.168.0.20
webserver1 - 192.168.0.10