目录
LVS简介
LVS是Linux Virtual Server的简称,也叫Linux虚拟服务器, 也就是现在常说的四层负载均衡。 它是一个由章文嵩博士发起的自由软件项目 。
现在LVS已经是 Linux标准内核的一部分,在Linux2.4内核以前,使用LVS时必须要重新编译内核以支持LVS功能模块,但是从Linux2.4内核以后,已经完全内置了LVS的各个功能模块,无需给内核打任何补丁,可以直接使用LVS提供的各种功能。
LVS是 Linux Virtual Server 的简称,也就是Linux虚拟服务器。这是一个由章文嵩博士发起的一个开源项目,它的官方网是 http://www.linuxvirtualserver.org 现在 LVS 已经是 Linux 内核标准的一部分。使用 LVS 可以达到的技术目标是:通过 LVS 达到的负载均衡技术和 Linux 操作系统实现一个高性能高可用的 Linux 服务器集群,它具有良好的可靠性、可扩展性和可操作性。从而以低廉的成本实现最优的性能。LVS 是一个实现负载均衡集群的开源软件项目,LVS架构从逻辑上可分为调度层、Server集群层和共享存储。
LVS相关术语
- DS:Director Server。指的是前端负载均衡器节点。
- RS:Real Server。后端真实的工作服务器。
- VIP:Virtual IP 向外部直接面向用户请求,作为用户请求的目标的IP地址。
- DIP:Director Server IP,主要用于和内部主机通讯的IP地址。
- RIP:Real Server IP,后端服务器的IP地址。
- CIP:Client IP,访问客户端的IP地址。
工作模式
1、NAT模式:
-
通过网络地址转换实现的虚拟服务器
-
大并发访问时,调度器的性能成为瓶颈
2、DR模式
-
直接使用路由技术实现虚拟服务器
-
节点服务器需要配置VIP,注意MAC地址广播
3、TUN模式
- 通过隧道方式实现虚拟服务
配置lvs-nat模式的httpd负载集群
环境
主机名称 | IP | VIP | 安装应用 |
Client | 192.168.153.150 | 客户端不需要VIP | 无 |
DR | 192.168.153.151 | 192.168.134.100 | ipvsadm |
RS1 | 192.168.153.152 | 网关为DR网关 | httpd |
RS2 | 192.168.153.153 | 网关为DR网关 | httpd |
给DR主机添加一个仅主机的网卡
[root@DR ~]# ifconfig //查看到仅主机网卡的名字为 ens38
ens33: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 192.168.91.129 netmask 255.255.255.0 broadcast 192.168.91.255
inet6 fe80::20c:29ff:feb8:3224 prefixlen 64 scopeid 0x20<link>
ether 00:0c:29:b8:32:24 txqueuelen 1000 (Ethernet)
RX packets 299161 bytes 120054672 (114.4 MiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 382902 bytes 85603867 (81.6 MiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
ens38: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
ether 00:0c:29:b8:32:2e txqueuelen 1000 (Ethernet)
RX packets 0 bytes 0 (0.0 B)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 0 bytes 0 (0.0 B)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
//查看虚拟网络编辑器种仅主机的网段为192.168.134.0
[root@DR ~]# nmcli connection add con-name ens38 ifname ens38 type ethernet
Connection 'ens38' (9b003222-efb6-4d19-8043-a625b3a9c154) successfully added.
[root@DR ~]# nmcli connection
NAME UUID TYPE DEVICE
ens33 af4d3903-2150-4bda-9723-f37666535088 ethernet ens33
ens38 9b003222-efb6-4d19-8043-a625b3a9c154 ethernet ens38
virbr0 95dd368f-e449-44b6-8fb2-cd0cbbb50c2f bridge virbr0
[root@DR ~]# nmcli connection modify ens38 ipv4.addresses 192.168.134.100/24 ipv4.method manual autoconnect yes
[root@DR ~]# systemctl restart NetworkManager
[root@DR ~]# nmcli connection up ens38
Connection successfully activated (D-Bus active path:/org/freedesktop/NetworkManager/ActiveConnection/5)
DR、RS1、RS2三台主机都关闭防火墙和selinux
[root@DR ~]# systemctl stop firewalld
[root@DR ~]# systemctl disable firewalld
[root@DR ~]# sed -i s/SELINUX=enforcing/SELINUX=disabled/g /etc/selinux/config
[root@DR ~]# setenforce 0
[root@RS1 ~]# systemctl stop firewalld
[root@RS1 ~]# systemctl disable firewalld
[root@RS1 ~]# sed -i s/SELINUX=enforcing/SELINUX=disabled/g /etc/selinux/config
[root@RS1 ~]# setenforce 0
[root@RS2 ~]# systemctl stop firewalld
[root@RS2 ~]# systemctl disable firewalld
[root@RS2 ~]# sed -i s/SELINUX=enforcing/SELINUX=disabled/g /etc/selinux/config
[root@RS2 ~]# setenforce 0
配置ip信息
//DR:
[root@DR ~]# vim /etc/sysconfig/network-scripts/ifcfg-ens33
IPADDR=192.168.153.151
PREFIX=24
GATEWAY=192.168.153.2
DNS1=8.8.8.8
//RS1:
[root@RS1 ~]# vim /etc/sysconfig/network-scripts/ifcfg-ens33
.....
IPADDR=192.168.153.152
PREFIX=24
GATEWAY=192.168.153.129
DNS1=8.8.8.8
//RS2:
[root@RS2 ~]# vim /etc/sysconfig/network-scripts/ifcfg-ens33
IPADDR=192.168.153.153
PREFIX=24
GATEWAY=192.168.153.129
DNS1=8.8.8.8
//后端RS1和RS2部署WEB服务器
RS1:
[root@RS1 ~]# yum -y install httpd
[root@RS1 ~]# echo RS1 > /var/www/html/index.html
[root@RS1 ~]# systemctl restart httpd
[root@RS1 ~]# systemctl enable httpd
//RS2:
[root@RS2 ~]# yum -y install httpd
[root@RS2 ~]# echo RS2 > /var/www/html/index.html
[root@RS2 ~]# systemctl restart httpd
[root@RS2 ~]# systemctl enable httpd
//配置DR
(1)开启IP转发功能
[root@DR ~]# vim /etc/sysctl.con
net.ipv4.ip_forward = 1
[root@DR ~]# sysctl -p
net.ipv4.ip_forward = 1
//安装ipvsadm并添加规则
[root@DR ~]# yum -y install ipvsadm
[root@DR ~]# ipvsadm -A -t 192.168.134.100:80 -s rr
[root@DR ~]# ipvsadm -a -t 192.168.134.100:80 -r 192.168.153.152:80 -m
[root@DR ~]# ipvsadm -a -t 192.168.134.100:80 -r 192.168.153.153:80 -m
[root@DR ~]# ipvsadm -Ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
TCP 192.168.134.100:80 rr
-> 192.168.153.152:80 Masq 1 0 0
-> 192.168.153.153:80 Masq 1 0 0
[root@DR ~]# ipvsadm -Sn > /etc/sysconfig/ipvsadm
[root@DR ~]# systemctl restart ipvsadm.service
[root@DR ~]# systemctl enable ipvsadm.service
//客户端测试
[root@Client ~]# curl http://192.168.134.100
RS2
[root@Client ~]# curl http://192.168.134.100
RS1
[root@Client ~]# curl http://192.168.134.100
RS2
[root@Client ~]# curl http://192.168.134.100
RS1
配置lvs-tun模式的httpd负载集群
DR:
//关闭防火墙和selinux
修改内核参数,开启IP转发
[root@DR ~]# vim /etc/sysctl.conf
net.ipv4.ip_forward = 1
[root@DR ~]# sysctl -p
net.ipv4.ip_forward = 1
[root@DR ~]# yum -y install ipvsadm
[root@DR ~]# ifconfig tunl0 192.168.153.151 broadcast 192.168.153.151 netmask 255.255.255.255 up
[root@DR ~]# ip a
.....
4: tunl0@NONE: <NOARP,UP,LOWER_UP> mtu 1480 qdisc noqueue state UNKNOWN group default qlen 1000
link/ipip 0.0.0.0 brd 0.0.0.0
inet 192.168.153.151/32 brd 192.168.153.151 scope global tunl0
valid_lft forever preferred_lft forever
//RS1和RS2(两台主机操作一样)
关闭防火墙和selinux,部署httpd
//启用ipip模块
RS1
[root@rs1 ~]# modprobe ipip
[root@rs1 ~]# ifconfig tunl0 192.168.153.151 broadcast 192.168.153.151 netmask 255.255.255.255 up
RS2
[root@rs2 ~]# modprobe ipip
[root@rs2 ~]# ifconfig tunl0 192.168.153.151 broadcast 192.168.153.151 netmask 255.255.255.255 up
//修改内核参数为
RS1:
[root@RS1 ~]# vim /etc/sysctl.conf
[root@RS1 ~]# sysctl -p
net.ipv4.conf.tunl0.arp_ignore = 1
net.ipv4.conf.tunl0.arp_announce = 2
net.ipv4.conf.all.arp_ignore = 1
net.ipv4.conf.all.arp_announce = 2
net.ipv4.conf.tunl0.rp_filter = 0
net.ipv4.conf.all.rp_filter = 0
RS2:
[root@RS2 ~]# vim /etc/sysctl.conf
[root@RS2 ~]# sysctl -p
net.ipv4.conf.tunl0.arp_ignore = 1
net.ipv4.conf.tunl0.arp_announce = 2
net.ipv4.conf.all.arp_ignore = 1
net.ipv4.conf.all.arp_announce = 2
net.ipv4.conf.tunl0.rp_filter = 0
net.ipv4.conf.all.rp_filter = 0
//DR上添加规则:
[root@DR ~]# ipvsadm -A -t 192.168.153.151:80 -s rr
[root@DR ~]# ipvsadm -a -t 192.168.153.151:80 -r 192.168.153.152 -i
[root@DR ~]# ipvsadm -a -t 192.168.153.151:80 -r 192.168.153.153 -i
[root@DR ~]# ipvsadm -Ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
TCP 192.168.153.151:80 rr
-> 192.168.153.152:80 Tunnel 1 0 0
-> 192.168.153.153:80 Tunnel 1 0 0
[root@DR ~]# ipvsadm -Sn > /etc/sysconfig/ipvsadm
[root@DR ~]# systemctl restart ipvsadm.service
//客户端验证:
[root@Client ~]# curl http://192.168.153.151
RS1
[root@Client ~]# curl http://192.168.153.151
RS2
[root@Client ~]# curl http://192.168.153.151
RS1
[root@Client ~]# curl http://192.168.153.151
RS2
配置lvs-dr模式的httpd负载集群
主机名称 | IP | VIP | 安装应用 |
Client | 192.168.153.150 | 客户端不需要VIP | 无 |
DR | 192.168.153.151 | 192.168.153.100 | ipvsadm |
RS1 | 192.168.153.152 | 192.168.153.100 | httpd |
RS2 | 192.168.153.153 | 192.168.153.100 | httpd |
//配置httpd
RS1:
关闭防火墙和selinux
[root@rs1 ~]# systemctl stop firewalld
[root@rs1 ~]# systemctl disable firewalld
[root@rs1 ~]# sed -i s/SELINUX=enforcing/SELINUX=disabled/g /etc/selinux/config
安装httpd
[root@rs1 ~]# yum -y install httpd
[root@rs1 ~]# echo "RS1" > /var/www/html/index.html
[root@rs1 ~]# systemctl restart httpd
[root@rs1 ~]# systemctl enable httpd
RS2:
关闭防火墙和selinux
[root@RS2 ~]# systemctl stop firewalld
[root@RS2 ~]# systemctl disable firewalld
[root@RS2 ~]# sed -i s/SELINUX=enforcing/SELINUX=disabled/g /etc/selinux/config
安装httpd
[root@RS2 ~]# yum -y install httpd
[root@RS2 ~]# echo "RS2" > /var/www/html/index.html
[root@RS2 ~]# systemctl restart httpd
[root@RS2 ~]# systemctl enable httpd
LVS上配置ip:
DR:
//关闭防火墙和selinux
[root@DR ~]# systemctl stop firewalld
[root@DR ~]# systemctl disable firewalld
[root@DR ~]# sed -i s/SELINUX=enforcing/SELINUX=disabled/g /etc/selinux/config
//临时生效
[root@DR ~]# ifconfig lo 192.168.153.100 broadcast 192.168.153.100 netmask 255.255.255.255 up
//永久生效
[root@DR ~]# vim /etc/rc.d/rc.local
ifconfig lo 192.168.153.100 broadcast 192.168.153.100 netmask 255.255.255.255 up
[root@DR ~]# chmod +x /etc/rc.d/rc.local
[root@DR ~]# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 192.168.153.100/32 brd 192.168.153.100 scope global lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: ens33: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
link/ether 00:0c:29:b8:32:24 brd ff:ff:ff:ff:ff:ff
inet 192.168.153.151/24 brd 192.168.153.255 scope global noprefixroute ens33
valid_lft forever preferred_lft forever
inet6 fe80::20c:29ff:feb8:3224/64 scope link noprefixroute
valid_lft forever preferred_lft forever
//RS上配置arp内核参数
RS1和RS2上都需要操作
vim /etc/sysctl.conf
#将对应网卡设置为只回应目标IP为自身接口地址的ARP请求
net.ipv4.conf.all.arp_ignore = 1
#将ARP请求的源IP设置为所有接口的IP,也就是RIP
net.ipv4.conf.all.arp_announce = 2
net.ipv4.conf.lo.arp_ignore = 1
net.ipv4.conf.lo.arp_announce = 2
//RS1
[root@RS1 ~]# vim /etc/sysctl.conf
net.ipv4.conf.lo.arp_ignore = 1
net.ipv4.conf.lo.arp_announce = 2
net.ipv4.conf.all.arp_ignore = 1
net.ipv4.conf.all.arp_announce = 2
[root@RS1 ~]# sysctl -p
net.ipv4.conf.lo.arp_ignore = 1
net.ipv4.conf.lo.arp_announce = 2
net.ipv4.conf.all.arp_ignore = 1
net.ipv4.conf.all.arp_announce = 2
//RS2
[root@RS2 ~]# vim /etc/sysctl.conf
net.ipv4.conf.lo.arp_ignore = 1
net.ipv4.conf.lo.arp_announce = 2
net.ipv4.conf.all.arp_ignore = 1
net.ipv4.conf.all.arp_announce = 2
[root@RS2 ~]# sysctl -p
net.ipv4.conf.lo.arp_ignore = 1
net.ipv4.conf.lo.arp_announce = 2
net.ipv4.conf.all.arp_ignore = 1
net.ipv4.conf.all.arp_announce = 2
//RS上配置VIP
一定要先配置好内核参数,再配置VIP,如果先配置VIP,VIP配置好后会立即通告给所有人,而修改内核参数就是为了不通告。
//LVS服务器的ens33网卡的ip:192.168.153.100作为VIP
两台RS都要做
RS1:
[root@RS1 ~]# ifconfig lo 192.168.153.100 broadcast 192.168.153.100 netmask 255.255.255.255 up
[root@RS1 ~]# ip a
[root@RS1 ~]# ifconfig lo 192.168.153.100 broadcast 192.168.153.100 netmask 255.255.255.255 up
[root@RS1 ~]# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 192.168.153.100/32 brd 192.168.153.100 scope global lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: ens33: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
link/ether 00:0c:29:70:9e:3b brd ff:ff:ff:ff:ff:ff
inet 192.168.153.152/24 brd 192.168.153.255 scope global noprefixroute ens33
valid_lft forever preferred_lft forever
inet6 fe80::20c:29ff:fe70:9e3b/64 scope link noprefixroute
valid_lft forever preferred_lft forever
[root@RS2 ~]# ifconfig lo 192.168.153.100 broadcast 192.168.153.100 netmask 255.255.255.255 up
[root@RS2 ~]# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 192.168.153.100/32 brd 192.168.153.100 scope global lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: ens33: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
link/ether 00:0c:29:33:c1:e3 brd ff:ff:ff:ff:ff:ff
inet 192.168.153.153/24 brd 192.168.153.255 scope global noprefixroute ens33
valid_lft forever preferred_lft forever
inet6 fe80::20c:29ff:fe33:c1e3/64 scope link noprefixroute
valid_lft forever preferred_lft forever
//添加路由信息
RS1:
[root@RS1 ~]# route add -host 192.168.100.100/32 dev lo
RS2:
[root@RS2 ~]# route add -host 192.168.100.100/32 dev lo
//添加并保存规则
[root@DR ~]# ipvsadm -A -t 192.168.153.100:80 -s rr
[root@DR ~]# ipvsadm -a -t 192.168.153.100:80 -r 192.168.153.152:80 -g
[root@DR ~]# ipvsadm -a -t 192.168.153.100:80 -r 192.168.153.153:80 -g
[root@DR ~]# ipvsadm -Ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
TCP 192.168.153.100:80 rr
-> 192.168.153.153:80 Route 1 0 0
-> 192.168.153.153:80 Route 1 0 0
[root@DR ~]# ipvsadm -Sn > /etc/sysconfig/ipvsadm
[root@DR ~]# cat /etc/sysconfig/ipvsadm
-A -t 192.168.153.100:80 -s rr
-a -t 192.168.153.100:80 -r 192.168.91.152:80 -g -w 1
-a -t 192.168.153.100:80 -r 192.168.91.153:80 -g -w 1
[root@DR ~]# systemctl restart ipvsadm.service
[root@DR ~]# systemctl enable ipvsadm.service
//客户端验证
[root@Client ~]# curl http://192.168.153.100
RS1
[root@Client ~]# curl http://192.168.153.100
RS2
[root@Client ~]# curl http://192.168.153.100
RS1
[root@Client ~]# curl http://192.168.153.100
RS2