lvs集群

## 负载均衡

一、好处：
1.提高了系统的整体性能
2.提高了系统的扩展性
3.提高了系统的可用性
二、负载均衡类型
1.DNS
2.硬件负载均衡
3.软件负载均衡

DNS实现负载均衡
DNS实现负载均衡是最基础简单的方式，一个域名通过DNS解析到多个IP，每个IP对应不同的服务器实例，这样就完成了流量的调度，完成了负载均衡的功能。

硬件负载均衡
硬件负载均衡的优点：
功能强大：全面支持各层级的负载均衡，支持全面的负载均衡算法。
性能强大：性能远超常见的软件负载均衡器。
稳定性高：商用硬件负载均衡，经过了良好的严格测试，经过大规模使用，稳定性高。
安全防护：除了具备负载均衡外，还具备防火墙、防 DDoS 攻击等安全功能，貌似还支持 SNAT 功能。
硬件负载均衡的缺点：
价格昂贵，就是贵。
扩展性差，无法进行扩展和定制。
调试和维护比较麻烦，需要专业人员。
软件负载均衡
软件负载均衡，可以在普通的服务器上运行负载均衡软件，实现负载均衡功能。目前常见的有 Nginx、HAproxy、LVS
软件负载均衡的优点：
简单：无论是部署还是维护都比较简单。
便宜：买个 Linux 服务器，装上软件即可。
灵活：4 层和 7 层负载均衡可以根据业务进行选择；也可以根据业务特点，比较方便进行扩展和定制功能。

LVS

简介：
LVS 是基于 Linux 内核中 netfilter 框架实现的负载均衡系统，所以要学习 LVS 之前必须要先简单了解 netfilter 基本工作原理。netfilter 其实很复杂也很重要，平时我们说的 Linux 防火墙就是 netfilter，不过我们平时操作的都是 iptables，iptables 只是用户空间编写和传递规则的工具而已，真正工作的是 netfilter。
LVS基本原理
LVS 是基于 netfilter 框架，主要工作于 INPUT 链上，在 INPUT 上注册 ip_vs_in HOOK 函数，进行 IPVS 主流程，大概原理如图所示：

LVS三种工作模式
1.DR模式
2.NAT模式
3.Tunnel模式
4.FULLNAT
相关操作

查看系统对ipvs的支持情况，包括算法
[root@localhost ~]# grep -i -A 2 'ipvs' /boot/config-3.10.0-514.el7.x86_64 
CONFIG_NETFILTER_XT_MATCH_IPVS=m
CONFIG_NETFILTER_XT_MATCH_LENGTH=m
CONFIG_NETFILTER_XT_MATCH_LIMIT=m
--
# IPVS transport protocol load balancing support
#
CONFIG_IP_VS_PROTO_TCP=y
--
# IPVS scheduler
#
CONFIG_IP_VS_RR=m
--
# IPVS SH scheduler
#
CONFIG_IP_VS_SH_TAB_BITS=8
--
# IPVS application helper
#
CONFIG_IP_VS_FTP=m
[root@localhost ~]#

lvs arch：
调度器：director，dispatcher，balancer
RS：Real Server

Client IP：CIP 客户端
Director Virutal IP：VIP 
Director IP：DIP
Real Server IP：RIP

环境

名称	IP
DR	192.168.175.100
RS1	192.168.175.150
RS2	192.168.175.151

安装ipvsadm包

[root@DR ~]# yum -y install ipvsadm

配置dip保证在同一个wangduan（此时已经是同一个网段，省略配置）

配置DR上的vip

[root@localhost ~]# ip a 
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN qlen 1
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: ens33: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 00:0c:29:1e:69:85 brd ff:ff:ff:ff:ff:ff
    inet 192.168.175.100/24 brd 192.168.175.255 scope global ens33
       valid_lft forever preferred_lft forever
    inet6 fe80::ea80:e71e:f5fc:9a2c/64 scope link 
       valid_lft forever preferred_lft forever
[root@localhost ~]# hostname DR
[root@localhost ~]# bash
[root@DR ~]# which ipvsadm
/usr/bin/which: no ipvsadm in (/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/root/bin)
[root@DR ~]# yum -y install ipvsadm
[root@DR ~]# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN qlen 1
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: ens33: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 00:0c:29:1e:69:85 brd ff:ff:ff:ff:ff:ff
    inet 192.168.175.100/24 brd 192.168.175.255 scope global ens33
       valid_lft forever preferred_lft forever
    inet6 fe80::ea80:e71e:f5fc:9a2c/64 scope link 
       valid_lft forever preferred_lft forever
[root@DR ~]# ip addr add 192.168.175.250/32 dev lo   添加网卡到lo
[root@DR ~]# ip a  查看是否添加成功
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN qlen 1
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet 192.168.175.250/32 scope global lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: ens33: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 00:0c:29:1e:69:85 brd ff:ff:ff:ff:ff:ff
    inet 192.168.175.100/24 brd 192.168.175.255 scope global ens33
       valid_lft forever preferred_lft forever
    inet6 fe80::ea80:e71e:f5fc:9a2c/64 scope link 
       valid_lft forever preferred_lft forever
[root@DR ~]# ping 192.168.175.250 看是否能够ping通
PING 192.168.175.250 (192.168.175.250) 56(84) bytes of data.
64 bytes from 192.168.175.250: icmp_seq=1 ttl=64 time=0.398 ms
64 bytes from 192.168.175.250: icmp_seq=2 ttl=64 time=0.066 ms
^C
--- 192.168.175.250 ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 1000ms
rtt min/avg/max/mdev = 0.066/0.232/0.398/0.166 ms
[root@DR ~]# 

配置rip（编辑物理网卡配置文件/etc/sysconfig/network-scripts/ifcfg-eth0）此时已经在同网段省略配置
修改RS上的网卡内核参数

[root@RS1 ~]# vim /etc/sysctl.conf
[root@RS1 ~]# sysctl -p
net.ipv4.conf.all.arp_ignore = 1
net.ipv4.conf.all.arp_announce = 2
RS2进行同样操作
[root@RS2 ~]# vim /etc/sysctl.conf
[root@RS2 ~]# sysctl -p
net.ipv4.conf.all.arp_ignore = 1
net.ipv4.conf.all.arp_announce = 2

配置vip地址
注意：此处必须先修改网卡内核参数然后再配置vip，因为如果先配vip，vip配好后就会立马通告给别人，而修改内核参数就是为了不通告

读取操作
[root@RS1 ~]# sysctl -p
net.ipv4.conf.all.arp_ignore = 1
net.ipv4.conf.all.arp_announce = 2
RS2重复操作
[root@RS2 ~]# sysctl -p
net.ipv4.conf.all.arp_ignore = 1
net.ipv4.conf.all.arp_announce = 2
RS1添加vip
[root@RS1 ~]# ip addr add 192.168.175.250 dev lo
[root@RS1 ~]# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN qlen 1
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet 192.168.175.250/32 scope global lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: ens33: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 00:0c:29:fe:26:ad brd ff:ff:ff:ff:ff:ff
    inet 192.168.175.150/24 brd 192.168.175.255 scope global ens33
       valid_lft forever preferred_lft forever
    inet6 fe80::4bd8:f5cb:d39:a60c/64 scope link 
       valid_lft forever preferred_lft forever
[root@RS1 ~]# 
RS2添加vip 
[root@RS2 ~]# ip addr add 192.168.175.250 dev lo
[root@RS2 ~]# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN qlen 1
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet 192.168.175.250/32 scope global lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: ens33: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 00:0c:29:6d:98:29 brd ff:ff:ff:ff:ff:ff
    inet 192.168.175.151/24 brd 192.168.175.255 scope global ens33
       valid_lft forever preferred_lft forever
    inet6 fe80::7790:6522:b7ac:8c7c/64 scope link 
       valid_lft forever preferred_lft forever
[root@RS2 ~]# 

配置路由信息：在director和所有RS上进行如下配置：

[root@DR ~]# route add -host 192.168.175.250 dev lo

[root@RS1 ~]# route add -host 192.168.175.250 dev lo

[root@RS2 ~]# route add -host 192.168.175.250 dev lo

在director上添加并保存规则：

[root@DR ~]# route add -host 192.168.175.250 dev lo
[root@DR ~]#  ipvsadm -A -t 192.168.175.250:80 -s wrr
[root@DR ~]# ipvsadm -a -t 192.168.175.250:80 -r 192.168.175.150:80 -g
[root@DR ~]# ipvsadm -a -t 192.168.175.250:80 -r 192.168.175.151:80 -g
[root@DR ~]#  ipvsadm -S > /etc/sysconfig/ipvsadm
[root@DR ~]# ipvsadm -Ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
  -> RemoteAddress:Port           Forward Weight ActiveConn InActConn
TCP  192.168.175.250:80 wrr
  -> 192.168.175.150:80           Route   1      0          0         
  -> 192.168.175.151:80           Route   1      0          0         
[root@DR ~]# 

RS1和RS2下载Apache,配置页面

RS1配置
[root@RS1 ~]# yum -y install httpd
安装过程省略。。。。
[root@RS1 ~]# cd /var/www/html/
[root@RS1 html]# ls
[root@RS1 html]# echo 'rs1' > index.html    配置页面
[root@RS1 html]# systemctl enable --now httpd
Created symlink from /etc/systemd/system/multi-user.target.wants/httpd.service to /usr/lib/systemd/system/httpd.service.
[root@RS1 html]# cd /etc/httpd/
[root@RS1 httpd]# ls
conf  conf.d  conf.modules.d  logs  modules  run
[root@RS1 httpd]# cd conf
[root@RS1 conf]# ls
httpd.conf  magic
[root@RS1 conf]# vim httpd.conf 
取消前面的#号
ServerName www.example.com:80

[root@RS1 conf]# systemctl start httpd
[root@RS1 conf]# systemctl stop firewalld   关闭防火墙
[root@RS1 conf]# systemctl disable firewalld
Removed symlink /etc/systemd/system/dbus-org.fedoraproject.FirewallD1.service.
Removed symlink /etc/systemd/system/basic.target.wants/firewalld.service.

RS2配置
[root@RS2 ~]# yum -y install httpd
安装过程省略
[root@RS2 ~]# cd /var/www/html/
[root@RS2 html]# echo 'rs2' > index.html
[root@RS2 html]# systemctl enable --now httpd
Created symlink from /etc/systemd/system/multi-user.target.wants/httpd.service to /usr/lib/systemd/system/httpd.service.
[root@RS2 html]# cd
[root@RS2 ~]# cd /etc/httpd/
[root@RS2 httpd]# ls
conf  conf.d  conf.modules.d  logs  modules  run
[root@RS2 httpd]# cd conf
[root@RS2 conf]# ls
httpd.conf  magic
[root@RS2 conf]# vim httpd.conf 
取消前面的#号
ServerName www.example.com:80

[root@RS2 conf]# systemctl stop firewalld
[root@RS2 conf]# systemctl disable firewalld
Removed symlink /etc/systemd/system/dbus-org.fedoraproject.FirewallD1.service.
Removed symlink /etc/systemd/system/basic.target.wants/firewalld.service.
[root@RS2 conf]# 

本机验证

C:\Users\Administrator>curl http://192.168.175.250
rs1

C:\Users\Administrator>curl http://192.168.175.250
rs2

C:\Users\Administrator>

LVS-NAT模式
lvs-nat配置：director要配置2块网卡
配置ip地址信息：
director（dip，vip）
RS（rip）
开启director的ip转发功能
在director上添加并保存规则：
ipvsadm -A -t vip:port -s rr
ipvsadm -a -t vip:port -r rip -m
ipvsadm -S > /etc/sysconfig/ipvsadm
LVS-tun模式

lvs-tun（ip tunneling）：ipip
lvs-tun模式不修改请求报文的ip首部，而是通过在原有的ip首部（CIP<–>VIP）之外再封装一个ip首部（DIP<–>RIP）
lvs-tun的特点：
RIP，DIP，VIP必须是公网地址
RS的网关不能指向DIP
请求报文必须经由director调度，但响应报文必须不能经由director
不支持端口映射
RS的OS必须支持隧道功能
LVS-fullnat模式
lvs-fullnat：keepalived
director通过同时修改请求报文的目标地址和源地址进行转发
lvs-fullnat的特点：
VIP是公网地址，RIP和DIP是私网地址，RIP与DIP无须在同一网络中
RS接收到的请求报文的源地址为DIP，因此要响应给DIP
请求报文和响应报文都必须经由Director
支持端口映射机制
RS可以使用任意OS
lvs scheduler：lvs调度器，即lvs挑选RS的算法
两种方法：
1.静态方法：仅根据算法本身进行调度
RR：round robin，轮调
WRR：weighted rr，加权的rr，根据一定的比例进行轮调，比如每次RS1给2个请求，RS2给1个请求
SH：source hash，源地址hash，实现session保持的机制，将来自于同一个IP的请求始终调度至同一RS，每个服务单独调度
DH：destination hash，目标地址hash，将对同一个目标（资源）的请求始终发往同一个RS

2.动态方法：根据算法及各RS的当前负载状态进行调度，根据指定的算法算出overhead（负载），最终挑选出overhead值最小的则为被选中的RS
LC：Least Connection，最少连接数，算法如下：
overhead = Active * 256 + Inactive
WLC：Weighted LC，加权的LC，算法如下：
overhead=（Active*256+Inactive）/ weight
SED：Shortest Expection Delay，最短期望延迟，算法如下：
overhead = （Active + 1） * 256 / weight
NQ：Nevel Queue，是SED算法的改进，根据SED算法每台主机第一次至少要均分配一次，然后再按SED算法来挑选
LBLC：Locality-Based LC，基于本地的最少连接数，即为动态的DH算法
正向代理情形下的cache server调度
LBLCR：Locality-Based Least-Connection with Replication，带复制功能的LBLC算法
ipvs的集群服务：
支持TCP，UDP，AH，EST，AH_EST，SCTP等诸多协议

ipvs集群服务的特点：
    一个ipvs主机可以同时定义多个cluster service
        定义时指明lvs-type（lvs的模式）以及lvs scheduler（调度器）
    一个cluster service上至少应该有一个real server        
    pvsadm的用法：
管理集群服务：
    ipvsadm -A|E -t|u|f service-address port [-s scheduler] [-p [timeout]] [-M netmask]
    ipvsadm -D -t|u|f service-address port
    
    常见的service-address：
        tcp：-t ip:port
        udp：-u ip:port
        fwm：-f mark
    -s scheduler：
        默认为wlc
    -p [timeout]：定义持久连接，timeout不指定时默认为300秒
管理集群服务中的RS
    ipvsadm -a|e -t|u|f service-address -r server-address [-g|i|m] [-w weight] [-x upper] [-y lower]
    ipvsadm -d -t|u|f service-address -r server-address
    ipvsadm -L|l [options]
    
    server-address：
        ip[:port]
    lvs-type：
        -g：gateway，dr模式
        -i：ipip，tun模式
        -m：masquerade，nat模式
-w：设定权重，可为0到正无穷数值，设为0时不会被调度，默认为1
    权重值越大则表示性能越好，被调度的资源也会更多
清空和查看：
    ipvsadm -C	清空
    ipvsadm -L|l [options]	查看
        options：
        -n：numeric，基于数字格式显示地址和端口
        -c：connection，显示当前ipvs连接
        --stats：统计数据
        --rate：输出速率信息
        --exact：显示精确值，不做单位换算
保存和重载：
    ipvsadm -R	重载
    ipvsadm -S [-n] 	保存
置零计数器：
    ipvsadm -Z [-t|u|f service-address]

相关操作

删除集群服务命令
[root@DR ~]# ipvsadm -Ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
  -> RemoteAddress:Port           Forward Weight ActiveConn InActConn
TCP  192.168.175.250:80 wrr
  -> 192.168.175.150:80           Route   1      0          1         
  -> 192.168.175.151:80           Route   1      0          1         
[root@DR ~]# ipvsadm -D -t 192.168.175.250:80
[root@DR ~]# ipvsadm -Ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
  -> RemoteAddress:Port           Forward Weight ActiveConn InActConn
[root@DR ~]# 

添加集群服务
[root@DR ~]# ipvsadm -A -t 192.168.175.250:80 -s wrr
[root@DR ~]# ipvsadm -Ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
  -> RemoteAddress:Port           Forward Weight ActiveConn InActConn
TCP  192.168.175.250:80 wrr

添加RS
[root@DR ~]# ipvsadm -a -t 192.168.175.250:80 -r 192.168.175.150:80 -w 2 -g          （-w 2 表示验证时出现的次数）
[root@DR ~]# ipvsadm -a -t 192.168.175.250:80 -r 192.168.175.151:80 -w 2 -g        （-w 2 表示验证时出现的次数）
[root@DR ~]# ipvsadm -Ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
  -> RemoteAddress:Port           Forward Weight ActiveConn InActConn
TCP  192.168.175.250:80 wrr
  -> 192.168.175.150:80           Route   2      0          0         
  -> 192.168.175.151:80           Route   2      0          0         
删除集群服务里面的某一台RS
[root@DR ~]# ipvsadm -Ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
  -> RemoteAddress:Port           Forward Weight ActiveConn InActConn
TCP  192.168.175.250:80 wrr
  -> 192.168.175.150:80           Route   2      0          0         
  -> 192.168.175.151:80           Route   2      0          0         
[root@DR ~]# ipvsadm -d -t 192.168.175.250:80 -r 192.168.175.150:80 
[root@DR ~]# ipvsadm -Ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
  -> RemoteAddress:Port           Forward Weight ActiveConn InActConn
TCP  192.168.175.250:80 wrr
  -> 192.168.175.151:80           Route   2      0          0         

添加集群服务
[root@DR ~]# ipvsadm -a -t 192.168.175.250:80 -r 192.168.175.150:80 -w 4 -g     修改调度值为4
[root@DR ~]# ipvsadm -Ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
  -> RemoteAddress:Port           Forward Weight ActiveConn InActConn
TCP  192.168.175.250:80 wrr
  -> 192.168.175.150:80           Route   4      0          0         
  -> 192.168.175.151:80           Route   2      0          0         
本机验证
C:\Users\Administrator>curl http://192.168.175.250
rs1
C:\Users\Administrator>curl http://192.168.175.250
rs1
C:\Users\Administrator>curl http://192.168.175.250
rs1
C:\Users\Administrator>curl http://192.168.175.250
rs1
C:\Users\Administrator>curl http://192.168.175.250
rs2
修改调度值为0
[root@DR ~]# ipvsadm -e -t 192.168.175.250:80 -r 192.168.175.150:80 -w 0 -g
[root@DR ~]# ipvsadm -Ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
  -> RemoteAddress:Port           Forward Weight ActiveConn InActConn
TCP  192.168.175.250:80 wrr
  -> 192.168.175.150:80           Route   0      0          0         
  -> 192.168.175.151:80           Route   2      0          0         
本机验证此时访问只能访问RS2的，因为上面设置成了0
C:\Users\Administrator>curl http://192.168.175.250
rs2
C:\Users\Administrator>curl http://192.168.175.250
rs2

查看连接状况
本机验证DR查看
C:\Users\Administrator>curl http://192.168.175.250
rs1
C:\Users\Administrator>curl http://192.168.175.250
rs2
C:\Users\Administrator>curl http://192.168.175.250
rs1
C:\Users\Administrator>curl http://192.168.175.250
rs1
[root@DR ~]# watch 'ipvsadm -lnc'
Every 2.0s: ipvsadm -lnc                                       Fri Jul 24 11:10:28 2020

IPVS connection entries
pro expire state       source             virtual            destination
TCP 01:44  FIN_WAIT    192.168.175.1:52965 192.168.175.250:80 192.168.175.150:80
TCP 01:49  FIN_WAIT    192.168.175.1:55026 192.168.175.250:80 192.168.175.150:80
TCP 01:48  FIN_WAIT    192.168.175.1:55025 192.168.175.250:80 192.168.175.151:80
TCP 01:51  FIN_WAIT    192.168.175.1:55029 192.168.175.250:80 192.168.175.150:80

清空
[root@DR ~]# ipvsadm -ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
  -> RemoteAddress:Port           Forward Weight ActiveConn InActConn
TCP  192.168.175.250:80 wrr
  -> 192.168.175.150:80           Route   3      0          0         
  -> 192.168.175.151:80           Route   2      0          0      
[root@DR ~]# ipvsadm -C
[root@DR ~]# ipvsadm -ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
  -> RemoteAddress:Port           Forward Weight ActiveConn InActConn

保存到文件中
[root@DR ~]# ipvsadm -S > /etc/sysconfig/ipvsadm
[root@DR ~]# cat /etc/sysconfig/ipvsadm
-A -t DR:http -s wrr
-a -t DR:http -r 192.168.175.150:http -g -w 1
-a -t DR:http -r 192.168.175.151:http -g -w 1

添加端口号保存
[root@DR ~]# ipvsadm -Sn > /etc/sysconfig/ipvsadm
[root@DR ~]# cat /etc/sysconfig/ipvsadm
-A -t 192.168.175.250:80 -s wrr
-a -t 192.168.175.250:80 -r 192.168.175.150:80 -g -w 1
-a -t 192.168.175.250:80 -r 192.168.175.151:80 -g -w 1

HTTPS
DR配置

DR安装ipvsadm服务添加网卡
[root@DR ~]# yum -y install ipvsadm
[root@DR ~]# ip addr add 192.168.175.250/32 dev lo 
[root@DR ~]# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN qlen 1
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet 192.168.175.250/32 scope global lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: ens33: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 00:0c:29:1e:69:85 brd ff:ff:ff:ff:ff:ff
    inet 192.168.175.100/24 brd 192.168.175.255 scope global ens33
       valid_lft forever preferred_lft forever
    inet6 fe80::ea80:e71e:f5fc:9a2c/64 scope link 
       valid_lft forever preferred_lft forever
   查看是否能够ping通
[root@DR ~]#  ping 192.168.175.250
PING 192.168.175.250 (192.168.175.250) 56(84) bytes of data.
64 bytes from 192.168.175.250: icmp_seq=1 ttl=64 time=0.036 ms
64 bytes from 192.168.175.250: icmp_seq=2 ttl=64 time=0.057 ms
^C
--- 192.168.175.250 ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 999ms
rtt min/avg/max/mdev = 0.036/0.046/0.057/0.012 ms

RS1和2配置

修改网卡内核参数
RS1修改
[root@RS1 ~]# vim /etc/sysctl.conf
[root@RS1 ~]# sysctl -p
net.ipv4.conf.all.arp_ignore = 1
net.ipv4.conf.all.arp_announce = 2
RS2修改
[root@RS2 ~]# vim /etc/sysctl.conf
[root@RS2 ~]# sysctl -p
net.ipv4.conf.all.arp_ignore = 1
net.ipv4.conf.all.arp_announce = 2
RS1添加网卡
[root@RS1 ~]# ip addr add 192.168.175.250/32 dev lo
[root@RS1 ~]# ping 192.168.175.250
PING 192.168.175.250 (192.168.175.250) 56(84) bytes of data.
64 bytes from 192.168.175.250: icmp_seq=1 ttl=64 time=0.071 ms
64 bytes from 192.168.175.250: icmp_seq=2 ttl=64 time=0.048 ms
^C
--- 192.168.175.250 ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 999ms
rtt min/avg/max/mdev = 0.048/0.059/0.071/0.013 ms

配置路由信息
[root@DR ~]# route add -host 192.168.175.250 dev lo
[root@RS1 ~]# route add -host 192.168.175.250 dev lo
[root@RS2 ~]# route add -host 192.168.175.250 dev lo

在DR上面添加并保持规则
[root@DR ~]# route add -host 192.168.175.250 dev lo
[root@DR ~]#  ipvsadm -A -t 192.168.175.250:80 -s wrr
[root@DR ~]# ipvsadm -a -t 192.168.175.250:80 -r 192.168.175.150:80 -g
[root@DR ~]# ipvsadm -a -t 192.168.175.250:80 -r 192.168.175.151:80 -g
[root@DR ~]#  ipvsadm -S > /etc/sysconfig/ipvsadm
[root@DR ~]# ipvsadm -Ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
  -> RemoteAddress:Port           Forward Weight ActiveConn InActConn
TCP  192.168.175.250:80 wrr
  -> 192.168.175.150:80           Route   1      0          0         
  -> 192.168.175.151:80           Route   1      0          0     

安装服务
[root@RS1 ~]# yum -y install httpd

配置web页面
[root@RS1 html]# echo 'RS1' >index.html
[root@RS1 html]# cat index.html
RS1

RS2添加网卡
[root@RS2 ~]# ip addr add 192.168.175.250/32 dev lo
[root@RS2 ~]# ping 192.168.175.250
PING 192.168.175.250 (192.168.175.250) 56(84) bytes of data.
64 bytes from 192.168.175.250: icmp_seq=1 ttl=64 time=0.039 ms
64 bytes from 192.168.175.250: icmp_seq=2 ttl=64 time=0.043 ms
^C
--- 192.168.175.250 ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 999ms
rtt min/avg/max/mdev = 0.039/0.041/0.043/0.002 ms
安装服务
[root@RS2 ~]# yum -y install httpd

配置RS2web页面
[root@RS2 ~]# cd /var/www/html/
[root@RS2 html]# echo 'RS2' >index.html
[root@RS2 html]# cat index.html
RS2

生成秘钥
[root@DR ~]# openssl genrsa -out server.key 2048 
Generating RSA private key, 2048 bit long modulus
......................................+++
.+++
e is 65537 (0x10001)

生成证书请求
[root@DR ~]# openssl req -new -key server.key -out server.csr 
You are about to be asked to enter information that will be incorporated
into your certificate request.
What you are about to enter is what is called a Distinguished Name or a DN.
There are quite a few fields but you can leave some blank
For some fields there will be a default value,
If you enter '.', the field will be left blank.
-----
Country Name (2 letter code) [XX]:cn
State or Province Name (full name) []:hubei
Locality Name (eg, city) [Default City]:wuhan
Organization Name (eg, company) [Default Company Ltd]:yc
Organizational Unit Name (eg, section) []:
Common Name (eg, your name or your server's hostname) []:www.yc.com
Email Address []:

Please enter the following 'extra' attributes
to be sent with your certificate request
A challenge password []:
An optional company name []:

生成证书crt
[root@RS1 ~]# openssl x509 -req -days 365 -in server.csr -signkey server.key -out server.crt
Signature ok
subject=/C=cn/ST=hubei/L=wuhan/O=yc/CN=www.yc.com
Getting Private key

复制证书到指定位置
[root@RS1 ~]# cp server.crt /etc/pki/tls/certs/ 
[root@RS1 ~# cp server.key /etc/pki/tls/private/ 
[root@RS1 ~# cp server.csr /etc/pki/tls/private/

ssl.conf配置文件导入证书，默认https服务使用此配置文件 
[root@RS1 ca]# vim /etc/httpd/conf.d/ssl.conf 
SSLCertificateFile /etc/pki/tls/certs/server.crt 
SSLCertificateKeyFile /etc/pki/tls/private/server.key

编辑虚拟主机文件
[root@RS1 vhost1]# vim /etc/httpd/conf.d/httpd-vhosts.conf
Listen 8800 
<VirtualHost 192.168.175.100:8800> 
 ServerAdmin root@localhost  
 ServerName www.yc.com  
 ServerAlias www.yc1.com
 
 SSLEngine on  
 SSLCertificateFile "/etc/pki/tls/certs/server.crt"  
 SSLCertificateKeyFile "/etc/pki/tls/private/server.key"
 
 ErrorLog "/var/log/httpd/error_log"  
 CustomLog "/var/log/httpd/access_log" combined  
 DocumentRoot "/var/www/vhost1/"  
 <Directory "/var/www/vhost1/">       
 <RequireAll>      
 Require all granted      
 Require not ip 192.168.1.1       
 </RequireAll>  
 </Directory> 
 </VirtualHost>
<VirtualHost 192.168.175.100:8800>  
ServerAdmin root@localhost  
ServerName www.yc.com  
ServerAlias www.yc1.com  
SSLEngine on  
SSLCertificateFile "/etc/pki/tls/certs/server.crt" 
 SSLCertificateKeyFile "/etc/pki/tls/private/server.key"  
 ErrorLog "/var/log/httpd/error_log"  
 CustomLog "/var/log/httpd/access_log" combined  
 DocumentRoot "/var/www/vhost2/"  
 <Directory "/var/www/vhost2/">       
 <RequireAll>      
 Require all granted      
 Require not ip 192.168.1.1      
  </RequireAll>  
  </Directory> 
  </VirtualHost>

下载mod_ssl包
[root@RS1 ~]# yum -y install mod_ssl
[root@RS1 ~# vim /etc/httpd/conf.modules.d/00-base.conf LoadModule ssl_module modules/mod_ssl.so

[root@RS1 ~# openssl genrsa -out server.key 2048
[root@RS1 ~# openssl req -new -key server.key -out server.csr
[root@RS1 ~# openssl x509 -req -days 365 -in server.csr -signkey server.key out server.crt
[root@RS1 ~# cp server.crt /etc/pki/tls/certs/ 
[root@RS1 ~# cp server.key /etc/pki/tls/private/ 
[root@RS1 ~]# cp server.csr /etc/pki/tls/private/

[root@RS1 ca]# vim /etc/httpd/conf.d/ssl.conf 
SSLCertificateFile /etc/pki/tls/certs/server.crt 
SSLCertificateKeyFile /etc/pki/tls/private/server.key 

编辑
[root@RS1 vhost1]# vim /etc/httpd/conf.d/httpd-vhosts.conf
SSLEngine on 
SSLCertificateFile "/etc/pki/tls/certs/server.crt" 
SSLCertificateKeyFile "/etc/pki/tls/private/server.key"

查看连接状况
本机验证DR查看
C:\Users\Administrator>curl https://192.168.175.250
rs1
C:\Users\Administrator>curl https://192.168.175.250
rs2
C:\Users\Administrator>curl https://192.168.175.250
rs1
C:\Users\Administrator>curl https://192.168.175.250
rs1

session保持的方法有以下三种实现方式：
session绑定：
对某一特定服务：可以使用lvs的sh算法进行session绑定
对多个共享同一组RS的服务器，需要统一进行绑定时：可以使用lvs的persistence持久连接来实现
功能：无论ipvs使用何种调度方法，其都能实现将来自于同一个Client的请求始终定向至第一次调度时挑选出的RS
lvs的persistence通过一个持久连接模板来实现持久连接，这个持久连接模板独立于算法之外，模板样式如下：
sourceip rs timer
持久连接的实现方式：
每端口持久：PPC，单服务持久调度。无论使用何种算法，可以使用此方式实现sh算法的功能，且能定义持久时长
每FWM持久：PFWMC，单FWM持久调度。将多种不同的服务通过iptables打标成同一种标记，然后通过FWM进行统一持久调度
每客户端持久：PCC，单客户端持久调度
director上会将用户的任何请求都识别为集群服务，并向RS进行调度
tcp：1-65535
udp：1-65535
session复制
session服务器：memcached，redis（key-value，kv store） （经常用到）
lvs缺点：
不具备健康状态检查功能，只是一个调度器而已