用vmware实现LVS

LVS实验之NAT

在这里插入代码片
@查看版本信息
[root@localhost ~]# yum info ipvsadm
Loaded plugins: fastestmirror
Repodata is over 2 weeks old. Install yum-cron? Or run: yum makecache fast
Loading mirror speeds from cached hostfile
Installed Packages
Name        : ipvsadm
Arch        : x86_64
Version     : 1.27
Release     : 8.el7
Size        : 75 k
Repo        : installed
From repo   : cdrom
Summary     : Utility to administer the Linux Virtual Server
URL         : https://kernel.org/pub/linux/utils/kernel/ipvsadm/
License     : GPLv2+
Description : ipvsadm is used to setup, maintain, and inspect the virtual server
            : table in the Linux kernel. The Linux Virtual Server can be used to
            : build scalable network services based on a cluster of two or more
            : nodes. The active node of the cluster redirects service requests to a
            : collection of server hosts that will actually perform the
            : services. Supported Features include:
            :   - two transport layer (layer-4) protocols (TCP and UDP)
            :   - three packet-forwarding methods (NAT, tunneling, and direct routing)
            :   - eight load balancing algorithms (round robin, weighted round robin,
            :     least-connection, weighted least-connection, locality-based
            :     least-connection, locality-based least-connection with
            :     replication, destination-hashing, and source-hashing)


@查看文件内容
[root@localhost ~]# rpm -ql ipvsadm
/etc/sysconfig/ipvsadm-config
/usr/lib/systemd/system/ipvsadm.service
/usr/sbin/ipvsadm
/usr/sbin/ipvsadm-restore（加载规则）
/usr/sbin/ipvsadm-save(保存规则)
/usr/share/doc/ipvsadm-1.27
/usr/share/doc/ipvsadm-1.27/README
/usr/share/man/man8/ipvsadm-restore.8.gz
/usr/share/man/man8/ipvsadm-save.8.gz
/usr/share/man/man8/ipvsadm.8.gz

@ipvsadm使用
[root@localhost ~]# ipvsadm --help
ipvsadm v1.27 2008/5/15 (compiled with popt and IPVS v1.2.1)
Usage:
  ipvsadm -A|E -t|u|f service-address [-s scheduler] [-p [timeout]] [-M netmask] [--pe persistence_engine] [-b sched-flags]
  ipvsadm -D -t|u|f service-address
  ipvsadm -C
  ipvsadm -R
  ipvsadm -S [-n]
  ipvsadm -a|e -t|u|f service-address -r server-address [options]
  ipvsadm -d -t|u|f service-address -r server-address
  ipvsadm -L|l [options]
  ipvsadm -Z [-t|u|f service-address]
  ipvsadm --set tcp tcpfin udp
  ipvsadm --start-daemon state [--mcast-interface interface] [--syncid sid]
  ipvsadm --stop-daemon state
  ipvsadm -h

Commands:
Either long or short options are allowed.
  --add-service     -A        add virtual service with options
  --edit-service    -E        edit virtual service with options
  --delete-service  -D        delete virtual service
  --clear           -C        clear the whole table
  --restore         -R        restore rules from stdin
  --save            -S        save rules to stdout
  --add-server      -a        add real server with options
  --edit-server     -e        edit real server with options
  --delete-server   -d        delete real server
  --list            -L|-l     list the table
  --zero            -Z        zero counters in a service or all services
  --set tcp tcpfin udp        set connection timeout values
  --start-daemon              start connection sync daemon
  --stop-daemon               stop connection sync daemon
  --help            -h        display this help message

Options:
  --tcp-service  -t service-address   service-address is host[:port]
  --udp-service  -u service-address   service-address is host[:port]
  --fwmark-service  -f fwmark         fwmark is an integer greater than zero
  --ipv6         -6                   fwmark entry uses IPv6
  --scheduler    -s scheduler         one of rr|wrr|lc|wlc|lblc|lblcr|dh|sh|sed|nq,
                                      the default scheduler is wlc.
  --pe            engine              alternate persistence engine may be sip,
                                      not set by default.
  --persistent   -p [timeout]         persistent service
  --netmask      -M netmask           persistent granularity mask
  --real-server  -r server-address    server-address is host (and port)
  --gatewaying   -g                   gatewaying (direct routing) (default)
  --ipip         -i                   ipip encapsulation (tunneling)
  --masquerading -m                   masquerading (NAT)
  --weight       -w weight            capacity of real server
  --u-threshold  -x uthreshold        upper threshold of connections
  --l-threshold  -y lthreshold        lower threshold of connections
  --mcast-interface interface         multicast interface for connection sync
  --syncid sid                        syncid for connection sync (default=255)
  --connection   -c                   output of current IPVS connections
  --timeout                           output of timeout (tcp tcpfin udp)
  --daemon                            output of daemon information
  --stats                             output of statistics information
  --rate                              output of rate information
  --exact                             expand numbers (display exact values)
  --thresholds                        output of thresholds information
  --persistent-conn                   output of persistent connection info
  --nosort                            disable sorting output of service/server entries
  --sort                              does nothing, for backwards compatibility
  --ops          -o                   one-packet scheduling
  --numeric      -n                   numeric output of addresses and ports
  --sched-flags  -b flags             scheduler flags (comma-separated)

----------------------------------------------------------------------------------

@添加集群
-A添加集群VIP:port，-t表示tcp，-s选择调度算法（默认算法为wlc）
[root@lvs-server ~]# ipvsadm -A -t 192.168.240.100:80 -s rr
@添加后端服务器
-a添加后端服务器VIP:port，-t表示tcp，-r表示后端服务器真实IP，-m为nat模式
[root@lvs-server ~]# ipvsadm -a -t 192.168.240.100:80 -r 192.168.241.201:80 -m
[root@lvs-server ~]# ipvsadm -a -t 192.168.240.100:80 -r 192.168.241.202:80 -m
r 192.168.242.102 -m
@查看策略
[root@lvs-server ~]# ipvsadm -Ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
  -> RemoteAddress:Port           Forward Weight ActiveConn InActConn
TCP  192.168.240.100:80 rr
  -> 192.168.241.201:80           Masq    1      0          0         
  -> 192.168.241.202:80           Masq    1      0          0       

@将策略保存到配置文件
[root@lvs-server ~]# ipvsadm-save -n > ipvsadm.rule 

[root@lvs-server ~]# cat ipvsadm.rule 
-A -t 192.168.240.100:80 -s rr
-a -t 192.168.240.100:80 -r 192.168.241.201:80 -m -w 1
-a -t 192.168.240.100:80 -r 192.168.241.202:80 -m -w 1

@清空策略利用配置文件恢复
[root@lvs-server ~]# ipvsadm -C
[root@lvs-server ~]# ipvsadm -Ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
  -> RemoteAddress:Port           Forward Weight ActiveConn InActConn

[root@lvs-server ~]# ipvsadm-restore < ipvsadm.rule

[root@lvs-server ~]# ipvsadm -Ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
  -> RemoteAddress:Port           Forward Weight ActiveConn InActConn
TCP  192.168.240.100:80 rr
  -> 192.168.241.201:80           Masq    1      0          0         
  -> 192.168.241.202:80           Masq    1      0          0        

@LVS开启forward
[root@lvs-server ~]# vim /etc/sysctl.conf
添加 net.ipv4.ip_forward=1
[root@lvs-server ~]# sysctl net.ipv4.ip_forward
net.ipv4.ip_forward = 0
[root@lvs-server ~]# sysctl -p
net.ipv4.ip_forward = 1

@修改调度方法为wrr
[root@lvs-server ~]# ipvsadm -E -t 192.168.240.100:80 -s wrr
[root@lvs-server ~]# ipvsadm -e -t 192.168.240.100:80 -r 192.168.241.101:80 -w 3 -m
[root@lvs-server ~]# ipvsadm -Ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
  -> RemoteAddress:Port           Forward Weight ActiveConn InActConn
TCP  192.168.240.100:80 wrr
  -> 192.168.241.101:80           Masq    3      0          0         
  -> 192.168.241.102:80           Masq    1      0          0       
在这里插入代码片

细节1：因为后端收到报文后转发经过LVS所以要写网关，网关为LVS的DIP。
细节2：因为转发请求报文都经过LVS，所以LVS需要开启forward。
细节3：在后端服务器nginx.access中可以看到客户端访问的真实地址。

LVS实验之DR

@配置VIP
[root@web01 ~]# ip addr add 192.168.240.203/32 dev lo label lo:1
[root@web02 ~]# ip addr add 192.168.240.203/32 dev lo label lo:1
[root@lvs-server ~]# ip addr add 192.168.240.203/32 dev ens33 label ens33:1

@修改后端服务器的内核参数arp_ignore跟arp_announce（lvs的内核参数不用修改）
限制响应级别：arp_ignore
0:默认值，表示可使用本地任意接口上配置的任意地址进行相应
1:仅在请求的目标IP配置在本地的主机的接收到请求报文的接口上时，才给予响应
限制通告级别：arp_announce
0:默认值，把本机所有接口的所有信息向每个接口的网络进行通告
1:尽量避免将接口信息向非直接连接网络进行通告
2:必须避免将接口信息向非本网络进行通告
[root@web01 ~]# echo 1 > /proc/sys/net/ipv4/conf/all/arp_ignore 
[root@web01 ~]# echo 2 > /proc/sys/net/ipv4/conf/all/arp_announce 
[root@web01 ~]# echo 1 > /proc/sys/net/ipv4/conf/lo/arp_ignore 
[root@web01 ~]# echo 2 > /proc/sys/net/ipv4/conf/lo/arp_announce

[root@web02 ~]# echo 1 > /proc/sys/net/ipv4/conf/all/arp_ignore
[root@web02 ~]# echo 2 > /proc/sys/net/ipv4/conf/all/arp_announce
[root@web02 ~]# echo 1 > /proc/sys/net/ipv4/conf/lo/arp_ignore
[root@web02 ~]# echo 2 > /proc/sys/net/ipv4/conf/lo/arp_announce


@ipvsadm配置
[root@lvs-server ~]# ipvsadm -A -t 192.168.240.203:80 -s rr
[root@lvs-server ~]# ipvsadm -a -t 192.168.240.203:80 -r 192.168.240.201 -g
[root@lvs-server ~]# ipvsadm -a -t 192.168.240.203:80 -r 192.168.240.202 -g

在这里插入代码片

细节1：在后端nginx中access日志中看到的也是客户端的真实IP
细节2：实验中后端服务器与LVS都为一个段不用配置网关

可能有人会问nginx怎么安装，下面是教学。

@配置镜像源
curl -o /etc/yum.repos.d/CentOS-Base.repo https://mirrors.aliyun.com/repo/Centos-7.repo
yum clean all && yum makecache

@安装gcc
yum -y install gcc

@pcre是一个perl库，包括perl兼容的正则表达式库，nginx的http模块使用pcre来解析正则表达式，所以需要安装pcre库。
yum install -y pcre pcre-devel

@zlib库提供了很多种压缩和解压缩方式nginx使用zlib对http包的内容进行gzip
yum install -y zlib zlib-devel

@下载nginx包(版本下最新的就行)
wget http://nginx.org/download/nginx-1.9.9.tar.gz

@解压
tar -zxvf nginx-1.9.9.tar.gz

@编译安装
cd nginx-1.9.9
./configure
make && make insatall

@启动
cd /usr/local/nginx/sbin
./nginx
在这里插入代码片