一.部署前说明:
CIP:10.10.1.1/24
VIP:10.10.1.222/24
DIR: Eth1:10.10.1.100/24 Eth0:192.168.1.2/24
RS server:(这里假设后端只有两台RS server)
RIP1( Eth0:192.168.1.12/24 && Eth1:10.10.1.10/24(提供nginx服务)
RIP2( Eth0:192.168.1.13/24 && Eth1:10.10.1.11/24(提供nginx服务)
二.部署操作:
负载均衡器上配置操作
(1)在DIR上安装ipvsadm软件包以及相关依赖包:
# yum install openssl-devel popt-devel libnl-devel ipvsadm -y
[root@node~]# ipvsadm –help
-A 添加虚拟服务器
-t 设置群集地址(VIP,Virtual IP)
-s 指定负载调度算法
-a 添加真实服务器
-d 删除真实服务器
-r 指定真实服务器(Real Server)的地址
-m 使用NAT模式;-g、-i分别对应DR、TUN模式
-w 为节点服务器设置权重,默认为1
(2)在DIR上新建一个shell脚本文件,如下操作所示:
# vim /etc/init.d/lvs-tun
#!/bin/sh
# Startup script handle the initialisation of LVS
# chkconfig: - 28 72
# description: Initialise the Linux Virtual Server for TUN
#
### BEGIN INIT INFO
# Provides: ipvsadm
# Required-Start: $local_fs $network $named
# Required-Stop: $local_fs $remote_fs $network
# Short-Description: Initialise the Linux Virtual Server
# Description: The Linux Virtual Server is a highly scalable and highly
# available server built on a cluster of real servers, with the load
# balancer running on Linux.
# description: start LVS of TUN
LOCK=/var/lock/lvs-tun.lock
VIP=10.10.1.222
RIP1=192.168.1.12
RIP2=192.168.1.13
ETH=ens33
. /etc/rc.d/init.d/functions
start() {
PID=`ipvsadm -Ln | grep ${VIP} | wc -l`
if [ $PID -gt 0 ];
then
echo "The LVS-TUN Server is already running !"
else
#Load the tun mod
/sbin/modprobe tun
/sbin/modprobe ipip
#Set the tun Virtual IP Address
/sbin/ifconfig tunl0 $VIP broadcast $VIP netmask 255.255.255.255 up
/sbin/route add -host $VIP dev tunl0
#Clear IPVS Table
/sbin/ipvsadm -C
#The icmp recruit setting
echo "0" >/proc/sys/net/ipv4/ip_forward
echo "0" >/proc/sys/net/ipv4/conf/all/send_redirects
echo "0" >/proc/sys/net/ipv4/conf/default/send_redirects
echo "0" >/proc/sys/net/ipv4/conf/$ETH/send_redirects
echo "0" >/proc/sys/net/ipv4/conf/$ETH/send_redirects
#Set Lvs
/sbin/ipvsadm -At $VIP:80 -s rr
/sbin/ipvsadm -at $VIP:80 -r $RIP1:80 -i -w 1
/sbin/ipvsadm -at $VIP:80 -r $RIP2:80 -i -w 1
/bin/touch $LOCK
#Run Lvs
echo "starting LVS-TUN-DIR Server is ok !"
fi
}
stop() {
#stop Lvs server
/sbin/ipvsadm -C
/sbin/ifconfig tunl0 down >/dev/null
#Remove the tun mod
/sbin/modprobe -r tun
/sbin/modprobe -r ipip
rm -rf $LOCK
echo "stopping LVS-TUN-DIR server is ok !"
}
status() {
if [ -e $LOCK ];
then
echo "The LVS-TUN Server is already running !"
else
echo "The LVS-TUN Server is not running !"
fi
}
case "$1" in
start)
start
;;
stop)
stop
;;
restart)
stop
sleep 1
start
;;
status)
status
;;
*)
echo "Usage: $1 {start|stop|restart|status}"
exit 1
esac
exit 0
授权并启动该脚本
# chmod 777 /etc/init.d/lvs-tun
# /etc/init.d/lvs-tun start
注意: VIP也可以配置为serve物理网卡已配置的ip,比如上述Eth1的网卡ip,不过广播必须是自己!
RS server 上配置操作
(1)分别在每个RIP(RIP1,RIP2)上新建一个shell脚本文件,如下操作所示:
# vim /etc/init.d/lvs-tun-dr
#!/bin/sh
#
# Startup script handle the initialisation of LVS
# chkconfig: - 28 72
# description: Initialise the Linux Virtual Server for TUN
#
### BEGIN INIT INFO
# Provides: ipvsadm
# Required-Start: $local_fs $network $named
# Required-Stop: $local_fs $remote_fs $network
# Short-Description: Initialise the Linux Virtual Server
# Description: The Linux Virtual Server is a highly scalable and highly
# available server built on a cluster of real servers, with the load
# balancer running on Linux.
# description: start LVS of TUN-RIP
LOCK=/var/lock/ipvsadm.lock
VIP=10.10.1.222
ETH=eno16777736
. /etc/rc.d/init.d/functions
start() {
PID=`ifconfig | grep tunl0 | wc -l`
if [ $PID -ne 0 ];
then
echo "The LVS-TUN-RIP Server is already running !"
else
#Load the tun mod
/sbin/modprobe tun
/sbin/modprobe ipip
#Set the tun Virtual IP Address
/sbin/ifconfig tunl0 $VIP netmask 255.255.255.255 broadcast $VIP up
/sbin/route add -host $VIP dev tunl0
echo "1" >/proc/sys/net/ipv4/conf/tunl0/arp_ignore
echo "2" >/proc/sys/net/ipv4/conf/tunl0/arp_announce
echo "1" >/proc/sys/net/ipv4/conf/$ETH/arp_ignore
echo "2" >/proc/sys/net/ipv4/conf/$ETH/arp_announce
echo "1" >/proc/sys/net/ipv4/conf/all/arp_ignore
echo "2" >/proc/sys/net/ipv4/conf/all/arp_announce
echo "0" > /proc/sys/net/ipv4/conf/tunl0/rp_filter
echo "0" > /proc/sys/net/ipv4/conf/all/rp_filter
/bin/touch $LOCK
echo "starting LVS-TUN-RIP server is ok !"
fi
}
stop() {
/sbin/ifconfig tunl0 down
echo "0" >/proc/sys/net/ipv4/conf/tunl0/arp_ignore
echo "0" >/proc/sys/net/ipv4/conf/tunl0/arp_announce
echo "0" >/proc/sys/net/ipv4/conf/$ETH/arp_ignore
echo "0" >/proc/sys/net/ipv4/conf/$ETH/arp_announce
echo "0" >/proc/sys/net/ipv4/conf/all/arp_ignore
echo "0" >/proc/sys/net/ipv4/conf/all/arp_announce
#Remove the tun mod
/sbin/modprobe -r tun
/sbin/modprobe -r ipip
rm -rf $LOCK
echo "stopping LVS-TUN-RIP server is ok !"
}
status() {
if [ -e $LOCK ];
then
echo "The LVS-TUN-RIP Server is already running !"
else
echo "The LVS-TUN-RIP Server is not running !"
fi
}
case "$1" in
start)
start
;;
stop)
stop
;;
restart)
stop
start
;;
status)
status
;;
*)
echo "Usage: $1 {start|stop|restart|status}"
exit 1
esac
exit 0
注:在LVS/TUN模式中,关于arp原则:
(1)若DIR和RIP在不同lan网络中,比如不同的网段,不同的IDC机房,就不需要设置arp仰制,不同网段中,arp会被屏蔽掉,所以只需设置 ip tunnel即可;
(2)若DIR和RIP在同一广播域中,需要和LVS/DR模式一样在所有的RIP上仰制arp,防止arp响应导致arp表混乱,这样lvs就不能正常工作!
授权并启动该脚本
# chmod 777 /etc/init.d/lvs-tun-dr
#/etc/init.d/lvs-tun-dr start
ipvsadm -Ln -c
[root@node1 ~]# ipvsadm -Ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
TCP 10.10.1.222:80 rr
-> 192.168.1.12:80 Tunnel 1 0 0
-> 192.168.1.13:80 Tunnel 1 0 0
总 结 : LVS/TUN是所有模式中最最适用于跨网络跨地域地理位置的一种模式,需要注意的是:
(1)若DIR和RIP在不同lan网络中,比如不同的网段,不同的IDC机房,就不需要设置arp仰制,不同网段中,arp会被屏蔽掉,所以只需设置 ip tunne以及报文反向验证即可;
(2)若DIR和RIP在同一广播域中,需要和LVS/DR模式一样在所有的RIP上仰制arp,防止arp响应导致arp表混乱,这样lvs就不能正常工作!
配置时除了配置DIR,还需要需要配置后端RS server,即在tunl上口配置vip地址(需要系统支持tunl才行),广播为为自己,此模式下无需开启路由转发功能!