Lvs-DR模式负载均衡

过程

client ->VS->RS->client(VS只做调度,RS为服务器)

DR模式:直接路由,VS改写请求报文的MAC地址,由调度算法分配真实的服务器,MAC地址改为真实服务器的MAC地址

      服务器将响应消息直接发送给客户端.

原理详情

1、通过在调度器VS上修改数据包的目的MAC地址实现转发,CIP不变,目的地址仍然是VIP地址;（cip(cm，m代表mac地址）->vip(vm->RM,在2层链路层改变了mac地址来定位)）
2、请求的报文经过调度器,而服务器响应处理后的报文无需经过调度器,因此并发访问量大时使用效率很高(和NAT模式比)
3、因为DR模式是通过MAC地址改写机制实现转发,因此所有RS节点和调度器只能在一个局域网里面
4、服务器主机需要绑定VIP地址在LO接口上,且需要配置ARP抑制。
5、服务器节点的默认网关不需要配置成LB,而是直接配置为上级路由的网关,能让服务器直接出网就可以。
6、因为DR模式的调度器仅做MAC地址的改写,所以调度器就不能改写目标端口,那么服务器就得使用和VIP相同的端口提供服务。

 

实验环境

iptables和selinux关闭
redhat6.5
VS：server1 172.25.254.1
RS：server2 172.25.254.2
RS：server3 172.25.254.3
vip：172.25.254.100

VS:

[root@server1 ~]# yum install -y ipvsadm
[root@server1 ~]# iptables -L   //查看iptables是否关闭
Chain INPUT (policy ACCEPT)
target     prot opt source               destination         

Chain FORWARD (policy ACCEPT)
target     prot opt source               destination         

Chain OUTPUT (policy ACCEPT)
target     prot opt source               destination         
[root@server1 ~]# ipvsadm -L
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
  -> RemoteAddress:Port           Forward Weight ActiveConn InActConn
[root@server1 ~]# ipvsadm -A -t 172.25.254.100:80 -s rr  
//-s调度算法，默认为wlc加权调度算法
[root@server1 ~]# ipvsadm -a -t 172.25.254.100:80 -r 172.25.254.2:80 -g
//-a表示在添加虚拟服务中添加,-g表示使用直连模式
[root@server1 ~]# ipvsadm -a -t 172.25.254.100:80 -r 172.25.254.3:80 -g
[root@server1 ~]# ipvsadm -ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
  -> RemoteAddress:Port           Forward Weight ActiveConn InActConn
TCP  172.25.254.100:80 rr
  -> 172.25.254.2:80               Route   1      0          0         
  -> 172.25.254.380               Route   1      0          0    
[root@server1 ~]# ip addr add 172.25.254.100/24 dev eth0  //添加虚拟ip
[root@server1 ~]# ip addr
link/ether 52:54:00:9f:e0:90     
inet 172.25.254.1/24
inet 172.25.254.100/24

RS：

注意： RS上必须有vip才能建立连接

Server2:
[root@server2 ~]# /etc/init.d/httpd start
[root@server2 html]# cat index.html
www.westos.org-Server2
[root@server2 ~]# ip addr add 172.25.254.100/24 dev eth0
//添加虚拟ip，这里是临时添加
[root@server2 ~]# ip addr
link/ether 52:54:00:74:05:fc
inet 172.25.254.2/24 brd
inet 172.25.254.100/24   

Server3:
[root@server3 ~]# /etc/init.d/httpd start
[root@server3 html]# cat index.html
bbs.westos.org
[root@server3 ~]# ip addr add 172.25.254.100/24 dev eth0
[root@server3 ~]# ip addr
link/ether 52:54:00:10:6f:56
inet 172.25.254.3/24
inet 172.25.254.100/24

客户端测试：

发现连接到的ip（VS和RS的ip都一样）是随机的，因为三台server在同一VLAN下具有相同的vip，故不能保证每次都会访问调度器server1

为了解决这个问题，需要设置禁止访问连接RS

RS：

server2：
[root@server2 ~]# yum install arptables_jf -y
[root@server2 ~]# arptables -A IN -d 172.25.253.100 -j DROP  
//-A控制arp协议,IN添加策略,mangle转换
[root@server2 ~]# arptables -A OUT -s 172.25.254.100 -j mangle --mangle-ip-s 172.25.254.2
[root@server2 ~]# /etc/init.d/arptables_jf save
[root@server2 ~]# cat /etc/sysconfig/arptables  
//此文件存有arptables的记录，关掉之后，重启依旧存在

server3：
[root@server3 ~]# yum install arptables_jf -y
[root@server3 ~]# arptables -A IN -d 172.25.254.100 -j DROP
[root@server3 ~]# arptables -A OUT -s 172.25.254.100 -j mangle --mangle-ip-s 172.25.254.3
[root@server3 ~]# /etc/init.d/arptables_jf save

Vs对后端没有健康检查

方式一：用ldirectord解决此问题

VS：

[root@server1 ~]# yum install ldirectord-3.9.5-3.1.x86_64.rpm -y
[root@server1 ~]# rpm -ql ldirectord   //查看配置文件
/usr/share/doc/ldirectord-3.9.5/ldirectord.cf
[root@server1 ~]# cp /usr/share/doc/ldirectord-3.9.5/ldirectord.cf /etc/ha.d/
[root@server1 ~]# cd /etc/ha.d
[root@server1 ha.d]# ls
ldirectord.cf  resource.d  shellfuncs
[root@server1 ha.d]# vim ldirectord.cf  
virtual=172.25.254.100:80
    real=172.25.254.2:80 gate
    real=172.25.254.3:80 gate
    fallback=127.0.0.1:80 gate
    service=http
    protoccol=tcp
    checkport=80
    request="index.html"

[root@server1 ha.d]# ipvsadm -C  //清理规则
[root@server1 ha.d]# ipvsadm -l  //查看是否清除
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
  -> RemoteAddress:Port           Forward Weight ActiveConn InActConn
[root@server1 ha.d]# /etc/init.d/ldirectord start //再次打开服务又可以加载出规则
[root@server1 ha.d]# ipvsadm -ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
  -> RemoteAddress:Port           Forward Weight ActiveConn InActConn
TCP  172.25.254.100:80 rr
  -> 172.25.254.2:80               Route   1      0          0         
  -> 172.25.254.3:80               Route   1      0          0    
[root@server1 ha.d]# cd /var/www/html
[root@server1 html]# vim index.html
系统维护中......
//在客户端curl 172.25.254.2测试时，RS轮询，当关闭server2时，只访问server3，RS都关闭时会访问本地，而显示“系统维护中......”

方式二：用keepalived软件解决

再建立一个子盘server4：172.25.254.4

server1:
[root@server1 ~]# tar zxf keepalived-2.0.6.tar.gz
[root@server1 ~]# cd keepalived-2.0.6
[root@server1 keepalived-2.0.6]# ./configure --prefix=/usr/local/keepalived --with-init=SYSV
[root@server1 keepalived-2.0.6]# yum install openssl-devel
[root@server1 keepalived-2.0.6]# ./configure --prefix=/usr/local/keepalived --with-init=SYSV
[root@server1 keepalived-2.0.6]# make  //编译
[root@server1 keepalived-2.0.6]# make install
[root@server1 keepalived-2.0.6]# ln -s /usr/local/keepalived/etc/rc.d/init.d/keepalived /etc/init.d/
[root@server1 keepalived-2.0.6]# ln -s /usr/local/keepalived/etc/keepalived/ /etc/
[root@server1 keepalived-2.0.6]# ln -s /usr/local/keepalived/etc/sysconfig/keepalived /etc/sysconfig/
[root@server1 keepalived-2.0.6]# ln -s /usr/local/keepalived/sbin/keepalived /sbin/
[root@server1 keepalived-2.0.6]# cd /usr/local/
[root@server1 local]# scp -r keepalived/ server4:/usr/local/  将keepalived传给server4
[root@server1 local]# cd /etc/init.d/
[root@server1 init.d]# chmod +x keepalived
[root@server1 init.d]# /etc/init.d/keepalived start  //开启服务

server4:
[root@server4 ~]# ln -s /usr/local/keepalived/etc/rc.d/init.d/keepalived /etc/init.d/
[root@server4 ~]# ln -s /usr/local/keepalived/etc/keepalived/ /etc/
[root@server4 ~]# ln -s /usr/local/keepalived/etc/sysconfig/keepalived /etc/sysconfig/
[root@server4 ~]# ln -s /usr/local/keepalived/sbin/keepalived /sbin/
[root@server4 ~]# chmod +x /usr/local/keepalived/etc/init.d/keepalived
[root@server4 local]# /etc/init.d/keepalived start

server1:
[root@server1 ~]# cd /etc/keepalived/
[root@server1 keepalived]# yum install mailx -y
[root@server1 keepalived]# vim keepalived.conf //内容如下
//Virtual_router_id虚拟路由id，Delay_loop后端的健康检查，Persistence_timeout持续连接，一直保持协议磋商
global_defs {
   notification_email {
        root@localhost
        }
   notification_email_from Alexandre.Cassen@firewall.loc
   smtp_server 127.0.0.1
   smtp_connect_timeout 30
   router_id LVS_DEVEL
   vrrp_skip_check_adv_addr
   #vrrp_strict             #注释以放其修改防火墙规则
   vrrp_garp_interval 0
   vrrp_gna_interval 0

vrrp_instance VI_1 {
    state MASTER        //server4上为BACKUP          
    interface eth0
    virtual_router_id 35
    priority 100        #数值越大，优先级越高 server4上为50
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass 1111
    }
    virtual_ipaddress {
        172.25.254.100
        }
}

virtual_server 172.25.254.100 80 {       #VS的vip，服务启动生效时自动添加
    delay_loop 3            #对后端的健康检查时间
    lb_algo rr              #调度算法
    lb_kind DR              #模式为DR
    #persistence_timeout 50     #注释持续连接
    protocol TCP

real_server 172.25.254.2 80{     #RS
        weight 1
        TCP_CHECK{
            connect_timeout 3
            retry 3
            delay_before_retry 3
        }
    }
    real_server 172.25.25.3 80{
        weight 1
        TCP_CHECK{
            connect_timeout 3
            retry 3
            delay_before_retry 3
       }
    }
}
[root@server1 keepalived]# scp keepalived.conf server4:/etc/keepalived/
[root@server1 keepalived]# ip addr del 172.25.254.100/24 dev eth0
[root@server1 keepalived]# /etc/init.d/keepalived restart

高可用

[root@server1 keepalived]# /etc/init.d/keepalived stop
//关掉server1(也可使用echo c > /proc/sysrq-trigger破坏server1的内核使其不能使用)，客户端查看内容不会变化，因为server4接替了server1
[root@server4 keepalived]# cat /var/log/messages  //查看server4的日志

[root@server3 ~]# /etc/init.d/httpd stop  
//将两个RS都关掉，则不能正常访问，与ldirectord不同的是本地不会接替让VS访问