LVS三种模式与ldirectord、keepalive的使用

centos6 或rhel6系统

模拟设备

server1  调度  172.25.11.1

server2  server3 为httpd 服务器172.25.11.2 172.25.11.3

server4 为keepalive的backup         172.25.11.4

 

 

改完yum源
[rhel-source]
name=Red Hat
baseurl=http://172.25.11.250/rhel6.5/LoadBalancer/
enabled=1
gpgcheck=0
yum install ipvsadm -y
/etc/init.d/ipvsadm start

模式
DR NAT  TUN（隧道）  FULLNAT(需要内核编译)

DR模式

在server1中
ipvsadm -A  -t  172.25.11.100:80   -s rr   虚拟ip  rr是轮询
ipvsadm -a -t 172.25.11.100:80 -r 172.25.11.2:80 -g
ipvsadm -a -t 172.25.11.100:80 -r 172.25.11.3:80 -g
ipvsadm -C  清空
保存 rule
service ipvsadm save

-g代表gateway，意思为直接路由，还可以有-i(internet)隧道模型，-m(masq)伪装，NAT模式

ipvsadm -ln  查看ip
ipvsadm -l 查看主机名
ipvsadm --help 自己查看帮助
ip addr add 172.25.11.100/24 dev eth0

server1本身的子网掩码 要和自身一致

server2 3也要加IP
server2   
ip addr add 172.25.11.100/32 dev eth0
server3
ip addr add 172.25.11.100/32 dev eth0

然后在真机中
curl 172.25.11.100

server1  curl 172.25.11.100是不通的
client -> vs ->rs -> client  
    虚拟系统  真实系统
ipvsadm -C 清空
arp  -an|grep 100
查看100的IP  自己设置的

[root@foundation11 html]# arp -an |grep 100
? (172.25.11.100) at 52:54:00:f0:91:43 [ether] on br0

清缓存 arp -d 172.25.11.100
arp 将地址转换为MAC

隐藏server2 3的MAC 地址  访问之后只显示server1调度服务器的MAC 地址
在server2 3添加arptables
yum install arptables_jf -y
arptables -A IN -d 172.25.11.100 -j DROP
arptables -A OUT -s 172.25.11.100 -j mangle --mangle-ip-s 172.25.11.2
arptables -L
/etc/init.d/arptables_jf  save
清除真机缓存arp -d 172.25.11.100
再次curl 172.25.11.100就可以实现
arp -an |grep 100
只显示server1的MAC地址了

NAT模式  
LVS-NAT模型的实现方式和iptables的DNAT相似，所以你懂的，Director节点不能和iptables同时使用，那么会有冲突，这就是我们后面为什么要将iptables的规则清空的目的之一。

server1
ipvsadm -C
ipvsadm -A  -t  172.25.11.100:80   -s rr   虚拟ip  rr是轮询
ipvsadm -a -t 172.25.11.100:80 -r 172.25.11.2:80 -i
ipvsadm -a -t 172.25.11.100:80 -r 172.25.11.3:80 -i
service ipvsadm save
添加路由模式
vim /etc/sysctl.conf
net.ipv4.ip_forward = 1
sysctl -p
添加nat模块
modprobe iptable_nat

 

 

 

 

 

 

 

TUN模式隧道模式
server1
ip addr del 172.25.11.100/24 dev eth0
ipvsadm -C
modprobe ipip
ip addr add 172.25.11.100/24 dev tunl0
ip link set up tunl0
再添加策略
ipvsadm -A  -t  172.25.11.100:80   -s rr   虚拟ip  rr是轮询
ipvsadm -a -t 172.25.11.100:80 -r 172.25.11.2:80 -i
ipvsadm -a -t 172.25.11.100:80 -r 172.25.11.3:80 -i
ipvsadm -L

server2 3也要
modprobe ipip
ip addr del 172.25.11.100/32 dev eth0
ip addr add 172.25.11.100/32 dev tunl0
ip link set up tunl0
ip addr show
sysctl -a |grep rp_filter
sysctl -w XXX=0  XXX表示上面显示为1的
sysctl -p  使他生效
vim /etc/sysctl.conf   将显示仍为1的选项改为0
sysctl -p
sysctl -a |grep rp_filter
真机
[root@foundation11 html]# arp -a |grep 100
? (172.25.11.100) at 52:54:00:f0:91:43 [ether] on br0

 

 

安全检查
采用DR模式
如果server3 http stop
就不会报错，而且server1打开http可以写index.html  正在维护中

server1
ipvsadm -C
modprobe -r ipip
server2 3
modprobe -r ipip

server1
ipvsadm -A  -t  172.25.11.100:80   -s rr   虚拟ip  rr是轮询
ipvsadm -a -t 172.25.11.100:80 -r 172.25.11.2:80 -g
ipvsadm -a -t 172.25.11.100:80 -r 172.25.11.3:80 -g
ip addr add 172.25.11.100/24 dev eth0

高可用的包
[rhel-source]
name=Red Hat
baseurl=http://172.25.11.250/rhel6.5/HighAvailability
enabled=1
gpgcheck=0

yum install ldirectord-3.9.5-3.1.x86_64.rpm -y
rpm -qpl  ldirectord-3.9.5-3.1.x86_64.rpm
 /usr/share/doc/ldirectord-3.9.5/ldirectord.cf
cp /usr/share/doc/ldirectord-3.9.5/ldirectord.cf .

vim   /etc/ha.d/ldirectord.cf
virtual=172.25.11.100:80
        real=172.25.11.3:80 gate  #server3
        real=172.25.11.2:80 gate  #server2
        fallback=127.0.0.1:80 gate
        service=http
        scheduler=rr
        #persistent=600
        #netmask=255.255.255.255
        protocol=tcp
        checktype=negotiate
        checkport=80
        request="index.html"
        #receive="Test Page"
        #virtualhost=www.x.y.z
/etc/init.d/ldirectord start

 

server2 3
ip addr add 172.25.11.100/32 dev eth0

 

keepalive的使用

DR模式要先设置好
创建第四个server4  172.25.11.4
停掉ldirectord
server1中
./configure --prefix=/usr/local/keepalived --with-init=SYSV
yum install openssl-devel -y
make && make install

cd /usr/local/keepalived/etc/rc.d/init.d/
chmod +x keepalive
ln -s /usr/local/keepalived/etc/rc.d/init.d/keepalived  /etc/init.d/
ln -s /usr/local/keepalived/etc/sysconfig/keepalived  /etc/sysconfig/
ln -s /usr/local/keepalived/etc/keepalived  /etc/
ln -s /usr/local/keepalived/sbin/keepalived  /sbin/
/etc/init.d/ldirectord stop
chkconfig ldirectord off  #开机不启动
ipaddr del 172.25.11.100/24  dev eth0
cd /etc/keepalived
vim keepalived.conf

global_defs {
   notification_email {
     root@localhost
   }
   notification_email_from Alexandre.Cassen@firewall.loc
   smtp_server 127.0.0.1
   smtp_connect_timeout 30
   router_id LVS_DEVEL
   vrrp_skip_check_adv_addr
   #vrrp_strict  不注释会出错
   vrrp_garp_interval 0
   vrrp_gna_interval 0
}

vrrp_instance VI_1 {
    state MASTER
    interface eth0
    virtual_router_id 11
    priority 100
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass 1111
    }
    virtual_ipaddress {
        172.25.11.100
    }
}
virtual_server 172.25.11.100 80 {
    delay_loop 6
    lb_algo rr
    lb_kind DR   #!!!!!
    #persistence_timeout 50
    protocol TCP

    real_server  172.25.11.2 80 {
        TCP_CHECK {
        weight 1
            connect_timeout 3
            retry 3
            delay_before_retry 3
        }
        }
    real_server  172.25.11.3 80 {
        TCP_CHECK {
        weight 1
            connect_timeout 3
            retry 3
            delay_before_retry 3
        }
        }
}

 

server4   
cd /usr/local/keepalived/etc/rc.d/init.d/
chmod +x keepalived
ln -s /usr/local/keepalived/etc/rc.d/init.d/keepalived  /etc/init.d/
ln -s /usr/local/keepalived/etc/sysconfig/keepalived  /etc/sysconfig/
ln -s /usr/local/keepalived/etc/keepalived  /etc/
ln -s /usr/local/keepalived/sbin/keepalived  /sbin/
cd /etc/keepalived
vim keepalived.conf

bal_defs {
   notification_email {
     root@localhost
   }
   notification_email_from Alexandre.Cassen@firewall.loc
   smtp_server 127.0.0.1
   smtp_connect_timeout 30
   router_id LVS_DEVEL
   vrrp_skip_check_adv_addr
   #vrrp_strict  不注释会出错
   vrrp_garp_interval 0
   vrrp_gna_interval 0
}

vrrp_instance VI_1 {
    state BACKUP  #主要修改了这里 别的和server1一样
    interface eth0
    virtual_router_id 11
    priority 100
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass 1111
    }
    virtual_ipaddress {
        172.25.11.100
    }
}

 

virtual_server 172.25.11.100 80 {
    delay_loop 6
    lb_algo rr
    lb_kind DR #!!!!
    #persistence_timeout 50
    protocol TCP

    real_server  172.25.11.2 80 {
        TCP_CHECK {
        weight 1
            connect_timeout 3
            retry 3
            delay_before_retry 3
        }
        }
    real_server  172.25.11.3 80 {
        TCP_CHECK {
        weight 1
            connect_timeout 3
            retry 3
            delay_before_retry 3
        }
        }
}

随便停掉一个server2 server3中的httpd
真机curl 172.25.11.100
等半分钟就不会报错了