CC00012.LBCHAC——|HAC构建.V3|——|5台server|-CSDN博客

一、LVS-DR+keepalived模式：10.10.10.11部署Keepalived服务(LVS主master)

### --- 10.10.10.11部署Keepalived服务(LVS主master)
### --- 将Keepalived.iso文件上传到10.10.10.11服务器的root目录下

### --- 创建一个/mnt/iso1目录用于挂载Keepalived.iso文件
[root@server11 ~]# mkdir  /mnt/iso1
[root@server11 ~]# mount -o loop Keepalived.iso /mnt/iso1/

~~~     将所有数据拷贝到当前目录
[root@server11 ~]# cp -a /mnt/iso1/* .                                      // 将所有数据拷贝到当前目录
[root@server11 ~]# ls   
keepalived-1.2.2.tar.gz  rkhunter-1.3.8.tar.gz                              // 得到Keepalived的源码包

### --- 需要源码编译安装；安装 gcc环境
[root@server11 ~]# yum install -y gcc gcc-c++
 
~~~     源码编译安装Keepalived
[root@server11 ~]# tar -zxvf keepalived-1.2.2.tar.gz
[root@server11 ~]# cd keepalived-1.2.2 
 
~~~     安装相关依赖
[root@server11 keepalived-1.2.2]# yum install -y kernel-devel openssl-devel popt-devel gcc*

~~~     生成Makefile文件

[root@server11 keepalived-1.2.2]# ./configure --prefix=/ --with-kernel-dir=/usr/src/kernels/2.6.32-642.el6.x86_64/
Keepalived configuration
------------------------
Keepalived version       : 1.2.2
Compiler                 : gcc
Compiler flags           : -g -O2
Extra Lib                : -lpopt -lssl -lcrypto 
Use IPVS Framework       : Yes
IPVS sync daemon support : Yes
IPVS use libnl           : No
Use VRRP Framework       : Yes
Use Debug flags          : No
[root@server11 keepalived-1.2.2]#  make && make install

~~~     把启动脚本做软连接，设置为开机自启
[root@server11 keepalived-1.2.2]# ll /etc/init.d/keepalived 
-rwxr-xr-x 1 root root 1288 Jan 21 00:35 /etc/init.d/keepalived

~~~     添加Keepalived自启规则    
[root@server11 keepalived-1.2.2]# chkconfig --add keepalived                // 添加Keepalived自启规则
[root@server11 keepalived-1.2.2]# chkconfig keepalived on                   // 设置开机自启

### --- 修改Keepalived配置文件

[root@server11 keepalived-1.2.2]# vim /etc/keepalived/keepalived.conf 
global_defs {
#   notification_email {                                                    // 删除
#   acassen@firewall.loc
#   failover@firewall.loc
#   sysadmin@firewall.loc
#   }
#   notification_email_from Alexandre.Cassen@firewall.loc
#   smtp_server 192.168.200.1
#   smtp_connect_timeout 30
   router_id R1                                                             // 保留router_id,可以这个集群服务在这台设备的别名设置主机的名称，可以自己定义，但是同一个组下名称不可以相同。
vrrp_instance VI_1 {
    state MASTER                                                            // 设置是主服务器，还是从服务器，master是主服务器
    interface eth0                                                          // 通过eth0网卡做心跳检测
    virtual_router_id 66                                                    // 以及虚拟的组，只有在同一个组里，它才是同一个高可用环境，组ID必须一致
    priority 80                                                             // 权重：最大不超过150，组合组之间最好差距为50，切换可能会更顺畅一点。官方建议；写80，留一定的余地
    advert_int 1                                                            // 检测间隔为1秒      
    authentication {                                                        // 认证：主和从之间的需要填写一定的身份认证，此处为密码认证
        auth_type PASS                                                      // 认证账户：
        auth_pass 1111                                                      // 认证密码
    }
    virtual_ipaddress {                                                     // 几群的IP地址
        10.10.10.100                                                        // 此环境只有一个集群

~~~     Keepalived以上为虚拟路由相关的协议设置OK
~~~     Keepalived以下为 虚拟服务的设置

 virtual_server 10.10.10.100 80 {                                           // 集群相关的参数；地址:端口号80：10.10.10.100:80
    delay_loop 6                                                            // 检测循环次数
    lb_algo rr                                                              // 当前使用的算法rr
    lb_kind DR                                                              // 当前的模式为DR模式
    nat_mask 255.255.255.0              
    persistence_timeout 50
    protocol TCP                                                            // 当前的检测方案为TCP连接模式

real_server 192.168.201.100 443 {                                           // 真实服务器
    weight 1                                                                // 权重，此环境使用的是r，没有用wr，所以写1：1即可
       TCP_CHECK {
         connect_port 80                                                    // TCP的检测方案，检测的端口是80
         connect_timeout 3                                                  // 检测的操作时间是3秒
         nb_get_retry 3                                                     // 重试3次
         delay_before_retry 4                                               // 重试以后间隔4秒

### --- 启动Keepalived服务
[root@server11 keepalived-1.2.2]# service keepalived start
Starting keepalived:                                       [  OK  ]

~~~     通过日志去查看一下
[root@server11 keepalived-1.2.2]# cat /var/log/messages
Jan 21 01:21:47 localhost Keepalived_vrrp: VRRP_Instance(VI_1) Sending gratuitous ARPs on eth0 for 10.10.10.100 //ARP监听到eth0，使用的是100地址
Jan 21 01:21:52 localhost Keepalived_vrrp: VRRP_Instance(VI_1) Sending gratuitous ARPs on eth0 for 10.10.10.100

附录一：keepalived.conf配置文件标准，上面为参数说明

### --- keepalived.conf配置文件标准，上面为参数说明

[root@server11 keepalived-1.2.2]# vim /etc/keepalived/keepalived.conf 
! Configuration File for keepalived

global_defs {
   router_id R1                                                             // 设置当前设备的名称
}

vrrp_instance VI_1 {                                                        // 设置服务器当前的名称，
    state MASTER
    interface eth0
    virtual_router_id 66
    priority 80
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass 1111
    }
    virtual_ipaddress {                                                     //虚拟IP的地址
        10.10.10.100
    }
}

virtual_server 10.10.10.100 80 {                                            //设置虚拟网络
    delay_loop 2
    lb_algo rr
    lb_kind DR
    protocol TCP            
    real_server 10.10.10.13 80 {                                            //真实服务器RS1
        weight 1
        TCP_CHECK {
            connect_port 80
            connect_timeout 3
            nb_get_retry 3
            delay_before_retry 4
        }
    } 
    real_server 10.10.10.14 80 {                                            //真实服务器RS2
        weight 1
        TCP_CHECK {
            connect_port 80
            connect_timeout 3
            nb_get_retry 3
            delay_before_retry 4
        }
    }
}

二、LVS-DR+keepalived模式：10.10.10.12部署Keepalived服务(LVS主master)

### --- 10.10.10.12部署Keepalived服务(LVS主master)
### --- 添加路由优化方案：关闭网卡的重定向功能

[root@server12 ~]# vim /etc/sysctl.conf 
net.ipv4.conf.all.send_redirects = 0
net.ipv4.conf.default.send_redirects = 0
net.ipv4.conf.eth0.send_redirects = 0
[root@server12 ~]# sysctl -p

### --- 启动10.10.10.100心跳IP地址
~~~     第一个问题：因为10.10.10.100在HA-server1中已经启动了，在HA-server2中是起不来的；能不能启动时在网卡的启动脚本设置的，只需要把网卡启动脚本的拒绝给关闭即可。
~~~     第二个问题：同一个局域网下有两个10.10.10.100地址，它会怎么处理，它就会在HA-server1和HA-server2中来回切换，交换机会刷新它的Mac地址
~~~     第三个问题：为什么要加入Keepalived：Keepalived会绑定你的ARP权限，若是HA-server1下的10.10.10.100是存活状态，HA-server2下的10.10.10.100是不允许被使用的。若是检测到HA-server1下的10.10.10.100不在了，HA-server2就可以使用10.10.10.100的IP权限了。就可以和外网进行通讯。

[root@server12 ~]# cd /etc/sysconfig/network-scripts/
[root@server12 network-scripts]# cp -a ifcfg-eth0 ifcfg-eth0:0  
[root@server12 network-scripts]# vim !$
DEVICE=eth0:0
ONBOOT=yes
BOOTPROTO=static
IPADDR=10.10.10.100
NETMASK=255.255.255.0

附录一：报错方案

### --- 故障现象：启动网卡会报错，已经有服务在使用该地址了
[root@server12 network-scripts]# ifup eth0:0
Determining if ip address 10.10.10.100 is already in use for device eth0...
Error, some other host (00:0C:29:FA:34:71) already uses address 10.10.10.100.

### --- 故障分析：
[root@server12 ~]# vim /etc/sysconfig/network-scripts/ifup-eth 
~~~     ：256行寻找修改参数，不同版本中它的位置是不一样的，找到如下内容
~~~     上面参数表示通过ARP协议检测在相同队列里有没有相同IP信息，若有相同就离开，返回值为1
if ! ARPING=$(/sbin/arping -c 2 -w ${ARPING_WAIT:-3} -D -I ${REALDEVICE} ${ipaddr[$idx]}) ; then
ARPINGMAC=$(echo $ARPING |  sed -ne 's/.*\[\(.*\)\].*/\1/p')
net_log $"Error, some other host ($ARPINGMAC) already uses address ${ipaddr[$idx]}."
exit 1          

### --- 解决方案：注释掉这几行内容
# if ! ARPING=$(/sbin/arping -c 2 -w ${ARPING_WAIT:-3} -D -I ${REALDEVICE} ${ipaddr[$idx]}) ; then
# ARPINGMAC=$(echo $ARPING |  sed -ne 's/.*\[\(.*\)\].*/\1/p')
# net_log $"Error, some other host ($ARPINGMAC) already uses address ${ipaddr[$idx]}."
# exit 1
# fi
~~~     注释信息后重新启动ifcfg-eth0:0网卡
[root@server12 ~]# ifup eth0:0                                              // 启动成功
Determining if ip address 10.10.10.100 is already in use for device eth0...
[root@server12 ~]# ifconfig 
eth0:0    Link encap:Ethernet  HWaddr 00:0C:29:DA:AF:AC  
          inet addr:10.10.10.100  Bcast:10.10.10.255  Mask:255.255.255.0
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1

### --- 构建Keepalived：将Keepalived.iso上传到服务器

[root@server12 ~]# mkdir /mnt/iso1
[root@server12 ~]# mount -o loop Keepalived.iso /mnt/iso1/
[root@server12 ~]# cp -a /mnt/iso1/* .
[root@server12 ~]# tar -zxvf keepalived-1.2.2.tar.gz
[root@server12 ~]# cd keepalived-1.2.2

### --- 安装make编译安装依赖包并
[root@server12 keepalived-1.2.2]# yum install -y kernel-devel openssl-devel popt-devel gcc*
~~~     生成makefile文件

[root@server12 keepalived-1.2.2]# ./configure --prefix=/ --with-kernel-dir=/usr/src/kernels/2.6.32-642.el6.x86_64/
Keepalived configuration
------------------------
Keepalived version       : 1.2.2
Compiler                 : gcc
Compiler flags           : -g -O2
Extra Lib                : -lpopt -lssl -lcrypto 
Use IPVS Framework       : Yes
IPVS sync daemon support : Yes
IPVS use libnl           : No
Use VRRP Framework       : Yes
Use Debug flags          : No

~~~     安装
[root@server12 keepalived-1.2.2]# make && make install

~~~     启动Keepalived
[root@server12 keepalived-1.2.2]# chkconfig --add keepalived
[root@server12 keepalived-1.2.2]# chkconfig keepalived on

### --- 修改Keepalived.conf配置文件
~~~     直接在HA-server1下推入配置文件

[root@server11 ~]# scp /etc/keepalived/keepalived.conf root@10.10.10.12:/etc/keepalived/keepalived.conf 
[root@server12 ~]# vim /etc/keepalived/keepalived.conf 
 ! Configuration File for keepalived
global_defs {
   router_id R2                                                             // 组号修改为R2,，同一个集群下不可一直
}
vrrp_instance VI_1 {
    state SLAVE                                                             // 名称为SLAVE或者BACKUP
    interface eth0
    virtual_router_id 66                                                    // 组号保持一致
    priority 20                                                             // 权重为20                        
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass 1111
    }

### --- 安装ipvsadm
[root@server12 ~]# yum install -y ipvsadm
[root@server12 ~]# service ipvsadm start
[root@server12 ~]# chkconfig ipvsadm on
 
### --- 启动Keepalived服务
[root@server12 ~]# service keepalived start
Starting keepalived:                                       [  OK  ]

### --- 验证是否配置成功：

[root@server12 ~]# ipvsadm -Ln                                          // 第一次没有查看到
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
  -> RemoteAddress:Port           Forward Weight ActiveConn InActConn

[root@server12 ~]# ipvsadm -Ln                                              // 间隔两秒重新查询后出现
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
  -> RemoteAddress:Port           Forward Weight ActiveConn InActConn
TCP  10.10.10.100:80 rr
  -> 10.10.10.13:80               Route   1      0          0         
  -> 10.10.10.14:80               Route   1      0          0

三、验证：通过IE浏览器访问http://10.10.10.100/

四、通过命令验证

### --- 验证LVS-DR+keepalived负载调度

[root@server11 ~]# ipvsadm -Ln --stats
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port               Conns   InPkts  OutPkts  InBytes OutBytes
  -> RemoteAddress:Port
TCP  10.10.10.100:80                    42      210        0    30051        0
  -> 10.10.10.13:80                     21      105        0    15015        0
  -> 10.10.10.14:80                     21      105        0    15036        0

~~~     刷新网页；发现HA-server2的数据没有发生变化，而HA-server1 的数据发生变化，说明HA-server1正在负载均衡

[root@server11 ~]# ipvsadm -Ln --stats                  
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port               Conns   InPkts  OutPkts  InBytes OutBytes
  -> RemoteAddress:Port
TCP  10.10.10.100:80                    56      280        0    40068        0
  -> 10.10.10.13:80                     28      140        0    20020        0
  -> 10.10.10.14:80                     28      140        0    20048        0

~~~     把HA-server1的网络断掉;HA-server2的数据发生变化，说明HA-server2切换为负载调度器实现负载调度。

[root@server12 ~]# ipvsadm -Ln --stats
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port               Conns   InPkts  OutPkts  InBytes OutBytes
  -> RemoteAddress:Port
TCP  10.10.10.100:80                    82      410        0    58670        0
  -> 10.10.10.13:80                     11       55        0     7865        0
  -> 10.10.10.14:80                     11       55        0     7875        0

### --- LVS-DR+keepalived结合后，Keepalived会后端的服务器进行健康状态检查，停掉HA-server3服务的Apache服务，查看状态

[root@server13 ~]# service httpd stop
[root@server11 ~]# ipvsadm -Ln --stats                                      //10.10.10.13服务已经不存在了
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port               Conns   InPkts  OutPkts  InBytes OutBytes
  -> RemoteAddress:Port
TCP  10.10.10.100:80                    56      280        0    40068        0
  -> 10.10.10.14:80                     28      140        0    20048        0
[root@server13 ~]# service httpd start

[root@server11 ~]# ipvsadm -Ln --stats                                      //10.10.10.13服务又重新恢复
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port               Conns   InPkts  OutPkts  InBytes OutBytes
  -> RemoteAddress:Port
TCP  10.10.10.100:80                    56      280        0    40068        0
  -> 10.10.10.13:80                      0        0        0        0        0
  -> 10.10.10.14:80                     28      140        0    20048        0

### --- 重点说明：

~~~     在此环境下真实服务器RS1/RS2宕机不会对业务造成中断
~~~     LVS-M/LVS-S宕机不会对业务造成中断
~~~     在此环境下不管是哪个节点宕机都不会对业务造成中断：
~~~     LVS-DR+keepalived高可用集群构建完成。