web集群学习：nginx+keepalived实现负载均衡高可用性

目录

项目架构

一，环境介绍

二，项目部署

在Web服务器上配置Web测试页面

nginx负载均衡配置

配置Nginx_Master

通过vrrp_script实现对集群资源的监控（1>通过killall命令探测服务运行状态）

通过vrrp_script实现对集群资源的监控（2、开发检测nginx存活的shell脚本）

三，项目测试

四，实现不抢占模式

---

项目架构

Nginx+Keepalived实现高可用

在 Keepalived + Nginx 高可用负载均衡架构中，keepalived 负责实现高可用。它是一个高性能的服务器高可用或者热备解决方案，Keepalived主要来防止服务器单点故障的发生问题，可以通过其与Nginx的配合实现Web服务器端的高可用。使用keepalived可以保证nginx的高可用，他能监控nginx的健康状态，当nginx出现宕机时自动主备切换。

一，环境介绍

服务器名称	IP	用途
Nginx_Master	172.16.90.111	提供负载均衡
Nginx_Backup	172.16.90.112	提供负载均衡
LVS-DR-VIP	172.16.90.200	网站的VIP地址
Web1服务器	172.16.90.113	提供Web服务
Web2服务器	172.16.90.114	提供Web服务

二，项目部署

在后端服务器上配置Web测试页面

web01配置：

cd /usr/share/nginx/html/
echo "web test page,`hostname -I`." > index.html 
systemctl restart nginx

web02配置同上

nginx负载均衡配置

nginx_master/backup做同样配置

1、安装nginx

yum install http://nginx.org/packages/rhel/7/x86_64/RPMS/nginx-1.10.0-1.el7.ngx.x86_64.rpm

 2、配置nginx反向代理

[root@nginx_master ~]# cd /etc/nginx/conf.d/
[root@nginx_master conf.d]# mv default.conf{,.bak}
[root@nginx_master conf.d]# vim web.conf

upstream webpools {
        server 172.16.90.113;
        server 172.16.90.114;
}

server {
        location / {
           proxy_pass http://webpools;
           index index.html;
        }
}

3，重启nginx服务，并测试访问nginx

nginx -t
systemctl restart nginx

客户端访问测试负载均衡：

for ((i=1;i<=10;i++)); do curl 172.16.90.111; done

配置Nginx_Master

安装keepalived

yum install keepalived -y

通过vrrp_script实现对集群资源的监控（1>通过killall命令探测服务运行状态）

配置keepalived

vim /etc/keepalived/keepalived.conf

vrrp_script chk_nginx {
        script "killall -0 nginx"
        #script "</dev/tcp/127.0.0.1/80"
        #script "if [ -f /var/run/httpd/httpd.pid ];then exit 0;else exit 1;fi"
        #script "/etc/keepalived/check_nginx.sh"
        interval 2       //间隔时间，单位为秒，默认1秒
        fall 2     //脚本几次失败转换为失败
        rise 1    //脚本连续监测成果后，把服务器从失败标记为成功的次数
        }

vrrp_instance VI_1 {
    state MASTER
    interface ens32 //填你对应的网卡名称
    virtual_router_id 51
    priority 150  //默认100，改大
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass 1111
    }

   notify_master "/etc/keepalived/master.sh"
   notify_backup "/etc/keepalived/backup.sh"
   notify_fault "/etc/keepalived/fault.sh"

   track_script{
         chk_nginx
    }
    virtual_ipaddress {
         172.16.90.200/24 dev ens32
     }
 }

相关参数说明：

notify的用法：

notify_master:当当前节点成为master时，通知脚本执行任务(一般用于启动某服务，比如 nginx,haproxy等)

notify_backup:当当前节点成为backup时，通知脚本执行任务(一般用于关闭某服务，比如nginx,haproxy等)

notify_fault：当当前节点出现故障，执行的任务;

根据提供的路径脚本路径，编写提供日志记录的脚本：

vim /etc/keepalived/master.sh
#!/bin/bash
LOGFILE=/etc/keepalived/nginx_state.log
echo "[master]" >> $LOGFILE
date >> $LOGFILE

vim /etc/keepalived/backup.sh
#!/bin/bash
LOGFILE=/etc/keepalived/nginx_state.sh
echo "[backup]" >> $LOGFILE
date >> $LOGFILE

vim /etc/keepalived/fault.sh
#!/bin/bash
LOGFILE=/etc/keepalived/nginx_state.log
echo "[dault]" >> $LOGFILE
date >> $LOGFILE

给文件赋予执行权限

chmod +x /etc/keepalived/*.sh

Nginx_Backup配置同上

通过vrrp_script实现对集群资源的监控（2、开发检测nginx存活的shell脚本）

vim /etc/keepalived/check_nginx.sh

#!/bin/bash
A=`ps -C nginx --no-header |wc -l`
if [ $A -eq 0 ];then
        systemctl start nginx
        sleep 3
if [ `ps -C nginx --no-header |wc -l` -eq 0 ]
then
        systemctl stop keepalived
        fi
fi

给文件赋予执行权限

chmod +x /etc/keepalived/*.sh

配置keepalived

vim /etc/keepalived/keepalived.conf

vrrp_script chk_nginx {
        #script "killall -0 nginx"
        #script "</dev/tcp/127.0.0.1/80"
        #script "if [ -f /var/run/httpd/httpd.pid ];then exit 0;else exit 1;fi"
        script "/etc/keepalived/check_nginx.sh"
        interval 2
        fall 2
        rise 1
        }

vrrp_instance VI_1 {
    state MASTER
    interface ens32 //填你对应的网卡名称
    virtual_router_id 51
    priority 100
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass 1111
    }

   notify_master "/etc/keepalived/master.sh"
   notify_backup "/etc/keepalived/backup.sh"
   notify_fault "/etc/keepalived/fault.sh"

   track_script{
         chk_nginx
    }
    virtual_ipaddress {
         172.16.90.200/24 dev ens32
     }
 }

Nginx_Backup配置文件

vrrp_script chk_nginx {
        #script "killall -0 nginx"
        #script "</dev/tcp/127.0.0.1/80"
        #script "if [ -f /var/run/httpd/httpd.pid ];then exit 0;else exit 1;fi"
        script "/etc/keepalived/check_nginx.sh"
        interval 2
        fall 2
        rise 1
        }

vrrp_instance VI_1 {
    state BACKUP  //改这里哟
    interface ens32 //填你对应的网卡名称
    virtual_router_id 51
    priority 100
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass 1111
    }

   notify_master "/etc/keepalived/master.sh"
   notify_backup "/etc/keepalived/backup.sh"
   notify_fault "/etc/keepalived/fault.sh"

   track_script{
         chk_nginx
    }
    virtual_ipaddress {
         172.16.90.200/24 dev ens32
     }
 }

三，项目测试

重启主从调度的nginx和keepalived

[root@nginx_master ~]# systemctl restart nginx keepalived
[root@nginx_backup ~]# systemctl restart nginx keepalived

Master，Backup都正常，只有Master对外提供服务

[root@nginx_master ~]# ip a | grep "172.16.90.200"
inet 172.16.90.200/24 scope global ens32

Master宕机，Backup接替Master对外提供服务

模拟Master的keepalived服务器停止
[root@nginx_master ~]# systemctl stop keepalived.service
此时VIP在Backup上
[root@nginx_backup ~]# ip a | grep "172.16.90.200"
inet 172.16.90.200/24 scope global ens32

在客户机上验证是否Backup接管后还能实现负载均衡

for ((i=1;i<=10;i++)); do curl 172.16.90.200; done

Master恢复正常，Master继续提供服务，Backup停止服务

模拟Master的keepalived服务恢复正常
[root@nginx_master ~]# systemctl start keepalived.service
此时VIP在Master上
[root@nginx_master ~]# ip a | grep "172.16.90.200"
inet 172.16.90.200/24 scope global ens32

四，实现不抢占模式

上述主节点一宕机，备节点就会接管，主节点修复好了，又会重新接管服务。服务的切换存在一定的风险和不稳定性，为了避免生产中多次网络抖动，要实现不抢占模式

nopreempt 设置的是高可用集群中的不抢占功能:设置 nopreempt可以实现主节点故障恢复后不再切回到主节点，让服务一直在备用节点下工作， 直到备用节点出现故障才会进行切换。在使用不抢占功能时，在“state” 状态为 “BACKUP” 的节点上设置,而且这个节点的优先级必须高于其他节点

Nginx_Master上

vim /etc/keepalived/keepalived.conf

vrrp_script chk_nginx {
        #script "killall -0 nginx"
        #script "</dev/tcp/127.0.0.1/80"
        #script "if [ -f /var/run/httpd/httpd.pid ];then exit 0;else exit 1;fi"
        script "/etc/keepalived/check_nginx.sh"
        interval 2
        fall 2
        rise 1
        }

vrrp_instance VI_1 {
    state BACKUP   //原来的MASTER改为BACKUP
    interface ens32 //填你对应的网卡名称
    virtual_router_id 51
    priority 100
    nopreept    //添加这行
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass 1111
    }

   notify_master "/etc/keepalived/master.sh"
   notify_backup "/etc/keepalived/backup.sh"
   notify_fault "/etc/keepalived/fault.sh"

   track_script{
         chk_nginx
    }
    virtual_ipaddress {
         172.16.90.200/24 dev ens32
     }
 }