Docker+Nginx+Keepalived实现架构高可用

# 安装centos 镜像
docker pull centos
# 运行centos容器
docker run -it centos /bash/bin

# 安装依赖和所需要的包　
#使用yum安装nginx需要包括Nginx的库，安装Nginx的库
rpm -Uvh http://nginx.org/packages/centos/7/noarch/RPMS/nginx-release-centos-7-0.el7.ngx.noarch.rpm
# 使用下面命令安装nginx
#yum install nginx
#安装网络包(需要使用ifconfig和ping命令)
yum install net-tools
#安装vim
yum install vim

# 在centos安装keepalvied
#安装keepalived环境依赖
yum install -y gcc openssl-devel popt-devel
#安装keepalived
yum install keepalived

#或者通过源码安装
wget http://124.205.69.132/files/90630000053A2BB4/www.keepalived.org/software/keepalived-1.3.4.tar.gz
tar zxvf keepalived-1.3.4.tar.gz 
cd keepalived-1.3.4
./configure --prefix=/usr/local/keepalived
make && make install

# 拷贝几个文件到CentOS7环境中：
cp keepalived-1.3.4/keepalived/etc/init.d/keepalived /etc/init.d/
mkdir /etc/keepalived
cp /usr/local/keepalived/etc/keepalived/keepalived.conf /etc/keepalived/
cp keepalived-1.3.4/keepalived/etc/sysconfig/keepalived /etc/sysconfig/
cp /usr/local/keepalived/sbin/keepalived /usr/sbin/

修改/etc/keepalived/keepalived.conf文件

! Configuration File for keepalived
global_defs {
    notification_email {
    	762357658@qq.com
    }
    notification_email_from itsection@example.com
    smtp_server mail.example.com
    smtp_connect_timeout 30
    router_id LVS_DEVEL  
}


vrrp_script chk_nginx {
    script "/etc/keepalived/nginx_check.sh"  
    interval 2
    weight -5
    fall 3
    rise 2
}


vrrp_instance VI_1 {
    state MASTER   
    interface eth0   
    virtual_router_id 2  
    priority 101  
    advert_int 2
    authentication {
        auth_type PASS  
        auth_pass 1111
    }
    virtual_ipaddress {
        172.17.0.210   
    }
    track_script {
       chk_nginx 
    }

}

/etc/keepalived/check_nginx.sh文件

A=`ps -ef | grep nginx | grep -v grep | wc -l`
if [ $A -eq 0 ];then
    nginx
    sleep 2
    if [ `ps -ef | grep nginx | grep -v grep | wc -l` -eq 0 ];then
        #killall keepalived
        ps -ef|grep keepalived|grep -v grep|awk '{print $2}'|xargs kill -9 
    fi

fi　　

对check_nginx.sh赋于执行权限：

chmod +x check_nginx.sh

keepalived是通过检测keepalived进程是否存在判断服务器是否宕机，如果keepalived进程在但是nginx进程不在了那么keepalived是不会做主备切换，所以我们需要写个脚本来监控nginx进程是否存在，如果nginx不存在就将keepalived进程杀掉。

在主nginx上需要编写nginx进程检测脚本（check_nginx.sh），判断nginx进程是否存在，如果nginx不存在就将keepalived进程杀掉，并将vip漂移到备份机器上

keepalived开机启动

chkconfig keepalived on
或者
systemctl enable keepalived.service  

# 启动
systemctl start keepalived.service 

安装所有需要的依赖和环境后，将容器新增的内容重新提交

docker commit 5d112 centos_keepalived_nginx:v1

5d112为，上述安装软件所对应的容器id

启动含有(keepalived+nginx)的容器

docker run --privileged  -tid --name  keepalived_master centos_keepalived_nginx:v1 /usr/sbin/init

进入keepalived_master容器

docker exec -it keepalived_master bash

进入/usr/share/nginx/html，修改index.html文件

修改标题为:Welcome to nginx Master!

启动keepalived_salve容器

#启动一个容器
docker run --privileged  -tid --name  keepalived_slave centos_keepalived_nginx:v1 /usr/sbin/init

#进入容器
docker exec -it keepalived_slave bash

# 修改keepalived_salve容器中nginx index.html文件
vim /usr/share/nginx/html/index.html

修改标题为:Welcome to nginx Slave!

修改keepalived_salve容器中keepalived.conf文件 (master容器中，保持和镜像中设置一样即可，不需要更改)

! Configuration File for keepalived
global_defs {
notification_email {
762357658@qq.com
}
notification_email_from itsection@example.com
smtp_server mail.example.com
smtp_connect_timeout 30
router_id LVS_DEVEL
}

vrrp_script chk_nginx {
script "/etc/keepalived/nginx_check.sh"
interval 2
weight -5
fall 3
rise 2
}


vrrp_instance VI_1 {
state BACKUP
interface eth0
virtual_router_id 2
priority 100
advert_int 2
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
172.17.0.210
}
track_script {
chk_nginx
}

}

由配置中可以看出，主要是state和priority两个参数的调整，其中master节点的priority值一定要比backup大才行！

原理说明：

1. 通过vrrp协议广播，每个keepalived vrrp都去争取master
2. 以virtual_router_id为组队标识。 同为一个vip服务的keepalived的virtual_router_id要保持相同
3. 以priority 为权值，同一个virtual_router_id下那个priority大那个就是master,其它为backup

改完之后，重新加载

systemctl daemon-reload 
systemctl restart keepalived.service

验证

查看两个容器中keepalived服务状态

systemctl status keepalived.service 

keepalived_master服务状态效果：

keepalived_slave服务状态效果图：

可以看到，keepalived服务运行正常

启动nginx: nginx
在master容器中 curl 172.17.0.210

在slave容器中 curl 172.17.0.210

可以看现，此时master和slave容器两边通过虚拟vip : 172.17.0.210 访问nginx数据，请求返回的数据都是master容器中nginx配置的数据： welcome to nginx master

继续验证，关掉master容器的keepalived服务：

验证得到的结果是当master容器中的keepalived服务关掉后，curl 172.17.0.210请求返回的数据来自slave，welcome to nginx slave

再继续验证，把关掉master容器的keepalived服务再开启：

可以看到，当master容器中的keepalived服务开启后，请求返回的数据会再次转到master中。

到此，所有的验证和预期的一致，也达到我们借助docker为基础来实现了整套基于Nginx+Keepalived高可用的方案了。

Keepalived服务命令

• systemctl daemon-reload 重新加载
• systemctl enable keepalived.service 设置开机自动启动
• systemctl disable keepalived.service 取消开机自动启动
• systemctl start keepalived.service 启动
• systemctl stop keepalived.service 停止
• systemctl status keepalived.service 查看服务状态