docker创建keepalived镜像实现高可用

必要条件：

• 1、在服务器中安装 centos 镜像
• 2、进入centos_01容器安装keepalived（或者keepalived+nginx），若有前端则加上nginx

环境：
前提条件 -> 已在 linux 中安装好docker环境。

1、在服务器中安装 centos 镜像

下载镜像：

1	docker  pull  centos:7.7.1908

查看镜像：

1	docker images

安装第一个centos容器：centos_01

#f1cb7c7d58b7  就是上图中的镜像ID

sudo docker run -it --name centos_01 f1cb7c7d58b7

#然后退出容器

exit;

2、进入centos_01容器安装keepalived+nginx

进入容器：

#efbad978d17f 是容器ID

docker exec -it efbad978d17f /bin/bash

安装keepalived 和安装nginx

#安装依赖环境

yum install -y gcc openssl-devel popt-devel

#安装keepalived

yum install -y keepalived

#或者本地安装指定版本

wget https://www.keepalived.org/software/keepalived-1.3.4.tar.gz

#使用yum安装nginx需要包括Nginx的库，安装Nginx的库

rpm -Uvh http://nginx.org/packages/centos/7/noarch/RPMS/nginx-release-centos-7-0.el7.ngx.noarch.rpm

# 使用下面命令安装nginx

yum install -y nginx

#安装网络包(需要使用ifconfig和ping命令)

yum install -y net-tool

修改/etc/keepalived/keepalived.conf文件:

#进入到 容器 centos_01 的根目录

cd /

#进入 keepalived 安装目录

cd /etc/keepalived

# 打开配置文件

vi keepalived.conf

修改/etc/keepalived/keepalived.conf文件        

! Configuration File for keepalived

global_defs {

    router_id LVS_DEVEL

}

vrrp_script chk_nginx {

    script "/etc/keepalived/nginx_check.sh" #检测nginx进程是否在线

    # script "</dev/tcp/127.0.0.1/8080" 检测服务端口是否在线

     interval 2 #检查脚本的频率,单位（秒）
     weight -30 #端口检查失败,优先级减少30,默认减少2[注意:两台主机配置的weight相同时候必须保证weight的值大于priority差值]

}

vrrp_instance VI_1 {

    state MASTER 

    interface eth0 

    virtual_router_id 51

    priority 101

    advert_int 2

    authentication {

        auth_type PASS

        auth_pass 1111

    }

    virtual_ipaddress {

        172.17.0.210 

    }

    track_script {

       chk_nginx

    }

}

创建 /etc/keepalived/check_nginx.sh 脚本文件

A=`ps -ef | grep nginx | grep -v grep | wc -l`

if [ $A -eq 0 ];then

    nginx

    sleep 2

    if [ `ps -ef | grep nginx | grep -v grep | wc -l` -eq 0 ];then

        #killall keepalived

        ps -ef|grep keepalived|grep -v grep|awk '{print $2}'|xargs kill -9

    fi

  

fi

再对check_nginx.sh赋于执行权限：

chmod +x check_nginx.sh

注：keepalived是通过检测keepalived进程是否存在判断服务器是否宕机，如果keepalived进程在但是nginx进程不在了那么keepalived是不会做主备切换，所以我们需要写个脚本来监控nginx进程是否存在，如果nginx不存在就将keepalived进程杀掉。
在主nginx上需要编写nginx进程检测脚本（check_nginx.sh），判断nginx进程是否存在，如果nginx不存在就将keepalived进程杀掉，并将vip漂移到备份机器上。

设置 keepalived 开机启动

1 2	#设置开机自动启动 systemctl enable keepalived.service

启动keepalived服务：

1 2	# 启动 systemctl start keepalived.service

安装所有需要的依赖和环境后，将容器新增的内容重新提交到镜像文件，制作成新的镜像以供使用

1 2 3 4

#查看容器ID docker ps #把这个容器打包为一个镜像，5d112 就是容器 centos_01 的ID docker commit 5d112 centos_keepalived_nginx:v1

执行完成之后，我们用 docker images 命令看一下

1	docker images

将镜像centos_keepalived_nginx:v1保存为keepalived.tar.gz

docker save -o keepalived.tar.gz centos_keepalived_nginx:v1    

这个镜像就是我们前面打包后的镜像。将keepalived.tar.gz放入另一台含有docker的服务器/home/docker/docker_test/images目录下，例如：192.168.2.2

在192.168.2.2服务器里操作：

cd home/docker/docker_test/images

从 eepalived.tar.gz 导入镜像

docker load -i keepalived.tar.gz

启动含有(keepalived+nginx)的容器。

1	docker run --privileged=true -tid --name  keepalived_master centos_keepalived_nginx:v1 /usr/sbin/init

进入keepalived_master容器：

1	docker exec -it keepalived_master bash

进入/usr/share/nginx/html，修改index.html文件

修改标题为:

Welcome to nginx Master!

启动keepalived_salve容器：

1 2 3 4 5

#启动一个容器 docker run --privileged=true -tid --name  keepalived_slave centos_keepalived_nginx:v1 /usr/sbin/init #进入容器 docker exec -it keepalived_slave bash

修改keepalived_salve容器中nginx index.html文件

1	vim /usr/share/nginx/html/index.html

修改标题为:

Welcome to nginx Slave!

修改keepalived_salve容器中keepalived.conf文件 (master容器中，保持和镜像中设置一样即可，不需要更改)

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38

! Configuration File for keepalived global_defs { notification_email { 762357658@qq.com } notification_email_from itsection@example.com smtp_server mail.example.com smtp_connect_timeout 30 router_id LVS_DEVEL } vrrp_script chk_nginx { script "/etc/keepalived/nginx_check.sh" interval 2 weight -5 fall 3 rise 2 } vrrp_instance VI_1 { state BACKUP interface eth0 virtual_router_id 2 priority 100 advert_int 2 authentication { auth_type PASS auth_pass 1111 } virtual_ipaddress { 172.17.0.210 } track_script { chk_nginx } }

其实，由配置中可以看出，主要是state和priority两个参数的调整，其中master节点的priority值一定要比backup大才行！
原理说明：
1、 通过vrrp协议广播，每个keepalived vrrp都去争取master
2、 以virtual_router_id为组队标识。 同为一个vip服务的keepalived的virtual_router_id要保持相同
3、 以priority 为权值，同一个virtual_router_id下那个priority大那个就是master,其它为backup

改完之后，重新加载

1 2	systemctl daemon-reload systemctl restart keepalived.service

验证

查看两个容器中keepalived服务状态，两台服务状态不一样

1	systemctl status keepalived.service

双机主备切换验证

 关闭主机

关闭主机后，备机自动出现服务ip
当主机再次启动后，服务ip切换至主机

关闭keepalived 服务

关闭keepalived 服务后，服务ip自动切换至备机
当主机keepalived服务再次启动后，服务ip自动切换至主机

关闭nginx服务

停止nginx服务后，服务ip自动切换至备机
当nginx服务重新启动后，服务ip自动切换至主机
 

关闭检测的端口8080服务

停止服务后，服务ip自动切换至备机
当服务重新启动后，服务ip自动切换至主机

以上是在192.168.2.2服务器进行命令操作，也可用docker-compse.yml操作

version: '3.9'
services:
  keepalivebase:
    image: 'centos:8'
    container_name: 'keepalivebase'
    network_mode: "host"
    restart: always
    volumes:
      - "/home/docker/volumes/xxx/_data/keepalived_data/keepalived.conf:/etc/keepalived/keepalived.conf"
      - "/etc/localtime:/etc/localtime"
      - "/etc/timezone:/etc/timezone"
    privileged: true

  portainer:
        image: portainer/portainer:latest
        container_name: portainer
        restart: always
        environment:
            TZ: Asia/Shanghai
            LANG: en_US.UTF-8
        ports:
            - "9000:9000"
        volumes:
            - /var/run/docker.sock:/var/run/docker.sock
            - "/etc/localtime:/etc/localtime"
            - "/etc/timezone:/etc/timezone"

version: '3.9'
services:  
    keepalive01:
        image: 'centos8_keepalived_nginx:v1'
        container_name: 'keepalive01'
        privileged: true
        entrypoint: "/sbin/init"
        environment:
            - TZ=Asia/Shanghai
        volumes:
            - ./keepalived_master.conf:/etc/keepalived/keepalived.conf
            - ./backup.sh:/etc/keepalived/backup.sh
            - ./fault.sh:/etc/keepalived/fault.sh
            - ./master.sh:/etc/keepalived/master.sh
            - ./index-master.html:/usr/share/nginx/html/index.html
        command: /bin/bash -c "chmod +x /etc/keepalived/*.sh && chmod -x /etc/keepalived/keepalived.conf"
        ports:
            - "80:80"
        networks:
            keepalive-ha:
                ipv4_address: '172.29.0.11'
    keepalive02:
        image: 'centos8_keepalived_nginx:v1'
        container_name: 'keepalive02'
        privileged: true
        environment:
            - TZ=Asia/Shanghai
        entrypoint: "/sbin/init"
        volumes:
            - ./keepalived_buckup.conf:/etc/keepalived/keepalived.conf
            - ./backup.sh:/etc/keepalived/backup.sh
            - ./fault.sh:/etc/keepalived/fault.sh
            - ./master.sh:/etc/keepalived/master.sh
            - ./index-slave.html:/usr/share/nginx/html/index.html
        command: /bin/bash -c "chmod +x /etc/keepalived/*.sh && chmod -x /etc/keepalived/keepalived.conf"
        ports:
            - "81:80"
        networks:
            keepalive-ha:
                ipv4_address: '172.29.0.12'
    cul_test:
        image: 'centos:8'
        container_name: 'cul_test'
        privileged: true
        environment:
            - TZ=Asia/Shanghai
        entrypoint: "/sbin/init"
        stdin_open: true
        tty: true
        networks:
            keepalive-ha:
                ipv4_address: '172.29.0.13'
    portainer:
        image: portainer/portainer:latest
        container_name: portainer
        restart: always
        environment:
            TZ: Asia/Shanghai
            LANG: en_US.UTF-8
        ports:
            - "9000:9000"
        volumes:
            - /var/run/docker.sock:/var/run/docker.sock
        networks:
            keepalive-ha:
                ipv4_address: '172.29.0.50'

networks:
    keepalive-ha:
        name: keepalive-ha
        driver: bridge
        # driver: overlay
        ipam:
            config:
                - subnet: '172.29.0.0/16'

docker load -i keepalived.tar.gz

docker-compose  -f /home/xxx/xxx/docker-compose.yml up -d 

    keepalived的主备状态与state值设置无关；
    主备机由priority值和vrrp_script中的weight值之和决定，大的为主；
    主备比较权值=priority值+weight值*标志位，当vrrp_script检测脚本为true时标志位为1，反之为0；
    为保证正常的主备切换，weight值应大于主备priority值之差。