Nginx+Keepalived搭建高可用

目录

1.背景

2.环境

1）操作系统和软件环境

2）虚拟机和IP规划

3）示意图

3.步骤

1）准备

2）配置

3）启动

4.测试

5.参考资料

---

1.背景

最近了解了一些Nginx的基础知识，如何配置反向代理、负载均衡，如何利用Nginx+Keepalived搭建一个简单的高可用环境。过程不算曲折，但也遇到一些些，记录一下，方便自己也方便他人。

2.环境

1）操作系统和软件环境

• OS：CentOS 7.3.1611 （# cat /etc/redhat-release）
• Java：v1.8.0_292
• Nginx：v1.14.2
• Keepalived：v1.3.5（# rpm -qa | grep keepalived）
• tomcat：v8.0.1

2）虚拟机和IP规划

虚拟机1	192.168.0.10	主Nginx+Keepalived
虚拟机2	192.168.0.11	从Nginx+Keepalived
虚拟机3	192.168.0.12	tomcat
VIP	192.168.0.20	虚IP，用于客户端访问

3）示意图

3.步骤

1）准备

操作系统和Java的安装这里就省略了~~

• 192.168.0.12

安装tomcat（https://archive.apache.org/dist/tomcat/），下载解压，如有特殊端口需求，修改/conf/server.xml中的默认端口8080，最后# sh startup.sh，启动tomcat即可。测试：http://192.168.0.12:8080/。出现小猫即为安装成功

• 192.168.0.10

安装nginx（http://nginx.org/en/download.html），可参考这个里面的步骤，已亲测有效（https://www.cnblogs.com/ligang0357/p/11362469.html）。测试：http://192.168.0.10/。出现“Welcome to Nginx...”，即安装成功。

安装keepalived。这里采用yum进行的。

# yum install -y keepalived

• 192.168.0.11 同上

2）配置

• 主Nginx上的Keepalived

# cd /etc/keepalived
# cp keepalived.conf keepalived.conf.init
# vim keepalived.conf

! Configuration File for keepalived


global_defs {
  notification_email {
    acassen@firewall.loc
    failover@firewall.loc
    sysadmin@firewall.loc
  }
  notification_email_from Alexandre.Cassen@firewall.loc
  smtp_server  192.168.0.10    # 主Nginx所在节点IP
  smtp_connect_timeout  30
  router_id  LVS_DEVEL        # 主从Nginx该值必须一致
  script_user  root
  enable_script_security
}

vrrp_script chk_http_port {
  script  "/usr/local/src/nginx_check.sh"        # 健康检查脚本
  interval  2
  weight  2
}

vrrp_instance VI_1 {
  state MASTER        # 主Nginx
  interface  eth0        # 网卡名称。可通过#ip a命令查看
  virtual_router_id  51
  priority  100
  advert_int  1
  authentication {
    auth_type  PASS
    auth_pass  1111
  }

  virtual_ipaddress {
    192.168.0.20        # VIP
  }
}

• 主Nginx

# cd /usr/local/nginx/conf
# cp nginx.conf nginx.conf.init
# vim nginx.conf

worker_processes  1;

events {
    worker_connections  1024;
}


http {
    include  mime.types;
    default_type  application/octet-stream;

    sendfile  on;
    keepalive_timeout  65;

    server {
        listen  80;
        server_name  192.168.0.10;

        location / {
            root html;
            proxy_pass  http://127.0.0.1:8080;
            index  index.html index.htm;
        }

        error_page  500 502 503 504 /50x.html;
        location = /50x.html {
            root html;
        }
    }  
  
    server {
        listen  9001;
        server_name  192.168.0.10;
        location ~ /tomcat/ {
            proxy_pass  http://127.0.0.1:8080;
        }
    }  
}

• 从keepalived

# cd /etc/keepalived
# cp keepalived.conf keepalived.conf.init
# vim keepalived.conf

! Configuration File for keepalived


global_defs {
  notification_email {
    acassen@firewall.loc
    failover@firewall.loc
    sysadmin@firewall.loc
  }
  notification_email_from Alexandre.Cassen@firewall.loc
  smtp_server  192.168.0.11    # 从Nginx所在节点IP
  smtp_connect_timeout  30
  router_id  LVS_DEVEL        # 主从Nginx该值必须一致
  script_user  root
  enable_script_security
}

vrrp_script chk_http_port {
  script  "/usr/local/src/nginx_check.sh"        # 健康检查脚本
  interval  2
  weight  2
}

vrrp_instance VI_1 {
  state BACKUP        # 主Nginx
  interface  eth0        # 网卡名称。可通过#ip a命令查看
  virtual_router_id  51
  priority  90        # 优先级。主大于从
  advert_int  1
  authentication {
    auth_type  PASS
    auth_pass  1111
  }

  virtual_ipaddress {
    192.168.0.20        # VIP
  }
}

•  从Nginx

# cd /etc/keepalived
# cp keepalived.conf keepalived.conf.init
# vim keepalived.conf

worker_processes  1;

events {
    worker_connections  1024;
}


http {
    include  mime.types;
    default_type  application/octet-stream;

    sendfile  on;
    keepalive_timeout  65;

    server {
        listen  80;
        server_name  192.168.0.11;

        location / {
            root html;
            index  index.html index.htm;
        }

        error_page  500 502 503 504 /50x.html;
        location = /50x.html {
            root html;
        }
    }  
  
    server {
        listen  9001;
        server_name  192.168.0.11;
        location ~ /tomcat/ {
            proxy_pass  http://192.168.0.11:8080;
        }
    }  
}

•  主从上均增加健康检查脚本

# vim /usr/local/src/nginx_check.sh

#!/bin/bash

A=`ps -C nginx -no-header | wc -l`
B=`ps -C nginx --no-header | wc -l`

if [ $A -eq 1 ];then
    /usr/local/nginx/sbin/nginx
    sleep 2
    if [ $B -eq 0 ];then
        killall keepalived
    fi
fi

修改脚本权限 chmode 764 nginx_check.sh

3）启动

分别启动主从Nginx

# /usr/local/nginx/sbin/nginx

分别启动主从节点上的keepalived

# systemctl start keepalived

4.测试

测试一：通过浏览器访问 http://192.168.0.20:9090/tomcat/

测试二：停止主Nginx后，访问 http://192.168.0.20:9090/tomcat/

均出现tomcat的欢迎界面，证明上述配置成功。

5.参考资料

https://www.cnblogs.com/yw-ah/p/9770971.html -- 安装tomcat

https://blog.csdn.net/natureqiu/article/details/117887620 -- keepalived的script脚本不执行解决办法

https://blog.csdn.net/Bb15070047748/article/details/106276491/ --  Keepalived监测脚本一直不执行