最新Nginx + Keepalived 高可用环境搭建
VMware + CentOS7的集群环境大家自行准备下
CentOS镜像名称:CentOS-7-x86_64-Minimal-1810
本文采用的是 淘宝的 Tengine (Nginx的加强版)
Tengine 2.3.0 安装
-
解压文件
tar -zxvf Tengine-2.3.0.tar.gz
-
配置相关环境
wget、gcc、pcre、openssl、zlib
yum -y install wget //wget 用于从外网上下载插件 yum -y install gcc yum -y install pcre pcre-devel yum -y install openssl openssl-deve //执行后会自动安装zlib
-
检查相关环境是否安装正确
进入tengine的根目录下执行 ./configure 无异常表示安装成功。 -
编译并安装
make && make install
-
检测是否安装成功
cd /usr/local/nginx/sbin // 进入nginxsbin目录 ./nginx // 启动nginx ps -f | grep nginx //查看nginx进程 浏览器输入: http://你的服务器ip/
keepalived 2.0.15 安装
-
解压文件
tar -zxvf keepalived-2.0.13.tar.gz
-
配置相关环境
yum -y install openssl openssl-devel yum -y install libnl libnl-devel yum -y install libnfnetlink-devel
-
编译并安装
cd keepalived-2.0.13/ // --prefix 指定安装地址 ./configure --prefix=/usr/local/keepalived/ make && make install
-
注册为系统服务
mkdir /etc/keepalived/ cd /usr/local/software/keepalived-2.0.13/keepalived/ cp etc/init.d/keepalived /etc/init.d/ cp etc/sysconfig/keepalived /etc/sysconfig/ cp etc/keepalived/keepalived.conf /etc/keepalived/
-
启动查看
service keepalived start systemctl status keepalived
Nginx + Keepalived 高可用环境搭建
-
系统规划
-
创建Nginx日志文件
mkdir -p /opt/data/nginx/logs
-
nginx.conf文件
- 主(Master)、备(Backup)配置都一样。
- 具体配置根据具体需求而定
#user nobody; worker_processes 1; error_log /opt/data/nginx/logs/error.log notice; pid /opt/data/nginx/logs/nginx.pid; events { worker_connections 1024; } http { include mime.types; default_type application/octet-stream; log_format main '$remote_addr - $remote_user [$time_local] "$request" ' '$status $body_bytes_sent "$http_referer" ' '"$http_user_agent" "$http_x_forwarded_for"'; access_log /opt/data/nginx/logs/access.log main; sendfile on; #tcp_nopush on; #keepalive_timeout 0; keepalive_timeout 65; #gzip on; upstream tomcat { server 192.168.8.100:8012 weight=1 max_fails=2 fail_timeout=30s; server 192.168.8.100:8013 weight=1 max_fails=2 fail_timeout=30s; } server { listen 80; server_name localhost; location / { root html; index index.html index.htm; proxy_set_header X-real-ip $remote_addr; proxy_pass http://tomcat; } error_page 500 502 503 504 /50x.html; location = /50x.html { root html; } } }
-
Keepalived配置
- 在/etc/keepalived目录下,添加check_nginx.sh(检查nginx存活的shell脚本)
- 安装工具 yum install -y psmisc
#!/bin/bash COUNT=$(ps -C nginx --no-header |wc -l) echo $COUNT if [ $COUNT -eq 0 ] then /usr/local/nginx/sbin/nginx start echo "restart nginx" sleep 5 COUNT=$(ps -C nginx --no-header |wc -l) if [ $COUNT -eq 0 ] then echo "stop keepalived" killall keepalived fi fi
- 在/etc/keepalived目录下,添加keepalived.conf(keepalived配置文件)
- 这里采用的是Keepalived抢占模式(抢占模式即MASTER从故障中恢复后,会将VIP从BACKUP节点中抢占过来。)
- Master 配置
vrrp_script chk_nginx { script "/etc/keepalived/check_nginx.sh" //检测nginx进程的脚本 interval 2 weight -20 } global_defs { notification_email { //可以添加邮件提醒 } } vrrp_instance VI_1 { state MASTER interface ens33 virtual_router_id 51 mcast_src_ip 192.168.146.11 priority 250 advert_int 1 nopreempt //非抢占模式 authentication { auth_type PASS uth_pass 123456 } track_script { chk_nginx } virtual_ipaddress { 192.168.146.100 } }
- Backup 配置
vrrp_script chk_nginx { script "/etc/keepalived/check_nginx.sh" //检测nginx进程的脚本 interval 2 weight -20 } global_defs { notification_email { //可以添加邮件提醒 } } vrrp_instance VI_1 { state BACKUP interface ens33 virtual_router_id 51 mcast_src_ip 192.168.146.12 priority 250 advert_int 1 nopreempt //非抢占模式 authentication { auth_type PASS uth_pass 123456 } track_script { chk_nginx } virtual_ipaddress { 192.168.146.100 } }
- 在/etc/keepalived目录下,添加check_nginx.sh(检查nginx存活的shell脚本)
-
测试环境是否配置成功
- 在node01
[root@node01 keepalived]# /usr/local/nginx/sbin/nginx [root@node01 keepalived]# /etc/init.d/keepalived start Starting keepalived (via systemctl): [ OK ] [root@node01 keepalived]# ip add 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo valid_lft forever preferred_lft forever inet6 ::1/128 scope host valid_lft forever preferred_lft forever 2: ens33: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000 link/ether 00:0c:29:4b:2d:2d brd ff:ff:ff:ff:ff:ff inet 192.168.146.11/24 brd 192.168.146.255 scope global noprefixroute ens33 valid_lft forever preferred_lft forever inet 192.168.146.100/32 scope global ens33 valid_lft forever preferred_lft forever inet6 fe80::6d51:9e88:4627:2640/64 scope link tentative noprefixroute dadfailed valid_lft forever preferred_lft forever inet6 fe80::4794:5f00:d68b:6e4f/64 scope link noprefixroute valid_lft forever preferred_lft forever
- 在node02
[root@node02 sbin]# /usr/local/nginx/sbin/nginx [root@node02 sbin]# /etc/init.d/keepalived start Starting keepalived (via systemctl): [ OK ] [root@node02 sbin]# ip add 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo valid_lft forever preferred_lft forever inet6 ::1/128 scope host valid_lft forever preferred_lft forever 2: ens33: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000 link/ether 00:0c:29:54:a6:67 brd ff:ff:ff:ff:ff:ff inet 192.168.146.12/24 brd 192.168.146.255 scope global noprefixroute ens33 valid_lft forever preferred_lft forever inet6 fe80::6d51:9e88:4627:2640/64 scope link noprefixroute valid_lft forever preferred_lft forever
- 观察运行 命令 ip add 后的结果
查看VIP是否被绑定到100机器上,可以看到192.168.146.100/32 scope global ens33已经被绑定到node01机器上了
inet 192.168.146.100/32 scope global ens33 valid_lft forever preferred_lft forever
- kill掉 node01 上的 keepalived 和 nginx 进程 , 192.168.146.100/32 scope global ens33 没有被绑定在node01机器上了
[root@node01 keepalived]# killall keepalived [root@node01 keepalived]# killall nginx [root@node01 keepalived]# ip add 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo valid_lft forever preferred_lft forever inet6 ::1/128 scope host valid_lft forever preferred_lft forever 2: ens33: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000 link/ether 00:0c:29:4b:2d:2d brd ff:ff:ff:ff:ff:ff inet 192.168.146.11/24 brd 192.168.146.255 scope global noprefixroute ens33 valid_lft forever preferred_lft forever inet6 fe80::6d51:9e88:4627:2640/64 scope link tentative noprefixroute dadfailed valid_lft forever preferred_lft forever inet6 fe80::4794:5f00:d68b:6e4f/64 scope link noprefixroute valid_lft forever preferred_lft forever
-
在 node02 机器上运行 ip add 后
可以看到192.168.146.100/32 scope global ens33已经被绑定到 node02 机器上了 -
恢复 node01 上的 nginx 和 keepalived 进程
在 node01 运行 ip add 后 (Keepalived抢占模式的效果)
可以看到192.168.146.100/32 scope global ens33已经被绑定到 node01 机器上了
在 node02 运行 ip add 后
可以看到192.168.146.100/32 scope global ens33已经没有绑定在 node02 机器上了
- 自此 keepalived 和 nginx 高可用环境搭建成功