nginx负载均衡高可用
文章目录
什么是负载均衡高可用
nginx作为负载均衡器,所有请求都到了nginx,可见nginx处于非常重点的位置,如果nginx服务器宕机后端web服务将无法提供服务,影响严重。
为了屏蔽负载均衡服务器的宕机,需要建立一个备份机。主服务器和备份机上都运行高可用(High Availability)监控程序,通过传送诸如“I am alive”这样的信息来监控对方的运行状况。当备份机不能在一定的时间内收到这样的信息时,它就接管主服务器的服务IP并继续提供负载均衡服务;当备份管理器又从主管理器收到“I am alive”这样的信息时,它就释放服务IP地址,这样的主服务器就开始再次提供负载均衡服务。
Nginx高可用方案
目前,比较流行的实现Nginx高可用方案就是:keepalived+nginx实现主备方案。
1、什么是keepalived
keepalived是集群管理中保证集群高可用的一个服务软件,用来防止单点故障。
Keepalived的作用是检测web服务器的状态,如果有一台web服务器死机,或工作出现故障,Keepalived将检测到,并将有故障的web服务器从系统中剔除,当web服务器工作正常后Keepalived自动将web服务器加入到服务器群中,这些工作全部自动完成,不需要人工干涉,需要人工做的只是修复故障的web服务器。
2、keepalived工作原理
keepalived是以VRRP协议为实现基础的,VRRP全称Virtual Router Redundancy Protocol,即虚拟路由冗余协议。
虚拟路由冗余协议,可以认为是实现路由器高可用的协议,即将N台提供相同功能的路由器组成一个路由器组,这个组里面有一个master和多个backup,master上面有一个对外提供服务的vip(VIP = Virtual IP Address,虚拟IP地址,该路由器所在局域网内其他机器的默认路由为该vip),master会发组播,当backup收不到VRRP包时就认为master宕掉了,这时就需要根据VRRP的优先级来选举一个backup当master。这样的话就可以保证路由器的高可用了。
keepalived主要有三个模块,分别是core、check和VRRP。core模块为keepalived的核心,负责主进程的启动、维护以及全局配置文件的加载和解析。check负责健康检查,包括常见的各种检查方式。VRRP模块是来实现VRRP协议的。
nginx负载均衡配置
环境说明
主机名 | IP地址 | 服务 |
---|---|---|
master | 192.168.205.144 | keepaliced、nginx |
backup | 192.168.205.147 | keepalived、nginx |
RS1 | 192.168.205.152 | httpd |
RS2 | 192.168.205.155 | nginx |
配置RS1
//关闭防火墙与SELINUX
[root@RS1 ~]# systemctl stop firewalld
[root@RS1 ~]# systemctl disable firewalld
Removed /etc/systemd/system/multi-user.target.wants/firewalld.service.
Removed /etc/systemd/system/dbus-org.fedoraproject.FirewallD1.service.
[root@RS1 ~]# setenforce 0
[root@RS1 ~]# sed -ri 's/^(SELINUX=).*/\1disabled/g' /etc/selinux/config
//安装httpd
[root@RS1 ~]# dnf -y install httpd
//启动httpd
[root@RS1 ~]# systemctl enable --now httpd
Created symlink /etc/systemd/system/multi-user.target.wants/httpd.service → /usr/lib/systemd/system/httpd.service.
[root@RS1 ~]# ss -antl
State Recv-Q Send-Q Local Address:Port Peer Address:Port Process
LISTEN 0 128 0.0.0.0:22 0.0.0.0:*
LISTEN 0 128 *:80 *:*
LISTEN 0 128 [::]:22 [::]:*
访问
配置RS2
//关闭防火墙与SELINUX
[root@RS2 ~]# systemctl stop firewalld
[root@RS2 ~]# systemctl disable firewalld
Removed /etc/systemd/system/multi-user.target.wants/firewalld.service.
Removed /etc/systemd/system/dbus-org.fedoraproject.FirewallD1.service.
[root@RS2 ~]# setenforce 0
[root@RS2 ~]# sed -ri 's/^(SELINUX=).*/\1disabled/g' /etc/selinux/config
//安装nginx
[root@RS2 ~]# dnf -y install nginx
//启功nginx
[root@RS2 ~]# systemctl enable --now nginx
Created symlink /etc/systemd/system/multi-user.target.wants/nginx.service → /usr/lib/systemd/system/nginx.service.
[root@RS2 ~]# ss -antl
State Recv-Q Send-Q Local Address:Port Peer Address:Port Process
LISTEN 0 128 0.0.0.0:80 0.0.0.0:*
LISTEN 0 128 0.0.0.0:22 0.0.0.0:*
LISTEN 0 128 [::]:80 [::]:*
LISTEN 0 128 [::]:22 [::]:*
访问
配置buckup
//关闭防火墙与SELINUX
[root@backup ~]# systemctl stop firewalld
[root@backup ~]# systemctl disable firewalld
Removed /etc/systemd/system/multi-user.target.wants/firewalld.service.
Removed /etc/systemd/system/dbus-org.fedoraproject.FirewallD1.service.
[root@backup ~]# setenforce 0
[root@backup ~]# sed -ri 's/^(SELINUX=).*/\1disabled/g' /etc/selinux/config
//安装nginx
[root@backup ~]# dnf -y install nginx
[root@backup ~]# vim /etc/nginx/nginx.conf
...........................
upstream webservers {
server 192.168.205.152 weight=2;
server 192.168.205.155;
}
server {
listen 80;
server_name _;
root /usr/share/nginx/html;
# Load configuration files for the default server block.
include /etc/nginx/default.d/*.conf;
location / {
proxy_pass http://webservers;
}
//启动nginx
[root@backup ~]# systemctl enable --now nginx
Created symlink /etc/systemd/system/multi-user.target.wants/nginx.service → /usr/lib/systemd/system/nginx.service.
[root@backup ~]# ss -antl
State Recv-Q Send-Q Local Address:Port Peer Address:Port Process
LISTEN 0 128 0.0.0.0:80 0.0.0.0:*
LISTEN 0 128 0.0.0.0:22 0.0.0.0:*
LISTEN 0 128 [::]:22 [::]:*
用backup访问RS1和RS2
nginx负载均衡高可用
配置主
//关闭防火墙与SELINUX
[root@master ~]# systemctl stop firewalld
[root@master ~]# systemctl disable firewalld
Removed /etc/systemd/system/multi-user.target.wants/firewalld.service.
Removed /etc/systemd/system/dbus-org.fedoraproject.FirewallD1.service.
[root@master ~]# setenforce 0
[root@master ~]# sed -ri 's/^(SELINUX=).*/\1disabled/g' /etc/selinux/config
//安装nginx并启动
[root@master ~]# dnf -y install nginx
[root@master ~]# systemctl enable --now nginx
Created symlink /etc/systemd/system/multi-user.target.wants/nginx.service → /usr/lib/systemd/system/nginx.service.
[root@master ~]# ss -antl
State Recv-Q Send-Q Local Address:Port Peer Address:Port Process
LISTEN 0 128 0.0.0.0:80 0.0.0.0:*
LISTEN 0 128 0.0.0.0:22 0.0.0.0:*
LISTEN 0 128 [::]:22 [::]:*
//安装keepalived
[root@master ~]# dnf -y install keepalived
配置备
//安装keepalived
[root@backup ~]# dnf -y install keepalived
配置主keepalived
[root@master ~]# cd /etc/keepalived/
[root@master keepalived]# mv keepalived.conf{,.bak}
[root@master keepalived]# ls
keepalived.conf.bak
[root@master keepalived]# vim keepalived.conf
! Configuration File for keepalived
global_defs {
router_id lb01
}
vrrp_instance VI_1 {
state MASTER
interface ens33
virtual_router_id 51
priority 100
advert_int 1
authentication {
auth_type PASS
auth_pass wangqing
}
virtual_ipaddress {
192.168.205.250
}
}
virtual_server 192.168.205.250 80 {
delay_loop 6
lb_algo rr
lb_kind DR
persistence_timeout 50
protocol TCP
real_server 192.168.205.144 80 {
weight 1
TCP_CHECK {
connect_port 80
connect_timeout 3
nb_get_retry 3
delay_before_retry 3
}
}
real_server 192.168.205.147 80 {
weight 1
TCP_CHECK {
connect_port 80
connect_timeout 3
nb_get_retry 3
delay_before_retry 3
}
}
}
[root@master keepalived]# systemctl start keepalived
[root@master keepalived]# systemctl enable keepalived
Created symlink /etc/systemd/system/multi-user.target.wants/keepalived.service → /usr/lib/systemd/system/keepalived.service.
配置备keepalived
[root@backup ~]# cd /etc/keepalived/
[root@backup keepalived]# mv keepalived.conf{,.bak}
[root@backup keepalived]# ls
keepalived.conf.bak
[root@backup keepalived]# vim keepalived.conf
! Configuration File for keepalived
global_defs {
router_id lb02
}
vrrp_instance VI_1 {
state BACKUP
interface ens33
virtual_router_id 51
priority 90
advert_int 1
authentication {
auth_type PASS
auth_pass wangqing
}
virtual_ipaddress {
192.168.205.250
}
}
virtual_server 192.168.205.250 80 {
delay_loop 6
lb_algo rr
lb_kind DR
persistence_timeout 50
protocol TCP
real_server 192.168.205.144 80 {
weight 1
TCP_CHECK {
connect_port 80
connect_timeout 3
nb_get_retry 3
delay_before_retry 3
}
}
real_server 192.168.205.147 80 {
weight 1
TCP_CHECK {
connect_port 80
connect_timeout 3
nb_get_retry 3
delay_before_retry 3
}
}
}
[root@backup keepalived]# systemctl enable keepalived
Created symlink /etc/systemd/system/multi-user.target.wants/keepalived.service → /usr/lib/systemd/system/keepalived.service.
查看VIP在哪里
在MASTER上查看
[root@master keepalived]# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: ens33: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
link/ether 00:0c:29:63:98:e5 brd ff:ff:ff:ff:ff:ff
inet 192.168.205.144/24 brd 192.168.205.255 scope global dynamic noprefixroute ens33
valid_lft 1047sec preferred_lft 1047sec
inet 192.168.205.250/32 scope global ens33
valid_lft forever preferred_lft forever
inet6 fe80::9e26:e42:e48e:6000/64 scope link noprefixroute
valid_lft forever preferred_lft forever
在backup上查看
[root@backup keepalived]# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: ens33: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
link/ether 00:0c:29:6a:fe:0a brd ff:ff:ff:ff:ff:ff
inet 192.168.205.147/24 brd 192.168.205.255 scope global dynamic noprefixroute ens33
valid_lft 1017sec preferred_lft 1017sec
inet6 fe80::9e26:e42:e48e:6000/64 scope link noprefixroute
valid_lft forever preferred_lft forever
让keepalived监控nginx负载均衡机
keepalived通过脚本来监控nginx负载均衡机的状态
在master上编写脚本
[root@master ~]# mkdir /scripts
[root@master ~]# cd /scripts/
[root@master scripts]# vim check_nginx.sh
#!/bin/bash
nginx=$(ps -A|grep -Ev "grep|$0" |grep '\bnginx\b'|wc -l)
if [ $nginx -lt 1 ];then
systemctl stop keepalived
fi
[root@master scripts]# chmod +x check_nginx.sh
[root@master scripts]# ll
total 4
-rwxr-xr-x. 1 root root 129 Oct 17 19:34 check_nginx.sh
[root@master scripts]# vim notify.sh
#!/bin/bash
case "$1" in
master)
nginx=$(ps -A|grep -Ev "grep|$0" |grep '\bnginx\b'|wc -l)
if [ $nginx -lt 1 ];then
systemctl start nginx.service
fi
;;
backup)
nginx=$(ps -A|grep -Ev "grep|$0" |grep '\bnginx\b'|wc -l)
if [ $nginx -gt 0 ];then
systemctl stop nginx.service
fi
;;
*)
echo "Usage:$0 master|backup VIP"
;;
esac
[root@master scripts]# chmod +x notify.sh
[root@master scripts]# ll
total 8
-rwxr-xr-x. 1 root root 129 Oct 17 19:34 check_nginx.sh
-rwxr-xr-x. 1 root root 407 Oct 17 19:43 notify.sh
在backup上编写脚本
[root@backup ~]# mkdir /scripts
[root@backup ~]# cd /scripts/
[root@backup scripts]# cat check_nginx.sh
#!/bin/bash
nginx=$(ps -A|grep -Ev "grep|$0" |grep '\bnginx\b'|wc -l)
if [ $nginx -lt 1 ];then
systemctl stop keepalived
fi
[root@backup scripts]# chmod +x check_nginx.sh
[root@backup scripts]# cat notify.sh
#!/bin/bash
case "$1" in
master)
nginx=$(ps -A|grep -Ev "grep|$0" |grep '\bnginx\b'|wc -l)
if [ $nginx -lt 1 ];then
systemctl start nginx.service
fi
;;
backup)
nginx=$(ps -A|grep -Ev "grep|$0" |grep '\bnginx\b'|wc -l)
if [ $nginx -gt 0 ];then
systemctl stop nginx.service
fi
;;
*)
echo "Usage:$0 master|backup VIP"
;;
esac
[root@backup scripts]# chmod +x notify.sh
配置keepalived加入监控脚本的配置
配置主keepalived
[root@master ~]# vim /etc/keepalived/keepalived.conf
! Configuration File for keepalived
global_defs {
router_id lb01
}
vrrp_script nginx_check {
script "/scripts/check_nginx.sh"
interval 1
weight -20
}
vrrp_instance VI_1 {
state MASTER
interface ens33
virtual_router_id 51
priority 100
advert_int 1
authentication {
auth_type PASS
auth_pass wangqing
}
virtual_ipaddress {
192.168.205.250
}
track_script {
nginx_check
}
notify_master "/scripts/notify.sh master"
notify_backup "/scripts/notify.sh backup"
}
virtual_server 192.168.205.250 80 {
delay_loop 6
lb_algo rr
lb_kind DR
persistence_timeout 50
protocol TCP
real_server 192.168.205.144 80 {
weight 1
TCP_CHECK {
connect_port 80
connect_timeout 3
nb_get_retry 3
delay_before_retry 3
}
}
real_server 192.168.205.147 80 {
weight 1
TCP_CHECK {
connect_port 80
connect_timeout 3
nb_get_retry 3
delay_before_retry 3
}
}
}
[root@master ~]# systemctl restart keepalived
配置备keepalived
[root@backup ~]# vim /etc/keepalived/keepalived.conf
! Configuration File for keepalived
global_defs {
router_id lb02
}
vrrp_instance VI_1 {
state BACKUP
interface ens33
virtual_router_id 51
priority 90
advert_int 1
authentication {
auth_type PASS
auth_pass wangqing
}
virtual_ipaddress {
192.168.205.250
}
notify_master "/scripts/notify.sh master"
notify_backup "/scripts/notify.sh backup"
}
virtual_server 192.168.205.250 80 {
delay_loop 6
lb_algo rr
lb_kind DR
persistence_timeout 50
protocol TCP
real_server 192.168.205.144 80 {
weight 1
TCP_CHECK {
connect_port 80
connect_timeout 3
nb_get_retry 3
delay_before_retry 3
}
}
real_server 192.168.205.147 80 {
weight 1
TCP_CHECK {
connect_port 80
connect_timeout 3
nb_get_retry 3
delay_before_retry 3
}
}
}
[root@backup ~]# systemctl restart keepalived