Linux安装Keepalived和配置Nginx主从热备

Linux安装Keepalived和配置Nginx主从热备

版本介绍

1. Linux: CentOS 7
2. keepalived: keepalived-2.0.6.tar.gz
3. nginx: nginx-1.14.0.tar.gz
4. 环境描述: 本次测试是在在本电脑中安装的VMware软件中的两台虚拟机, ip地址分别为192.168.10.129和192.168.10.196
5. 参考文档: 安装参考 配置参考1 配置参考2
6. keepalived-2.0.6.tar.gz下载地址: https://pan.baidu.com/s/1A1aSmFqpTWzE8sEBqb9oIQ 提取码: edrq
7. nginx-1.14.0.tar.gz下载地址 https://pan.baidu.com/s/1kRvMlK7c3iBKTyMMZVpriA 提取码: xkd8

题外话

第一次尝试的时候我并不是在本地电脑中用多台虚拟机进行尝试, 而是在租借的云服务中进行的尝试, 安装步骤均一模一样, 唯独因为ip和端口的原因导致在云服务器中未成功, 因为云服务器中需要开通浮动虚拟IP的服务,不同的云服务器厂商可能有不同的方式, 后续需要继续探究, 在本地尝试的目的在于测试安装方式是否正确

本文已省略nginx的安装

1. keepalived的安装

1.1 解压安装包

我的安装包放在 /usr/local/src/ 下

[root@masteryee-one src]# tar -zxvf keepalived-2.0.6.tar.gz -C /usr/local/

1.2 编译安装

1.2.1 进入安装目录

[root@masteryee-one src]# cd /usr/local/keepalived-2.0.6/

1.2.2 指定编译时程序存放路径

[root@masteryee-one keepalived-2.0.6]# /configure --prefix=/usr/local/keepalived

注意:

如果提示　 WARNING - this build will not support IPVS with IPv6.Please install libnl/libnl-3 dev libraries to support IPv6 with IPVS.

安装以下插件

[root@masteryee-one keepalived-2.0.6]# yum -y install libnl libnl-devel

如果提示　　configure: error: libnfnetlink headers missing

安装以下插件

[root@masteryee-one keepalived-2.0.6]# yum install -y libnfnetlink-devel

再执行

[root@masteryee-one keepalived-2.0.6]# /configure --prefix=/usr/local/keepalived

1.2.3 安装

[root@masteryee-one keepalived-2.0.6]# make && make install

查看是否安装成功, 在/usr/local/ 下查看是否产生keepalived文件夹

1.3 配置开机启动服务

因为没有使用 keepalived 的默认路径安装（默认是/usr/local） ,安装完成之后，需要做一些工作复制默认配置文件到默认路径

1.3.1 在系统的 etc 下创建文件夹

[root@masteryee-one keepalived-2.0.6]# mkdir /etc/keepalived

1.3.2 复制 keepalived 服务脚本到默认的地址

[root@masteryee-one keepalived-2.0.6]# cp /usr/local/keepalived/etc/keepalived/keepalived.conf /etc/keepalived/keepalived.conf

[root@masteryee-one keepalived-2.0.6]# cp /usr/local/keepalived-2.0.6/keepalived/etc/init.d/keepalived /etc/rc.d/init.d/keepalived

[root@masteryee-one keepalived-2.0.6]# cp /usr/local/keepalived/etc/sysconfig/keepalived /etc/sysconfig/keepalived

1.3.3 配置开机启动

如果这一步没有成功也没关系

// 进入相应目录
[root@masteryee-one keepalived-2.0.6]# cd /usr/lib/systemd/system/
// 设置开机启动
[root@masteryee-one system]# systemctl enable keepalived.service
// 查看是否开机启动  enabled表示是开机启动
[root@masteryee-one system]# systemctl is-enabled keepalived.service  
enabled

2. 配置nginx的主从热备

2.1 修改 Keepalived 配置文件

配置文件解读: vrrp_script 下的 weight 的配置权重减多少, 要注意配合 vrrp_instance 下的 priority 的配置, 相当于如果主宕机后, 主节点的keepalived的优先级priority 需要减少 weight 数据, 而且减少后的priority数, 要低于从节点的priority数.

我主节点的priority是100, 从节点的priority是90, 主节点宕机后 100-20 = 80, 所以此时从节点的 priority 优先级要高

另外, 从节点没有 vrrp_instance 下的 nopreempt 这个配置

2.1.1 MASTER 节点配置文件（192.168.10.129）

[root@masteryee-one system]# # vi /etc/keepalived/keepalived.conf

配置文件详情

! Configuration File for keepalived
global_defs {
	## keepalived 自带的邮件提醒需要开启 sendmail 服务。 建议用独立的监控或第三方 SMTP
	router_id masteryee-one ## 标识本节点的字条串，通常为 hostname
} 
## keepalived 会定时执行脚本并对脚本执行的结果进行分析，动态调整 vrrp_instance 的优先级。如果脚本执行结果为 0，并且 weight 配置的值大于 0，则优先级相应的增加。如果脚本执行结果非 0，并且 weight配置的值小于 0，则优先级相应的减少。其他情况，维持原本配置的优先级，即配置文件中 priority 对应的值。
vrrp_script chk_nginx {
	script "/etc/keepalived/nginx_check.sh" ## 检测 nginx 状态的脚本路径
	interval 2 ## 检测时间间隔
	weight -20 ## 如果条件成立，权重-20
}
## 定义虚拟路由， VI_1 为虚拟路由的标示符，自己定义名称
vrrp_instance VI_1 {
	state MASTER ## 主节点为 MASTER， 对应的备份节点为 BACKUP
	interface ens33 ## 绑定虚拟 IP 的网络接口，与本机 IP 地址所在的网络接口相同， 我的是 eth1
	virtual_router_id 129 ## 虚拟路由的 ID 号， 两个节点设置必须一样， 可选 IP 最后一段使用, 相同的 VRID 为一个组，他将决定多播的 MAC 地址
	mcast_src_ip 192.168.10.129 ## 本机 IP 地址
	priority 100 ## 节点优先级， 值范围 0-254， MASTER 要比 BACKUP 高
	nopreempt ## 优先级高的设置 nopreempt 解决异常恢复后再次抢占的问题
	advert_int 1 ## 组播信息发送间隔，两个节点设置必须一样， 默认 1s
	## 设置验证信息，两个节点必须一致
	authentication {
		auth_type PASS
		auth_pass 1111 ## 真实生产，按需求对应该过来
	}
	## 将 track_script 块加入 instance 配置块
	track_script {
		chk_nginx ## 执行 Nginx 监控的服务
	} #
	# 虚拟 IP 池, 两个节点设置必须一样
	virtual_ipaddress {
		192.168.10.1 ## 虚拟 ip，可以定义多个
	}
}

2.1.2 BACKUP 节点配置文件（192.168.10.196）

[root@masteryee-one system]# vi /etc/keepalived/keepalived.conf

配置文件详情

! Configuration File for keepalived
global_defs {
	router_id masteryee-two
} 
vrrp_script chk_nginx {
	script "/etc/keepalived/nginx_check.sh"
	interval 2
	weight -20
}

vrrp_instance VI_1 {
	state BACKUP
	interface ens33
	virtual_router_id 129
	mcast_src_ip 192.168.100.196
	priority 90
	advert_int 1
	authentication {
		auth_type PASS
		auth_pass 1111
	}

	track_script {
		chk_nginx
	}

	virtual_ipaddress {
		192.168.10.1
	}
}

2.2 编写 Nginx 状态检测脚本

编写 Nginx 状态检测脚本 /etc/keepalived/nginx_check.sh (已在 keepalived.conf 中配置)脚本要求：如果 nginx 停止运行，尝试启动，如果无法启动则杀死本机的 keepalived 进程， keepalied将虚拟 ip 绑定到 BACKUP 机器上。 内容如下：

[root@masteryee-one system]# vi /etc/keepalived/nginx_check.sh

配置详情

#!/bin/bash
A=`ps -C nginx –no-header |wc -l`
if [ $A -eq 0 ];then
/usr/local/nginx/sbin/nginx
sleep 2
if [ `ps -C nginx --no-header |wc -l` -eq 0 ];then
	killall keepalived
fi
fi

保存后，给脚本赋执行权限：

[root@masteryee-one system]# chmod +x /etc/keepalived/nginx_check.sh

2.3 启动keepalived

前提是已经安装好了nginx, 并且已经启动好了nginx

启动

[root@masteryee-one ~]# systemctl start keepalived  

查看启动状态, running表示正在运行中

[root@masteryee-one ~]# systemctl status keepalived 
● keepalived.service - LVS and VRRP High Availability Monitor
   Loaded: loaded (/usr/lib/systemd/system/keepalived.service; enabled; vendor preset: disabled)
   Active: active (running) since Mon 2019-03-04 10:21:32 CST; 25s ago
  Process: 61489 ExecStart=/usr/local/keepalived/sbin/keepalived $KEEPALIVED_OPTIONS (code=exited, status=0/SUCCESS)
 Main PID: 61492 (keepalived)
   CGroup: /system.slice/keepalived.service
           ├─15138 nginx: master process /usr/local/nginx/sbin/nginx
           ├─15140 nginx: worker process
           ├─61492 /usr/local/keepalived/sbin/keepalived -D
           ├─61494 /usr/local/keepalived/sbin/keepalived -D
           ├─61538 /bin/bash /etc/keepalived/nginx_check.sh
           └─61542 /usr/local/nginx/sbin/nginx

Mar 04 10:21:41 masteryee-one Keepalived_vrrp[61494]: Sending gratuitous ARP on ens33 for 192.168.10.1
Mar 04 10:21:41 masteryee-one Keepalived_vrrp[61494]: Sending gratuitous ARP on ens33 for 192.168.10.1
Mar 04 10:21:42 masteryee-one Keepalived_vrrp[61494]: kill -15 of process /etc/keepalived/nginx_check.sh(61510) with new state 2 failed with errno 0
Mar 04 10:21:42 masteryee-one Keepalived_vrrp[61494]: Track script chk_nginx is being timed out, expect idle - skipping run
Mar 04 10:21:46 masteryee-one Keepalived_vrrp[61494]: kill -15 of process /etc/keepalived/nginx_check.sh(61515) with new state 2 failed with errno 0
Mar 04 10:21:46 masteryee-one Keepalived_vrrp[61494]: Track script chk_nginx is being timed out, expect idle - skipping run
Mar 04 10:21:50 masteryee-one Keepalived_vrrp[61494]: kill -15 of process /etc/keepalived/nginx_check.sh(61520) with new state 2 failed with errno 0
Mar 04 10:21:50 masteryee-one Keepalived_vrrp[61494]: Track script chk_nginx is being timed out, expect idle - skipping run
Mar 04 10:21:54 masteryee-one Keepalived_vrrp[61494]: kill -15 of process /etc/keepalived/nginx_check.sh(61525) with new state 2 failed with errno 0
Mar 04 10:21:54 masteryee-one Keepalived_vrrp[61494]: Track script chk_nginx is being timed out, expect idle - skipping run
[root@masteryee-one ~]# 

3. 测试

为方便查看是访问的哪一台服务器的nginx, 建议将nginx安装目录下的html下的index.html文件中添加相关醒目标识, 我是在Welcome to nginx! 后添加一个数字, 如Welcome to nginx!1 或 Welcome to nginx!2

3.1 测试主节点nginx

http://192.168.10.129/
测试是否成功启动了nginx服务

3.2 测试从节点nginx

http://192.168.10.196/
测试是否成功启动了nginx服务

3.3 测试浮动虚拟VIP

http://192.168.10.1/

此时不出意外, 访问的是主服务器上的nginx服务, 也就是页面中出现的是Welcome to nginx!1

3.3.1 停掉nginx服务

现在我将两个节点上的ngixn停掉, 他会又自动重启了nginx, 因为有nginx的check脚本会监测nginx状态, 当nginx关闭是会自动重启nginx

3.3.2 停掉主节点的keepalived

现在我将主节点上的keepalived停掉

[root@masteryee-one ~]# systemctl stop keepalived

在访问http://192.168.10.1/ 应该访问的是从服务器的nginx页面, 页面中出现的是Welcome to nginx!2

3.3.3 重新启动主节点的keepalived

现在我又将主服务器的keepalived启动

[root@masteryee-one ~]# systemctl start keepalived  

在访问http://192.168.10.1/ 应该访问的又是主服务器的nginx页面, 页面中出现的是Welcome to nginx!1