keepalived高可用方案部署
keepalived高可用软件
Keepalived软件起初是专为LVS负载均衡软件设计的,用来管理并监控LVS集群系统中各个服务节点的状态,后来又加入了可以实现高可用的VRRP功能。因此,keepalived除了能够管理LVS软件外,还可以作为其他服务的高可用解决方案软件。
keepalived软件主要是通过VRRP协议实现高可用功能的。VRRP是Virtual Router Redundancy Protocol(虚拟路由冗余协议)的缩写,VRRP出现的目的就是为了解决静态路由的单点故障问题的,它能保证当个别节点宕机时,整个网络可以不间断地运行。所以,keepalived一方面具有配置管理LVS的功能,同时还具有对LVS下面节点进行健康检查的功能,另一方面也可以实现系统网络服务的高可用功能。
keepalived高可用故障切换转移原理
Keepalived高可用服务对之间的故障切换转移,是通过VRRP来实现的。在keepalived服务工作时,主Master节点会不断地向备节点发送(多播的方式)心跳消息,用来告诉备Backup节点自己还活着。当主节点发生故障时,就无法发送心跳的消息了,备节点也因此无法继续检测到来自主节点的心跳了。于是就会调用自身的接管程序,接管主节点的IP资源和服务。当主节点恢复时,备节点又会释放主节点故障时自身接管的IP资源和服务,恢复到原来的备用角色。
高可用部署架构
- 架构
用户 ----> 浏览器 --172.30.10.10--> nginx(代理:172.30.10.10:80,指向业务VIP)
--> keepalived+web服务器1(VIP:172.30.10.11,172.30.10.12)
--> keepalived+web服务器2(VIP:172.30.10.11,172.30.10.12)
+ nginx作为反向代理,将用户访问的172.30.10.10地址代理到对于keepalived生成的虚拟IP上(172.30.10.11, 172.30.10.12)
+ keepalived+web服务器1,keepalived生成虚拟IP:172.30.10.11(主),172.30.10.12(备),web服务器即业务服务,既可以是nginx、也可以是foreignserver等其它web服务
+ keepalived+web服务器2,keepalived生成虚拟IP:172.30.10.11(备),172.30.10.12(主),web服务器即业务服务,既可以是nginx、也可以是foreignserver等其它web服务
安装并启用keepalived
- 安装
yum install keepalived -y
- 启用keepalived,并加入到开机启动的脚本
/etc/init.d/keepalived start
echo "/etc/init.d/keepalived start" >>/etc/rc.local
修改keepalived配置文件并且重启keepalived服务
/etc/init.d/keepalived stop #关闭keepalived服务
vim /etc/keepalived/keepalived.conf #用vim打开编辑
部署示例
前置nginx配置
user nginx;
worker_processes auto;
worker_cpu_affinity auto;
worker_rlimit_nofile 1024000;
error_log /var/log/nginx/error.log warn;
pid /var/run/nginx.pid;
events {
worker_connections 10240;
}
http {
include /etc/nginx/mime.types;
default_type application/octet-stream;
log_format main '[$time_local] $remote_addr $remote_port "$request" '
'$status $body_bytes_sent $request_time '
'$upstream_addr $upstream_response_time';
# access_log /var/log/nginx/access.log main;
access_log off;
sendfile on;
keepalive_timeout 300;
#include /etc/nginx/conf.d/*.conf;
upstream iot_cluster{
server 172.30.10.11:80 max_fails=5 fail_timeout=60s;
server 172.30.10.12:80 max_fails=5 fail_timeout=60s;
keepalive 300;
}
server {
listen 80;
listen [::]:80 ipv6only=on;
server_name iot_cluster;
keepalive_timeout 75;
client_body_buffer_size 128k;
proxy_max_temp_file_size 0;
location / {
proxy_pass http://iot_cluster;
proxy_http_version 1.1;
proxy_redirect off;
proxy_pass_header Server;
proxy_set_header Connection "";
proxy_set_header Host $http_host;
proxy_set_header x-real-ip $remote_addr;
proxy_set_header x-real-port $remote_port;
proxy_set_header X-Scheme $scheme;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_read_timeout 300;
recursive_error_pages on;
}
}
}
keepalived+web服务器1配置
- keepalived 配置
! Configuration File for keepalived
global_defs {
}
vrrp_script chk_nginx {
script "/etc/keepalived/nginx.sh"
interval 1
}
vrrp_instance VI_1 {
state MASTER
interface ens33
virtual_router_id 46
priority 100
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
172.16.10.11/24
}
track_script {
chk_nginx
}
}
vrrp_instance VI_2 {
state BACKUP
interface ens33
virtual_router_id 47
priority 100
advert_int 1
authentication {
auth_type PASS
auth_pass 2222
}
virtual_ipaddress {
172.16.10.12/24
}
track_script {
chk_nginx
}
}
- 检测脚本配置
#!/bin/bash
if [ $(netstat -tlnp|grep nginx|wc -l) -ne 1 ]
then
/etc/init.d/keepalived stop
fi
keepalived+web服务器2配置
- keepalived 配置
! Configuration File for keepalived
global_defs {
}
vrrp_script chk_nginx {
script "/etc/keepalived/nginx.sh"
interval 1
}
vrrp_instance VI_1 {
state MASTER
interface ens33
virtual_router_id 46
priority 100
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
172.16.10.11/24
}
track_script {
chk_nginx
}
}
vrrp_instance VI_2 {
state BACKUP
interface ens33
virtual_router_id 47
priority 100
advert_int 1
authentication {
auth_type PASS
auth_pass 2222
}
virtual_ipaddress {
172.16.10.12/24
}
track_script {
chk_nginx
}
}
- 检测脚本配置
#!/bin/bash
if [ $(netstat -tlnp|grep nginx|wc -l) -ne 1 ]
then
/etc/init.d/keepalived stop
fi
进行主备切换实验
- 正常运行两个节点,可以看到11的节点有虚拟IP:172.16.10.11,可以看到12的节点有虚拟IP:172.16.10.12
- 节点1状态
[root@localhost ~]# ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: ens33: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
link/ether 00:0c:29:6d:e4:84 brd ff:ff:ff:ff:ff:ff
inet 172.16.10.8/24 brd 172.16.10.255 scope global noprefixroute dynamic ens33
valid_lft 86329sec preferred_lft 86329sec
inet 172.16.10.11/24 scope global secondary ens33
valid_lft forever preferred_lft forever
inet6 fe80::e839:bf7:465:d378/64 scope link noprefixroute
valid_lft forever preferred_lft forever
+ 节点2状态
[root@localhost ~]# ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: ens33: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
link/ether 00:0c:29:46:8b:88 brd ff:ff:ff:ff:ff:ff
inet 172.16.10.9/24 brd 172.16.10.255 scope global noprefixroute dynamic ens33
valid_lft 86321sec preferred_lft 86321sec
inet 172.16.10.12/24 scope global secondary ens33
valid_lft forever preferred_lft forever
inet6 fe80::e839:bf7:465:d378/64 scope link tentative noprefixroute dadfailed
valid_lft forever preferred_lft forever
inet6 fe80::608:f4ad:c8c8:f19/64 scope link noprefixroute
valid_lft forever preferred_lft forever
- 停掉节点1上的nginx服务,可以看到节点1中的172.16.10.11已经取消,然后漂移到节点2上
- 节点1
[root@localhost ~]# ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: ens33: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
link/ether 00:0c:29:6d:e4:84 brd ff:ff:ff:ff:ff:ff
inet 172.16.10.8/24 brd 172.16.10.255 scope global noprefixroute dynamic ens33
valid_lft 85562sec preferred_lft 85562sec
inet6 fe80::e839:bf7:465:d378/64 scope link noprefixroute
valid_lft forever preferred_lft forever
+ 节点2
[root@localhost ~]# ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: ens33: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
link/ether 00:0c:29:46:8b:88 brd ff:ff:ff:ff:ff:ff
inet 172.16.10.9/24 brd 172.16.10.255 scope global noprefixroute dynamic ens33
valid_lft 85533sec preferred_lft 85533sec
inet 172.16.10.11/24 scope global secondary ens33
valid_lft forever preferred_lft forever
inet 172.16.10.12/24 scope global secondary ens33
valid_lft forever preferred_lft forever
inet6 fe80::e839:bf7:465:d378/64 scope link tentative noprefixroute dadfailed
valid_lft forever preferred_lft forever
inet6 fe80::608:f4ad:c8c8:f19/64 scope link noprefixroute
valid_lft forever preferred_lft forever
- 启动节点1上的nginx服务,节点1重新抢占到172.16.10.11
- 节点1状态
[root@localhost ~]# ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: ens33: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
link/ether 00:0c:29:6d:e4:84 brd ff:ff:ff:ff:ff:ff
inet 172.16.10.8/24 brd 172.16.10.255 scope global noprefixroute dynamic ens33
valid_lft 86329sec preferred_lft 86329sec
inet 172.16.10.11/24 scope global secondary ens33
valid_lft forever preferred_lft forever
inet6 fe80::e839:bf7:465:d378/64 scope link noprefixroute
valid_lft forever preferred_lft forever
+ 节点2状态
[root@localhost ~]# ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: ens33: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
link/ether 00:0c:29:46:8b:88 brd ff:ff:ff:ff:ff:ff
inet 172.16.10.9/24 brd 172.16.10.255 scope global noprefixroute dynamic ens33
valid_lft 86321sec preferred_lft 86321sec
inet 172.16.10.12/24 scope global secondary ens33
valid_lft forever preferred_lft forever
inet6 fe80::e839:bf7:465:d378/64 scope link tentative noprefixroute dadfailed
valid_lft forever preferred_lft forever
inet6 fe80::608:f4ad:c8c8:f19/64 scope link noprefixroute
valid_lft forever preferred_lft forever
keepalived配置说明
- global_defs 全局配置,全局配置以配置邮件通知为主
- vrrp_instance VRRP配置实例,可配置多个
- state #角色类型MASTER|BACKUP,MASTER主角色,BACKUP备份角色
- interface #网卡名称
- virtual_router_id #虚拟路由id,同一个VRRP实例,必须保证虚拟路由id保持一致
- priority #优先级,0-100之间,如果采用MASTER+BACKUP模式时,MASTER的优先级高于BACKUP时,则为抢占式
- advert_int #每一秒检查一次
- authentication #认证信息
- auth_type PASS #认证类型 主备之间必须一样
- auth_pass 2222 #认证密码 主备之间必须一样
- virtual_ipaddress #虚拟IP,vip
- track_script #检查脚本
参考:https://zhuanlan.zhihu.com/p/108577218
参考:https://www.cnblogs.com/fengzi7314/p/9912101.html