集群介绍, keepalived配置高可用集群, 基于nginx服务

Linux集群概述

  • 根据功能划分为两大类:高可用和负载均衡
  • 高可用集群通常为两台服务器,一台工作,另外一台作为冗余,当提供服务的机器宕机,冗余将接替继续提供服务
  • 实现高可用的开源软件有:heartbeat、keepalived
  • 负载均衡集群,需要有一台服务器作为分发器,它负责把用户的请求分发给后端的服务器处理,在这个集群里,除了分发器外,就是给用户提供服务的服务器了,这些服务器数量至少为2
  • 实现负载均衡的开源软件有LVS、keepalived、haproxy、nginx,商业的有F5、Netscaler

keepalived介绍

  • 在这里我们使用keepalived来实现高可用集群,因为heartbeat在centos6上有一些问题,影响实验效果
  • keepalived通过VRRP(Virtual Router Redundancy Protocl)来实现高可用。
  • 在这个协议里会将多台功能相同的路由器组成一个小组,这个小组里会有1个master角色和N(N>=1)个backup角色。
  • master会通过组播的形式向各个backup发送VRRP协议的数据包,当backup收不到master发来的VRRP数据包时,就会认为master宕机了。此时就需要根据各个backup的优先级来决定谁成为新的mater。
  • Keepalived要有三个模块,分别是core、check和vrrp。其中core模块为keepalived的核心,负责主进程的启动、维护以及全局配置文件的加载和解析,check模块负责健康检查,vrrp模块是来实现VRRP协议的。

用keepalived配置高可用

  • 配置是针对nginx服务,两台服务器分别安装nginx和keepalived,实现master机上的nginx服务不正常,启动backup机上的nginx;
  • 对外使用VIP实现服务,keepalived根据机器情况(脚本配置)把VIP配置给其中一个服务器;
  • 测试:两个服务器的真实IP显示不同内容;连接VIP显示的内容;master出现问题时,连接VIP显示的内容;master恢复,连接VIP显示的内容;相关的日志信息;
  • 主机MASTER配置
  1. 安装nginx,可以yum安装,也可以编译安装,安装后实现以下状态
[root@draft ~]# curl -x127.0.0.1:8080 localhost  #编缉虚拟主机文件,显示如下信息;
master server in use.
  1. 安装和配置keepalived
yum install -y keepalived
vi /etc/keepalived/keepalived.conf  #写入以下内容;

global_defs {
   notification_email {			#这一部分用于发送通知邮件,可忽略;
     aming@aminglinux.com
   }
   notification_email_from root@aminglinux.com
   smtp_server 127.0.0.1
   smtp_connect_timeout 30
   router_id LVS_DEVEL
}

vrrp_script chk_nginx {
    script "/usr/local/sbin/check_ng.sh"		#检查服务是否启动的脚本;
    interval 3			#检测间隔;
}	

vrrp_instance VI_1 {		#定义一个虚拟路由;
    state MASTER		
    #当前节点在此此虚拟路由器上的初始状态,只能有一个是MASTER,余下的都应该为BACKUP,启动后根据priority竞选;
    interface ens33			#监听的实际网口;
    virtual_router_id 51	#组播ID;
    priority 100				#主机即本机的权重;
    advert_int 1				#发送组播包的间隔时间,默认为1秒;
    authentication {		#认证配置;
        auth_type PASS
        auth_pass aminglinux>com
    }
    virtual_ipaddress {		#指定漂移地址;
        192.168.87.222
    }

    track_script {				
        chk_nginx
    }

}
vi /usr/local/sbin/check_ng.sh  #写入以下内容;

#!/bin/bash
#时间变量,用于记录日志
d=`date --date today +%Y%m%d_%H:%M:%S`
#计算nginx进程数量
n=`ps -C nginx --no-heading|wc -l`
#如果进程为0,则启动nginx,并且再次检测nginx进程数量,
#如果还为0,说明nginx无法启动,此时需要关闭keepalived,释放VIP,同时所有BACKUP就收不到MASTER的信息,BACKUP根据优先级获取VIP,提供服务;
if [ $n -eq "0" ]; then
        systemctl start nginx #安装方式不一样,启动方式不一样;
        n2=`ps -C nginx --no-heading|wc -l`
        if [ $n2 -eq "0"  ]; then
                echo "$d nginx down,keepalived will stop" >> /var/log/check_ng.log
                systemctl stop keepalived
        fi
fi

chmod 755 /usr/local/sbin/check_ng.sh  #要修改脚本权限,不然脚本不能使用;
systemctl start keepalived  #启动服务;
  1. 单机检查:
    检查服务进程:
[root@draft ~]# ps aux |grep keepalived
root      22501  0.0  0.1 122888  1412 ?        Ss   16:04   0:00 /usr/sbin/keepalived -D
root      22502  0.0  0.3 133848  3340 ?        S    16:04   0:00 /usr/sbin/keepalived -D
root      22503  0.0  0.2 133788  2900 ?        S    16:04   0:00 /usr/sbin/keepalived -D
root      25218  0.0  0.0 112728   980 pts/0    S+   16:27   0:00 grep --color=auto keepalived
[root@draft ~]# ps aux |grep nginx			#关闭也会由keepalived启动;
root      22417  0.0  0.0  46344   972 ?        Ss   16:04   0:00 nginx: master process nginx
nginx     22421  0.0  0.2  46752  2168 ?        S    16:04   0:00 nginx: worker process
root      25236  0.0  0.0 112728   968 pts/0    R+   16:27   0:00 grep --color=auto nginx

[root@draft ~]# curl -x192.168.87.222:8080 192.168.87.222
master server in use.
  • 从机BACKUP配置
  1. 安装nginx,可以yum安装,也可以编译安装,安装后实现以下状态
[root@second ~]# curl -x127.0.0.1:8080 localhost   #编缉虚拟主机文件,显示如下信息;
back up server in use.
  1. 安装和配置keepalived
yum install -y keepalived
vi /etc/keepalived/keepalived.conf  #写入以下内容;

global_defs {
   notification_email {			#这一部分用于发送通知邮件,可忽略;
     aming@aminglinux.com
   }
   notification_email_from root@aminglinux.com
   smtp_server 127.0.0.1
   smtp_connect_timeout 30
   router_id LVS_DEVEL
}

vrrp_script chk_nginx {
    script "/usr/local/sbin/check_ng.sh"		#检查服务是否启动的脚本;
    interval 3			#检测间隔;
}	

vrrp_instance VI_1 {		#定义一个虚拟路由;
    state BACKUP		
    #当前节点在此此虚拟路由器上的初始状态,只能有一个是MASTER,余下的都应该为BACKUP,启动后根据priority竞选;
    interface ens33			#监听的实际网口;
    virtual_router_id 51	#组播ID;
    priority 90				#从机即本机的权重;
    advert_int 1				#发送组播包的间隔时间,默认为1秒;
    authentication {		#认证配置;
        auth_type PASS
        auth_pass aminglinux>com
    }
    virtual_ipaddress {		#指定漂移地址;
        192.168.87.222
    }

    track_script {				
        chk_nginx
    }

}
vi /usr/local/sbin/check_ng.sh  #跟主机的检测脚本一样;

chmod 755 /usr/local/sbin/check_ng.sh  #要修改脚本权限,不然脚本不能使用;
systemctl start keepalived  #启动服务;
  1. 单机检查:
    检查服务进程:
[root@second ~]# ps aux | grep keepalived
root      32973  0.0  0.1 122888  1408 ?        Ss   16:40   0:00 /usr/sbin/keepalived -D
root      32974  0.0  0.2 127088  2420 ?        S    16:40   0:00 /usr/sbin/keepalived -D
root      32975  0.0  0.2 127088  2452 ?        S    16:40   0:00 /usr/sbin/keepalived -D
root      32991  0.0  0.0 112728   980 pts/1    R+   16:40   0:00 grep --color=auto keepalived
[root@second ~]# ps aux | grep nginx
root      32928  0.0  0.0  46344   976 ?        Ss   16:21   0:00 nginx: master process /usr/sbin/nginx -c /etc/nginx/nginx.conf
nginx     32929  0.0  0.2  46752  2168 ?        S    16:21   0:00 nginx: worker process
root      32993  0.0  0.0 112728   968 pts/1    R+   16:40   0:00 grep --color=auto nginx
[root@second ~]# pkill nginx				
[root@second ~]# ps aux | grep nginx			#杀死后,会再次启动nginx;
root      33010  0.0  0.0  46344   968 ?        Ss   16:40   0:00 nginx: master process nginx
nginx     33014  0.0  0.1  46752  1924 ?        S    16:40   0:00 nginx: worker process
root      33016  0.0  0.0 112728   972 pts/1    R+   16:40   0:00 grep --color=auto nginx
[root@second ~]# curl -x192.168.87.222:8080 192.168.87.222		#从机上访问VIP;
master server in use.
  • 将nginx程序转移位置后,检测脚本的日志情况
[root@second ~]# mv /usr/sbin/nginx /usr/sbin/nginx.bak
[root@second ~]# pkill nginx  #杀死后,keepalived不能再将nginx启动,脚本停止keepalived;
[root@second ~]# tail /var/log/check_ng.log 
20191107_16:49:06 nginx down,keepalived will stop

综合测试

[root@draft ~]# systemctl stop keepalived		#MASTER停止keepalived,BACKUP将会提供服务;
[root@draft ~]# !curl
curl -x192.168.87.222:8080 192.168.87.222
back up server in use.
[root@second ~]# ip addr
2: ens33: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
    link/ether 00:0c:29:97:47:ed brd ff:ff:ff:ff:ff:ff
    inet 192.168.87.150/24 brd 192.168.87.255 scope global dynamic ens33
       valid_lft 1519sec preferred_lft 1519sec
    inet 192.168.87.222/32 scope global ens33			#VIP在BACKUP中使用;
       valid_lft forever preferred_lft forever
    inet6 fe80::50a5:f454:5189:9942/64 scope link noprefixroute 
       valid_lft forever preferred_lft forever
MASTER日志:
Nov  7 16:52:42 draft Keepalived[22501]: Stopping				#停止服务的细节;
Nov  7 16:52:42 draft systemd: Stopping LVS and VRRP High Availability Monitor...
Nov  7 16:52:42 draft Keepalived_vrrp[22503]: VRRP_Instance(VI_1) sent 0 priority
Nov  7 16:52:42 draft Keepalived_vrrp[22503]: VRRP_Instance(VI_1) removing protocol VIPs.		#释放VIP;
Nov  7 16:52:42 draft Keepalived_healthcheckers[22502]: Stopped
Nov  7 16:52:43 draft Keepalived_vrrp[22503]: Stopped
Nov  7 16:52:43 draft Keepalived[22501]: Stopped Keepalived v1.3.5 (03/19,2017), git commit v1.3.5-6-g6fa32f2
Nov  7 16:52:43 draft systemd: Stopped LVS and VRRP High Availability Monitor.

BACKUP日志:
Nov  7 16:52:42 draft Keepalived_vrrp[34156]: VRRP_Instance(VI_1) Transition to MASTER STATE
Nov  7 16:52:43 draft Keepalived_vrrp[34156]: VRRP_Instance(VI_1) Entering MASTER STATE		#BACKUP进入服务状态;
Nov  7 16:52:43 draft Keepalived_vrrp[34156]: VRRP_Instance(VI_1) setting protocol VIPs.			#设置VIPs;
Nov  7 16:52:43 draft Keepalived_vrrp[34156]: Sending gratuitous ARP on ens33 for 192.168.87.222
Nov  7 16:52:43 draft Keepalived_vrrp[34156]: VRRP_Instance(VI_1) Sending/queueing gratuitous ARPs on ens33 for 192.168.87.222
Nov  7 16:52:43 draft Keepalived_vrrp[34156]: Sending gratuitous ARP on ens33 for 192.168.87.222
[root@draft ~]# systemctl start keepalived			#MASTER恢复服务;
[root@draft ~]# curl -x192.168.87.222:8080 192.168.87.222
master server in use.
[root@draft ~]# ip addr
2: ens33: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
    link/ether 00:0c:29:bd:e9:03 brd ff:ff:ff:ff:ff:ff
    inet 192.168.87.149/24 brd 192.168.87.255 scope global noprefixroute ens33
       valid_lft forever preferred_lft forever
    inet 192.168.87.222/32 scope global ens33			#VIP在MASTER中使用;
       valid_lft forever preferred_lft forever
    inet6 fe80::d8da:c84:7947:a438/64 scope link noprefixroute 
       valid_lft forever preferred_lft forever
MASTER日志:
Nov  7 17:03:50 draft systemd: Starting LVS and VRRP High Availability Monitor...		#启动过程...
Nov  7 17:03:50 draft Keepalived[28407]: Starting Keepalived v1.3.5 (03/19,2017), git commit v1.3.5-6-g6fa32f2
Nov  7 17:03:50 draft Keepalived[28407]: Opening file '/etc/keepalived/keepalived.conf'.
Nov  7 17:03:50 draft systemd: PID file /var/run/keepalived.pid not readable (yet?) after start.
Nov  7 17:03:50 draft Keepalived[28408]: Starting Healthcheck child process, pid=28409
Nov  7 17:03:50 draft Keepalived[28408]: Starting VRRP child process, pid=28410
Nov  7 17:03:50 draft systemd: Started LVS and VRRP High Availability Monitor.
Nov  7 17:03:50 draft Keepalived_healthcheckers[28409]: Opening file '/etc/keepalived/keepalived.conf'.
Nov  7 17:03:50 draft Keepalived_vrrp[28410]: Registering Kernel netlink reflector
Nov  7 17:03:50 draft Keepalived_vrrp[28410]: Registering Kernel netlink command channel
Nov  7 17:03:50 draft Keepalived_vrrp[28410]: Registering gratuitous ARP shared channel
Nov  7 17:03:50 draft Keepalived_vrrp[28410]: Opening file '/etc/keepalived/keepalived.conf'.
Nov  7 17:03:50 draft Keepalived_vrrp[28410]: WARNING - default user 'keepalived_script' for script execution does not exist - please create.
Nov  7 17:03:50 draft Keepalived_vrrp[28410]: Truncating auth_pass to 8 characters
Nov  7 17:03:50 draft Keepalived_vrrp[28410]: SECURITY VIOLATION - scripts are being executed but script_security not enabled.
Nov  7 17:03:50 draft Keepalived_vrrp[28410]: VRRP_Instance(VI_1) removing protocol VIPs.
Nov  7 17:03:50 draft Keepalived_vrrp[28410]: Using LinkWatch kernel netlink reflector...
Nov  7 17:03:50 draft Keepalived_vrrp[28410]: VRRP sockpool: [ifindex(2), proto(112), unicast(0), fd(10,11)]
接上
Nov  7 17:03:50 draft Keepalived_vrrp[28410]: VRRP_Script(chk_nginx) succeeded
Nov  7 17:03:51 draft Keepalived_vrrp[28410]: VRRP_Instance(VI_1) Transition to MASTER STATE
Nov  7 17:03:52 draft Keepalived_vrrp[28410]: VRRP_Instance(VI_1) Entering MASTER STATE				#进入MASTER状态;
Nov  7 17:03:52 draft Keepalived_vrrp[28410]: VRRP_Instance(VI_1) setting protocol VIPs.					#设置VIPs;
Nov  7 17:03:52 draft Keepalived_vrrp[28410]: Sending gratuitous ARP on ens33 for 192.168.87.222
Nov  7 17:03:52 draft Keepalived_vrrp[28410]: VRRP_Instance(VI_1) Sending/queueing gratuitous ARPs on ens33 for 192.168.87.222
Nov  7 17:03:52 draft Keepalived_vrrp[28410]: Sending gratuitous ARP on ens33 for 192.168.87.222
BACKUP日志:
Nov  7 17:03:51 draft Keepalived_vrrp[34156]: VRRP_Instance(VI_1) Received advert with higher priority 100, ours 90		#收到更高优先级的信息;
Nov  7 17:03:51 draft Keepalived_vrrp[34156]: VRRP_Instance(VI_1) Entering BACKUP STATE			#进入BACKUP状态;
Nov  7 17:03:51 draft Keepalived_vrrp[34156]: VRRP_Instance(VI_1) removing protocol VIPs.
  • 建议重装服务后,本例是nginx,要重装keepalived,可能会检查在用的服务;重装nginx后,不重装keepalived使用不正常;
  • 脑裂

脑裂是因为cluster分裂导致的,cluster集群中节点因为处理器忙或者其他原因暂时停止响应时,其他节点可能误认为该节点“已死”,从而夺取共享磁盘(即资源)的访问权,此时极有可能假死节点重新对共享文件系统产生读写操作,从而导致共享磁盘文件系统损坏。

评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值