keepalived简介
keepalived是一个类似于layer3, 4 & 5交换机制的软件，Keepalived的作用是检测服务器的状态，如果有一台web服务器宕机，或工作出现故障，Keepalived将检测到，并将有故障的服务器从系统中剔除，同时使用其他服务器代替该服务器的工作，当服务器工作正常后Keepalived自动将服务器加入到服务器群中，这些工作全部自动完成，不需要人工干涉，需要人工做的只是修复故障的服务器。
环境
两台Redhat7虚拟机
server1 192.168.56.132 keepalived,nginx,tomcat 作为master节点
server2 192.168.56.133 keepalived,nginx,tomcat 作为slave节点
Firewall关闭
selinux关闭
原理
nginx本身具有反向代理的功能，用户访问的请求通过nginx代理转发到后端的tomcat，具体的负载均衡策略在nginx中进行配置
nginx+tomcat已经实现了用户请求的负载均衡，但nginx存在单点的风险，当nginx宕掉之后整个系统就会瘫痪掉，keepalived通过虚拟冗余协议实现对nginx高可用。
Keepalived 以 VRRP 协议为实现基础，用 VRRP 协议来实现高可用性(HA)。
一组VRRP路由器协同工作，共同构成一台虚拟路由器。该虚拟路由器对外表现为一个具有唯一固定的IP地址和MAC地址的逻辑路由器。处于同一个VRRP组中的路由器具有两种互斥的角色：主控路由器和备份路由器，一个VRRP组中有且只有一台处于主控角色的路由器，可以有一个或者多个处于备份角色的路由器VRRP协议从路由器组中选出一台作为主控路由器，负责ARP解析和转发IP数据包，组中的其他路由器作为备份的角色并处于待命状态，当由于某种原因主控路由器发生故障时，其中的一台备份路由器能在瞬间的时延后升级为主控路由器，由于此切换非常迅速而且不用改变IP地址和MAC地址，故对终端使用者系统是透明的。
安装JDK
可以用yum安装也可以用二进制包安装，我的虚拟机已经安装JDK就不在重复安装了。
[root@server1 bin]# java -version
java version "1.8.0_171"
Java(TM) SE Runtime Environment (build 1.8.0_171-b11)
Java HotSpot(TM) 64-Bit Server VM (build 25.171-b11, mixed mode)
注意配置Java环境变量
[root@server1 bin]# vim /etc/profile
export JAVA_HOME=/usr/local/jdk1.8.0_171
export PATH=$PATH:$JAVA_HOME/bin
安装Tomcat
1.在server1和server2上分别安装Tomcat
[root@server1 bin]# tar -zxvf apache-tomcat-8.5.46.tar.gz -C /usr/local
[root@server2 conf]# tar -zxvf apache-tomcat-8.5.46.tar.gz -C /usr/local
[root@server1 bin]# mv apache-tomcat-8.5.46/ tomcat
[root@server2 conf]# mv apache-tomcat-8.5.46/ tomcat
2.修改Tomcat的默认页面用于区分两个Tomcat
Tomcat的页面在tomcat/webapps/ROOT目录下，删除或移走文件（server2上同样操作）
[root@server1 ROOT]# pwd
/usr/local/tomcat/webapps/ROOT
[root@server1 ROOT]# mkdir tomcat
root@server1 ROOT]# mv * tomcat
在ROOT下新建index.html文件
这里配置IP进行区分两个tomcat，X-NGINX值将在后面配置NGINX时传入
[root@server1 ROOT]# vim index.jsp
<div id="asf-box">
<h1>${pageContext.servletContext.serverInfo}(192.168.56.132)<%=request.getHeader("X-NGINX")%></h1>
</div>
[root@server2 ROOT]# vim index.jsp
<div id="asf-box">
<h1>${pageContext.servletContext.serverInfo}(192.168.56.133)<%=request.getHeader("X-NGINX")%></h1>
</div>
启动tomcat1和tomcat2
[root@server1 tomcat]# bin/startup.sh
[root@server2 tomcat]# bin/startup.sh
分别访问tomcat1和tomcat2
安装nginx
1.安装编译工具及库文件（server1和server2同时安装）
yum -y install zlib zlib-devel gcc-c++ libtool openssl openssl-devel
2.下载Nginx
下载地址：http://nginx.org/download/ 选择合适的版本进行下载
[root@localhost Downloads]# ls
nginx-1.10.0.tar.gz
解压nginx至/usr/local目录下
[root@localhost Downloads]# tar -zxf nginx-1.10.0.tar.gz -C /usr/local/
3.编译安装
[root@localhost nginx-1.10.0]# ./configure --prefix=/usr/local/nginx --with-http_ssl_module
[root@localhost nginx-1.10.0]# make && make install
查看nginx版本
[root@localhost conf]# /usr/local/nginx/sbin/nginx -v
nginx version: nginx/1.10.0
4.启动nginx
[root@server1 sbin]# ./nginx
[root@server1 sbin]# ps -ef |grep nginx
root 45461 1 0 22:30 ? 00:00:00 nginx: master process ./nginx
nobody 45463 45461 0 22:30 ? 00:00:00 nginx: worker process
root 45475 25292 0 22:30 pts/0 00:00:00 grep --color=auto nginx
5.配置nginx文件，添加负载均衡
[root@server1 conf]# pwd
/usr/local/nginx/conf
[root@server1 conf]# vim nginx.conf
http {
upstream redhat {
server 192.168.56.132:8080;
server 192.168.56.133:8080;
}
include mime.types;
default_type application/octet-stream;
server {
listen 80;
server_name localhost;
}
location / {
proxy_pass http://redhat;
proxy_set_header X-NGINX "NGINX-1";
}
}
server2上同样安装nginx,我这里直接将server1的nginx复制到server2上
[root@server1 local]# scp -r nginx/ root@192.168.56.133:/usr/local/
在server1和server2的/etc/hosts文件中添加对www.123.com的解析
[root@server1 sbin]# vim /etc/hosts #server2同样步骤
192.168.56.132 www.123.com
192.168.56.133 www.123.com
6.启动nginx
[root@server1 sbin]# ./nginx
[root@server1 sbin]# netstat -antlpe |grep nginx
tcp 0 0 0.0.0.0:80 0.0.0.0:* LISTEN 0 346136 45461/nginx: master
7.访问localhost:80验证nginx负载均衡是否生效
刷新后就变成tomcat2了，说明nginx配置负载均衡已经生效
安装keepalived
1.安装keepalived
可以到官网下载tar包解压安装也可以用yum安装，这里我用yum安装
yum源配置可以参考yum源配置方法
[root@server1 ~]# yum install keepalived -y
[root@server2 ~]# yum install keepalived -y
2.修改keepalived配置文件
修改server1的keepalived配置文件
[root@server1 ~]# vim /etc/keepalived/keepalived.conf
vrrp_instance VI_1 {
state MASTER #master节点
interface eno16777736 #网卡
virtual_router_id 51 #必须和slave相同
priority 100 #优先级，master优先级大于slave
advert_int 1
authentication {
auth_type PASS
auth_pass 1111 #设置验证信息，两个节点必须一致
}
virtual_ipaddress {
192.168.56.140 #虚拟IP
}
track_script {
check_nginx # nginx存活状态检测脚本
}
}
修改server2上keepalived配置文件
[root@server2 yum.repos.d]# vim /etc/keepalived/keepalived.conf
vrrp_instance VI_1 {
state BACKUP #slave节点
interface eno16777736 #网卡
virtual_router_id 51 #必须和master相同
priority 90
advert_int 1
authentication {
auth_type PASS
auth_pass 1111 #设置验证信息，两个节点必须一致
}
virtual_ipaddress {
192.168.56.140 #虚拟IP
}
track_script {
check_nginx # nginx存活状态检测脚本
}
}
3.创建nginx检测脚本（server1和server2相同）
[root@server1 local]# vim check_nginx
#!/bin/bash
ng=`ps -C nginx --no-header | wc -l`
if [ $A -eq 0 ];then
/usr/local/nginx/sbin/nginx #尝试重新启动nginx
sleep 2 #睡眠2秒
if [ `ps -C nginx --no-header | wc -l` -eq 0 ];then
killall keepalived #启动失败，将keepalived服务杀死。将vip漂移到其它备份节点
fi
fi
4.启动keepalived
[root@server1 local]# systemctl start keepalived.service
[root@server2 nginx]# systemctl restart keepalived
[root@server1 local]# ps -ef |grep keepalived
root 48456 1 0 22:56 ? 00:00:00 /usr/sbin/keepalived -D
root 48457 48456 0 22:56 ? 00:00:00 /usr/sbin/keepalived -D
root 48458 48456 14 22:56 ? 00:00:01 /usr/sbin/keepalived -D
[root@server2 nginx]# ps -ef |grep keepalived
root 22872 1 0 21:17 ? 00:00:00 /usr/sbin/keepalived -D
root 22873 22872 0 21:17 ? 00:00:00 /usr/sbin/keepalived -D
root 22874 22872 0 21:17 ? 00:00:00 /usr/sbin/keepalived -D
查看虚拟IP是在master上
[root@server1 local]# ip add
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: eno16777736: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
link/ether 00:0c:29:0e:c1:04 brd ff:ff:ff:ff:ff:ff
inet 192.168.56.132/24 brd 192.168.56.255 scope global noprefixroute dynamic eno16777736
valid_lft 1722sec preferred_lft 1722sec
inet 192.168.56.140/32 scope global eno16777736 这里已经能看到虚拟IP了
valid_lft forever preferred_lft forever
inet6 fe80::20c:29ff:fe0e:c104/64 scope link noprefixroute
valid_lft forever preferred_lft forever
[root@server2 local]# ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: eno16777736: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
link/ether 00:0c:29:59:21:53 brd ff:ff:ff:ff:ff:ff
inet 192.168.56.133/24 brd 192.168.56.255 scope global dynamic eno16777736
valid_lft 1519sec preferred_lft 1519sec
inet6 fe80::20c:29ff:fe59:2153/64 scope link
valid_lft forever preferred_lft forever
4.停掉master节点，看虚拟IP是否漂移
[root@server1 local]# systemctl stop keepalived.service
[root@server2 local]# ip addr
eno16777736: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
link/ether 00:0c:29:59:21:53 brd ff:ff:ff:ff:ff:ff
inet 192.168.56.133/24 brd 192.168.56.255 scope global dynamic eno16777736
valid_lft 1350sec preferred_lft 1350sec
inet 192.168.56.140/32 scope global eno16777736 虚拟IP已经漂移到slave了
valid_lft forever preferred_lft forever
inet6 fe80::20c:29ff:fe59:2153/64 scope link
valid_lft forever preferred_lft forever
启动master，虚拟IP再次回到master
[root@server1 local]# systemctl start keepalived.service
[root@server1 local]# ip addr
eno16777736: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
link/ether 00:0c:29:0e:c1:04 brd ff:ff:ff:ff:ff:ff
inet 192.168.56.132/24 brd 192.168.56.255 scope global noprefixroute dynamic eno16777736
valid_lft 1471sec preferred_lft 1471sec
inet 192.168.56.140/32 scope global eno16777736
valid_lft forever preferred_lft forever
inet6 fe80::20c:29ff:fe0e:c104/64 scope link noprefixroute
valid_lft forever preferred_lft forever
5.模拟业务测试单点故障
现在两个nginx都是启动状态，测试负载均衡是否正常
访问虚拟IP不通
[root@server1 nginx]# ping 192.168.56.140
PING 192.168.56.140 (192.168.56.140) 56(84) bytes of data.
^C
--- 192.168.56.140 ping statistics ---
71 packets transmitted, 0 received, 100% packet loss, time 70022ms
在网上查找原因是keepalived.conf配置中默认vrrp_strict打开了，注释掉vrrp_strict
[root@server1 nginx]# vim /etc/keepalived/keepalived.conf
notification_email_from Alexandre.Cassen@firewall.loc
smtp_server 192.168.200.1
smtp_connect_timeout 30
router_id LVS_DEVEL
vrrp_skip_check_adv_addr
# vrrp_strict 注释掉
vrrp_garp_interval 0
vrrp_gna_interval 0
重启keepalived,可以ping通了
[root@server1 nginx]# ping 192.168.56.140
PING 192.168.56.140 (192.168.56.140) 56(84) bytes of data.
64 bytes from 192.168.56.140: icmp_seq=1 ttl=64 time=0.040 ms
64 bytes from 192.168.56.140: icmp_seq=2 ttl=64 time=0.054 ms
64 bytes from 192.168.56.140: icmp_seq=3 ttl=64 time=0.036 ms
访问虚拟IP
刷新变成tomcat2，说明nginx功能正常。
此时负责分发请求的nginx是nginx1，因为虚拟IP在master上
停掉master上的nginx,再次测试
[root@server1 nginx]# sbin/nginx -s stop
访问虚拟IP
再次刷新，页面变成tomcat2。此时虚拟IP漂移到slave上，所以nginx2来分发请求
启动master上的nginx,停掉slave上的nginx,再次测试
[root@server2 nginx]# sbin/nginx -s stop
[root@server1 nginx]# sbin/nginx
访问虚拟IP
再次刷新，页面变成tomcat2。此时master已启动，虚拟IP回到master，此时nginx1进行分发请求
总结
通过上面对master和slave的nginx停启，虚拟IP在master和slave之间漂移，证明了keepalived能够实现对nginx高可用，再加上nginx对tomcat进行负载均衡，这样就完成了一个简单的高可用负载均衡集群。
扫一扫
520
被折叠的 条评论
为什么被折叠?
到【灌水乐园】发言
