拓扑图
![](https://img-blog.csdnimg.cn/img_convert/88958519b643da500fd29aa25ee3ec7d.png)
1实现目标:
使用tomcat+nginx+keepalived搭建高可用集群
2环境准备:
2.1软件下载地址及版本
Keepalived-2.2.7.tar.gz
下载地址:https://www.keepalived.org/download.html
Apache-Tomcat8.5.70.tar.gz
下载地址:https://tomcat.apache.org/download-80.cgi
Nginx-1.22.0.tar.gz
下载地址:http://nginx.org/download
jdk-8u291-linux-x64.tar.gz
下载地址:
https://www.oracle.com/java/technologies/javse/javase8u211-later-archive-downloads.html
2.2编译环境
yum groupinstall "Development Tools"
yum-y install openssl-devel libnl libnl-devel
2.3 配置要求
四台centos7.9虚拟机(两台安装nginx+keepalived,两台安装tomcat)
2.4网段规划
nginx+keepalived(主) 192.168.205.150
nginx+keepalived(备) 192.168.205.151
Tomcat(1) 192.168.205.152
Tomcat(2) 192.168.205.153
2.5 模块安装
Nginxweb ui模块nginx-module-vts
下载地址:https://github.com/vozlt/nginx-module-vts/releases/tag/v0.2.1
负载均衡模块nginx-goodies-nginx-sticky
下载地址:
https://bitbucket.org/nginx-goodies/nginx-sticky-module-ng/downloads/
3.操作步骤:
注意:3.1-3.3.8步骤在152和153同步执行
3.1配置jdk
3.1.1创建jdk目录
mkdir -p /usr/java/jvm
![](https://img-blog.csdnimg.cn/img_convert/6e04731c6b751fbd1fa23f15be4df263.png)
3.1.2解压jdk压缩包到/usr/java/jvm
![](https://img-blog.csdnimg.cn/img_convert/2b54e77120eafa4293e3f36c7b419f2e.png)
3.1.3配置并更新环境变量
vim /etc/profile
![](https://img-blog.csdnimg.cn/img_convert/9a9b6484ef8f516c06865e1f9c06098c.png)
exportJAVA_HOME=/usr/java/jvm/jdk1.8.0_291
exportJRE_HOME=${JAVA_HOME}/jre
exportCLASSPATH=.:${JAVA_HOME}/lib:${JRE_HOME}/lib
exportPATH=${JAVA_HOME}/bin:$PATH
![](https://img-blog.csdnimg.cn/img_convert/51980a026ce1739e7b204a28aa7fd47b.png)
更新:source/etc/profile
![](https://img-blog.csdnimg.cn/img_convert/2063e395f7640fbed71bcfc4823fd86f.png)
3.1.4 查看是否安装成功
javac -version
![](https://img-blog.csdnimg.cn/img_convert/76884e3966697a10a2aaabca28d9a78a.png)
3.2配置tomcat
3.2.1创建tomcat目录
mkdir -p /usr/java/tomcat
![](https://img-blog.csdnimg.cn/img_convert/381498c106a99e838e888b71e64294be.png)
3.2.2解压tomcat压缩包到/usr/java/tomcat
tar -zxf apache-tomcat-8.5.70.tar.gz -C/usr/java/tomcat
![](https://img-blog.csdnimg.cn/img_convert/39f7062494c97462f6b5818b93671991.png)
注意:以下操作都在/usr/ava/tomcat/apache-tomcat-8.5.70里执行
3.2.3创建模板目录
mkdir web-template
![](https://img-blog.csdnimg.cn/img_convert/f2e75d25af64ff4fcada451fa7f28d8c.png)
3.2.4将conf、logs、temp、webapps、work移动到web-template目录下
![](https://img-blog.csdnimg.cn/img_convert/277b366f618a9b1215266459a85b5fdc.png)
3.2.5创建实例(两个实例)
cp-rp web-template web-8080(配置默认)
cp-rp web-template web-8081(配置里面修改端口号)
(8081配置
原本的8005改为9005
![](https://img-blog.csdnimg.cn/img_convert/df52f4247a8c3aad9be5af1a953f9e55.png)
原本的8080改为8081
![](https://img-blog.csdnimg.cn/img_convert/20b9910fbdc867b5c2319579cc2dd918.png)
3.2.6 创建访问界面
cd /usr/java/tomcat/apache-tomcat-8.5.70/web-8080/webapps
![](https://img-blog.csdnimg.cn/img_convert/7f8551b1a320a969c7b5b7bd465f351d.png)
mkdir testweb
![](https://img-blog.csdnimg.cn/img_convert/d3fd8a6da3ae438a0a143f1f51447bd5.png)
cd testweb
![](https://img-blog.csdnimg.cn/img_convert/cd3835ce93695afaaad12eb5e7188a96.png)
vim /indes.jsp
![](https://img-blog.csdnimg.cn/img_convert/46129bf18e36bff354ce313ab202b55e.png)
<%@page language="java" contentType="text/html; charset=utf-8"
pageEncoding="utf-8"%>
<!DOCTYPEhtml PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN""" target="_blank">http://www.w3.org/TR/html4/loose.dtd">
<html>
<head>
<metahttp-equiv="Content-Type" content="text/html;charset=ISO-8859-1">
<title>测试</title>
</head>
<body>
<h1>192.168.205.152</h1>
</body>
</html>
![](https://img-blog.csdnimg.cn/img_convert/4f43c795f2f7fe6ad2033c038df83362.png)
注意:
![](https://img-blog.csdnimg.cn/img_convert/068c2ef63f5520d3f5a97ca9ec4fe8f8.png)
3.2.7创建实例脚本并执行(有几个实例就写几个脚本)
vim/etc/tomcat_8080.sh
RETVAL=$?
#tomcat instance directory
export CATALINA_HOME="/usr/java/tomcat/apache-tomcat-8.5.70"
#tomcat installation directory, change to your own
export CATALINA_BASE="$CATALINA_HOME/web-8080"
#Optional
exportJVM_OPTIONS="-Xms128m -Xmx1024m -XX:PermSize=128m-XX:MaxPermSize=512m"
case"$1" in
start)
if[ -f $CATALINA_HOME/bin/startup.sh ];then
echo$"Start Tomcat"
$CATALINA_HOME/bin/startup.sh
fi
;;
stop)
if[ -f $CATALINA_HOME/bin/shutdown.sh ];then
echo$"Stop Tomcat"
$CATALINA_HOME/bin/shutdown.sh
fi
;;
*)
echo$"Usage:$0 {start|stop}"
exit1
;;
esac
exit$RETVAL
vim /etc/tomcat_8081.sh
RETVAL=$?
#tomcat instance directory
export CATALINA_HOME="/usr/java/tomcat/apache-tomcat-8.5.70"
#tomcat installation directory, change to your own
export CATALINA_BASE="$CATALINA_HOME/web-8081"
#Optional
exportJVM_OPTIONS="-Xms128m -Xmx1024m -XX:PermSize=128m -XX:MaxPermSize=512m"
case"$1" in
start)
if[ -f $CATALINA_HOME/bin/startup.sh ];then
echo$"Start Tomcat"
$CATALINA_HOME/bin/startup.sh
fi
;;
stop)
if[ -f $CATALINA_HOME/bin/shutdown.sh ];then
echo$"Stop Tomcat"
$CATALINA_HOME/bin/shutdown.sh
fi
;;
*)
echo$"Usage:$0 {start|stop}"
exit1
;;
esac
exit$RETVAL
![](https://img-blog.csdnimg.cn/img_convert/b4a96856ac32cbd64745418ff5f26a9d.png)
赋予权限
chmod +x tomcat_8080.sh
chmod +x tomcat_8081.sh
![](https://img-blog.csdnimg.cn/img_convert/0538cf95e3df8ad811ad2f28f726b3b9.png)
执行
./tomcat_8080.sh start
./tomcat_8081.sh start
![](https://img-blog.csdnimg.cn/img_convert/42d1e927f468dc7710364731b4a310f9.png)
如要停止请执行:./tomcat_8080.shstop
3.2.8测试
192.168.205.152:8080
192.168.205.153:8080
![](https://img-blog.csdnimg.cn/img_convert/6886c13a38617e6184e413298e29ed6d.png)
静态页面
192.168.205.152:8080/testweb/index.jsp
![](https://img-blog.csdnimg.cn/img_convert/f03a6225424c93ceefd405d364b04a46.png)
192.168.205.153:8080/testweb/index.jsp
![](https://img-blog.csdnimg.cn/img_convert/43bddb5686bd4e70190ee7c0940917f0.png)
注意:3.3-3.3.8步骤在150和151同步执行
3.3 配置nginx
3.3.1安装编译环境
yum groupinstall "Development Tools"(安装之前先打下面两条命令不然安不上)
yum groups mark install "DevelopmentTools"
yum groups mark convert "DevelopmentTools"
![](https://img-blog.csdnimg.cn/img_convert/bce5916d0be2cc47b73b6d8e78125ec1.png)
yum-y install openssl-devel libnl libnl-devel
![](https://img-blog.csdnimg.cn/img_convert/1fd7a4042c1c645be6460b089477fbfe.png)
3.3.2创建nginx用户和组(不允许登录)
groupaddnginx && useradd -s /sbin/nologin -g nginx nginx
![](https://img-blog.csdnimg.cn/img_convert/3103e9b9858774da1bb0be70219d4e54.png)
3.3.3解压nginx压缩包,进入并编译安装
tar-zxf nginx-1.22.0.tar.gz && cd nginx-1.22.0
![](https://img-blog.csdnimg.cn/img_convert/fe0b3b697260c36e7ccf55b8a8a8c4cc.png)
Nginxweb ui模块nginx-module-vts
负载均衡模块nginx-goodies-nginx-sticky
这两个模块放在nginx父目录(环境准备里有下载地址)
3.3.4预编译
./configure\
--prefix=/usr/local/nginx\
--user=nginx--group=nginx \
--with-http_stub_status_module\
--with-http_ssl_module\
--with-http_realip_module\
--add-module=../nginx-module-vts\
--add-module=../nginx-sticky-module-ng
执行完
![](https://img-blog.csdnimg.cn/img_convert/61dea1f02dbd6ba776436562f5b56bcb.png)
3.3.5编译并安装
make -j8 && make install
![](https://img-blog.csdnimg.cn/img_convert/cb1420869491d80ae82f794eb73d3da2.png)
3.3.6修改nginx配置文件
vim /usr/local/nginx/conf/nginx.conf
添加: include/usr/local/nginx/conf/conf.d/*.conf;
![](https://img-blog.csdnimg.cn/img_convert/5b7b0a2f2cee2707ec76a979fa8d5c4a.png)
3.3.7添加server模块
vim /usr/local/nginx/conf/conf.d/tomcat.conf
set_real_ip_from 192.168.205.150/32;
set_real_ip_from 192.168.205.151/32;
real_ip_header X-Forwarded-For;
real_ip_recursiveon;
vhost_traffic_status_zone;
upstreamtomcat{
sticky;
server 192.168.205.152:8080;
server 192.168.205.153:8081;
server 192.168.205.152:8080;
server 192.168.205.153:8081;
}
server{
listen 8090;
server_name localhost;
location / {
proxy_pass http://tomcat;
root html;
index index.html index.htm;
}
location /status {
vhost_traffic_status_display;
vhost_traffic_status_display_formathtml;
}
error_log /var/nginx/logs/error.log error;
access_log /var/nginx/logs/access.log main;
}
![](https://img-blog.csdnimg.cn/img_convert/fd3cee705485a046ed66f5c33dedc0a0.png)
3.3.8 启动,停止,重启,检测命令
/usr/local/nginx/sbin/nginx -t //检测配置语法
/usr/local/nginx/sbin/nginx -s reload //重启nginx
/usr/local/nginx/sbin/nginx -s stop //停止nginx
/usr/local/nginx/sbin/nginx //启动nginx
3.4部署keepalived
3.4.1解压keepalived压缩包并进入
![](https://img-blog.csdnimg.cn/img_convert/bc382f277cae7365c8fcf6087fa4be32.png)
3.4.2预编译(以下操作在keepalived 2.2.7目录中运行)
./configure--prefix=/usr/local/keepalived
![](https://img-blog.csdnimg.cn/img_convert/30a0e112d7e781b38aea5bc86453be13.png)
3.4.3编译与安装
make-j8 && make install
![](https://img-blog.csdnimg.cn/img_convert/839351f0c3178a706dd3452d290cc516.png)
3.4.4创建配置文件目录并复制二进制启动文件,配置文件,keepalived系统配置文件,keepalived指令
mkdir -p /etc/keepalived
![](https://img-blog.csdnimg.cn/img_convert/eb881ca3e88ef27bfcee08bf1942d87e.png)
二进制启动文件: cp keepalived/etc/init.d/keepalived /etc/init.d/
配置文件: cp /usr/local/keepalived/etc/keepalived/keepalived.conf/etc/keepalived/
keepalived系统配置文件:cpkeepalived/etc/sysconfig/keepalived /etc/sysconfig/keepalived
keepalived指令: cp /usr/local/keepalived/sbin/keepalived /usr/sbin/
3.4.5创建nginx检测脚本并执行
创建脚本目录
mkdir -p /opt/nginx_check
![](https://img-blog.csdnimg.cn/img_convert/6c7099ef3b7222c024e8cefa8d5a1201.png)
cat>> /opt/nginx_check/nginx_check.sh << EOF
#!/bin/bash
A=`ps-C nginx --no-header |wc -l`
if[ $A -eq 0 ];then
/usr/local/nginx/sbin/nginx
sleep 2
if [ `ps -C nginx --no-header |wc -l` -eq 0];then
killall keepalived
fi
fi
EOF
![](https://img-blog.csdnimg.cn/img_convert/bdec5a16c93aa19d4a28df2965107e5d.png)
执行权限
chmod755 /opt/nginx_check/nginx_check.sh
![](https://img-blog.csdnimg.cn/img_convert/f1b47a652580a3b32767e6f57257a2d5.png)
3.4.6配置keepalived
192.168.205.150(主) vim /etc/keepalived/keepalived.conf
! Configuration File for keepalived
global_defs {
notification_email {
}
notification_email_from Alexandre.Cassen@firewall.loc
smtp_server 192.168.200.1
smtp_connect_timeout 30
router_id LVS_DEVEL1
vrrp_skip_check_adv_addr
!vrrp_strict
vrrp_garp_interval 0
vrrp_gna_interval 0
}
vrrp_instance VI_1 {
state MASTER
interface ens33
virtual_router_id 51
priority 100
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
192.168.205.159
}
}
![](https://img-blog.csdnimg.cn/img_convert/10ef41be3bbfd6aeb4006581d03ade50.png)
192.168.205.151(备) vim /etc/keepalived/keepalived.conf
! Configuration File for keepalived
global_defs {
notification_email {
}
notification_email_from Alexandre.Cassen@firewall.loc
smtp_server 192.168.200.1
smtp_connect_timeout 30
router_id LVS_DEVEL2
vrrp_skip_check_adv_addr
!vrrp_strict
vrrp_garp_interval 0
vrrp_gna_interval 0
}
vrrp_instance VI_1 {
state BACKUP
interface ens33
virtual_router_id 51
priority 90
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
192.168.205.159
}
}
![](https://img-blog.csdnimg.cn/img_convert/27dfd9ba50a1cfe0c1ee59c23c8ee872.png)
3.4.7启动keepalived服务并让其开机自启动
systemctl start keepalived && systemctl enable keepalived
![](https://img-blog.csdnimg.cn/img_convert/f5fd0fd506332f99cc71684cc4528fbc.png)
3.4.8查看配置结果
![](https://img-blog.csdnimg.cn/img_convert/f7de22f243ca5ed5b4b1de60a9a0ff23.png)
把192.168.205.150 关闭
![](https://img-blog.csdnimg.cn/img_convert/03245cfc79f98d0f2650495295c42ba8.png)
可以看到已经飘到192.168.205.151
![](https://img-blog.csdnimg.cn/img_convert/a15e9b3c6b9c416cda8835d62501174e.png)
4.访问测试
192.168.205.150测试
192.168.205.159:8090
![](https://img-blog.csdnimg.cn/img_convert/aea9beb70e24015e445f993610faa5d3.png)
访问静态页面
http://192.168.205.159:8090/testweb/index.jsp
![](https://img-blog.csdnimg.cn/img_convert/752469ae0b4c5df2c112a3b0f9e88b5c.png)
把192.168.205.150宕机
![](https://img-blog.csdnimg.cn/img_convert/0ac7a241f516fcac948a65c168d4b47a.png)
可以看到已经漂移到192.168.205.151
![](https://img-blog.csdnimg.cn/img_convert/89c305a43324c58ce32529a205183746.png)
测试前把浏览器缓存清理一下
![](https://img-blog.csdnimg.cn/img_convert/657c45716929478166bac38c203747d0.png)
Shift+ctrl+delete 清理缓存
192.168.205.151测试
![](https://img-blog.csdnimg.cn/img_convert/6d3e60346cd10c71694332cd456e6bb0.png)
访问静态网页
http://192.168.205.159:8090/testweb/index.jsp
![](https://img-blog.csdnimg.cn/img_convert/27f0ac46d70a9fedccdf626cee7115ea.png)