tomcat+nginx+keepalived高可用实现详细部署

minmax329

已于 2023-02-05 09:45:56 修改

阅读量325

点赞数

文章标签： linux 运维 tomcat nginx 分布式 Powered by 金山文档

于 2023-02-05 09:40:44 首次发布

本文链接：https://blog.csdn.net/minmax329/article/details/128888279

版权

系统

两台Nginx：

CentOS6.5 x86_64

两台tomcat：

CentOS6.5 x86_64

拓扑

机器IP	安装软件	角色	虚拟ip	描述
192.168.15.132	Nginx、keepalived	Nginx主机	192.168.15.135	反向代理到tomcat1和tomcat2
192.168.15.133	Nginx、keepalived	Nginx备机	主机挂了切换虚拟ip 192.168.15.135	反向代理到tomcat1和tomcat2
192.168.15.128	Tomcat	Tomcat1	无
192.168.15.30	Tomcat	Tomcat2	无

IP地址

nginx(主LB):192.168.15.132

nginx(备LB):192.168.15.133

VIP地址:192.168.15.135

Real1的IP:192.168.15.128

Real2的IP:192.168.15.30

一．修改启动web服务器

在192.168.15.128和192.168.15.30上分别解压修改tomcat的index页面：

vi webapps/ROOT/index.jsp

修改页面html部分

<html>
    <head>
    </head>
    <body>
SessionID:<%=session.getId()%>
<br/>
SessionIP:<%=request.getServerName()%>
<br/>
<h1>tomcat1 page</h1>
</body>
</html>

这样可以通过访问index来判断访问是不是轮询的

二．安装配置nginx

1.安装GCC编译器等工具：

yum install -y gcc gcc-c++ autoconf automake libtool make openssl openssl-devel pcre pcre-devel

2.下载安装Nginx:

wget http://nginx.org/download/nginx-1.6.3.tar.gz

注：这里也可以下载tengine压缩包，比一般nginx多一些功能

tar -zxvf nginx-1.6.3.tar.gz 

cd nginx-1.6.3/  

./configure --prefix=/usr/local/nginx
--sbin-path=/usr/local/nginx/sbin/nginx
--conf-path=/usr/local/nginx/conf/nginx.conf
--pid-path=/usr/local/nginx/logs/nginx.pid \
--with-http_ssl_module \
--with-http_stub_status_module \
--with-http_gzip_static_module \ 

make && make install

注：查询"./configure --help"相关模块，按需求指定启用

3.配置Nginx.conf

配置文件，二个nginx负载均衡器的文件一样

user nobody;

worker_processes 1;

error_log /usr/local/nginx/logs/error.log notice;

pid /usr/local/nginx/logs/nginx.pid;

worker_rlimit_nofile 51200;

events {

use epoll;

worker_connections 51200;

}

http {

include mime.types;

default_type application/octet-stream;

log_format main '$remote_addr - $remote_user [$time_local] "$request" '

'$status $body_bytes_sent "$http_referer" '

'"$http_user_agent" "$http_x_forwarded_for"';

access_log logs/access.log main;

server_names_hash_bucket_size 128;

client_header_buffer_size 32k;

large_client_header_buffers 4 32k;

client_max_body_size 8m;

sendfile on;

tcp_nopush on;

server_tokens off;

keepalive_timeout 60;

fastcgi_connect_timeout 300;

fastcgi_send_timeout 300;

fastcgi_read_timeout 300;

fastcgi_buffer_size 64k;

fastcgi_buffers 4 64k;

fastcgi_busy_buffers_size 128k;

fastcgi_temp_file_write_size 128k;

gzip on;

upstream backend

{

server 192.168.15.128;

server 192.168.15.130;

}

server {

listen 80;

server_name 192.168.15.135;

location / {

root html;

index index.php index.html index.htm;

proxy_redirect off;

proxy_set_header Host $host;

proxy_set_header X-Real-IP $remote_addr;

#后端的Web服务器可以通过X-Forwarded-For获取用户真实IP

proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;

proxy_pass http://backend;

}

error_page 500 502 503 504 /50x.html;

location = /50x.html {

root html;

}

location /nginx_status {

stub_status on;

auth_basic "NginxStatus";

auth_basic_user_file /usr/local/nginx/htpasswd;

#allow 127.0.0.1;

#deny all;

}

location ~* \.(ini|docx|txt|doc|pdf)$ {

#禁止访问文档性文件

root /usr/share/nginx/html;

deny all;

}

location ~ .*\.(gif|jpg|jpeg|png|bmp|swf|js|html|htm|css)$ {

root /home/image;

proxy_store on;

proxy_store_access user:rw group:rw all:rw;

proxy_temp_path /home/image;

if ( !-e $request_filename) {

proxy_pass http://backend;

}

这里面配置较多，其中比较有用的已经标红了，可以直接将此部分配置在默认的nginx的配置文件里面即可

三．安装及配置keepalived

1.安装keepalived

在两台nginx服务器张安装keepalived：

wget http://www.keepalived.org/software/keepalived-1.2.15.tar.gz

tar -zxvf keepalived-1.2.15.tar.gz

cd keepalived-1.2.15

./configure --sysconf=/etc/  --with-kernel-dir=/usr/src/kernels/2.6.32-573.8.1.el6.x86_64
make && make install

ln -s /usr/local/sbin/keepalived  /sbin/

#这一步很重要，不执行ln -s会报错“Starting keepalived: /bin/bash: keepalived: command not found”

service keepalived start

上述步骤也可以直接通过yum –y install keepalived 来代替，不同安装方式而已，不是重点

二台Nginx上keepalived.conf配置文件如下，配置完成后分别service keepalived start启动｡检验keepalived配置是否成功

两台keepalived的配置如下：可以看出两者之间的区别仅仅是主备的权重不同，主为100备为66，其余一些有作用的配置已经用红色标注

主：

 global_defs {
   notification_email {
     test@163.com
   }
   notification_email_from keepalived@localhost 
   smtp_server 127.0.0.1
   smtp_connect_timeout 30
   router_id LVS_MASTER
}
vrrp_script chk_http_port {
script "/usr/local/src/check_nginx_pid.sh"
interval 2                           #（检测脚本执行的间隔）
weight 2
}
vrrp_instance VI_1 {
   #state MASTER
    state BACKUP
    nopreempt                       
    #设置非抢占模式时，修改“state MASTER”为“state BACKUP”，添加“nopreempt“
    interface bond0
    virtual_router_id 51
    priority 100
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass 1111
}
track_script {
chk_http_port            #（调用检测脚本）
}
    virtual_ipaddress {
        192.168.15.135
    }
}

备：

 global_defs {
   notification_email {
     test@163.com
   }
   notification_email_from keepalived@localhost 
   smtp_server 127.0.0.1
   smtp_connect_timeout 30
   router_id LVS_BACKUP
}
vrrp_script chk_http_port {
script "/usr/local/src/check_nginx_pid.sh"
interval 2                           #（检测脚本执行的间隔）
weight 2
}
vrrp_instance VI_1 {
    state BACKUP
    interface bond0
    virtual_router_id 51
    priority 66
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass 1111
}
track_script {
chk_http_port            #（调用检测脚本）
}
    virtual_ipaddress {
        192.168.15.135
    }
}

以下是针对nginx状态进行检测的脚本，第一次nginx服务死掉时，会重新启动，如果Nginx服务无法正常启动，则杀掉keepalived进程

vim  /usr/local/src/check_nginx_pid.sh

check_nginx_pid.sh 文件内容：

#!/bin/bash
A=`ps -C nginx --no-header |wc -l`        
if [ $A -eq 0 ];then                            
      /usr/local/nginx/sbin/nginx        
      if [ `ps -C nginx --no-header |wc -l` -eq 0 ];then  
              killall keepalived                    
      fi
fi

开始nginx负载均衡测试，停掉其中一台的任何服务，不影响整个系统的运作｡