Nginx+Tomcat+Redis负载均衡集群服务搭建

转载请注明出处:http://blog.csdn.net/l1028386804/article/details/52216000

        在之前的一篇《Nginx+Tomcat+Memcached负载均衡集群服务搭建》中,向大家详细介绍了,基于Nginx+Tomcat+Memcached实现负载均衡集群服务搭建,今天向大家介绍如何基于Nginx+Tomcat+Redis实现负载均衡集群服务的搭建。我们直接进入正题:

插曲:

官方截止到2015-10-12前是不支持Tomcat8的,详情见官网:https://github.com/jcoleman/tomcat-redis-session-manager 

下面我们将它修改为支持Tomcat8,

修改的源代码:RedisSessionManager.java


 
 
  1. @SuppressWarnings( "deprecation")
  2. private void initializeSerializer() throws ClassNotFoundException, IllegalAccessException, InstantiationException {
  3. log.info( "Attempting to use serializer :" + serializationStrategyClass);
  4. serializer = (Serializer) Class.forName(serializationStrategyClass).newInstance();
  5. Loader loader = null;
  6. if (getContainer() != null) {
  7. loader = getContainer().getLoader();
  8. }
  9. ClassLoader classLoader = null;
  10. if (loader != null) {
  11. classLoader = loader.getClassLoader();
  12. }
  13. serializer.setClassLoader(classLoader);
  14. }
修改后的内容

 
 
  1. private void initializeSerializer() throws ClassNotFoundException, IllegalAccessException, InstantiationException {
  2. log.info( "Attempting to use serializer :" + serializationStrategyClass);
  3. serializer = (Serializer) Class.forName(serializationStrategyClass).newInstance();
  4. Loader loader = null;
  5. Context context = this.getContext();
  6. if (context != null) {
  7. loader = context.getLoader();
  8. }
  9. ClassLoader classLoader = null;
  10. if (loader != null) {
  11. classLoader = loader.getClassLoader();
  12. }
  13. serializer.setClassLoader(classLoader);
  14. }

一、前言

         nginx 作为目前最流行的开源反向代理HTTP Server,用于实现资源缓存、web server负载均衡等功能,由于其轻量级、高性能、高可靠等特点在互联网项目中有着非常普遍的应用,相关概念网上有丰富的介绍。分布式web server集群部署后需要实现session共享,针对 tomcat 服务器的实现方案多种多样,比如 tomcat cluster session 广播、nginx IP hash策略、nginx sticky module等方案,本文主要介绍了使用 redis 服务器进行 session 统一存储管理的共享方案。
相关应用结构参照下图:

二、环境配置

  测试环境基于 Linux CentOS 6.5,请先安装 tomcat、redis、nginx 相关环境,不作详细描述,本文测试配置如下:

版本

名称IP与端口
nginx1.9.13192.168.106.130:80
Tomcat1 7.0.54192.168.106.130:8081
Tomcat2 7.0.54192.168.106.130:8082
Tomcat3 7.0.54192.168.106.130:8083
 redis2.8.19192.168.106.130:6379
注:安装nginx请参见《 Nginx+Tomcat+Memcached负载均衡集群服务搭建》,安装redis请参见《 Redis之——CentOS 6.5安装配置redis

三、构建 tomcat-redis-session-manager-master

1、由于源码构建基于 gradle,请先配置 gradle 环境。
2、从 github 获取 tomcat-redis-session-manager-master 源码,地址如下:

https://github.com/jcoleman/tomcat-redis-session-manager
3、找到源码中的 build.gradle 文件,由于作者使用了第三方仓库(sonatype),需要注册帐号,太麻烦,注释后直接使用maven中央仓库,同时注释签名相关脚本并增加依赖包的输出脚本 copyJars(dist目录),修改后的 build.gradle 文件如下:

apply plugin: 'java'
apply plugin: 'maven'
apply plugin: 'signing'

group = ‘com.orangefunction’
version = ‘2.0.0’

repositories {
mavenCentral()
}

compileJava {
sourceCompatibility = 1.7
targetCompatibility = 1.7
}

dependencies {
compile group: ‘org.apache.tomcat’, name: ‘tomcat-catalina’, version: ‘7.0.27’
compile group: ‘redis.clients’, name: ‘jedis’, version: ‘2.5.2’
compile group: ‘org.apache.commons’, name: ‘commons-pool2’, version: ‘2.2’
//compile group: ‘commons-codec’, name: ‘commons-codec’, version: ‘1.9’

testCompile group: ‘junit’, name: ‘junit’, version: ‘4.+’
testCompile ‘org.hamcrest:hamcrest-core:1.3’
testCompile ‘org.hamcrest:hamcrest-library:1.3’
testCompile ‘org.mockito:mockito-all:1.9.5’
testCompile group: ‘org.apache.tomcat’, name: ‘tomcat-coyote’, version: ‘7.0.27’
}

task javadocJar(type: Jar, dependsOn: javadoc) {
classifier = ‘javadoc’
from ‘build/docs/javadoc’
}

task sourcesJar(type: Jar) {
from sourceSets.main.allSource
classifier = ‘sources’
}

artifacts {
archives jar

archives javadocJar
archives sourcesJar
}

//signing {
// sign configurations.archives
//}

task copyJars(type: Copy) {
from configurations.runtime
into ‘dist’
}

uploadArchives {
repositories {
mavenDeployer {
beforeDeployment { MavenDeployment deployment -> signing.signPom(deployment) }

  <span class="hljs-comment">//repository(url: "https://oss.sonatype.org/service/local/staging/deploy/maven2/") {</span>
  <span class="hljs-comment">//  authentication(userName: sonatypeUsername, password: sonatypePassword)</span>
  <span class="hljs-comment">//}</span>
  <span class="hljs-comment">//repository(url: "https://oss.sonatype.org/content/repositories/snapshots") {</span>
  <span class="hljs-comment">//  authentication(userName: sonatypeUsername, password: sonatypePassword)</span>
  <span class="hljs-comment">//}</span>

  pom.project {
    name <span class="hljs-string">'tomcat-redis-session-manager'</span>
    packaging <span class="hljs-string">'jar'</span>
    description <span class="hljs-string">'Tomcat Redis Session Manager is a Tomcat extension to store sessions in Redis'</span>
    url <span class="hljs-string">'https://github.com/jcoleman/tomcat-redis-session-manager'</span>

    issueManagement {
      url <span class="hljs-string">'https://github.com:jcoleman/tomcat-redis-session-manager/issues'</span>
      system <span class="hljs-string">'GitHub Issues'</span>
    }

    scm {
      url <span class="hljs-string">'https://github.com:jcoleman/tomcat-redis-session-manager'</span>
      connection <span class="hljs-string">'scm:git:git://github.com/jcoleman/tomcat-redis-session-manager.git'</span>
      developerConnection <span class="hljs-string">'scm:git:git@github.com:jcoleman/tomcat-redis-session-manager.git'</span>
    }

    licenses {
      license {
        name <span class="hljs-string">'MIT'</span>
        url <span class="hljs-string">'http://opensource.org/licenses/MIT'</span>
        distribution <span class="hljs-string">'repo'</span>
      }
    }

    developers {
      developer {
        id <span class="hljs-string">'jcoleman'</span>
        name <span class="hljs-string">'James Coleman'</span>
        email <span class="hljs-string">'jtc331@gmail.com'</span>
        url <span class="hljs-string">'https://github.com/jcoleman'</span>
      }
    }
  }
}

}
}4、执行gradle命令构建源码,编译输出tomcat-redis-session-manager-master 及依赖jar包

gradle build -x test  copyJars
所有输出列表文件如下:

四、tomcat 配置

安装配置三台 tomcat web服务器,分别修改 Connector 端口号为8081、8082、8083,并确保都能正常工作,当然如果分布在不同的主机则可以使用相同端口号。

五、编写测试页面

  为了区别3台tomcat的访问,分别编写页面并打包部署:
  1、为tomcat_1编写测试页面,显示 “ response from tomcat_1 ”,同时页面提供按钮显示当前session值,打包并发布到 tomcat_1 服务器;
  2、为tomcat_2编写测试页面,显示 “ response from tomcat_2 ”,同时页面提供按钮显示当前session值,打包并发布到 tomcat_2 服务器;
   3、为tomcat_3编写测试页面,显示 “ response from tomcat_3 ”,同时页面提供按钮显示当前session值,打包并发布到 tomcat_3 服务器;
  此时分别访问 http://192.168.106.130:8081 、http://192.168.106.130:8082、http://192.168.106.130:8083 地址,因为访问的是不同web服务器,所以各自显示不同的页面内容及session值肯定不同。

六、tomcat session manager 配置

  修改配置使用 tomcat-redis-session-manager-master 作为 tomcat session 管理器
  1、分别将第三步生成的 tomcat-redis-session-manager-master 及依赖jar包覆盖到 tomcat 安装目录的 lib 文件夹
  2、分别修改3台 tomcat 的 context.xml 文件,使 tomcat-redis-session-manager-master 作为session管理器,同时指定redis地址和端口。
  context.xml 增加以下配置:


 
 
  1. <Context>
  2. <Valve className="com.orangefunction.tomcat.redissessions.RedisSessionHandlerValve" />
  3. <Manager className="com.orangefunction.tomcat.redissessions.RedisSessionManager"
  4. host= "localhost"
  5. port= "6379"
  6. database= "0"
  7. maxInactiveInterval= "60" />
  8. </Context>
3、分别重启3台 tomcat 服务器

七、nginx 配置

1、配置nginx.conf

#创建进程的用户和用户组

user php php;

#服务进程数量,一般等于CPU数量

worker_processes 8;

#全局错误日志定义,建议开启error级别日志.[ debug | info | notice | warn | error | crit ]

worker_rlimit_nofile 655350;

error_log logs/error.log error;

#error_log logs/error.log notice;

#error_log logs/error.log info;

#记录进程ID的文件

#pid logs/nginx.pid;

events {

<span class="hljs-meta">#epoll是多路复用IO(I/O Multiplexing)中的一种方式,但是仅用于linux2.6以上内核,可以大大提高nginx的性能.Linux建议使用epoll,FreeBSD建议使用kqueue.</span>

use epoll;

<span class="hljs-meta">#一个worker_processe允许的最近并发连接数量</span>

worker_connections 1024;

}

http {

include mime.types;

default_type application/octet-stream;

client_max_body_size     <span class="hljs-number">15</span>m;
   client_body_buffer_size  <span class="hljs-number">128</span>k;
   proxy_connect_timeout    <span class="hljs-number">600</span>;
   proxy_read_timeout       <span class="hljs-number">600</span>;
   proxy_send_timeout       <span class="hljs-number">6000</span>;
   proxy_buffer_size        <span class="hljs-number">16</span>k;
   proxy_buffers            <span class="hljs-number">4</span> <span class="hljs-number">64</span>k;
   proxy_busy_buffers_size <span class="hljs-number">64</span>k;
   proxy_temp_file_write_size <span class="hljs-number">64</span>k;

#log_format main '$remote_addr - r e m o t e u s e r [ remote_user [ remoteuser[time_local]"$request" ’

<span class="hljs-meta">#                  '$status $body_bytes_sent"$http_referer" '</span>

<span class="hljs-meta">#                  '"$http_user_agent""$http_x_forwarded_for"';</span>



<span class="hljs-meta">#access_log  logs/access.log  main;</span>

sendfile on;

#tcp_nopush on;

<span class="hljs-meta">#http连接的持续时间</span>

keepalive_timeout 65;

<span class="hljs-meta">#gzip压缩设置</span>

gzip  <span class="hljs-keyword">on</span>;           <span class="hljs-meta">#开启gzip</span>

gzip_min_length 1k; #最小压缩文件大小

gzip_buffers 4 16k; #压缩缓冲区

<span class="hljs-meta">#http的协议版本(1.0/1.1),默认1.1,前端如果是squid2.5请使用1.0</span>

gzip_http_version 1.1;

<span class="hljs-meta">#gzip压缩比,1压缩比最小处理速度最快,9压缩比最大但处理速度最慢(传输快但比较消耗cpu)</span>

gzip_comp_level 2;

<span class="hljs-meta">#和http头有关系,加个vary头,给代理服务器用的,有的浏览器支持压缩,有的不支持,所以避免浪费不支持的也压缩,所以根据客户端的HTTP头来判断,是否需要压缩</span>

gzip_vary <span class="hljs-keyword">on</span>;

<span class="hljs-meta">#gzip压缩类型,不用添加text/html,否则会有警告信息</span>

gzip_types text/plain text/javascript text/css application/xmlapplication/x-javascript application/json;

<span class="hljs-meta">#设定负载均衡的服务器列表,可以设置多个upstream,但mysvr名字要区分</span>

upstream myClusterServer1 {

   <span class="hljs-meta">#weigth参数表示权值,权值越高被分配到的几率越大</span>

    <span class="hljs-meta">#本机上的Squid开启3128端口</span>

   server <span class="hljs-number">127.0</span><span class="hljs-number">.0</span><span class="hljs-number">.1</span>:<span class="hljs-number">8081</span>  weight=<span class="hljs-number">5</span>;

   server <span class="hljs-number">127.0</span><span class="hljs-number">.0</span><span class="hljs-number">.1</span>:<span class="hljs-number">8082</span>  weight=<span class="hljs-number">5</span>;

   server <span class="hljs-number">127.0</span><span class="hljs-number">.0</span><span class="hljs-number">.1</span>:<span class="hljs-number">8083</span>  weight=<span class="hljs-number">5</span>;

}


server {

   <span class="hljs-meta">#nginx监听的端口号</span>

   listen       <span class="hljs-number">80</span>;

    <span class="hljs-meta">#域名可以有多个,用空格隔开</span>

   server_name  <span class="hljs-number">127.0</span><span class="hljs-number">.0</span><span class="hljs-number">.1</span>;

   

    <span class="hljs-meta">#字符编码方式</span>

   charset utf<span class="hljs-number">-8</span>;



    <span class="hljs-meta">#设定本虚拟主机的访问日志。关闭日志可以减少IO,提高性能。</span>

   <span class="hljs-meta">#access_log logs/host.access.log  main;</span>



    <span class="hljs-meta">#默认请求</span>

   location / {

        <span class="hljs-meta">#定义服务器的默认网站根目录位置</span>

       root   html;

        <span class="hljs-meta">#定义首页索引文件的名称</span>

       index  index.html index.htmindex.jsp;

        <span class="hljs-meta">#请求转向mysvr 定义的服务器列表</span>

       proxy_pass    http:<span class="hljs-comment">//myClusterServer1;</span>

       proxy_redirect <span class="hljs-keyword">default</span>;

        <span class="hljs-meta">#跟代理服务器连接的超时时间,必须留意这个time out时间不能超过75秒,当一台服务器当掉时,过10秒转发到另外一台服务器。</span>

       proxy_connect_timeout <span class="hljs-number">10</span>;

    }



   <span class="hljs-meta">#error_page  404              /404.html;</span>



    <span class="hljs-meta">#redirect server <span class="hljs-meta-keyword">error</span> pages to the static page /50x.html</span>

    <span class="hljs-meta">#</span>

   error_page   <span class="hljs-number">500</span> <span class="hljs-number">502</span> <span class="hljs-number">503</span> <span class="hljs-number">504</span>  /<span class="hljs-number">50</span>x.html;

   location = /<span class="hljs-number">50</span>x.html {

       root   html;

    }



    <span class="hljs-meta">#proxy the PHP scripts to Apache listening on 127.0.0.1:80</span>

    <span class="hljs-meta">#</span>

   <span class="hljs-meta">#location ~ \.php$ {</span>

   <span class="hljs-meta">#    proxy_pass   http://127.0.0.1;</span>

    <span class="hljs-meta">#}</span>



    <span class="hljs-meta">#pass the PHP scripts to FastCGI server listening on 127.0.0.1:9000</span>

    <span class="hljs-meta">#</span>

   <span class="hljs-meta">#location ~ \.php$ {</span>

   <span class="hljs-meta">#    root           html;</span>

   <span class="hljs-meta">#    fastcgi_pass   127.0.0.1:9000;</span>

   <span class="hljs-meta">#    fastcgi_index  index.php;</span>

   <span class="hljs-meta">#    fastcgi_param  SCRIPT_FILENAME  /scripts$fastcgi_script_name;</span>

   <span class="hljs-meta">#    include        fastcgi_params;</span>

    <span class="hljs-meta">#}</span>



    <span class="hljs-meta">#deny access to .htaccess files, <span class="hljs-meta-keyword">if</span> Apache's document root</span>

    <span class="hljs-meta">#concurs with nginx's one</span>

    <span class="hljs-meta">#</span>

   <span class="hljs-meta">#location ~ /\.ht {</span>

   <span class="hljs-meta">#    deny  all;</span>

    <span class="hljs-meta">#}</span>

}

}2、nginx 重新加载配置

nginx -s reload

八、测试结果

1、访问 http://192.168.106.130:8081 直接请求到tomcat_1服务器,
    显示 “ response from tomcat_1 ”, session 值为 ‘56E2FAE376A47F1C0961D722326B8423’;
2、访问 http://192.168.106.130:8082 直接请求到tomcat_2服务器,
    显示 “ response from tomcat_2 ”, session 值为 ‘56E2FAE376A47F1C0961D722326B8423’;
3、访问 http://192.168.106.130:8083 直接请求到tomcat_3服务器,
    显示 “ response from tomcat_3 ”, session 值为 ‘56E2FAE376A47F1C0961D722326B8423’;
4、访问 http://192.168.106.130 (默认80端口)请求到 nginx 反向代理到指定Web服务器,由于默认使用轮询负载方式,
    反复刷新页面显示的内容在“ response from tomcat_1 ” 、 “ response from tomcat_2 ”和“ response from tomcat_3 ”之间切换,但 session 值保持为 ‘56E2FAE376A47F1C0961D722326B8423’;
5、使用 redis-cli 连接 redis 服务器,查看会显示有 “56E2FAE376A47F1C0961D722326B8423” key的 session 数据,value为序列化数据。

至此实现了基于nginx负载均衡下 tomcat 集群的 session 一致性


评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值