nginx+tomcat+memcached

主机角色

	releases	Iptables	Selinux	IP	service
Server1	Rhel6	Disabled	Disabled	172.25.37.1	Nginx Memecached tomcat
Server2	Rhel6	Disabled	Disabled	172.25.37.2	Memecached tomcat

 

结构图：

Tomcat-1 (T1) 将 session 存储在 memcached-2 (T2)上。只有当 M2 不可用时,T1 才将 session 存储在 memcached-1 上(M1是T1的failoverNode)。使用这种配置的好处是,当 T1 和 M1 同时崩溃时也不会丢失 session 会话,避免单点故障。

以下步骤在 server1 与 server2 上实施:

【1】tomcat 安装

[root@server2 ~]# tar zxf jdk-7u79-linux-x64.tar.gz 

[root@server2 ~]# mv jdk1.7.0_79/ /usr/local/jdk

[root@server2 ~]# vim /etc/profile

在文件末尾添加下面三项：
export  JAVA_HOME=/usr/local/jdk
export  CLASSPATH=:$JAVA_HOME/lib
export  PATH=$PATH:$JAVA_HOME/bin

[root@server2 ~]# source  /etc/profile

[root@server2 ~]# which java

/usr/local/jdk/bin/java

[root@server2 ~]# tar zxf apache-tomcat-7.0.37.tar.gz

[root@server2 ~]# mv apache-tomcat-7.0.37 /usr/local/tomcat

session 的序列化方案官方推荐的有 4 种:

1. java serialization

2. msm-kryo-serializer

3. msm-javolution-serializer

4. msm-xstream-serializer

其中性能最好的序列化方案是 Kryo,此实验我们采用 kryo 方式。

把如下软件包放置到/usr/local/tomcat/lib 目录中

kryo-1.03.jar

kryo-serializers-0.8.jar

memcached-2.5.jar

memcached-session-manager-1.5.1.jar

memcached-session-manager-tc7-1.5.1.jar

minlog-1.2.jar

msm-kryo-serializer-1.5.1.jar

reflectasm-0.9.jar

[root@server2 ~]# cd memcache_soft/

[root@server2 memcache_soft]# ls

asm-3.2.jar                              minlog-1.2.jar

kryo-1.04.jar                            msm-kryo-serializer-1.6.3.jar

kryo-serializers-0.10.jar                reflectasm-1.01.jar

memcached-session-manager-1.6.3.jar      spymemcached-2.7.3.jar

memcached-session-manager-tc7-1.6.3.jar

[root@server2 memcache_soft]# mv * /usr/local/tomcat/lib/

[root@server2 memcache_soft]# vim /usr/local/tomcat/conf/context.xml

[root@server1 conf]# pwd
/usr/local/tomcat/conf
[root@server1 conf]# cat context.xml 
<?xml version='1.0' encoding='utf-8'?>
<Context>

    <WatchedResource>WEB-INF/web.xml</WatchedResource>
    <Valve className="org.apache.catalina.valves.CometConnectionManagerValve" />
    -->
    <Manager className="de.javakaffee.web.msm.MemcachedBackupSessionManager"
    memcachedNodes="n1:172.25.37.1:11211,n2:172.25.37.2:11211"
    failoverNodes="n1"   注：server2这里改为n2
    requestUriIgnorePattern=".*\.(ico|png|gif|jpg|css|js)$"
    transcoderFactoryClass="de.javakaffee.web.msm.serializer.kryo.KryoTranscoderFactory"
    />
</Context>

启动 tomcat

/usr/local/tomcat/bin/startup.sh

关闭 tomcat

/usr/local/tomcat/bin/shutdown.sh

【2】memcached 安装

[root@server2 ~]# yum install -y libevent-devel-1.4.13-4.el6.x86_64.rpm libevent-doc-1.4.13-4.el6.noarch.rpm libevent-headers-1.4.13-4.el6.noarch.rpm 

[root@server2 ~]# tar zxf memcached-1.5.8.tar.gz

[root@server2 ~]# cd memcached-1.5.8

./configure --prefix=/usr/local/memcached

&& make && make install

[root@server2 memcached-1.5.8]# ln -s /usr/local/memcached/bin/memcached /usr/local/sbin/memcached

打开memcached:

[root@server2 ~]# memcached -u root &

【3】nginx 安装

以下步骤在 server1上实施:

安装依赖软件：

[root@server1 ~]# yum install -y pcre-devel openssl-devel

[root@server1 ~]# tar zxf nginx-1.14.0.tar.gz 

[root@server1 ~]# tar zxf nginx-sticky-module-ng.tar.gz 

[root@server1 ~]# useradd -s /sbin/nologin nginx

[root@server1 ~]# cd nginx-1.14.0

[root@server1 nginx-1.14.0]# ./configure --user=nginx --group=nginx --with-http_ssl_module --with-http_stub_status_module --add-module=../nginx-sticky-module-ng

[root@server1 ~]# make && make install

注: nginx-sticky-module 为 nginx 的第三方模块,使 nginx 支持 sticky 模式,所谓 sticky 模式就是指同一个用户的访问请求都被发送到同一个 tomcat 实例上处理。

[root@server1 conf]# ln -s /usr/local/nginx/sbin/nginx

[root@server1 conf]# pwd

/usr/local/nginx/conf

[root@server1 conf]# cat nginx.conf
user  nginx;
worker_processes  1;
pid        logs/nginx.pid;

events {
    worker_connections  1024;
}

http {
    include       mime.types;
    default_type  application/octet-stream;
    
    upstream tomcat {
        sticky;
        server 172.25.37.1:8080;
        server 172.25.37.2:8080;
    }
    sendfile        on;
    tcp_nopush     on;
    keepalive_timeout  65;
    gzip  on;

    server {
        listen       80;
        server_name  tomcat;
        location / {
            root   html;
            index  index.html index.htm;
        }
        error_page   500 502 503 504  /50x.html;
        location = /50x.html {
            root   html;
        }
#所有 jsp 页面交给 tomcat 处理,动静分离
        location ~ \.jsp$ {
             proxy_pass    http://tomcat;
        }
    }
}

启动nginx:

[root@server1 conf]# nginx

到此配置完成,可以测试了:

以下为测试页面,保存到/usr/local/tomcat/webapps/ROOT/test.jsp

[root@server1 conf]# cd /usr/local/tomcat/webapps/ROOT/
[root@server1 ROOT]# cat test.jsp 
<%@ page contentType="text/html; charset=GBK" %>
<%@ page import="java.util.*" %>
<html><head><title>Cluster App Test</title></head>
<body>
Server Info:
<%
out.println(request.getLocalAddr() + " : " + request.getLocalPort()+"<br>");%>
<%
out.println("<br> ID " + session.getId()+"<br>");
String dataName = request.getParameter("dataName");
if (dataName != null && dataName.length() > 0) {
String dataValue = request.getParameter("dataValue");
session.setAttribute(dataName, dataValue);
}
out.print("<b>Session list</b>");
Enumeration e = session.getAttributeNames();
while (e.hasMoreElements()) {
String name = (String)e.nextElement();
String value = session.getAttribute(name).toString();
out.println( name + " = " + value+"<br>");
System.out.println( name + " = " + value);
}
%>
<form action="test.jsp" method="POST">
name:<input type=text size=20 name="dataName">
<br>
key:<input type=text size=20 name="dataValue">
<br>
<input type=submit>
</form>
</body>
</html>

访问 http://172.25.37.1/test.jsp,不同的主机访问时会调度到不同的 tomcat 实例上处理

来自同一主机的请求会交给同一个 tomcat 实例处理,如下图添加几组键值对：

此时你 down 掉当前正在响应的 tomcat 实例

[root@server2 ~]# /usr/local/tomcat/bin/shutdown.sh 

nginx 会自动把用户的请求调度到另一个 tomcat 实例上,同时 session 也没有丢掉。

你也可以在终端通过查看日志的方式查看到session信息

[root@server2 ~]#tail  -f  /usr/local/tomcat/log/catalina.out