tomcat msm部署

本篇讲述tomcat msm，实现由memcached集中式管理会话模式。

实验环境

主机	端口	开源软件
192.168.161.73	8081	tomcat
192.168.161.73	8080	tomcat
192.168.161.73	11213	memcached
192.168.161.73	11214	memcached
192.168.161.73	8888	nginx

 

说明：

 8080、8081 分别为tomcat两实例;

11212、11213 分别为 memcached两实例;

web应用示例工程casdemo部署在tomcat两实例上。

Nginx 8888端口，如果为非80端口，用ngnix分发tomcat中J2ee应用重定向会自动跳转到80端口，需要做特殊处理。

MSM介绍

传统tomcat集群，会话复制随着结点数增多，扩展性成为瓶颈。Msm使用memcached完成统一管理tomcat会话，避免tomcat结点间过多会话复制。MSM会话分为sticky与no-ticky模式。

sticky ： 会话粘连模式。客户端在一台tomcat实例上完成登录后，以后的请求均会根据IP直接绑定到该tomcat实例。

no-sticky:会话非粘连模式。客户端的请求是随机分发，多台tomcat实例均会收到请求。

MSM依赖包

spymemcached-2.11.1.jar

reflectasm-1.01.jar

msm-kryo-serializer-1.8.3.jar

msm-javolution-serializer-1.8.3.jar

msm-flexjson-serializer-1.8.3.jar

minlog-1.2.jar

memcached-session-manager-tc8-1.8.3.jar      --tc8为tomcat的版本号。不同版本号tomcat，对应的包不同。此处为tomcat8的jar包

memcached-session-manager-1.8.3.jar

kryo-serializers-0.11.jar

kryo-1.04.jar

asm-3.2.jar

 放到tomcat/lib下

tomcat配置

1. 8080 端口tomcat 实例关键配置

修改tomcat目录下conf/server.xml。修改的内容下面用红色加粗标注。

<Server port="8005" shutdown="SHUTDOWN">       
 
 
    <Connector port="8080" protocol="HTTP/1.1"
               connectionTimeout="20000"
               redirectPort="8443" />
    
    <Connector port="8009" protocol="AJP/1.3" redirectPort="8443" />
  
 
    <Engine name="Catalina" defaultHost="localhost" jvmRoute="tomcat1">   ---非必选 ，只有选择了sticky模式才加入jvmRoute属性。不同的tomcat实例 jvmRoute取值不能相同。例：8080端口的tomcat实例jvmRoute=tomcat1,8081端口的tomcat实例jvmRoute=tomcat2

2. 8081 端口tomcat 实例关键配置

<Server port="9005" shutdown="SHUTDOWN">       
 
 
    <Connector port="8081" protocol="HTTP/1.1"
               connectionTimeout="20000"
               redirectPort="8443" />
    
    <Connector port="9009" protocol="AJP/1.3" redirectPort="8443" />
  
 
    <Engine name="Catalina" defaultHost="localhost" jvmRoute="tomcat2">   ---非必选 ，只有选择了sticky模式才加入jvmRoute属性。不同的tomcat实例 jvmRoute取值不能相同。例：8080端口的tomcat实例jvmRoute=tomcat1,8081端口的tomcat实例jvmRoute=tomcat2

3. msm配置

修改tomcat目录下conf/context.xml。修改的内容下面用红色加粗标注。

 

No-Stick模式

<Manager className="de.javakaffee.web.msm.MemcachedBackupSessionManager"
memcachedNodes="n1:192.168.161.73:11213,n2:192.168.161.73:11214" 
lockingMode="auto"
 sticky="false" 
requestUriIgnorePattern=".*\.(png|gif|jpg|css|js|ico|jpeg|htm|html)$" 
 sessionBackupAsync="false" 
sessionBackupTimeout="1800000"
 copyCollectionsForSerialization="false" transcoderFactoryClass="de.javakaffee.web.msm.serializer.kryo.KryoTranscoderFactory" />

 

Stick模式

<Manager 
 className="de.javakaffee.web.msm.MemcachedBackupSessionManager"      memcachedNodes="n1:192.168.161.73:11213,n2:192.168.161.73:11214" 
 lockingMode="auto"
sticky="true" 
failoverNodes="n1"
 requestUriIgnorePattern=".*\.(png|gif|jpg|css|js|ico|jpeg|htm|html)$"  transcoderFactoryClass="de.javakaffee.web.msm.serializer.kryo.KryoTransco
derFactory" />

运行memcached

1. 运行11213端口memcached实例 

./memcached -d -m 128 -p 11213 -u root

 

2. 运行11214端口memcached实例 

./memcached -d -m 128 -p 11214 -u root

运行ngnix

cd sbin

./ngnix

conf/nginx.conf供参考

#user  nobody;
user root root;
worker_processes  2;
worker_rlimit_nofile 65535;
#error_log  logs/error.log;
error_log  logs/error.log  notice;
#error_log  logs/error.log  info;
 
#pid        logs/nginx.pid;
 
 
events {
    use epoll;
    worker_connections  65535;
}
 
http {
    include       mime.types;
    default_type  application/octet-stream;
 
    log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
                      '$status $body_bytes_sent "$http_referer" '
                      '"$http_user_agent" "$http_x_forwarded_for"';
 
     
    access_log  logs/access.log  main;
#控制缓冲区溢出攻击
client_body_buffer_size  1K;
client_header_buffer_size 1k;
client_max_body_size 1k;
large_client_header_buffers 2 1k;
##cache##
proxy_connect_timeout 5;
proxy_read_timeout 60;
proxy_send_timeout 5;
proxy_buffer_size 16k;
proxy_buffers 4 64k;
gzip_proxied any;  
proxy_busy_buffers_size 128k;
proxy_temp_file_write_size 128k;
proxy_temp_path /home/temp_dir;
proxy_cache_path /home/cache levels=1:2 keys_zone=cache_one:200m inactive=1d max_size=1g;
  
    #gzip#
gzip    on;
gzip_vary on;
gzip_min_length   1k;
gzip_buffers   4 8k;
gzip_comp_level 4;
gzip_http_version  1.0;
    gzip_types   text/plain  text/css application/json application/x-javascript text/xml application/xml application/xml+rss text/javascript;
gzip_disable "MSIE [1-6]\.";
  
    sendfile        on;
    #tcp_nopush     on;
 
    #keepalive_timeout  0;
    keepalive_timeout  65;
 
    #gzip  on;
    upstream tc{
#ip_hash;
         server 192.168.161.73:8080;  
     server 192.168.161.73:8081;
    }
    server {
        listen       8888;
        server_name  localhost;
        charset utf-8;
 
        #access_log  logs/host.access.log  main;
 
        location /casdemo {
proxy_pass http://tc/casdemo/;
# $server_port 可以不要，只有nginx的端口是非80情况下有效
proxy_set_header        Host $host:$server_port; 
#proxy_set_header        X-Real-IP $remote_addr;
#proxy_set_header        X-Forwarded-For $proxy_add_x_forwarded_for;
}
location ~ .*\.(jsp|do|action)?$
{
# $server_port 可以不要，只有nginx的端口是非80情况下有效
proxy_set_header Host $host:$server_port;
#proxy_set_header        X-Real-IP $remote_addr;
#proxy_set_header X-Forwarded-For $remote_addr;
proxy_pass http://tc;
}
# location ~ .*\.(js)?$
# {
# proxy_pass http://tc;
        #                proxy_redirect off;
        #                proxy_cache_key $host$uri$is_args$args;
        #                proxy_set_header Host $host;
         #               proxy_cache cache_one;
          #              proxy_cache_valid 200 302 1h;
           #             proxy_cache_valid 301 1d;
            #            proxy_cache_valid any 1m;
# expires   1h;
# }  
        #error_page  404              /404.html;
 
        # redirect server error pages to the static page /50x.html
        #
        error_page   500 502 503 504  /50x.html;
        location = /50x.html {
            root   html;
        }
 
    
    }
 
}

casdemo应用

login.jsp

<%@ page language="java" contentType="text/html; charset=ISO-8859-1"
    pageEncoding="ISO-8859-1"%>
<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd">
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=ISO-8859-1">
<title>Insert title here</title>
</head>
<body>  <h1>TOMCAT实例1</h1>     <!--此处在不同8080与8081端口tomcat实例上分别为TOMCAT实例1 、TOMCAT实例2-->
<form action="login" method="post" >
<input type="text" name="username"/>
<input type="submit" name="login" value="login" />
</form>
</body>
</html>
 
usr/index.jsp
<%@ page language="java" contentType="text/html; charset=ISO-8859-1"
    pageEncoding="ISO-8859-1"%>
<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd">
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=ISO-8859-1">
<title>Insert title here</title>
</head>
<body>
<h1>TOMCAT实例2</h1>     <!--此处在不同8080与8081端口tomcat实例上分别为TOMCAT实例1 、TOMCAT实例2-->
Hello <%=request.getSession().getAttribute("user")%>!!!
<a href="../login">exit</a>
</body>
</html>

web.xml

<?xml version="1.0" encoding="UTF-8"?>
<web-app xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns="http://xmlns.jcp.org/xml/ns/javaee" xsi:schemaLocation="http://xmlns.jcp.org/xml/ns/javaee http://xmlns.jcp.org/xml/ns/javaee/web-app_3_1.xsd" id="WebApp_ID" version="3.1">
  <display-name>casdemo</display-name>
  <welcome-file-list>
    <welcome-file>index.html</welcome-file>
    <welcome-file>index.htm</welcome-file>
    <welcome-file>index.jsp</welcome-file>
    <welcome-file>default.html</welcome-file>
    <welcome-file>default.htm</welcome-file>
    <welcome-file>default.jsp</welcome-file>
  </welcome-file-list>
 
  <filter>
    <filter-name>CheckLoginFilter</filter-name>
    <filter-class>casdemo.CheckLoginFilter</filter-class>
  </filter>
  <listener>
    <listener-class>casdemo.DebugSessionListener</listener-class>
  </listener>
  <servlet>
     <servlet-name>login</servlet-name>
     <servlet-class>casdemo.Login</servlet-class>
</servlet>
   <servlet-mapping>
     <servlet-name>login</servlet-name>
     <url-pattern>/login</url-pattern>
</servlet-mapping>
  <filter-mapping>
    <filter-name>CheckLoginFilter</filter-name>
    <url-pattern>/usr/*</url-pattern>
  </filter-mapping>
</web-app>

CheckLoginFilter

package casdemo;
 
import java.io.IOException;
import javax.servlet.Filter;
import javax.servlet.FilterChain;
import javax.servlet.FilterConfig;
import javax.servlet.ServletException;
import javax.servlet.ServletRequest;
import javax.servlet.ServletResponse;
import javax.servlet.http.HttpServlet;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;
import javax.servlet.http.HttpServletResponseWrapper;
import javax.servlet.http.HttpSession;
public class CheckLoginFilter implements Filter {
 
@Override
public void destroy() {
// TODO Auto-generated method stub
}
 
@Override
public void doFilter(ServletRequest request, ServletResponse response,
FilterChain chain) throws IOException, ServletException {
//System.out.println("=====");
HttpServletRequest req=((HttpServletRequest) request);
//System.out.println("getRequestURL  :"+req.getRequestURL());
//System.out.println("getQueryString  :"+req.getQueryString());
System.out.println((req.getSession(false)==null)+"isRequestedSessionIdFromCookie :"+req.isRequestedSessionIdFromCookie());
System.out.println((req.getSession(false)==null)+"isRequestedSessionIdValid :"+req.isRequestedSessionIdValid());
if(req.getSession(false)!=null&&!req.isRequestedSessionIdValid()){
System.out.println("====session is not valid");
}
HttpSession session=req.getSession();
session.setMaxInactiveInterval(1000*60*30);
if(session.getAttribute("user")!=null&&!session.getAttribute("user").equals("")){
System.out.println("alreay login");
chain.doFilter(request, response);
}else{
System.out.println("not login");
//HttpServletResponse resp=((HttpServletResponse) response);
HttpServletResponseWrapper wrapper = new HttpServletResponseWrapper(
(HttpServletResponse) response);
wrapper.sendRedirect("/casdemo/login.jsp");
}
}
 
@Override
public void init(FilterConfig arg0) throws ServletException {
// TODO Auto-generated method stub
}

 
}

Login

package casdemo;
 
import java.io.IOException;
 
import javax.servlet.ServletConfig;
import javax.servlet.ServletException;
import javax.servlet.annotation.WebServlet;
import javax.servlet.http.HttpServlet;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;
import javax.servlet.http.HttpSession;
 
 
/**
 * Servlet implementation class Login
 */
public class Login extends HttpServlet {
private static final long serialVersionUID = 1L;
     
    /**
     * @see HttpServlet#HttpServlet()
     */
    public Login() {
    
        super();
        // TODO Auto-generated constructor stub
    }
    private ApplicationContext applicationContext; 
/**
 * @see HttpServlet#doGet(HttpServletRequest request, HttpServletResponse response)
 */
protected void doGet(HttpServletRequest request, HttpServletResponse response) throws ServletException, IOException {
HttpSession session=request.getSession();
session.invalidate();
response.sendRedirect("login.jsp");
}
public void init(ServletConfig config) throws ServletException { 
       // TODO Auto-generatedmethod stub 
    super.init(config); 
  
} 
/**
 * @see HttpServlet#doPost(HttpServletRequest request, HttpServletResponse response)
 */
protected void doPost(HttpServletRequest request, HttpServletResponse response) throws ServletException, IOException {
HttpSession session=request.getSession();
String username=request.getParameter("username");
session.setAttribute("user",username);
response.sendRedirect("usr/index.jsp");
}
 
}

DebugSessionListener

package casdemo;

import javax.servlet.http.HttpSession;
import javax.servlet.http.HttpSessionEvent;
import javax.servlet.http.HttpSessionListener;

public class DebugSessionListener  implements HttpSessionListener {
	
	public void sessionCreated(HttpSessionEvent event) {
		HttpSession session = event.getSession();
		String sessionId = session.getId();
		System.out.println(">>>>>>>>>>>create session id " + sessionId);
	}

	public void sessionDestroyed(HttpSessionEvent event) {
		HttpSession session = event.getSession();
		String sessionId = session.getId();
		System.out.println(">>>>>>>>>>>destory session id" + sessionId);
	}
}

源码地址

部署测试

访问http://localhost:8888/casdemo 

1. 测试负载分发：

测试网页是来自于tomcat1或者tomcat2的页面请求，如果是来自于tomcat1，登录页面（login.jsp）显示“TOMCAT1”，登录成功后跳转页面（index.jsp），显示“TOMCAT1”。

 

 

2. 测试tomcat单点故障

登录成功后跳转页面（index.jsp），根据网页上显示的tomcat实例号，手工关闭该tomcat实例。

刷新跳转页面（index.jsp），查看页面是否还能维持会话。如果跳转到登录页，说明会话丢失。如果不跳转，显示新tomcat实例 ，说明会话已经在新的tomcat实例完成共享。

3. 测试memcached单点故障

关闭某台memcached，测试会话是否正常 。如果两台memcached没有保持同步,关闭会话没有缺失的memcached，会造成会话丢失,因为余下的memcached会话数据不完整。