nginx+tomcat+memcached配置+脚本

最新推荐文章于 2024-07-12 02:15:27 发布

阿冰Firice

最新推荐文章于 2024-07-12 02:15:27 发布

阅读量561

点赞数

分类专栏： lnmp 文章标签： nginx memcached tomcat

本文链接：https://blog.csdn.net/firice_1379/article/details/43939115

版权

lnmp 专栏收录该内容

2 篇文章 0 订阅

订阅专栏

研究了下nginx+tomcat+memcached 的配置，练习了下脚本的书写，参考了好几篇文章，在此记录一下，有不对的地方敬请指正。

系统环境：
RHEL6.5 x64
iptables -F & selinux is disabled

主机角色：
node1 :192.168.122.101 :nginx tomcat memcached
node2 :192.168.122.102 :tomcat memcached

https://code.google.com/memcached-session-manager
memcached的session管理

nginx做反向代理两台tomcat，用memcached同步session，防止数据丢失

Tomcat1将session存储在memcacted2上.tomcat和Memcached使用交叉存储,只有当M2不可用时T1才将数据存储在M1上(M1是T1的failoverNode),使用这种方式可以避免单点故障.从而实现应用的高可用性.

注意:两台node上的tomcat应完全一致

1.在两台node主机上配置tomcat环境

#./jdk-6u32-linux-x64.bin

# mv jdk1.6.0_32/ /usr/local/lnmp/jdk


#vim /etc/profile              编写环境变量

export JAVA_HOME=/usr/local/lnmp/jdk

export CLASSPATH=:$JAVA_HOME/lib:$JAVA_HOME/jre/lib

export PATH=$PATH:$JAVA_HOME/bin

#source /etc/profile

测试java能否正常工作
#vim test.java

public class test{

public static void main(String[] args)

{System.out.println("Hello!");  }

}

#javac test.java          编译

#java test               执行后出现Hello!说明java环境配置好

安装tomcat服务器

#tar zxf apache-tomcat-7.0.37.tar.gz      解压就能用,不需要编译

#mv apache-tomcat-7.0.37 tomcat

#/TOMCAT_ROOT_DIR/tomcat/webapps/ROOT                tomcat的默认发布目录

#/TOMCAT_ROOT_DIR/tomcat/bin/startup.sh(shutdown.sh)    tomcat默认的启动和关闭脚本

tomcat默认开启8080端口,

测试http://192.168.122.101:8080 访问到tomcat默认的测试页

#cd tomcat/webapps/ROOT

#cat test.jsp           #测试页

this time is: <%=new java.util.Date()%>

测试访问;http://192.168.122.101:8080 显示当前时间

tomcat访问8080端口,使用nginx反向代理。

使用nginx发布jsp动态网页:

nginx配置文件:

cat /usr/local/nginx/conf/nginx.conf

#user  nginx nginx;
worker_processes  4;

#error_log  logs/error.log;
#error_log  logs/error.log  notice;
#error_log  logs/error.log  info;

#pid        logs/nginx.pid;


events {
    use epoll;
    worker_connections  1024;
}


http {
    upstream tomcat {  #两台tomcat负载均衡
    sticky;               #session同步，为nginx的模块
        server 192.168.0.1:8080;
        server 192.168.0.2:8080;
    }

    include       mime.types;
    default_type  application/octet-stream;

    #log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
    #                  '$status $body_bytes_sent "$http_referer" '
    #access_log  logs/access.log  main;

    sendfile        on;
    #tcp_nopush     on;

    #keepalive_timeout  0;
    keepalive_timeout  65;

    #gzip  on;

    server {
        listen       80;
        server_name  localhost;

        #charset koi8-r;

        #access_log  logs/host.access.log  main;

        location / {
            root   html;
            proxy_pass   http://tomcat;
            index  index.html index.htm;
        }

        #error_page  404              /404.html;

        # redirect server error pages to the static page /50x.html
        error_page   500 502 503 504  /50x.html;
        location = /50x.html {
            root   html;
        }

        location ~ \.jsp$ {
            proxy_pass   http://tomcat;
        }
}

nginx -t && nginx -s reload

访问:http://192.168.122.101/test.jsp 测试

同步两台tomcat的内容，并且修改java环境变量/etc/profile

测试：http://192.168.122.102:8080/test.jsp

两台node(tomcat)准备好了

2、在node1上的nginx环境中发布网页

使用nginx的负载均衡功能,

体现在nginx.conf中：

upstream tomcat-lb {

　　server 192.168.122.101:8080;

　　server 192.168.122.102:8080;

}

location ~ \.jsp$ {

　　proxy_pass http://tomcat-lb;

}

nginx -t && nginx -s reload

测试：http：//192.168.122.101/test.jsp
实现两台主机的负载均衡(访问nginx所在的node)

问题：在后台实现了nginx的负载均衡，但是当一个用户刷新数据时，数据总是变化的，试想，在动态页面，如果用户提交数据刷新后并没有提交到服务器上，并且必须要重新填写表格，这样会造成糟糕的用户体验。

解决:给nginx增加一个sticky模块.(需要重新编译nginx)

重新在nginx在进行模块化编译：

nginx-sticky-modules.tar.gz(解压就行)

#tar zxf nginx-sticky-modules.tar.gz -C /root/nginx-1.4.2
#cd nginx-1.4.2/
#make clean
#./configure --prefix=/usr/local/nginx --add-module=$NGINX_PKG_DIR/$DIR/nginx-sticky-module-1.0 --with-http_ssl_module --with-http_stub_status_module


#make && make install

#vim nginx.conf

upstream linux {
　　sticky;   #装了nginx-sticky后的功能
　　server 192.168.122.101:8080 ;
　　server 192.168.122.102:8080 ;
}

测试：http://192.168.122.101/test.jsp 后发现刷新不会来回负载(每个用户看到的其实不是一个tomcat上的数据,但是对用户是透明的)

3.nginx负载tomcat的jsp时,需要解决session共享:

使用memcache进行缓存(用户)后端数据,但是又要想到解决单点故障问题,因而可以采用两台memcache作为后端负载.

memcached默认端口11211，后端使用交叉存储(tomcat会将session同步，session自动寻找存储的memcached，但是默认是交叉存储，当一个memcached服务器坏掉，tomcat都会存到存活的memcached服务器上)

只要tomcat不宕掉,一切数据都还存在

但是当memcached宕掉,tomcat会向存活的memcached上存取

session 的序列化方案官方推荐的有 4 种:

java serialization
msm-kryo-serializer
msm-javolution-serializer
msm-xstream-serializer

其中性能最好的是Kryo,我们使用kryo来做

mecached服务器node1 and node2

#yum install memcached -y

#/etc/init.d/memcached start

memcached默认开启11211端口

测试访问:telnet localhost 11211

stats 查看状态
set user 0 0 3 存储新值(add replace)
get user 获取值

配置两台memcached服务器(tomcat),下载jar包(必须支持相关的java程序)
提供测试环境的包，解压即可用：
链接: http://pan.baidu.com/s/1mgIF9NU 密码: n5wu

asm-3.2.jar
couchbase-client-1.2.2.jar
kryo-1.03.jar
kryo-serializers-0.11.jar
memcached-session-manager-1.6.5.jar
memcached-session-manager-tc7-1.6.5.jar
minlog-1.2.jar
msm-kryo-serializer-1.6.5.jar
reflectasm-0.9.jar
spymemcached-2.10.3.jar

node1 and node2 同步内容

#cd tomcat/lib/

#mget jar/*    下载tomcat的session共享管理包

#cd tomcat/conf/

# vim context.xml

<Manager className="de.javakaffee.web.msm.MemcachedBackupSessionManager"
memcachedNodes="n1:192.168.122.101:11211,n2:192.168.122.102:11211"
failoverNodes="n1"    #tomcat2需要写成n2
requestUriIgnorePattern=".*\.(ico|png|gif|jpg|css|js)$" transcoderFactoryClass="de.javakaffee.web.msm.serializer.kryo.KryoTranscoderFactory"
/>


#tomcat/bin/shutdown.sh #重启tomcat,以识别memcached-session-manager

# tail -f logs/catalina.out    #默认日志

INFO: MemcachedSessionService finished initialization, sticky true, operation timeout 1000, with node ids [n2] and failover node ids [n1]

正常启动

编辑测试文件:(提交用户信息的jsp测试页)

# vim tomcat/webapps/ROOT/test.jsp             

<%@ page contentType="text/html; charset=GBK" %>

<%@ page import="java.util.*" %>

<html><head><title>Cluster App Test</title></head>

<body>

Server Info:

<%

out.println(request.getLocalAddr() + " : " + request.getLocalPort()+"<br>");%>

<%

out.println("<br> ID " + session.getId()+"<br>");

String dataName = request.getParameter("dataName");

if (dataName != null && dataName.length() > 0) {

String dataValue = request.getParameter("dataValue");

session.setAttribute(dataName, dataValue);

}

out.print("<b>Session list</b>");

Enumeration e = session.getAttributeNames();

while (e.hasMoreElements()) {

String name = (String)e.nextElement();

String value = session.getAttribute(name).toString();

out.println( name + " = " + value+"<br>");

System.out.println( name + " = " + value);

}

%>

<form action="test.jsp" method="POST">

name:<input type=text size=20 name="dataName">

<br>

key:<input type=text size=20 name="dataValue">

<br>

<input type=submit>

</form>

</body>

</html>

session共享完成:两台tomcat和memcached做相同的配置(jdk,memcache,tomcat)

测试:

http://192.168.122.101/test.jsp
使用一台tomcat和另外一台的memcached进行session共享

任何一台tomcat或者memcached挂了都无所谓

session会记录并且保持用户的数据信息

我同时写了一键安装的脚本，还有一些问题，也分享出来，希望有大神能指点～～

第一个，在有nginx的机器上运行：

#!/bin/bash

setenforce 0 > /dev/null
iptables -F > /dev/null
sed -i 's/^SELINUX=enforcing/SELINUX=disabled/g' /etc/selinux/config

###############  nginx + tomcat + memcached

DIR_NOW=`pwd`

IPADDR_NTM='192.168.122.101'
IPADDR_TM='192.168.122.102'

NGINX_PKG_DIR='/root/one_key_install'
NGINX_PKG_NAME='nginx-1.6.1.tar.gz'
NGINX_DIR='/usr/local/nginx'

TOMCAT_PKG_DIR='/root/one_key_install'
TOMCAT_PKG_NAME='apache-tomcat-7.0.37.tar.gz'
TOMCAT_DIR='/usr/local/tomcat'

STICKY_PKG_DIR='/root/one_key_install'
STICKY_PKG_NAME='nginx-sticky-module-1.0.tar.gz'

JDK_BIN_DIR='/root/one_key_install'
JDK_BIN_NAME='jdk-6u32-linux-x64.bin'

###################  nginx + sticky install

cd $NGINX_PKG_DIR
tar zxf $NGINX_PKG_NAME
DIR=`ls -F | grep /$ | grep nginx | awk -F '/' '{print $1}'`
tar zxf $STICKY_PKG_DIR/$STICKY_PKG_NAME -C $NGINX_PKG_DIR/$DIR

sed -i 's/^CFLAGS="$CFLAGS -g"/#CFLAGS="$CFLAGS -g"/g' /$NGINX_PKG_DIR/$DIR/auto/cc/gcc
sed -i 's/^#define NGINX_VER          \"nginx\/\" NGINX_VERSION/#define NGINX_VER          \"nginx\/\"/g' /$NGINX_PKG_DIR/$DIR/src/core/nginx.h

yum install gcc pcre-devel openssl-devel -y

cd $NGINX_PKG_DIR/$DIR
./configure --prefix=/usr/local/nginx --add-module=$NGINX_PKG_DIR/$DIR/nginx-sticky-module-1.0 --with-http_ssl_module --with-http_stub_status_module
make && make install

ln -s /usr/local/nginx/sbin/nginx /usr/local/sbin/
useradd -M -d /usr/local/nginx/ -s /sbin/nologin nginx

cd $DIR_NOW
rm -f /usr/local/nginx/conf/nginx.conf
cp nginx.conf.exp /usr/local/nginx/conf/nginx.conf
sed -i "s/server 192.168.0.1:8080;/server ${IPADDR_NTM}:8080;/g" /usr/local/nginx/conf/nginx.conf
sed -i "s/server 192.168.0.2:8080;/server ${IPADDR_TM}:8080;/g" /usr/local/nginx/conf/nginx.conf

#####################  tomcat install
cd $JDK_BIN_DIR
sh $JDK_BIN_NAME
DIR=`ls -F | grep /$ | grep jdk | awk -F '/' '{print $1}'`
mv $DIR /usr/local/
mv /usr/local/$DIR /usr/local/jdk
echo "export JAVA_HOME=/usr/local/jdk
export CLASSPATH=:\$JAVA_HOME/lib
export PATH=\$PATH:\$JAVA_HOME/bin
">>/etc/profile

cd $TOMCAT_PKG_DIR
tar zxf $TOMCAT_PKG_NAME -C /usr/local
cd /usr/local
mv /usr/local/`ls | grep tomcat` /usr/local/tomcat
ln -s /usr/local/tomcat/bin/startup.sh /usr/local/sbin/tomcat-start
ln -s /usr/local/tomcat/bin/shutdown.sh /usr/local/sbin/tomcat-stop

cd $DIR_NOW
tar zxf kryo_pkgs.tar.gz
cd kryo_pkgs
cp * /usr/local/tomcat/lib

cd $DIR_NOW
rm -f /usr/local/tomcat/conf/context.xml
cp context.xml.exp /usr/local/tomcat/conf/context.xml
 cd $DIR_NOW
rm -f /usr/local/tomcat/conf/context.xml
cp context.xml.exp /usr/local/tomcat/conf/context.xml
sed -i "s#memcachedNodes=\"n1:192.168.0.1:11211,n2:192.168.0.2:11211\"#memcachedNodes=\"n1:${IPADDR_NTM}:11211,n2:${IPADDR_TM}:11211\"#" /usr/local/tomcat/conf/context.xml

####################  memcached install

yum install memcached -y

####################  start services
/etc/init.d/memcached start
source /etc/profile
tomcat-start
nginx

第二个，在只有tomcat和memcached的机器上运行

#!/bin/bash

setenforce 0 > /dev/null
iptables -F > /dev/null
sed -i 's/^SELINUX=enforcing/SELINUX=disabled/g' /etc/selinux/config

DIR_NOW=`pwd`

IPADDR_NTM='192.168.122.101'
IPADDR_TM='192.168.122.102'

TOMCAT_PKG_DIR='/root/no_ngx'
TOMCAT_PKG_NAME='apache-tomcat-7.0.37.tar.gz'
TOMCAT_DIR='/usr/local/tomcat'

STICKY_PKG_DIR='/root/no_ngx'
STICKY_PKG_NAME='nginx-sticky-module-1.0.tar.gz'

JDK_BIN_DIR='/root/no_ngx'
JDK_BIN_NAME='jdk-6u32-linux-x64.bin'



cd $JDK_BIN_DIR
sh $JDK_BIN_NAME

DIR=`ls -F | grep /$ | grep jdk | awk -F '/' '{print $1}'`
mv $DIR /usr/local/
mv /usr/local/$DIR /usr/local/jdk
echo "export JAVA_HOME=/usr/local/jdk
export CLASSPATH=:\$JAVA_HOME/lib
export PATH=\$PATH:\$JAVA_HOME/bin
">>/etc/profile

cd $TOMCAT_PKG_DIR
tar zxf $TOMCAT_PKG_NAME -C /usr/local/
cd /usr/local
mv /usr/local/`ls | grep tomcat` /usr/local/tomcat


cd $DIR_NOW
tar zxf kryo_pkgs.tar.gz
cd kryo_pkgs
cp * /usr/local/tomcat/lib

cd $DIR_NOW
rm -f /usr/local/tomcat/conf/context.xml
cp context.xml.exp /usr/local/tomcat/conf/
mv /usr/local/tomcat/conf/context.xml.exp /usr/local/tomcat/conf/context.xml
sed -i "s#memcachedNodes=\"n1:192.168.0.1:11211,n2:192.168.0.2:11211\"#memcachedNodes=\"n1:${IPADDR_NTM}:11211,n2:${IPADDR_TM}:11211\"#" /usr/local/tomcat/conf/context.xml
sed -i 's/failoverNodes="n1"/failoverNodes="n2"/' /usr/local/tomcat/conf/context.xml

yum install memcached -y

/etc/init.d/memcached start

source /etc/profile
ln -s /usr/local/tomcat/bin/startup.sh /usr/local/sbin/tomcat-start
ln -s /usr/local/tomcat/bin/shutdown.sh /usr/local/sbin/tomcat-stop

tomcat-start