CSA in Clustered Enviroment

Cluster 机制可以使我们在服务器上并行运行应用. 负载分布在几个服务器上.在容错和扩展性上都有很大的优势.
JBoss 支持cluster机制,最简单的办法就是在同一网段上运行几个JBoss的实例, 用"run -c all " 命令启动每一个实例,就可以实现了clustering.

Cluster 的定义

一个cluster 就是一组节点,对JBoss而言,一个节点就是一个JBoss的实例.


几个节点组成一个"partition", 或成为一个Cluster.


Load Balancer


客户端(浏览器等)发送通过一定的协议发送到服务器,这时候,所有的请求被派发到不同的节点.

HTTP Services

HTTP session replication is used to replicate the state associated with your web clients on other nodes of a cluster. Thus, in the event one of your node crashes, another node in the cluster will be able to recover. Two distinct functions must be performed:

? Session state replication

? Load-balance of incoming invocations

State replication is directly handled by JBoss. When you run JBoss in the all configuration, session state replication is enabled by default. Just deploy your web application and its session state is already replicated across all JBoss instances in the cluster. However, Load-balancing is a different story, it is not handled by JBoss itself and requires additional software. As a very common scenario, we will demonstrate how to setup Apache and mod_jk. This activity could be either performed by specialized hardware switches or routers (Cisco LoadDirector for example) or any other dedicated software.

Apache is a well-known web server which can be extended by plugging modules. One of these modules, mod_jk (and the newest mod_jk2) has been specifically designed to allow forward requests from Apache to a Servlet container. Furthermore, it is also able to load-balance HTTP calls to a set of Servlet containers while maintaining sticky sessions, and this is what is actually interesting for us.

Using mod_jk 2.x with JBoss 4.0.0 and Apache2.2

Quick Overview

1. Download Apache2.2
2. Download modjk 2.x
3. Change the main Apache config to include modjk config
4. Create the modjk config
5. Configure the modjk workers (which JBoss nodes Apache uses)
6. Configure the Apache URIs served by modjk (the applications served by JBoss)
7. Restart Apache
8. Configure JBoss (Give each JBoss a jvmRoute for session stickness)
9. Restart JBoss (with run -c all)
10. Test it

More Details

This wiki outlines the various steps required to install a basic load-balancing solution based on JBoss/Tomcat and mod_jk 2.2.

Step #1: Download Apache2 Web Server
Get the latest Apache2 package from Apache.org and install it. We require no special configuration, just use the default settings. In the following steps, APACHE_HOME will represent the Apache install directory.

Step #2: Download mod_jk 2.x (DOWNLOAD)
Or Download the latest package available from Tomcats's 'Download Tomcat connector section' page.
Always download the latest stable release if possible (currently 2.2.4).
Rename the lib mod_jk.so and drop it in APACHE_HOME/modules directory.
NOTE: Don't use any release prior to mod_jk 1.2.15. Earlier releases are fairly buggy.

Step #3: Setup Apache to use modjk
Add this line at the very bottom in APACHE_HOME/conf/httpd.conf : # Include mod_jk configuration file
Include conf/mod-jk.conf
Step #4: Create the modjk config
Under APACHE_HOME/conf, create mod-jk.conf and populate it as follows:

 # Load mod_jk module
 # Specify the filename of the mod_jk lib
 LoadModule jk_module modules/mod_jk.so              
 
 #Where to find workers.properties
 JkWorkersFile conf/workers.properties             
 
 # Where to put jk logs
 JkLogFile logs/mod_jk.log            
 
 # Set the jk log level debug/error/info
 JkLogLevel info             
 
 # Select the log format
 JkLogStampFormat  "%a %b %d %H:%M:%S %Y"            
 
 # JkOptions indicates to send SSK KEY SIZE
 JkOptions +ForwardKeySize +ForwardURICompat -ForwardDirectories            
 
 # JkRequestLogFormat
 JkRequestLogFormat "%w %V %T"              
 
 # Mount your applications
 JkMount /application/* loadbalancer              
 
 # You can use external file for mount points.
 # It will be checked for updates each 60 seconds.
 # The format of the file is: /url=worker
 # /examples/*=loadbalancer              
 #JkMountFile conf/uriworkermap.properties                
 
 # Add shared memory.
 # This directive is present with 1.2.10 and
 # later versions of mod_jk, and is needed for
 # for load balancing to work properly
   JkShmFile logs/jk.shm             
 
 # Deny direct access to WEB-INF and META-INF      
   JkMount /CSA_CS/* loadbalance
 
 # Deny direct access to WEB-INF and META-INF     
      
    
    
     
     "/CSA_CS/WEB-INF/*">
     
     
          AllowOverride None
     
     
         deny from all
     
     
     
    
    

mod_jk is ready to forward requests to JBoss instances. We need now to setup the workers

Step #5: Configuring workers
Under APACHE_HOME/conf, create workers.properties and populate it as follows:

# Define list of workers that will be used
#for mapping requests The configuration directives are valid
#for the mod_jk version 1.2.18 and laterworker.list=loadbalancer,status#Define Node1
#modify the host as your host IP or DNS name.worker.node1.port=8009
worker.node1.host=node1.mydomain.com
worker.node1.type=ajp13
worker.node1.lbfactor=1
#worker.node1.connection_pool_size=10 (1)
 
#Define Node2
#modify the host as your host IP or DNS name.
worker.node2.port=8009
worker.node2.host= node2.mydomain.com
worker.node2.type=ajp13
worker.node2.lbfactor=1
#worker.node1.connection_pool_size=10 (1)#Load-balancing behaviour
worker.loadbalancer.type=lb
worker.loadbalancer.balance_workers=node1,node2#Status worker for managing load balancer
worker.status.type=status

Important: Please review http://tomcat.apache.org/connectors-doc/reference/workers.htmlfor the directive descriptions. Especially lookout for the comments on cachsize for Apache 1.3.x.
(1) You should only set the connection_pool_size if the number of allowed connection to the Httpd is higher than maxThreads in server.xml
If you specify worker.loadbalancer.sticky_session=Off, each request will be load balanced between node1 and node2. But when a user opens a Session on one server, it is a good idea to always forward this user's requests to the same server. Otherwise the user's session data would need to be synchronized between both servers. This is called a "sticky session", as the client is always using the same server he reached on his first request. Session stickiness is enabled by default.
Side Note: a non-loadbalanced setup with a single node required the "worker.list=node1" entry before mod_jk
would function correctly. Without this setting I would only get a 500 error and no other useful messages in log or otherwise. -Harlequin516
Side Note: I tried both loadbalanced and single node methods on Fedora 4. Both setups causing jk.shm errno=13 and jk-runtime-status errno=13 in the mod_jk.log. Could only get 500 errors. As a last resort disabled selinux on apache server. Restarted service and connection was made first try. -paulbrown

Step #6: Create the URI to worker map file
Create a uriworkermap.properties file in the APACHE_HOME/conf directory. This file should contain the URL mappings you want Apache to forward to Tomcat. The format of the file is /url=worker_name. To get things started, paste this example into the file you created:

   # Simple worker configuration file
   # Mount the Servlet context to the ajp13 worker   /jmx-console=loadbalancer
   /jmx-console/*=loadbalancer
   /web-console=loadbalancer
   /web-console/*=loadbalancer
   /CSA_CS=loadbalancer
   / CSA_CS/*=loadbalancer 

This will configure mod_jk to forward requests to /jmx-console , /web-console and /CSA_CS to JBoss.

Step #7: Restart Apache

Step #8: Configure JBoss
To complete the configuration, we also need to name each node accordingly to the names specified in workers.properties.
Edit JBOSS_HOME/server/all/deploy/ jbossweb-tomcat50.sar/server.xml (replace /all with your own server name)
Locate the element and add an attribute jvmRoute:

    
    
     
     "jboss.web" defaultHost=
     
     "localhost" jvmRoute=
     
     "node1">
     
     
 ..........
     
     

    
    

The jvmRoute attribute must match the name specified in workers.properties.
In the server.xml file, make sure that the AJP 1.3 Connector is uncommented, e.g.:

    
    

    
    
     
     "8009" address=
     
     "${jboss.bind.address}"     emptySessionPath=
     
     "true" enableLookups=
     
     "false" redirectPort=
     
     "8443"
     
     
      protocol=
     
     "AJP/1.3"/>
    
    

If you are only accepting requests via mod_jk, you can comment out the regular HTTP Connector; Tomcat then won't listen on port 8080.
Finally, we need to tell Tomcat to add the jvmRoute value to its session cookies so that mod_jk can route
incoming requests.
Edit JBOSS_HOME/server/all/deploy/jbossweb-tomcat50.sar/META-INF/jboss-service.xml (replace /all with your own server name)
Locate the element with a name of UseJK, and set its value to "true": true
Note that tomcat version can be 50 or 55 depending on the AS that you use.

Step #9: Restart JBoss AS with run -c all option.

Step #10: Access the JBoss AS web-console through Apache by browsing to http://localhost/web-consoleand you should see the JBoss web console page.

Additional Steps to deploy CSA on JBoss with clustering:

Step #1:
Remove commons-beanutils.jar and commons-collections.jar and put commons-beanutils-1.7.0.jar and commons-collections-3.1.jar from JBOSS_HOME/server/all/deploy/jbossweb-tomcat50.sar
If you face any problem while run CSA.

Step #2: You must deploy your application into JBOSS_HOME/server/all/deploy.