扩展CXF, 支持LoadBalance负载均衡

原文：http://scud.blogjava.net

CXF是一个比较流行的Web Service框架. ( 当然如果追求更高效, 还可以去搜索ice, thrift, protobuff之类的)
近一个月, 断断续续地又好好看了看CXF的一些代码, CXF的文档还是很欠缺,特别是关于内部实现的东西. 从我的感觉来说, 内部实现还是挺复杂的. Inteceptor, Feature, ConduitSelector 这些概念一大堆, 又差不多可以做类似的事情, 真是让人头晕.

CXF本身提供了一个FailoverFeature, 可以在调用服务出错时切换到其他服务器, 但是无法做到负载均衡, 我研究了几天, 在FailoverFeature的基础上改出来一个LoadBalanceFeature, 当然也同时支持Failover.

首先我们来看看如何使用CXF的FailoverFeature: (下载示例中包括使用xml和代码两种方式, 当然CXF自己还提供了使用wsdl内部定义的方式)

我们需要先准备一个HelloService, 非常简单的一个Web Service, 这里不在贴出, 具体可以看下载包
调用代码示例:

package org.javascud.extensions.cxf.testfailover;

import org.apache.cxf.clustering.FailoverFeature;
import org.apache.cxf.clustering.RandomStrategy;
import org.apache.cxf.feature.AbstractFeature;
import org.apache.cxf.jaxws.JaxWsProxyFactoryBean;
import org.javascud.extensions.cxf.service.Hello;

import java.util.ArrayList;
import java.util.List;

public class HelloServiceFailOverClient
{
public static void main(String[]args)
{
StringhelloFirst = " http://localhost:8080/service/Hello " ;
StringhelloSecond = " http://localhost:8081/service/Hello " ;
StringhelloThird = " http://localhost:8082/service/Hello " ;
StringhelloFour = " http://localhost:8083/service/Hello " ;

List < String > serviceList = new ArrayList < String > ();
serviceList.add(helloFirst);
serviceList.add(helloSecond);
serviceList.add(helloThird);
// serviceList.add(helloFour);

RandomStrategystrategy = new RandomStrategy();
strategy.setAlternateAddresses(serviceList);

FailoverFeatureff = new FailoverFeature();
ff.setStrategy(strategy);

JaxWsProxyFactoryBeanfactory = new JaxWsProxyFactoryBean();

List < AbstractFeature > features = new ArrayList < AbstractFeature > ();
features.add(ff);

factory.setFeatures(features);
factory.initFeatures();

factory.setServiceClass(Hello. class );
// factory.setAddress(" http://localhost :8080/service/Hello");

Helloclient = (Hello)factory.create();
Stringresult = client.sayHello( " felix " );
System.out.println( " resultis: " + result);
}
}

在遇到错误时可以自动使用下一个服务器, 但是必须要自己设置一个地址, 如果不设置的话也可以, 但是会出错然后failover.


下面我们自己来看看我们的 LoadBalanceFeature

1. 首先我们创建一个LoadBalanceFeature (完全和FailoverFeature一样)

Feature是用来定制Server, Client, Bus的一个组件, 具体可以查看AbstractFeature, 我们使用initialize方法来定制Client, 修改Client的Conduit选择器达到负载均衡的目的.

LoadBalanceFeature代码如下:

/**
*ThisfeaturemaybeappliedtoaClientsoastoenable
*loadbalance,useanycompatibleendpointforthetargetservice.
*
* @author FelixZhangDate:2010-10-322:58
* @see org.apache.cxf.clustering.FailoverFeature
*/
public class LoadBalanceFeature extends AbstractFeature{
private LoadBalanceStrategyloadBalanceStrategy;

@Override
public void initialize(Clientclient,Busbus){
LoadBalanceTargetSelectorselector = new LoadBalanceTargetSelector();
selector.setEndpoint(client.getEndpoint());
selector.setStrategy(getStrategy());
client.setConduitSelector(selector);
}

public void setStrategy(LoadBalanceStrategystrategy){
loadBalanceStrategy = strategy;
}

public LoadBalanceStrategygetStrategy(){
return loadBalanceStrategy;
}

}

2. 定制一个LoadBalanceStrategy 负载均衡策略
负载均衡策略有很多种, 例如随机选择, 顺序选择等, FailoverFeature提供了三种策略, 总之很简单, 我们在这里就先实现随机策略, 其他的策略都很简单, 几行代码就可以实现了.

这个类主要用来设置/获取所有的提供服务的地址列表, 为了方便控制, 我新增了2个选项:
A: alwaysChangeEndpoint 是否每次请求都切换地址:　如果只有一个客户端, 可以分担负载. 缺省为true
B: removeFailedEndpoint 是否从全局的地址列表中移除失败服务地址 -- 如果你没有监测服务器状态的程序

关于动态增删服务地址

• 可以使用zookeeper等服务实时监测服务器状态, 或者自己写程序实现, 调用strategy.setAlternateAddresses即可.
• removeFailedEndpoint 如果设置为true, 但没有监测服务器状态的程序, 新增的或者复活的服务器则无法被恢复到地址列表中.
• 考虑到效率和支持failover, 设置地址列表, 移除地址等没有同步锁.
• 自动移除失败服务地址时, 目前仅支持手动地址列表, 没有考虑wsdl中的多服务地址.
• 后续我会写一个使用zookeeper增删服务地址列表的示例. (最近也在看zookeeper)

主要的代码都在AbstractLoadBalanceStrategy 中, 基本和 AbstractStaticFailoverStrategy 一样, 添加了一个removeAlternateAddress 用于移除失败的服务地址.

LoadBalanceStrategy 接口的代码如下:

/**
*Supportspluggablestrategiesforalternateendpointselectionon
*loadbalance.
*<p/>
*Random,Retries,Mod(later)
*<p/>
*1.supportloadbalance2.supportfailover.
*
* @author FelixZhangDate:2010-10-118:14
* @see org.apache.cxf.clustering.FailoverStrategy
*/
public interface LoadBalanceStrategy{

/**
*Getthealternateendpointsforthisinvocation.
*
* @param exchangethecurrentExchange
* @return afailoverendpointifoneisavailable
*/
List < Endpoint > getAlternateEndpoints(Exchangeexchange);

/**
*Selectoneofthealternateendpointsforaretriedinvocation.
*
* @param alternatesListofalternateendpointsifavailable
* @return theselectedendpoint
*/
EndpointselectAlternateEndpoint(List < Endpoint > alternates);

/**
*Getthealternateaddressesforthisinvocation.
*Theseaddressesover-rideanyaddressesspecifiedintheWSDL.
*
* @param exchangethecurrentExchange
* @return afailoverendpointifoneisavailable
*/
List < String > getAlternateAddresses(Exchangeexchange);

/**
*Selectoneofthealternateaddressesforaretriedinvocation.
*
* @param addressesListofalternateaddressesifavailable
* @return theselectedaddress
*/
StringselectAlternateAddress(List < String > addresses);

/**
*shouldremovefailedendpointornot.
*onlyworkforuserdefinedaddresseslist.
* @return trueorfalse
*/
boolean isRemoveFailedEndpoint();

/**
*changeendpointeverytimeornot.
* @return boolean
*/
boolean isAlwaysChangeEndpoint();

/**
*removefailedaddressfromlist.
* @param addressthefailedaddress
*/
void removeAlternateAddress(Stringaddress);
}

RandomLoadBalanceStrategy继承自 AbstractLoadBalanceStrategy, 和 RandomStrategy的区别就是获取下一个服务地址时并不从列表中移除此地址, 否则就做不到负载均衡了.


3. 最重要的 LoadBalanceTargetSelector
A: 这个类比较复杂, 我们为了实现负载均衡, 修改了 prepare来动态设置调用的endpoint, 替换策略取决于LoadBalanceStrategy
主要代码如下:

boolean existsEndpoint = false ;
// checkcurrentendpointisnotnull
EndpointtheEndpoint = exchange.get(Endpoint. class );
if (theEndpoint.getEndpointInfo().getAddress() != null ){
existsEndpoint = true ;
}

EndpointnextEndpoint;
if (getStrategy().isAlwaysChangeEndpoint() || ! existsEndpoint){
// getaendpointandsettocurrentendpoint
EndpointloadBalanceTarget = getLoadBalanceTarget(exchange);
if (loadBalanceTarget != null ){
logger.info( " switchtonexttarget: " + loadBalanceTarget.getEndpointInfo().getAddress());
setEndpoint(loadBalanceTarget);

// updateexchange.org.apache.cxf.message.Message.ENDPOINT_ADDRESS---不设置这个就用上次的奇怪
message.put(Message.ENDPOINT_ADDRESS,loadBalanceTarget.getEndpointInfo().getAddress());
}

nextEndpoint = loadBalanceTarget;
} else {
// usecurrentendpoint
nextEndpoint = theEndpoint;
}

B:为了和原有Failover特性兼容, 我们修改了 getFailoverTarget函数, 在此函数中要移除失败的服务地址, 因为在之前我们修改了LoadBalanceStrategy, 它在获取地址时不再移除当前地址, 所以我们需要手动移除.

部分代码如下:

StringcurrentAddress = getEndpoint().getEndpointInfo().getAddress();

// failovershouldremovecurrentendpointfirst,thengetnext--根据定义的策略来决定是否从全局地址列表中移除
if (getStrategy().isRemoveFailedEndpoint()){
logger.warn( " removecurrentfailedaddress: " + currentAddress);
// removeforclient,notforcurrentinvocation--没有同步锁
getStrategy().removeAlternateAddress(currentAddress);
}

// removeforcurrentinvocation:当前请求中总是移除失败服务地址
alternateAddresses.remove(currentAddress);

StringalternateAddress =
getStrategy().selectAlternateAddress(alternateAddresses);

4. 调用实例:

此处我们采用XML定义方式:

<? xmlversion="1.0"encoding="UTF-8" ?>
< beans xmlns ="http://www.springframework.org/schema/beans"
xmlns:xsi ="http://www.w3.org/2001/XMLSchema-instance"
xmlns:jaxws ="http://cxf.apache.org/jaxws"
xmlns:clustering ="http://cxf.apache.org/clustering"
xmlns:util ="http://www.springframework.org/schema/util"
xsi:schemaLocation ="
http://cxf.apache.org/jaxwshttp://cxf.apache.org/schemas/jaxws.xsd
http://www.springframework.org/schema/beanshttp://www.springframework.org/schema/beans/spring-beans.xsd
http://www.springframework.org/schema/utilhttp://www.springframework.org/schema/util/spring-util.xsd" >

< util:list id ="addressList" >
< value > http://localhost:8081/service/Hello </ value >
< value > http://localhost:8082/service/Hello </ value >
< value > http://localhost:8083/service/Hello </ value >
< value > http://localhost:8086/service/Hello </ value >
< value > http://localhost:8087/service/Hello </ value >
< value > http://localhost:8088/service/Hello </ value >
</ util:list >

< bean id ="SequentialAddresses" class ="org.apache.cxf.clustering.SequentialStrategy" >
< property name ="alternateAddresses" >
< ref bean ="addressList" />
</ property >
</ bean >

< bean id ="randomAddresses" class ="org.javascud.extensions.cxf.RandomLoadBalanceStrategy" >
< property name ="alternateAddresses" >
< ref bean ="addressList" />
</ property >
< property name ="removeFailedEndpoint" value ="true" />
</ bean >

< bean id ="loadBalanceFeature" class ="org.javascud.extensions.cxf.LoadBalanceFeature" >
< property name ="strategy" ref ="randomAddresses" />
</ bean >


< jaxws:client name ="helloClient"
serviceClass ="org.javascud.extensions.cxf.service.Hello" >
< jaxws:features >
< ref bean ="loadBalanceFeature" />
</ jaxws:features >
</ jaxws:client >


</ beans >

8081, 8082, 8083是实际存在的服务, 其他的不存在.


调用的Java代码:

package org.javascud.extensions.cxf.loadbalance;

import org.apache.cxf.endpoint.Client;
import org.apache.cxf.frontend.ClientProxy;
import org.javascud.extensions.cxf.LoadBalanceStrategy;
import org.javascud.extensions.cxf.service.Hello;
import org.springframework.context.support.ClassPathXmlApplicationContext;


public class HelloLoadBalanceAndFailOverClientByXML
{
public static void main(String[]args)
{
ClassPathXmlApplicationContextcontext
= new ClassPathXmlApplicationContext( new String[]
{ " org/javascud/extensions/cxf/loadbalance/loadbalance_fail.xml " });
Helloclient = (Hello)context.getBean( " helloClient " );

LoadBalanceStrategystrategy = (LoadBalanceStrategy)context.getBean( " randomAddresses " );

Clientmyclient = ClientProxy.getClient(client);
Stringaddress = myclient.getEndpoint().getEndpointInfo().getAddress();

System.out.println(address);

for ( int i = 1 ;i <= 20 ;i ++ )
{
Stringresult1 = client.sayHello( " Felix " + i);
System.out.println( " Call " + i + " : " + result1);

int left = strategy.getAlternateAddresses( null ).size();
System.out.println( " ==================left " + left + " =========================== " );
}


}
}

此处仅仅为模拟测试.


5. 关于测试用例
没想好如何写单元测试, test里面目前都是随意测试的代码, 基本照顾到所有功能.



6. 下载
代码下载: http://cnscud.googlecode.com/files/extensions-cxf_20101015.zip
源码位置: http://cnscud.googlecode.com/svn/trunk/extensions/ 其中cxf目录是此文章相关的源码.

7. 有任何问题请留言.