1 概述
Dubbo中的路由策略的作用是服务消费端使用路由策略对服务提供者进行过滤和选择，最终获取符合路由规则的服务提供者。
Dubbo中的路由策略主要分为两类，StateRouter和普通Router。
StateRouter （如TagStateRouter、ConditionStateRouter等）是在Dubbo 3中引入的路由实现，是Dubbo 3中主要使用的路由策略。它可根据服务提供者的状态信息（如负载、响应时间等）进行路由决策。
普通Router是指根据预设的路由规则进行路由决策。
2 路由策略的加载过程
2.1 加载过程剖析
服务消费端启动时，会加载远程服务对应的路由规则，并创建路由规则链。
具体而言，在服务消费端启动时，会将远程服务转换为Invoker，在此过程中，会调用 RegistryProtocol 的 doCreateInvoker 方法，其中会调用 RegistryDirectory 的 buildRouterChain() 方法创建路由规则链 RouterChain。具体实现细节如下所示。
// org.apache.dubbo.registry.integration.RegistryProtocol#getInvoker
public <T> ClusterInvoker<T> getInvoker(Cluster cluster, Registry registry, Class<T> type, URL url) {
// FIXME, this method is currently not used, create the right registry before enable.
DynamicDirectory<T> directory = new RegistryDirectory<>(type, url);
return doCreateInvoker(directory, cluster, registry, type);
}
// org.apache.dubbo.registry.integration.RegistryProtocol#doCreateInvoker
protected <T> ClusterInvoker<T> doCreateInvoker(DynamicDirectory<T> directory, Cluster cluster, Registry registry, Class<T> type) {
directory.setRegistry(registry);
directory.setProtocol(protocol);
// all attributes of REFER_KEY
Map<String, String> parameters = new HashMap<>(directory.getConsumerUrl().getParameters());
URL urlToRegistry = new ServiceConfigURL(
parameters.get(PROTOCOL_KEY) == null ? CONSUMER : parameters.get(PROTOCOL_KEY),
parameters.remove(REGISTER_IP_KEY),
0,
getPath(parameters, type),
parameters
);
urlToRegistry = urlToRegistry.setScopeModel(directory.getConsumerUrl().getScopeModel());
urlToRegistry = urlToRegistry.setServiceModel(directory.getConsumerUrl().getServiceModel());
if (directory.isShouldRegister()) {
directory.setRegisteredConsumerUrl(urlToRegistry);
registry.register(directory.getRegisteredConsumerUrl());
}
// 1、建立路由规则链
directory.buildRouterChain(urlToRegistry);
// 2、订阅服务提供者地址，生成invoker
directory.subscribe(toSubscribeUrl(urlToRegistry));
// 3、封装集群容错策略到invoker
return (ClusterInvoker<T>) cluster.join(directory, true);
}以下为加载路由规则和创建路由规则链的实现细节。
org.apache.dubbo.registry.integration.DynamicDirectory#buildRouterChain
-> org.apache.dubbo.rpc.cluster.RouterChain#buildChain
// org.apache.dubbo.registry.integration.DynamicDirectory#buildRouterChain
public void buildRouterChain(URL url) {
this.setRouterChain(RouterChain.buildChain(getInterface(), url));
}
// org.apache.dubbo.rpc.cluster.RouterChain#buildChain
public static <T> RouterChain<T> buildChain(Class<T> interfaceClass, URL url) {
SingleRouterChain<T> chain1 = buildSingleChain(interfaceClass, url);
SingleRouterChain<T> chain2 = buildSingleChain(interfaceClass, url);
return new RouterChain<>(new SingleRouterChain[]{chain1, chain2});
}
public static <T> SingleRouterChain<T> buildSingleChain(Class<T> interfaceClass, URL url) {
ModuleModel moduleModel = url.getOrDefaultModuleModel();
List<RouterFactory> extensionFactories = moduleModel.getExtensionLoader(RouterFactory.class)
.getActivateExtension(url, ROUTER_KEY);
// 加载 Router 路由规则
List<Router> routers = extensionFactories.stream()
.map(factory -> factory.getRouter(url))
.sorted(Router::compareTo)
.collect(Collectors.toList());
// 加载 StateRouter 路由规则
List<StateRouter<T>> stateRouters = moduleModel
.getExtensionLoader(StateRouterFactory.class)
.getActivateExtension(url, ROUTER_KEY)
.stream()
.map(factory -> factory.getRouter(interfaceClass, url))
.collect(Collectors.toList());
boolean shouldFailFast = Boolean.parseBoolean(ConfigurationUtils.getProperty(moduleModel, Constants.SHOULD_FAIL_FAST_KEY, "true"));
RouterSnapshotSwitcher routerSnapshotSwitcher = ScopeModelUtil.getFrameworkModel(moduleModel).getBeanFactory().getBean(RouterSnapshotSwitcher.class);
// 创建路由链
return new SingleRouterChain<>(routers, stateRouters, shouldFailFast, routerSnapshotSwitcher);
}
public SingleRouterChain(List<Router> routers, List<StateRouter<T>> stateRouters, boolean shouldFailFast, RouterSnapshotSwitcher routerSnapshotSwitcher) {
initWithRouters(routers);
initWithStateRouters(stateRouters);
this.shouldFailFast = shouldFailFast;
this.routerSnapshotSwitcher = routerSnapshotSwitcher;
}以下为构建 StateRouter 路由链的实现细节。
private void initWithStateRouters(List<StateRouter<T>> stateRouters) {
StateRouter<T> stateRouter = TailStateRouter.getInstance();
for (int i = stateRouters.size() - 1; i >= 0; i--) {
StateRouter<T> nextStateRouter = stateRouters.get(i);
// 设置当前路由节点的下一个路由节点
nextStateRouter.setNextRouter(stateRouter);
stateRouter = nextStateRouter;
}
this.headStateRouter = stateRouter;
this.stateRouters = Collections.unmodifiableList(stateRouters);
}2.2 默认加载的路由策略
消费端启动时，默认加载的 StateRouter 路由策略有7个，并构成一个路由策略链，包括 TagStateRouter、ServiceStateRouter 等；Router 为 0 个。可以通过以下测试类获悉。
/**
* verify the router and state router loaded by default
*/
@Test
void testBuildRouterChain() {
RouterChain<DemoService> routerChain = createRouterChanin();
Assertions.assertEquals(0, routerChain.getRouters().size());
Assertions.assertEquals(7, routerChain.getStateRouters().size());
}
private RouterChain<DemoService> createRouterChanin() {
Map<String, String> parameters = new HashMap<>();
parameters.put(INTERFACE_KEY, DemoService.class.getName());
parameters.put("registry", "zookeeper");
URL url = new ServiceConfigURL("dubbo",
"127.0.0.1",
20881,
DemoService.class.getName(),
parameters);
RouterChain<DemoService> routerChain = RouterChain.buildChain(DemoService.class, url);
return routerChain;
}
3 路由策略的使用过程
服务消费端根据负载均衡策略获取服务提供者前，会先根据路由策略获取符合路由规则的服务提供者。具体细节如下所示。
（1）以下是服务消费端发起远程调用的主要过程。
org.apache.dubbo.rpc.cluster.support.AbstractClusterInvoker#invoke
public Result invoke(final Invocation invocation) throws RpcException {
checkWhetherDestroyed();
// binding attachments into invocation.
// Map<String, Object> contextAttachments = RpcContext.getClientAttachment().getObjectAttachments();
// if (contextAttachments != null && contextAttachments.size() != 0) {
// ((RpcInvocation) invocation).addObjectAttachmentsIfAbsent(contextAttachments);
// }
InvocationProfilerUtils.enterDetailProfiler(invocation, () -> "Router route.");
// 1、获取服务提供者列表-Invokers
List<Invoker<T>> invokers = list(invocation);
InvocationProfilerUtils.releaseDetailProfiler(invocation);
checkInvokers(invokers, invocation);
// 2、获取负载均衡策略。根据url参数找LoadBalance扩展，默认RandomLoadBalance
LoadBalance loadbalance = initLoadBalance(invokers, invocation);
RpcUtils.attachInvocationIdIfAsync(getUrl(), invocation);
InvocationProfilerUtils.enterDetailProfiler(invocation, () -> "Cluster " + this.getClass().getName() + " invoke.");
try {
// 3、执行远程调用。子类实现，会有不同的集群容错方式
return doInvoke(invocation, invokers, loadbalance);
} finally {
InvocationProfilerUtils.releaseDetailProfiler(invocation);
}
}
protected List<Invoker<T>> list(Invocation invocation) throws RpcException {
return getDirectory().list(invocation);
}（2）以下是根据路由策略获取符合路由规则的服务提供者。
org.apache.dubbo.rpc.cluster.directory.AbstractDirectory#list
public List<Invoker<T>> list(Invocation invocation) throws RpcException {
if (destroyed) {
throw new RpcException("Directory of type " + this.getClass().getSimpleName() + " already destroyed for service " + getConsumerUrl().getServiceKey() + " from registry " + getUrl());
}
BitList<Invoker<T>> availableInvokers;
SingleRouterChain<T> singleChain = null;
try {
try {
if (routerChain != null) {
routerChain.getLock().readLock().lock();
}
// use clone to avoid being modified at doList().
if (invokersInitialized) {
availableInvokers = validInvokers.clone();
} else {
availableInvokers = invokers.clone();
}
// 获取路由规则链
if (routerChain != null) {
singleChain = routerChain.getSingleChain(getConsumerUrl(), availableInvokers, invocation);
singleChain.getLock().readLock().lock();
}
} finally {
if (routerChain != null) {
routerChain.getLock().readLock().unlock();
}
}
// 根据路由规则信息和invoker列表，获取经过路由规则筛选后的服务提供者列表
List<Invoker<T>> routedResult = doList(singleChain, availableInvokers, invocation);
if (routedResult.isEmpty()) {
// 2-2 - No provider available.
logger.warn(CLUSTER_NO_VALID_PROVIDER, "provider server or registry center crashed", "",
"No provider available after connectivity filter for the service " + getConsumerUrl().getServiceKey()
+ " All routed invokers' size: " + routedResult.size()
+ " from registry " + this
+ " on the consumer " + NetUtils.getLocalHost()
+ " using the dubbo version " + Version.getVersion() + ".");
}
return Collections.unmodifiableList(routedResult);
} finally {
if (singleChain != null) {
singleChain.getLock().readLock().unlock();
}
}
}org.apache.dubbo.registry.integration.DynamicDirectory#doList
public List<Invoker<T>> doList(SingleRouterChain<T> singleRouterChain,
BitList<Invoker<T>> invokers, Invocation invocation) {
if (forbidden && shouldFailFast) {
// 1. No service provider 2. Service providers are disabled
throw new RpcException(RpcException.FORBIDDEN_EXCEPTION, "No provider available from registry " +
this + " for service " + getConsumerUrl().getServiceKey() + " on consumer " +
NetUtils.getLocalHost() + " use dubbo version " + Version.getVersion() +
", please check status of providers(disabled, not registered or in blacklist).");
}
if (multiGroup) {
return this.getInvokers();
}
try {
// 执行路由策略，获取服务提供者列表
// Get invokers from cache, only runtime routers will be executed.
List<Invoker<T>> result = singleRouterChain.route(getConsumerUrl(), invokers, invocation);
return result == null ? BitList.emptyList() : result;
} catch (Throwable t) {
// 2-1 - Failed to execute routing.
logger.error(CLUSTER_FAILED_SITE_SELECTION, "", "",
"Failed to execute router: " + getUrl() + ", cause: " + t.getMessage(), t);
return BitList.emptyList();
}
}org.apache.dubbo.rpc.cluster.SingleRouterChain#route
public List<Invoker<T>> route(URL url, BitList<Invoker<T>> availableInvokers, Invocation invocation) {
if (invokers.getOriginList() != availableInvokers.getOriginList()) {
logger.error(INTERNAL_ERROR, "", "Router's invoker size: " + invokers.getOriginList().size() +
" Invocation's invoker size: " + availableInvokers.getOriginList().size(),
"Reject to route, because the invokers has changed.");
throw new IllegalStateException("reject to route, because the invokers has changed.");
}
if (RpcContext.getServiceContext().isNeedPrintRouterSnapshot()) {
return routeAndPrint(url, availableInvokers, invocation);
} else {
return simpleRoute(url, availableInvokers, invocation);
}
}
public List<Invoker<T>> simpleRoute(URL url, BitList<Invoker<T>> availableInvokers, Invocation invocation) {
BitList<Invoker<T>> resultInvokers = availableInvokers.clone();
// 1. route state router
resultInvokers = headStateRouter.route(resultInvokers, url, invocation, false, null);
if (resultInvokers.isEmpty() && (shouldFailFast || routers.isEmpty())) {
printRouterSnapshot(url, availableInvokers, invocation);
return BitList.emptyList();
}
if (routers.isEmpty()) {
return resultInvokers;
}
List<Invoker<T>> commonRouterResult = resultInvokers.cloneToArrayList();
// 2. route common router
for (Router router : routers) {
// Copy resultInvokers to a arrayList. BitList not support
RouterResult<Invoker<T>> routeResult = router.route(commonRouterResult, url, invocation, false);
commonRouterResult = routeResult.getResult();
if (CollectionUtils.isEmpty(commonRouterResult) && shouldFailFast) {
printRouterSnapshot(url, availableInvokers, invocation);
return BitList.emptyList();
}
// stop continue routing
if (!routeResult.isNeedContinueRoute()) {
return commonRouterResult;
}
}
if (commonRouterResult.isEmpty()) {
printRouterSnapshot(url, availableInvokers, invocation);
return BitList.emptyList();
}
return commonRouterResult;
}使用基于BitMap实现的BitList，对不同路由策略之间的结果取交集（&），得到最终的路由结果。具体实现如下所示。
org.apache.dubbo.rpc.cluster.router.state.AbstractStateRouter#route
public final BitList<Invoker<T>> route(BitList<Invoker<T>> invokers, URL url, Invocation invocation, boolean needToPrintMessage, Holder<RouterSnapshotNode<T>> nodeHolder) throws RpcException {
if (needToPrintMessage && (nodeHolder == null || nodeHolder.get() == null)) {
needToPrintMessage = false;
}
RouterSnapshotNode<T> currentNode = null;
RouterSnapshotNode<T> parentNode = null;
Holder<String> messageHolder = null;
// pre-build current node
if (needToPrintMessage) {
parentNode = nodeHolder.get();
currentNode = new RouterSnapshotNode<>(this.getClass().getSimpleName(), invokers.clone());
parentNode.appendNode(currentNode);
// set parent node's output size in the first child invoke
// initial node output size is zero, first child will override it
if (parentNode.getNodeOutputSize() < invokers.size()) {
parentNode.setNodeOutputInvokers(invokers.clone());
}
messageHolder = new Holder<>();
nodeHolder.set(currentNode);
}
BitList<Invoker<T>> routeResult;
routeResult = doRoute(invokers, url, invocation, needToPrintMessage, nodeHolder, messageHolder);
if (routeResult != invokers) {
// 对不同的路由策略之间的结果取交集（&）
routeResult = invokers.and(routeResult);
}
// check if router support call continue route by itself
if (!supportContinueRoute()) {
// use current node's result as next node's parameter
if (!shouldFailFast || !routeResult.isEmpty()) {
routeResult = continueRoute(routeResult, url, invocation, needToPrintMessage, nodeHolder);
}
}
// post-build current node
if (needToPrintMessage) {
currentNode.setRouterMessage(messageHolder.get());
if (currentNode.getNodeOutputSize() == 0) {
// no child call
currentNode.setNodeOutputInvokers(routeResult.clone());
}
currentNode.setChainOutputInvokers(routeResult.clone());
nodeHolder.set(parentNode);
}
return routeResult;
}4 路由策略介绍和实践案例
StateRouter 主要包含 TagStateRouter、ConditionStateRouter 等。
4.1 TagStateRouter
4.1.1 概述
TagStateRouter 指标签路由规则。标签路由通过将某一个或多个服务提供者划分到同一个分组，设置对应的分组标签，可以约束流量只在指定分组中流转，从而实现流量隔离的目的，可以作为蓝绿发布、灰度发布等场景的能力基础。
简单来说就是服务消费者可以设置只调用指定标签的服务提供者。另外服务消费者启动时默认会加载 TagStateRouter 。
4.1.2 源码剖析
下面为 TagStateRouter 进行路由决策的主要代码，TagStateRouter 的 doRoute() 方法。
public BitList<Invoker<T>> doRoute(BitList<Invoker<T>> invokers, URL url, Invocation invocation, boolean needToPrintMessage, Holder<RouterSnapshotNode<T>> nodeHolder, Holder<String> messageHolder) throws RpcException {
if (CollectionUtils.isEmpty(invokers)) {
if (needToPrintMessage) {
messageHolder.set("Directly Return. Reason: Invokers from previous router is empty.");
}
return invokers;
}
// since the rule can be changed by config center, we should copy one to use.
final TagRouterRule tagRouterRuleCopy = tagRouterRule;
if (tagRouterRuleCopy == null || !tagRouterRuleCopy.isValid() || !tagRouterRuleCopy.isEnabled()) {
if (needToPrintMessage) {
messageHolder.set("Disable Tag Router. Reason: tagRouterRule is invalid or disabled");
}
return filterUsingStaticTag(invokers, url, invocation);
}
BitList<Invoker<T>> result = invokers;
// 获取服务消费者调用时指定的tag
String tag = StringUtils.isEmpty(invocation.getAttachment(TAG_KEY)) ? url.getParameter(TAG_KEY) :
invocation.getAttachment(TAG_KEY);
// 获取指定tag的服务提供者
// if we are requesting for a Provider with a specific tag
if (StringUtils.isNotEmpty(tag)) {
Set<String> addresses = tagRouterRuleCopy.getTagnameToAddresses().get(tag);
// filter by dynamic tag group first
if (addresses != null) { // null means tag not set
result = filterInvoker(invokers, invoker -> addressMatches(invoker.getUrl(), addresses));
// if result is not null OR it's null but force=true, return result directly
if (CollectionUtils.isNotEmpty(result) || tagRouterRuleCopy.isForce()) {
if (needToPrintMessage) {
messageHolder.set("Use tag " + tag + " to route. Reason: result is not null OR it's null but force=true");
}
return result;
}
} else {
// dynamic tag group doesn't have any item about the requested app OR it's null after filtered by
// dynamic tag group but force=false. check static tag
result = filterInvoker(invokers, invoker -> tag.equals(invoker.getUrl().getParameter(TAG_KEY)));
}
// If there's no tagged providers that can match the current tagged request. force.tag is set by default
// to false, which means it will invoke any providers without a tag unless it's explicitly disallowed.
if (CollectionUtils.isNotEmpty(result) || isForceUseTag(invocation)) {
if (needToPrintMessage) {
messageHolder.set("Use tag " + tag + " to route. Reason: result is not empty or ForceUseTag key is true in invocation");
}
return result;
}
// FAILOVER: return all Providers without any tags.
else {
BitList<Invoker<T>> tmp = filterInvoker(
invokers,
invoker -> addressNotMatches(invoker.getUrl(), tagRouterRuleCopy.getAddresses())
);
if (needToPrintMessage) {
messageHolder.set("FAILOVER: return all Providers without any tags");
}
return filterInvoker(tmp, invoker -> StringUtils.isEmpty(invoker.getUrl().getParameter(TAG_KEY)));
}
} else {
// List<String> addresses = tagRouterRule.filter(providerApp);
// return all addresses in dynamic tag group.
Set<String> addresses = tagRouterRuleCopy.getAddresses();
if (CollectionUtils.isNotEmpty(addresses)) {
result = filterInvoker(invokers, invoker -> addressNotMatches(invoker.getUrl(), addresses));
// 1. all addresses are in dynamic tag group, return empty list.
if (CollectionUtils.isEmpty(result)) {
if (needToPrintMessage) {
messageHolder.set("all addresses are in dynamic tag group, return empty list");
}
return result;
}
// 2. if there are some addresses that are not in any dynamic tag group, continue to filter using the
// static tag group.
}
if (needToPrintMessage) {
messageHolder.set("filter using the static tag group");
}
return filterInvoker(result, invoker -> {
String localTag = invoker.getUrl().getParameter(TAG_KEY);
return StringUtils.isEmpty(localTag);
});
}
}4.1.3 实践
标签路由规则的使用主要分为两步：
- 对服务提供者进行分组，设置分组标签；
- 服务消费者调用远程服务时指定分组标签。
（1）对服务提供者进行分组，设置分组标签
目前有两种方式可以完成实例分组，分别是动态规则打标和静态规则打标，其中动态规则相较于静态规则优先级更高，而当两种规则同时存在且出现冲突时，将以动态规则为准。
动态规则打标，可随时在dubbo服务治理控制台上进行设置。分组配置举例如下。
# test-provider应用增加了两个标签分组tag1和tag2
# tag1包含一个实例 127.0.0.1:20880
# tag2包含一个实例 127.0.0.1:20881
force: false
runtime: true
enabled: true
key: test-provider
tags:
- name: tag1
addresses: ["127.0.0.1:20880"]
- name: tag2
addresses: ["127.0.0.1:20881"]
其中参数的含义如下所示。
key 明确规则体作用到哪个应用。必填。
enabled=true 当前路由规则是否生效，可不填，缺省生效。
force=false 当路由结果为空时，是否强制执行，如果不强制执行，路由结果为空的路由规则将自动失效，可不填，缺省为 false。
runtime=false 是否在每次调用时执行路由规则，否则只在提供者地址列表变更时预先执行并缓存结果，调用时直接从缓存中获取路由结果。如果用了参数路由，必须设为 true，需要注意设置会影响调用的性能，可不填，缺省为 false。
tags 定义具体的标签分组内容，可定义任意n（n>=1）个标签并为每个标签指定实例列表。必填。
name 标签名称
addresses 当前标签包含的实例列表
（2）服务消费者调用远程服务时指定分组标签
可以使用注解或在xml中设置。举例如下。
@DubboReference(tag = "tag1")
private HelloService helloService;下面是使用该路由规则来选择服务提供者的测试用例。
@Test
void testTagRouteWithDynamicRuleV4() {
TagStateRouter router = (TagStateRouter) new TagStateRouterFactory().getRouter(TagRouterRule.class, url);
router = Mockito.spy(router);
List<Invoker<String>> originInvokers = new ArrayList<>();
// 给服务提供者设置 tag
URL url1 = URL.valueOf("test://192.168.1.8:7777/DemoInterface?application=foo&dubbo.tag=tag1").setScopeModel(moduleModel);
URL url2 = URL.valueOf("test://192.168.1.8:7778/DemoInterface?application=foo&dubbo.tag=tag2").setScopeModel(moduleModel);
URL url3 = URL.valueOf("test://192.168.1.8:7779/DemoInterface?application=foo").setScopeModel(moduleModel);
Invoker<String> invoker1 = new MockInvoker<>(url1, true);
Invoker<String> invoker2 = new MockInvoker<>(url2, true);
Invoker<String> invoker3 = new MockInvoker<>(url3, true);
originInvokers.add(invoker1);
originInvokers.add(invoker2);
originInvokers.add(invoker3);
BitList<Invoker<String>> invokers = new BitList<>(originInvokers);
// 服务消费者调用时设置只调用指定 tag 的服务提供者
RpcInvocation invocation = new RpcInvocation();
invocation.setAttachment(TAG_KEY, "tag2");
TagRouterRule rule = getTagRule();
Mockito.when(router.getInvokers()).thenReturn(invokers);
rule.init(router);
router.setTagRouterRule(rule);
List<Invoker<String>> filteredInvokers = router.route(invokers, invokers.get(0).getUrl(), invocation, false, new Holder<>());
Assertions.assertEquals(1, filteredInvokers.size());
}
920
被折叠的 条评论
为什么被折叠?
到【灌水乐园】发言
