Soul网关源码分析-1期

今日任务

---

将一个http服务跑起来, 看看有什么关键类在作用, 以及最重要的网关server和client如何工作.

重点可以关注 DividePlugin 的 doExecute()

分析项目启动

---

今天切到了tag2.2.1 , 启动 soul-admin 和 soul-bootstrap , 看到 soul-admin 的一行信息很有意思

2021-01-14 16:38:23.704  INFO 4600 --- [0.0-9095-exec-5] o.d.s.a.l.AbstractDataChangedListener    : update config cache[PLUGIN], old:{group='PLUGIN', md5='ff9f3045505109e66c403fcc6a7a9a12', lastModifyTime=1610613405746}, updated:{group='PLUGIN', md5='ff9f3045505109e66c403fcc6a7a9a12', lastModifyTime=1610613503704}

看信息得知是连上网关之后, 配置的一些修改, 找到关键监听类 AbstractDataChangedListener

等了一段时间, soul-admin 接着又打了一些信息

2021-01-14 16:41:45.858  INFO 4600 --- [-long-polling-1] a.l.h.HttpLongPollingDataChangedListener : http sync strategy refresh config start.
2021-01-14 16:41:45.882  INFO 4600 --- [-long-polling-1] o.d.s.a.l.AbstractDataChangedListener    : update config cache[APP_AUTH], old:{group='APP_AUTH', md5='d751713988987e9331980363e24189ce', lastModifyTime=1610613405689}, updated:{group='APP_AUTH', md5='d751713988987e9331980363e24189ce', lastModifyTime=1610613705882}
2021-01-14 16:41:45.890  INFO 4600 --- [-long-polling-1] o.d.s.a.l.AbstractDataChangedListener    : update config cache[PLUGIN], old:{group='PLUGIN', md5='ff9f3045505109e66c403fcc6a7a9a12', lastModifyTime=1610613503704}, updated:{group='PLUGIN', md5='ff9f3045505109e66c403fcc6a7a9a12', lastModifyTime=1610613705890}
2021-01-14 16:41:45.895  INFO 4600 --- [-long-polling-1] o.d.s.a.l.AbstractDataChangedListener    : update config cache[RULE], old:{group='RULE', md5='d751713988987e9331980363e24189ce', lastModifyTime=1610613405772}, updated:{group='RULE', md5='d751713988987e9331980363e24189ce', lastModifyTime=1610613705895}
2021-01-14 16:41:45.899  INFO 4600 --- [-long-polling-1] o.d.s.a.l.AbstractDataChangedListener    : update config cache[SELECTOR], old:{group='SELECTOR', md5='d751713988987e9331980363e24189ce', lastModifyTime=1610613405799}, updated:{group='SELECTOR', md5='d751713988987e9331980363e24189ce', lastModifyTime=1610613705899}
2021-01-14 16:41:45.904  INFO 4600 --- [-long-polling-1] o.d.s.a.l.AbstractDataChangedListener    : update config cache[META_DATA], old:{group='META_DATA', md5='d751713988987e9331980363e24189ce', lastModifyTime=1610613405835}, updated:{group='META_DATA', md5='d751713988987e9331980363e24189ce', lastModifyTime=1610613705904}
2021-01-14 16:41:45.904  INFO 4600 --- [-long-polling-1] a.l.h.HttpLongPollingDataChangedListener : http sync strategy refresh config success.

看信息得知, http长轮询更新了些配置, 更新的配置组成有 APP_AUTH 、PLUGIN 、RULE 、SELECTOR 、META_DATA

启动 soul-test-http , soul-admin 如猜测打印了更新配置的信息, 且 soul-test-http 也打印了注册的服务信息

2021-01-14 17:33:33.709  INFO 5147 --- [pool-1-thread-1] s.c.s.i.SpringMvcClientBeanPostProcessor : http client register success :{} {"appName":"http","context":"/http","path":"/http/test/**","pathDesc":"","rpcType":"http","host":"172.20.71.40","port":8187,"ruleName":"/http/test/**","enabled":true,"registerMetaData":false}
2021-01-14 17:33:33.923  INFO 5147 --- [pool-1-thread-1] s.c.s.i.SpringMvcClientBeanPostProcessor : http client register success :{} {"appName":"http","context":"/http","path":"/http/order/save","pathDesc":"订单保存","rpcType":"http","host":"172.20.71.40","port":8187,"ruleName":"/http/order/save","enabled":true,"registerMetaData":true}
2021-01-14 17:33:34.055  INFO 5147 --- [pool-1-thread-1] s.c.s.i.SpringMvcClientBeanPostProcessor : http client register success :{} {"appName":"http","context":"/http","path":"/http/order/findById","pathDesc":"根据id获取","rpcType":"http","host":"172.20.71.40","port":8187,"ruleName":"/http/order/findById","enabled":true,"registerMetaData":true}
2021-01-14 17:33:34.152  INFO 5147 --- [pool-1-thread-1] s.c.s.i.SpringMvcClientBeanPostProcessor : http client register success :{} {"appName":"http","context":"/http","path":"/http/order/path/**","pathDesc":"","rpcType":"http","host":"172.20.71.40","port":8187,"ruleName":"/http/order/path/**","enabled":true,"registerMetaData":false}

这个test项目用到了 soul-spring-boot-starter-client-springmvc 的jar, 发送http注册服务的类也能看到: SpringMvcClientBeanPostProcessor, 在管理后台也可检查到这些服务.

@Override
public Object postProcessBeforeInitialization(@NonNull final Object bean, @NonNull final String beanName) throws BeansException {
    if (soulSpringMvcConfig.isFull()) {
        return bean;
    }
    Controller controller = AnnotationUtils.findAnnotation(bean.getClass(), Controller.class);
    RestController restController = AnnotationUtils.findAnnotation(bean.getClass(), RestController.class);
    RequestMapping requestMapping = AnnotationUtils.findAnnotation(bean.getClass(), RequestMapping.class);
    if (controller != null || restController != null || requestMapping != null) {
        String contextPath = soulSpringMvcConfig.getContextPath();
        SoulSpringMvcClient clazzAnnotation = AnnotationUtils.findAnnotation(bean.getClass(), SoulSpringMvcClient.class);
        String prePath = "";
        if (Objects.nonNull(clazzAnnotation)) {
            if (clazzAnnotation.path().indexOf("*") > 1) {
                String finalPrePath = prePath;
                executorService.execute(() -> post(buildJsonParams(clazzAnnotation, contextPath, finalPrePath)));
                return bean;
            }
            prePath = clazzAnnotation.path();
        }
        final Method[] methods = ReflectionUtils.getUniqueDeclaredMethods(bean.getClass());
        for (Method method : methods) {
            SoulSpringMvcClient soulSpringMvcClient = AnnotationUtils.findAnnotation(method, SoulSpringMvcClient.class);
            if (Objects.nonNull(soulSpringMvcClient)) {
                String finalPrePath = prePath;
                executorService.execute(() -> post(buildJsonParams(soulSpringMvcClient, contextPath, finalPrePath)));
            }
        }
    }
    return bean;
}

粗略看下 SpringMvcClientBeanPostProcessor 的实现, 借助 spring的生命周期, 在初始化的BeanPostProcessor阶段, 拿到定义了@SoulSpringMvcClient注解的所有类, 将注解的其他信息通过http发送给 soul-admin.

后台开放的获取服务的地址: /soul-client/springmvc-register , 不继续跟踪, 在研究后台时可以翻这个路径.

分析网关调用服务

---

调用注册在网关上的服务, 在 soul-bootstrap 查看打印信息:

2021-01-14 18:46:17.455  INFO 4629 --- [-work-threads-4] o.d.soul.plugin.base.AbstractSoulPlugin  : divide selector success match , selector name :/http
2021-01-14 18:46:17.456  INFO 4629 --- [-work-threads-4] o.d.soul.plugin.base.AbstractSoulPlugin  : divide rule success match ,rule name :/http/test/**
2021-01-14 18:46:17.457  INFO 4629 --- [-work-threads-4] o.d.s.plugin.httpclient.WebClientPlugin  : you request,The resulting urlPath is :http://172.20.71.40:8187/test/findByUserId?userId=1

看信息, 经过DividePlugin 匹配, 调用WebClientPlugin 插件访问服务.

此类在项目 soul-plugin-httpclient 下, 看到还有个reponse相关的WebClientResponsePlugin, 应该是返回响应的插件, 在这两个类, 和服务项目分别debug下, 看看调用顺序.

结果如同猜测, 另外, 响应返回的线程与请求线程不一致, 没有阻塞点, 可以之后借此扩展研究下spring-webflux .

开始分析DividePlugin 的代码

@Override
protected Mono<Void> doExecute(final ServerWebExchange exchange, final SoulPluginChain chain, final SelectorData selector, final RuleData rule) {
    final SoulContext soulContext = exchange.getAttribute(Constants.CONTEXT);
    assert soulContext != null;
    final DivideRuleHandle ruleHandle = GsonUtils.getInstance().fromJson(rule.getHandle(), DivideRuleHandle.class);
    final List<DivideUpstream> upstreamList = UpstreamCacheManager.getInstance().findUpstreamListBySelectorId(selector.getId());
    if (CollectionUtils.isEmpty(upstreamList)) {
      LOGGER.error("divide upstream configuration error：{}", rule.toString());
      Object error = SoulResultWarp.error(SoulResultEnum.CANNOT_FIND_URL.getCode(), SoulResultEnum.CANNOT_FIND_URL.getMsg(), null);
      return WebFluxResultUtils.result(exchange, error);
    }
    final String ip = Objects.requireNonNull(exchange.getRequest().getRemoteAddress()).getAddress().getHostAddress();
    DivideUpstream divideUpstream = LoadBalanceUtils.selector(upstreamList, ruleHandle.getLoadBalance(), ip);
    if (Objects.isNull(divideUpstream)) {
      LOGGER.error("divide has no upstream");
      Object error = SoulResultWarp.error(SoulResultEnum.CANNOT_FIND_URL.getCode(), SoulResultEnum.CANNOT_FIND_URL.getMsg(), null);
      return WebFluxResultUtils.result(exchange, error);
    }
    //设置一下 http url
    String domain = buildDomain(divideUpstream);
    String realURL = buildRealURL(domain, soulContext, exchange);
    exchange.getAttributes().put(Constants.HTTP_URL, realURL);
    //设置下超时时间
    exchange.getAttributes().put(Constants.HTTP_TIME_OUT, ruleHandle.getTimeout());
    return chain.execute(exchange);
}

1. 首先看到它将真实要访问的url等信息, 加到了ServerWebExchange 中, 并且在下个链里传递, 所有链上插件都共享这个上下文, 用来存储Http访问相关资源.

2. 第二个参数SoulPluginChain 则是插件链式调用的接口定义, 在 SoulWebHandler 中有其具体实现

   @Override
   public Mono<Void> execute(final ServerWebExchange exchange) {
     return Mono.defer(() -> {
       if (this.index < plugins.size()) {
         SoulPlugin plugin = plugins.get(this.index++);
         Boolean skip = plugin.skip(exchange);
         if (skip) {
           return this.execute(exchange);
         } else {
           return plugin.execute(exchange, this);
         }
       } else {
         return Mono.empty();
       }
     });
   }
   

   反应式的一次次调用plugins集合, 从索引0开始, 一个个的向后调用插件的execute().

整个构建真实Domain的路径, 由3个部分组成: DivideRuleHandle 规则处理、UpstreamCacheManager 资源缓存列表获取、LoadBalanceUtils 负载均衡类.

通过UpstreamCacheManager 以及 selector, 获得对应服务集群在缓存中的资源集合, 再通过 LoadBalanceUtils 和DivideRuleHandle 获得一个具体真实资源类. 最后构建为http调用的url, 装入上下文 ServerWebExchange 中并继续向下调用插件.

这里的LoadBalanceUtils 会根据传入的规则, 构造出不同的负载均衡策略的类, 看了下继承关系, 有hash、随机、循环三种策略.

这里还有个关键类没研究, 就是 SelectorData , 用来找到真实服务集群的, 猜测是 soul-admin 的服务信息同步到网关的缓存 UpstreamCacheManager 中, 根据配置的 selector 选择器策略做了一些隔离措施, 需要研究下管理后台的选择器了, 这里先暂缓.

总结欠缺处

---

今天分析了 DividePlugin 的流程, 但没有继续往下分析 WebClientPlugin 插件这一环了, 想debug看看中间经历的过程, 但回到家里的环境, 有些问题没跑起来项目, 没折腾了, 这个留给明天补足.

今天看到了同步配置信息的关键类AbstractDataChangedListener , 这块是soul网关动态更新配置的关键, 可以结合提供的同步配置的url /soul-client/springmvc-register 仔细研究, 分析同步策略以及之前在文档里看到的, 配置元数据的定义, 学习下.

说到同步配置, 没有使用 eureka时, 当服务项目接入网关后, 即在网关上完成了服务注册, 这点相当于注册中心了, 既然是注册中心, 就有CAP一说, 到底soul是保证高可用还是一致性, 也是我的疑问点, 看看它有哪些这方面的特性, 这块可以后面点再研究.

现在总共启动了3个项目, soul-admin 、soul-bootstrap 、soul-test-http , 这里的test项目, 可以换成 doubbo 和 springcloud, 测测不同点, 看看有什么其他关键类, 尤其是, 和 doubbo的对接是怎么做的, 这块我很好奇. 可以当做明后天的研究内容.