Soul网关源码阅读08-使用http长轮询同步数据(01)

最新推荐文章于 2021-02-03 20:51:01 发布

lq9616

最新推荐文章于 2021-02-03 20:51:01 发布

阅读量210

点赞数

分类专栏： Soul 网关源码阅读

本文链接：https://blog.csdn.net/liquan961646757/article/details/112992256

版权

Soul 网关源码阅读专栏收录该内容

18 篇文章 0 订阅

订阅专栏

前面两篇文章分析了zookeeper的数据同步原理，本篇体验http长轮询同步数据

一、环境配置

1、soul-admin 网关配置管理服务
soul-admin/src/main/resources/application.yml
打开http长轮询配置。
注意：soul-admin–>pom.xml 中的 websocket、zookeeper、nacos 等依赖无需删除，否则服务无法启动，仅需要把相应配置注释即可。

soul:
  sync:
#    websocket:
#      enabled: true
#      zookeeper:
#          url: localhost:2181
#          sessionTimeout: 5000
#          connectionTimeout: 2000
      http:
        enabled: true
  #    nacos:
  #      url: localhost:8848
  #      namespace: 1c10d748-af86-43b9-8265-75f487d20c6c

2、soul-bootstrap 网关服务
soul-bootstrap/src/main/resources/application-local.yml
打开http长轮询配置。

soul :
    sync:
#        websocket :
#             urls: ws://localhost:9095/websocket
#        zookeeper:
#             url: localhost:2181
#             sessionTimeout: 5000
#             connectionTimeout: 2000
        http:
             url : http://localhost:9095
#        nacos:
#              url: localhost:8848
#              namespace: 1c10d748-af86-43b9-8265-75f487d20c6c
......

soul-bootstrap/pom.xml ，http同步依赖是默认加入的

<!--soul data sync start use http-->
<dependency>
    <groupId>org.dromara</groupId>
    <artifactId>soul-spring-boot-starter-sync-data-http</artifactId>
    <version>${project.version}</version>
</dependency>

3、配置完成后，记得分别重启 soul-admin、soul-bootstrap

4、soul-admin 启动后的打印日志

HttpLongPollingDataChangedListener 应该是用来 http长轮询处理的，这里的同步机制是每300000ms同步一次。

2021-01-22 14:01:07.163  INFO 62771 --- [ main] a.l.h.HttpLongPollingDataChangedListener : http sync strategy refresh interval: 300000ms
2021-01-22 14:16:08.227  INFO 62771 --- [-long-polling-2] a.l.h.HttpLongPollingDataChangedListener : http sync strategy refresh config start.
2021-01-22 14:16:08.898  INFO 62771 --- [-long-polling-2] a.l.h.HttpLongPollingDataChangedListener : http sync strategy refresh config success.

5、soul-bootstrap 启动后的打印日志

根据打印日志可以看出，HttpSyncDataService类根据配置的soul-admin地址，发起了http数据同步请求。http://localhost:9095/configs/fetch?groupKeys=APP_AUTH&groupKeys=PLUGIN&groupKeys=RULE&groupKeys=SELECTOR&groupKeys=META_DATA

2021-01-22 14:03:43.532  INFO 62950 --- [           main] .s.s.b.s.s.d.h.HttpSyncDataConfiguration : you use http long pull sync soul data
2021-01-22 14:03:44.176  INFO 62950 --- [           main] o.d.s.s.data.http.HttpSyncDataService    : request configs: [http://localhost:9095/configs/fetch?groupKeys=APP_AUTH&groupKeys=PLUGIN&groupKeys=RULE&groupKeys=SELECTOR&groupKeys=META_DATA]
2021-01-22 14:03:45.436  INFO 62950 --- [onPool-worker-2] o.d.s.s.d.h.refresh.AppAuthDataRefresh   : clear all appAuth data cache
2021-01-22 14:03:45.436  INFO 62950 --- [onPool-worker-3] o.d.s.s.d.http.refresh.MetaDataRefresh   : clear all metaData cache
2021-01-22 14:03:45.449  INFO 62950 --- [           main] o.d.s.s.data.http.HttpSyncDataService    : get latest configs: [{"code":200,"message":"success","data":{"META_DATA":{"md5":"d751713988987e9331980363e24189ce","lastModifyTime":1611295267161,"data":[]},"SELECTOR":
......
{"id":"1352160464805158912","name":"/http/order/findById","pluginName":"divide","selectorId":"1352160417631821824","matchMode":0,"sort":2,"enabled":true,"loged":true,"handle":"{\"requestVolumeThreshold\":\"0\",\"errorThresholdPercentage\":\"0\",\"maxConcurrentRequests\":\"0\",\"sleepWindowInMilliseconds\":\"0\",\"loadBalance\":\"random\",\"timeout\":3000,\"retry\":\"0\"}","conditionDataList":[{"paramType":"uri","operator":"=","paramName":"/","paramValue":"/http/order/findById"}]}]}}}]
2021-01-22 14:03:45.586  INFO 62950 --- [           main] o.s.b.a.e.web.EndpointLinksResolver      : Exposing 2 endpoint(s) beneath base path '/actuator'
2021-01-22 14:03:46.642  INFO 62950 --- [           main] o.s.b.web.embedded.netty.NettyWebServer  : Netty started on port(s): 9195
2021-01-22 14:03:46.650  INFO 62950 --- [           main] o.d.s.b.SoulBootstrapApplication         : Started SoulBootstrapApplication in 8.173 seconds (JVM running for 9.86)

6、尝试在网关后台更改插件规则信息
在这里插入图片描述
soul-admin 发送了规则变更的响应

2021-01-22 14:46:00.368  INFO 62771 --- [0.0-9095-exec-7] o.d.s.a.l.AbstractDataChangedListener    : update config cache[RULE], old: {group='RULE', md5='55ac8e48340aae787d6d0bd6cd644578', lastModifyTime=1611297671016}, updated: {group='RULE', md5='d59a425a1cdbb06d0212544517a2767d', lastModifyTime=1611297960368}
2021-01-22 14:46:00.391  INFO 62771 --- [-long-polling-2] a.l.h.HttpLongPollingDataChangedListener : send response with the changed group,ip=127.0.0.1, group=RULE, changeTime=1611297960370

soul-bootstrap 同时收到了，插件变更的通知

2021-01-22 14:46:00.418  INFO 62950 --- [-long-polling-1] o.d.s.s.data.http.HttpSyncDataService    : Group config changed: [RULE]
2021-01-22 14:46:00.466  INFO 62950 --- [-long-polling-1] o.d.s.s.data.http.HttpSyncDataService    : request configs: [http://localhost:9095/configs/fetch?groupKeys=RULE]
2021-01-22 14:46:00.564  INFO 62950 --- [-long-polling-1] o.d.s.s.d.h.refresh.AbstractDataRefresh  : update RULE config: {"md5":"d59a425a1cdbb06d0212544517a2767d","lastModifyTime":1611297960368,"data":
......
{\"requestVolumeThreshold\":\"0\",\"errorThresholdPercentage\":\"0\",\"maxConcurrentRequests\":\"0\",\"sleepWindowInMilliseconds\":\"0\",\"loadBalance\":\"random\",\"timeout\":3000,\"retry\":\"0\"}","conditionDataList":[{"paramType":"uri","operator":"\u003d","paramName":"/","paramValue":"/http/order/findById"}]}]}
2021-01-22 14:46:00.635  INFO 62950 --- [-long-polling-1] o.d.s.s.data.http.HttpSyncDataService    : get latest configs: [{"code":200,"message":"success","data":{"RULE":{"md5":"d59a425a1cdbb06d0212544517a2767d","lastModifyTime":1611297960368,"data":[{"id":"1352160436929814528","name":"/http/test/**","pluginName":"divide","selectorId":"1352160417631821824","matchMode":0,"sort":2,"enabled":true,"loged":true,"handle":"
......
\"}","conditionDataList":[{"paramType":"uri","operator":"=","paramName":"/","paramValue":"/http/order/findById"}]}]}}}]

7、观察上面日志打印的过程，大概梳理一下调用过程

1）启动服务：soul-admin，soul-bootstrap 建立http长轮询连接。定时同步策略为300秒。

2）soul-admin 更改网关数据，发送一个响应给到 soul-bootstrap，告知发生了 “Group config changed: [RULE]”。

3）soul-bootstrap 收到变更通知，重新 request configs: [http://localhost:9095/configs/fetch?groupKeys=RULE]，同步数据至内存。

二、探究soul http长轮询

通过上面的调试观察，我们大概知道了http长轮询同步数据的过程，下面我们继续深入到源码一探究竟。

1、soul-admin --> ConfigController

org.dromara.soul.admin.controller.ConfigController

由注释可知，此类会根据 HttpLongPollingDataChangedListener为条件是否注入容器，HttpLongPollingDataChangedListener 由类名可知通过http长轮询监听数据的变化。

fetchConfigs 方法负责同步数据

listener 方法负责监听

/**
 * This Controller only when HttpLongPollingDataChangedListener exist, will take effect.
 * @author huangxiaofeng
 * @author xiaoyu
 */
@ConditionalOnBean(HttpLongPollingDataChangedListener.class)
@RestController
@RequestMapping("/configs")
@Slf4j
public class ConfigController {
    @Resource
    private HttpLongPollingDataChangedListener longPollingListener;
    /**
     * Fetch configs soul result.
     * @param groupKeys the group keys
     * @return the soul result
     */
    @GetMapping("/fetch")
    public SoulAdminResult fetchConfigs(@NotNull final String[] groupKeys) {
        Map<String, ConfigData<?>> result = Maps.newHashMap();
        for (String groupKey : groupKeys) {
            ConfigData<?> data = longPollingListener.fetchConfig(ConfigGroupEnum.valueOf(groupKey));
            result.put(groupKey, data);
        }
        return SoulAdminResult.success(SoulResultMessage.SUCCESS, result);
    }
    /**
     * Listener.
     * @param request  the request
     * @param response the response
     */
    @PostMapping(value = "/listener")
    public void listener(final HttpServletRequest request, final HttpServletResponse response) {
        longPollingListener.doLongPolling(request, response);
    }
}

2、HttpLongPollingDataChangedListener http长轮询实现

org.dromara.soul.admin.listener.http.HttpLongPollingDataChangedListener

此类实现的内容较多，我们还是从方法的调用关系来看

fetchConfigs 方法的实现在抽象类 AbstractDataChangedListener

HttpLongPollingDataChangedListener 继承 AbstractDataChangedListener

fetchConfigs 方法主要负责从内存中获取 APP_AUTH，PLUGIN，RULE，SELECTOR，META_DATA为key的网关数据并返回。

soul-bootstrap启动后就是从这个方法获取的全部网关数据。

/**
 * fetch configuration from cache.
 * @param groupKey the group key
 * @return the configuration data
 */
public ConfigData<?> fetchConfig(final ConfigGroupEnum groupKey) {
    ConfigDataCache config = CACHE.get(groupKey.name());
    switch (groupKey) {
        case APP_AUTH:
            List<AppAuthData> appAuthList = GsonUtils.getGson().fromJson(config.getJson(), new TypeToken<List<AppAuthData>>() {
            }.getType());
            return new ConfigData<>(config.getMd5(), config.getLastModifyTime(), appAuthList);
        case PLUGIN:
            List<PluginData> pluginList = GsonUtils.getGson().fromJson(config.getJson(), new TypeToken<List<PluginData>>() {
            }.getType());
            return new ConfigData<>(config.getMd5(), config.getLastModifyTime(), pluginList);
        case RULE:
            List<RuleData> ruleList = GsonUtils.getGson().fromJson(config.getJson(), new TypeToken<List<RuleData>>() {
            }.getType());
            return new ConfigData<>(config.getMd5(), config.getLastModifyTime(), ruleList);
        case SELECTOR:
            List<SelectorData> selectorList = GsonUtils.getGson().fromJson(config.getJson(), new TypeToken<List<SelectorData>>() {
            }.getType());
            return new ConfigData<>(config.getMd5(), config.getLastModifyTime(), selectorList);
        case META_DATA:
            List<MetaData> metaList = GsonUtils.getGson().fromJson(config.getJson(), new TypeToken<List<MetaData>>() {
            }.getType());
            return new ConfigData<>(config.getMd5(), config.getLastModifyTime(), metaList);
        default:
            throw new IllegalStateException("Unexpected groupKey: " + groupKey);
    }
}

3、HttpLongPollingDataChangedListener --> doLongPolling

由方法的注释和实现可知，方法收到请求后会先比较是否有数据变化，有立即返回，否则会阻塞请求。

1）asyncContext.setTimeout(0L) 设置请求超时为0，方便自由控制，也就是阻塞调用的前提。

2）创建一个线程，负责请求端阻塞处理

scheduler.execute(new LongPollingClient(asyncContext, clientIp, HttpConstants.SERVER_MAX_HOLD_TIMEOUT));

/**
 * If the configuration data changes, the group information for the change is immediately responded.
 * Otherwise, the client's request thread is blocked until any data changes or the specified timeout is reached.
 * @param request  the request
 * @param response the response
 */
public void doLongPolling(final HttpServletRequest request, final HttpServletResponse response) {
    // compare group md5
    List<ConfigGroupEnum> changedGroup = compareChangedGroup(request);
    String clientIp = getRemoteIp(request);
    // response immediately.
    if (CollectionUtils.isNotEmpty(changedGroup)) {
        this.generateResponse(response, changedGroup);
        log.info("send response with the changed group, ip={}, group={}", clientIp, changedGroup);
        return;
    }
    // listen for configuration changed.
    final AsyncContext asyncContext = request.startAsync();
    // AsyncContext.settimeout() does not timeout properly, so you have to control it yourself
    asyncContext.setTimeout(0L);
    // block client's thread.
    scheduler.execute(new LongPollingClient(asyncContext, clientIp, HttpConstants.SERVER_MAX_HOLD_TIMEOUT));
}

4、HttpLongPollingDataChangedListener --> LongPollingClient

LongPollingClient 是一个内部类，实现了 Runnable 接口。

LongPollingClient 初始化传入了3个参数：request AsyncContext，客户端请求ip，超时时间。

run()方法中将 LongPollingClient 加入BlocingQueue中，然后通过定时任务调度执行，如果60s之内没有配置变更，则60s后执行，响应http请求。

class LongPollingClient implements Runnable {
    ......
    LongPollingClient(final AsyncContext ac, final String ip, final long timeoutTime) {
        this.asyncContext = ac;
        this.ip = ip;
        this.timeoutTime = timeoutTime;
    }
    
    @Override
    public void run() {
        // 加入定时任务，如果60s之内没有配置变更，则60s后执行，响应http请求
        this.asyncTimeoutFuture = scheduler.schedule(() -> {
            clients.remove(LongPollingClient.this);
            List<ConfigGroupEnum> changedGroups = compareChangedGroup((HttpServletRequest) asyncContext.getRequest());
            sendResponse(changedGroups);
        }, timeoutTime, TimeUnit.MILLISECONDS);
        clients.add(this);
    }
    /**
     * Send response.
     * @param changedGroups the changed groups
     */
    void sendResponse(final List<ConfigGroupEnum> changedGroups) {
        // cancel scheduler
        if (null != asyncTimeoutFuture) {
            asyncTimeoutFuture.cancel(false);
        }
        generateResponse((HttpServletResponse) asyncContext.getResponse(), changedGroups);
        asyncContext.complete();
    }
}

三、总结

在这里插入图片描述
下一篇接着分析…

官网：soul 数据同步原理

lq9616

关注

0
点赞
踩
0

收藏

觉得还不错? 一键收藏
0
评论
Soul网关源码阅读08-使用http长轮询同步数据(01)

前面两篇文章分析了zookeeper的数据同步原理，今天看一下体验http长轮询同步数据一、环境配置1、soul-admin 网关配置管理服务soul-admin/src/main/resources/application.yml打开http长轮询配置。还有 soul-admin–>pom.xml 中的 websocket、zookeeper、nacos 等依赖无需删除，否则服务无法启动，仅需要把相应配置注释即可。soul: sync:# websocket:#
复制链接

扫一扫