soul网关系列(八)：soul-admin与网关数据同步之zookeeper

最新推荐文章于 2021-08-31 16:33:41 发布

HDGM3

最新推荐文章于 2021-08-31 16:33:41 发布

阅读量386

点赞数

分类专栏： # soul网关系列

本文链接：https://blog.csdn.net/bo_book/article/details/113004035

版权

soul网关系列专栏收录该内容

17 篇文章 6 订阅

订阅专栏

本篇文章主要介绍soul网关admin与bootstrap数据同步的另外三种方式(zookeeper/http长连接/nacos)

一、zookeeper同步方式简述

1.1 内容简述

在这里插入图片描述
按之前理解的数据流向，如图所示，今天主要介绍zookeeper/Http长连接/nacos三种同步方式，整体流程上应该和之前的websocket方式基本一致。

在这里插入图片描述

zookeeper 同步策略，将变更数据更新到 zookeeper，而 ZookeeperSyncCache 会监听到 zookeeper 的数据变更，并予以处理

1.2 几个疑惑点

根据官网的文档也解答了之前的几个比较疑惑的地方，不过随着后面soul的房展，可能支持的同步方式、缓存方式、配置中心会越来越多。

四种同步方式的支持顺序和原因？

1.x只支持zookeeper，2.0版本后来逐步的引入了http长轮询/websocket/nacos三种方式，理由是为了满足更多的用户，用户的注册中心可能是etcd、consul、nacos，甚至于没有。
为啥不用redis？

本地缓存hashmap使得响应更快
为什么不用配置中心？

额外的成本和运营，数据格式不可用

二、zookeeper同步方式的配置

soul-admin的pom.xml引入依赖，默认已添加

<dependency>
    <groupId>com.101tec</groupId>
    <artifactId>zkclient</artifactId>
    <exclusions>
        <exclusion>
            <groupId>org.slf4j</groupId>
            <artifactId>slf4j-log4j12</artifactId>
        </exclusion>
        <exclusion>
            <groupId>log4j</groupId>
            <artifactId>log4j</artifactId>
        </exclusion>
    </exclusions>
</dependency>

soul-admin的yml配置文件打开

soul没有显示多种同步方式并存的情况，这里先把昨日分析的websocket同步方式给屏蔽掉
soul-bootstrap的pom.xml引入依赖
和websocket一样的引入zookeeper的starter

<dependency>
    <groupId>org.dromara</groupId>
    <artifactId>soul-spring-boot-starter-sync-data-zookeeper</artifactId>
    <version>${project.version}</version>
</dependency>

soul-bootstrap的yml配置文件打开

zookeeper.url: localhost:2181(zk的url)
zookeeper.sessionTimeout: 5000(session的超时时间)
zookeeper.connectionTimeout: 2000(l链接的超时时间)

重启soul-getway和重启boot-bootstrap服务

三、zookeeper同步方式源码解读

验证初始化ZookeeperDataChangedListener

昨日分析了几种同步方式的DataChangedListeners的初始化，今天可以再验证一下，随便web管理端修改下配置
直接在DataChangedEventDispatcher的循环listeners处打个断点
在这里插入图片描述发现里面有两个listener,按理说关了websocket，初始化的地方看一下，发现有一个matchIfMissing = true配置，默认的话websocket会必然打开的，这块不影响后续分析，先忽略

2. soul-admin端

初次加载(判断条件，有点简单，感觉理论上重启之后无论如何都要全量同步)

    @Override
    public void run(final String... args) {
        String pluginPath = ZkPathConstants.PLUGIN_PARENT;
        String authPath = ZkPathConstants.APP_AUTH_PARENT;
        String metaDataPath = ZkPathConstants.META_DATA;
        //判断几个节点是否存在，进行全量同步
        if (!zkClient.exists(pluginPath) && !zkClient.exists(authPath) && !zkClient.exists(metaDataPath)) {
            syncDataService.syncAll(DataEventTypeEnum.REFRESH);
        }
    }

zkClient.exists(),看zkclient的源码，如果不存在节点，就新建一个节点,并初始化watch流程
在这里插入图片描述

这里复用的也是一套消息同步机制

    @Override
    public boolean syncAll(final DataEventTypeEnum type) {
        appAuthService.syncData();
        List<PluginData> pluginDataList = pluginService.listAll();
        eventPublisher.publishEvent(new DataChangedEvent(ConfigGroupEnum.PLUGIN, type, pluginDataList));
        List<SelectorData> selectorDataList = selectorService.listAll();
        eventPublisher.publishEvent(new DataChangedEvent(ConfigGroupEnum.SELECTOR, type, selectorDataList));
        List<RuleData> ruleDataList = ruleService.listAll();
        eventPublisher.publishEvent(new DataChangedEvent(ConfigGroupEnum.RULE, type, ruleDataList));
        metaDataService.syncData();
        return true;
    }

增量更新

/*
 * Licensed to the Apache Software Foundation (ASF) under one or more
 * contributor license agreements.  See the NOTICE file distributed with
 * this work for additional information regarding copyright ownership.
 * The ASF licenses this file to You under the Apache License, Version 2.0
 * (the "License"); you may not use this file except in compliance with
 * the License.  You may obtain a copy of the License at
 *
 *     http://www.apache.org/licenses/LICENSE-2.0
 *
 * Unless required by applicable law or agreed to in writing, software
 * distributed under the License is distributed on an "AS IS" BASIS,
 * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 * See the License for the specific language governing permissions and
 * limitations under the License.
 */

package org.dromara.soul.admin.listener.zookeeper;

import java.net.URLEncoder;
import java.util.List;
import lombok.SneakyThrows;
import org.I0Itec.zkclient.ZkClient;
import org.dromara.soul.admin.listener.DataChangedListener;
import org.dromara.soul.common.constant.ZkPathConstants;
import org.dromara.soul.common.dto.AppAuthData;
import org.dromara.soul.common.dto.MetaData;
import org.dromara.soul.common.dto.PluginData;
import org.dromara.soul.common.dto.RuleData;
import org.dromara.soul.common.dto.SelectorData;
import org.dromara.soul.common.enums.DataEventTypeEnum;

/**
 * Use zookeeper to push data changes.
 *
 * @author huangxiaofeng
 * @author xiaoyu
 */
public class ZookeeperDataChangedListener implements DataChangedListener {

    private final ZkClient zkClient;

    public ZookeeperDataChangedListener(final ZkClient zkClient) {
        this.zkClient = zkClient;
    }

    @Override
    public void onAppAuthChanged(final List<AppAuthData> changed, final DataEventTypeEnum eventType) {
        for (AppAuthData data : changed) {
            // delete
            if (eventType == DataEventTypeEnum.DELETE) {
                String pluginPath = ZkPathConstants.buildAppAuthPath(data.getAppKey());
                if (zkClient.exists(pluginPath)) {
                    zkClient.delete(pluginPath);
                }
                continue;
            }

            // create or update
            String appAuthPath = ZkPathConstants.buildAppAuthPath(data.getAppKey());
            if (!zkClient.exists(appAuthPath)) {
                zkClient.createPersistent(appAuthPath, true);
            }
            zkClient.writeData(appAuthPath, data);
        }
    }

    @SneakyThrows
    @Override
    public void onMetaDataChanged(final List<MetaData> changed, final DataEventTypeEnum eventType) {
        for (MetaData data : changed) {
            // delete
            if (eventType == DataEventTypeEnum.DELETE) {
                String path = ZkPathConstants.buildMetaDataPath(URLEncoder.encode(data.getPath(), "UTF-8"));
                if (zkClient.exists(path)) {
                    zkClient.delete(path);
                }
                continue;
            }
            // create or update
            String metaDataPath = ZkPathConstants.buildMetaDataPath(URLEncoder.encode(data.getPath(), "UTF-8"));
            if (!zkClient.exists(metaDataPath)) {
                zkClient.createPersistent(metaDataPath, true);
            }
            zkClient.writeData(metaDataPath, data);
        }
    }

    @Override
    public void onPluginChanged(final List<PluginData> changed, final DataEventTypeEnum eventType) {
        for (PluginData data : changed) {
            // delete
            if (eventType == DataEventTypeEnum.DELETE) {
                String pluginPath = ZkPathConstants.buildPluginPath(data.getName());
                if (zkClient.exists(pluginPath)) {
                    zkClient.deleteRecursive(pluginPath);
                }
                String selectorParentPath = ZkPathConstants.buildSelectorParentPath(data.getName());
                if (zkClient.exists(selectorParentPath)) {
                    zkClient.deleteRecursive(selectorParentPath);
                }
                String ruleParentPath = ZkPathConstants.buildRuleParentPath(data.getName());
                if (zkClient.exists(ruleParentPath)) {
                    zkClient.deleteRecursive(ruleParentPath);
                }
                continue;
            }

            // update
            String pluginPath = ZkPathConstants.buildPluginPath(data.getName());
            if (!zkClient.exists(pluginPath)) {
                zkClient.createPersistent(pluginPath, true);
            }
            zkClient.writeData(pluginPath, data);
        }
    }

    @Override
    public void onSelectorChanged(final List<SelectorData> changed, final DataEventTypeEnum eventType) {
        if (eventType == DataEventTypeEnum.REFRESH) {
            String selectorParentPath = ZkPathConstants.buildSelectorParentPath(changed.get(0).getPluginName());
            if (zkClient.exists(selectorParentPath)) {
                zkClient.deleteRecursive(selectorParentPath);
            }
        }
        for (SelectorData data : changed) {
            if (eventType == DataEventTypeEnum.DELETE) {
                deleteSelector(data);
                continue;
            }
            createSelector(data);
        }
    }

    @Override
    public void onRuleChanged(final List<RuleData> changed, final DataEventTypeEnum eventType) {
        if (eventType == DataEventTypeEnum.REFRESH) {
            String selectorParentPath = ZkPathConstants.buildRuleParentPath(changed.get(0).getPluginName());
            if (zkClient.exists(selectorParentPath)) {
                zkClient.deleteRecursive(selectorParentPath);
            }
        }
        for (RuleData data : changed) {
            if (eventType == DataEventTypeEnum.DELETE) {
                final String rulePath = ZkPathConstants.buildRulePath(data.getPluginName(), data.getSelectorId(), data.getId());
                if (zkClient.exists(rulePath)) {
                    zkClient.delete(rulePath);
                }
                continue;
            }
            String ruleParentPath = ZkPathConstants.buildRuleParentPath(data.getPluginName());
            if (!zkClient.exists(ruleParentPath)) {
                zkClient.createPersistent(ruleParentPath, true);
            }
            String ruleRealPath = ZkPathConstants.buildRulePath(data.getPluginName(), data.getSelectorId(), data.getId());
            if (!zkClient.exists(ruleRealPath)) {
                zkClient.createPersistent(ruleRealPath, true);
            }
            zkClient.writeData(ruleRealPath, data);
        }
    }

    private void deleteSelector(final SelectorData data) {
        String selectorRealPath = ZkPathConstants.buildSelectorRealPath(data.getPluginName(), data.getId());
        if (zkClient.exists(selectorRealPath)) {
            zkClient.delete(selectorRealPath);
        }
    }

    private void createSelector(final SelectorData data) {
        String selectorParentPath = ZkPathConstants.buildSelectorParentPath(data.getPluginName());
        if (!zkClient.exists(selectorParentPath)) {
            zkClient.createPersistent(selectorParentPath, true);
        }
        String selectorRealPath = ZkPathConstants.buildSelectorRealPath(data.getPluginName(), data.getId());
        if (!zkClient.exists(selectorRealPath)) {
            zkClient.createPersistent(selectorRealPath, true);
        }
        zkClient.writeData(selectorRealPath, data);
    }

}

大致看zk的API文档，zk的节点就是path，对于数据的同步就是对这些zk path的增删，和watch机制同步数据。

soul-bootstrap端

从soul-spring-boot-starter-sync-data-zookeeper开始，这个搞各种zk sync的初始化的

@Configuration
// 加载ZookeeperSyncDataService类
@ConditionalOnClass(ZookeeperSyncDataService.class)
// 如果配置文件有soul.sync.zookeeper.url就注入ZookeeperSyncDataConfiguration 
@ConditionalOnProperty(prefix = "soul.sync.zookeeper", name = "url")
// 加载配置属性ZookeeperConfig类
@EnableConfigurationProperties(ZookeeperConfig.class)
@Slf4j
public class ZookeeperSyncDataConfiguration {

    /**
     * Sync data service sync data service.
     *
     * @param zkClient          the zk client
     * @param pluginSubscriber the plugin subscriber
     * @param metaSubscribers   the meta subscribers
     * @param authSubscribers   the auth subscribers
     * @return the sync data service
     */
    @Bean
    public SyncDataService syncDataService(final ObjectProvider<ZkClient> zkClient, final ObjectProvider<PluginDataSubscriber> pluginSubscriber,
                                           final ObjectProvider<List<MetaDataSubscriber>> metaSubscribers, final ObjectProvider<List<AuthDataSubscriber>> authSubscribers) {
        log.info("you use zookeeper sync soul data.......");
        return new ZookeeperSyncDataService(zkClient.getIfAvailable(), pluginSubscriber.getIfAvailable(),
                metaSubscribers.getIfAvailable(Collections::emptyList), authSubscribers.getIfAvailable(Collections::emptyList));
    }

    /**
     * register zkClient in spring ioc. 
     * 从配置文件读取zk配置信息，并注入
     * @param zookeeperConfig the zookeeper configuration
     * @return ZkClient {@linkplain ZkClient}
     */
    @Bean
    public ZkClient zkClient(final ZookeeperConfig zookeeperConfig) {
        return new ZkClient(zookeeperConfig.getUrl(), zookeeperConfig.getSessionTimeout(), zookeeperConfig.getConnectionTimeout());
    }

}

soul-sync-data-zookeeper服务

就只有一个类，就是上面初始化的ZookeeperSyncDataService
在这里插入图片描述
这里挑选一个AppAuth进行说明,设计的很精巧

    private void watchAppAuth() {
        final String appAuthParent = ZkPathConstants.APP_AUTH_PARENT;
        //不存在则创建节点
        if (!zkClient.exists(appAuthParent)) {
            zkClient.createPersistent(appAuthParent, true);
        }
        final List<String> childrenList = zkClient.getChildren(appAuthParent);
        if (CollectionUtils.isNotEmpty(childrenList)) {
            childrenList.forEach(children -> {
                String realPath = buildRealPath(appAuthParent, children);
                cacheAuthData(zkClient.readData(realPath));
                subscribeAppAuthDataChanges(realPath);
            });
        }
        // 订阅authpath变化通知
        zkClient.subscribeChildChanges(appAuthParent, (parentPath, currentChildren) -> {
            if (CollectionUtils.isNotEmpty(currentChildren)) {
                final List<String> addSubscribePath = addSubscribePath(childrenList, currentChildren);
                addSubscribePath.stream().map(children -> {
                    final String realPath = buildRealPath(parentPath, children);
                    cacheAuthData(zkClient.readData(realPath));
                    return realPath;
                }).forEach(this::subscribeAppAuthDataChanges);
            }
        });
    }
    private void subscribeAppAuthDataChanges(final String realPath) {
        zkClient.subscribeDataChanges(realPath, new IZkDataListener() {
            @Override
            public void handleDataChange(final String dataPath, final Object data) {
                cacheAuthData((AppAuthData) data);
            }
            
            @Override
            public void handleDataDeleted(final String dataPath) {
                unCacheAuthData(dataPath);
            }
        });
    }
    //更新缓存数据
    private void cacheAuthData(final AppAuthData appAuthData) {
        Optional.ofNullable(appAuthData).ifPresent(data -> authDataSubscribers.forEach(e -> e.onSubscribe(data)));
    }
    //删除缓存数据
    private void unCacheAuthData(final String dataPath) {
        final String key = dataPath.substring(ZkPathConstants.APP_AUTH_PARENT.length() + 1);
        AppAuthData appAuthData = new AppAuthData();
        appAuthData.setAppKey(key);
        authDataSubscribers.forEach(e -> e.unSubscribe(appAuthData));
    }

这块的authDataSubscribers都会把相应实现了的插件都更新
在这里插入图片描述

无论soul-admin的插件开没开，这个是一个后端逻辑，保证随时开启后，相应的插件有缓存，能够随时工作。每一个插件的具体实现里都有一个ConcurrentMap<String, MetaData> META_DATA。