Soul 网关源码分析（七）使用zookeeper同步数据到网关

与 websocket 数据同步方式类似，采用 zookeeper 的同步也很简单， 由于 zookeeper 的 watch 机制，sou-bootstrap 会监听配置的node，soul-admin 在启动的时候，也会跟websocker一样将数据全量写入 zookeeper。数据发生变更时，会对zookeeper 的节点做增量更新 ，并更新本地缓存。

流程演示

首先启动zookeeper，笔者使用 docker 启动。

推荐使用 ZooInspector 来直观地看到 zookeeper 中节点的数据。
下载地址：
https://issues.apache.org/jira/secure/attachment/12436620/ZooInspector.zip
使用方法：
在build 文件夹下执行 java -jar zookeeper-dev-ZooInspector.jar

下面我们来修改相关的配置使用 zookeeper来同步数据。先改 soul-bootstrap的配置：

soul :
    file:
      enabled: true
    corss:
      enabled: true
    dubbo :
      parameter: multi
    sync:
#        websocket :
#             urls: ws://localhost:9095/websocket

        zookeeper:
             url: localhost:2181
             sessionTimeout: 5000
             connectionTimeout: 2000

再改soul-admin 的配置：

soul:
  database:
    dialect: mysql
    init_script: "META-INF/schema.sql"
    init_enable: true
  sync:
#    websocket:
#      enabled: true
      zookeeper:
          url: localhost:2181
          sessionTimeout: 5000
          connectionTimeout: 2000

注释掉websocket 下的配置并解除注释 zookeeper 相关配置。
启动 soul-bootstrap 和 soul-admin
可以看到 数据 已经写入了 zookeeper

尝试修改数据：

发现一个奇怪的bug，跟到源码发现了问题，喜提issue：


重新添加 selector，再点击同步，成功同步到了zookeeper。

深入源码

/**
     * The type Zookeeper listener.
     */
    @Configuration
    @ConditionalOnProperty(prefix = "soul.sync.zookeeper", name = "url")
    @Import(ZookeeperConfiguration.class)
    static class ZookeeperListener {

        /**
         * 这里如果开启了zookeeper相关配置，会加载 ZookeeperDataChangedListener bean。
         *
         * @param zkClient the zk client
         * @return the data changed listener
         */
        @Bean
        @ConditionalOnMissingBean(ZookeeperDataChangedListener.class)
        public DataChangedListener zookeeperDataChangedListener(final ZkClient zkClient) {
            return new ZookeeperDataChangedListener(zkClient);
        }

        /**
         * 负责同步初始化数据。
         *
         * @param zkClient        the zk client
         * @param syncDataService the sync data service
         * @return the zookeeper data init
         */
        @Bean
        @ConditionalOnMissingBean(ZookeeperDataInit.class)
        public ZookeeperDataInit zookeeperDataInit(final ZkClient zkClient, final SyncDataService syncDataService) {
            return new ZookeeperDataInit(zkClient, syncDataService);
        }
    }

然后是 ZookeeperDataChangedListener中对应事件监听处理的逻辑：

public class ZookeeperDataChangedListener implements DataChangedListener {
	//zookeeper 客户端
    private final ZkClient zkClient;

    public ZookeeperDataChangedListener(final ZkClient zkClient) {
        this.zkClient = zkClient;
    }
	//监听到 应用鉴权变更
    @Override
    public void onAppAuthChanged(final List<AppAuthData> changed, final DataEventTypeEnum eventType) {
        for (AppAuthData data : changed) {
            final String appAuthPath = ZkPathConstants.buildAppAuthPath(data.getAppKey());
            // delete
            if (eventType == DataEventTypeEnum.DELETE) {
                deleteZkPath(appAuthPath);
                continue;
            }
            // create or update
            upsertZkNode(appAuthPath, data);
        }
    }

	//监听到 元数据变更
    @SneakyThrows
    @Override
    public void onMetaDataChanged(final List<MetaData> changed, final DataEventTypeEnum eventType) {
        for (MetaData data : changed) {
            final String metaDataPath = ZkPathConstants.buildMetaDataPath(URLEncoder.encode(data.getPath(), "UTF-8"));
            // delete
            if (eventType == DataEventTypeEnum.DELETE) {
                deleteZkPath(metaDataPath);
                continue;
            }
            // create or update
            upsertZkNode(metaDataPath, data);
        }
    }
	//监听到 插件变更
    @Override
    public void onPluginChanged(final List<PluginData> changed, final DataEventTypeEnum eventType) {
        for (PluginData data : changed) {
            final String pluginPath = ZkPathConstants.buildPluginPath(data.getName());
            // delete
            if (eventType == DataEventTypeEnum.DELETE) {
            	//先删除再更新
                deleteZkPathRecursive(pluginPath);
                final String selectorParentPath = ZkPathConstants.buildSelectorParentPath(data.getName());
                deleteZkPathRecursive(selectorParentPath);
                final String ruleParentPath = ZkPathConstants.buildRuleParentPath(data.getName());
                deleteZkPathRecursive(ruleParentPath);
                continue;
            }
            //create or update
            upsertZkNode(pluginPath, data);
        }
    }
	//监听到 选择器变更
    @Override
    public void onSelectorChanged(final List<SelectorData> changed, final DataEventTypeEnum eventType) {
        if (eventType == DataEventTypeEnum.REFRESH) {
            final String selectorParentPath = ZkPathConstants.buildSelectorParentPath(changed.get(0).getPluginName());
            deleteZkPathRecursive(selectorParentPath);
        }
        for (SelectorData data : changed) {
            final String selectorRealPath = ZkPathConstants.buildSelectorRealPath(data.getPluginName(), data.getId());
            if (eventType == DataEventTypeEnum.DELETE) {
                deleteZkPath(selectorRealPath);
                continue;
            }
            final String selectorParentPath = ZkPathConstants.buildSelectorParentPath(data.getPluginName());
            createZkNode(selectorParentPath);
            //create or update
            upsertZkNode(selectorRealPath, data);
        }
    }
	//监听到 规则变更
    @Override
    public void onRuleChanged(final List<RuleData> changed, final DataEventTypeEnum eventType) {
        if (eventType == DataEventTypeEnum.REFRESH) {
            final String selectorParentPath = ZkPathConstants.buildRuleParentPath(changed.get(0).getPluginName());
            deleteZkPathRecursive(selectorParentPath);
        }
        for (RuleData data : changed) {
            final String ruleRealPath = ZkPathConstants.buildRulePath(data.getPluginName(), data.getSelectorId(), data.getId());
            if (eventType == DataEventTypeEnum.DELETE) {
                deleteZkPath(ruleRealPath);
                continue;
            }
            final String ruleParentPath = ZkPathConstants.buildRuleParentPath(data.getPluginName());
            createZkNode(ruleParentPath);
            //create or update
            upsertZkNode(ruleRealPath, data);
        }
    }
    //创建节点
    private void createZkNode(final String path) {
        if (!zkClient.exists(path)) {
            zkClient.createPersistent(path, true);
        }
    }
    
    /**
     * 有则更新、无则创建节点
     * @param path node path
     * @param data node data 
     */
    private void upsertZkNode(final String path, final Object data) {
        if (!zkClient.exists(path)) {
            zkClient.createPersistent(path, true);
        }
        zkClient.writeData(path, data);
    }
    //删除路径
    private void deleteZkPath(final String path) {
        if (zkClient.exists(path)) {
            zkClient.delete(path);
        }
    }
    //递归删除
    private void deleteZkPathRecursive(final String path) { 
        if (zkClient.exists(path)) {
            zkClient.deleteRecursive(path);
        }
    }

总结

zookeeper 的同步方式相对与websocket 性能上差距应该不大，实现原理也类似，比较麻烦的是需要维护 zookeeper，而使用 websocket 则没有这样的问题。