【源码】Sharding-JDBC源码分析之Yaml分片配置转换原理

JingAi_jia917

于 2024-08-22 08:30:00 发布

阅读量617

点赞数 17

分类专栏： ShardingSphere 源码文章标签： SpringBoot分库分表分库分表 Spring分库分表 ShardingJDBC源码 Sharding-JDBC

本文链接：https://blog.csdn.net/JingAi_jia917/article/details/141254069

版权

ShardingSphere 同时被 2 个专栏收录

6 篇文章 0 订阅

订阅专栏

源码

6 篇文章 0 订阅

订阅专栏

Sharding-JDBC系列

1、Sharding-JDBC分库分表的基本使用

2、Sharding-JDBC分库分表之SpringBoot分片策略

3、Sharding-JDBC分库分表之SpringBoot主从配置

4、SpringBoot集成Sharding-JDBC-5.3.0分库分表

5、SpringBoot集成Sharding-JDBC-5.3.0实现按月动态建表分表

6、【源码】Sharding-JDBC源码分析之JDBC

7、【源码】Sharding-JDBC源码分析之SPI机制

8、【源码】Sharding-JDBC源码分析之Yaml分片配置文件解析原理

9、【源码】Sharding-JDBC源码分析之Yaml分片配置原理（一）

10、【源码】Sharding-JDBC源码分析之Yaml分片配置原理（二）

11、【源码】Sharding-JDBC源码分析之Yaml分片配置转换原理

前言

在《Sharding-JDBC系列》中用了三篇介绍了Yaml分片配置。在ShardingSphere的源码设计中，对于Yaml中最核心的配置信息rules和mode，解析后的对象的类名为YamlXxxConfiguration，且都实现了YamlConfiguration。

Yaml中的配置都是文本信息，在使用配置信息时，需要对一些配置信息进行转换（如通过type类型，匹配对应的类），对于每个YamlXxxConfiguration类，都有一个对应的配置转换后的类，类名为XxxConfiguration（如：YamlModeConfiguration对应ModeConfiguration）。

每个YamlXxxConfiguration都有一个对应的YamlXxxConfigurationSwapper，这些类都实现YamlConfigurationSwapper，用于YamlXxxConfiguration与XxxConfiguration的相互转换。

dataSources配置转换DataSource对象回顾

在【源码】Sharding-JDBC源码分析之Yaml分片配置原理（一）-CSDN博客中介绍了dataSources的配置以及转换为对应的DataSource对象。

1）通过YamlDataSourceConfigurationSwapper，将dataSources配置信息转换为DataSourceProperties。在DataSourceProperties中，对不同的dataSourceClassName进行兼容性处理；

2）通过DataSourcePoolCreator，使用反射机制，将DataSourceProperties创建为对应的DataSource对象。在DataSource对象中，添加了配置的数据源信息以及对应数据源的默认配置信息；

通过dataSourceClassName找到对应DataSourcePoolMetaData，在其中指定了默认配置项。如HikariDataSourcePoolMetaData。

rules配置转换

在YamlShardingSphereDataSourceFactory的createDataSource(final byte[] yamlBytes)方法中，先解析Yaml配置文件，得到YamlRootConfiguration对象。然后使用YamlDataSourceConfigurationSwapper将dataSources配置转换为DataSource对象。之后调用同名方法，对mode配置及rules配置进行转换。

3.1 YamlShardingSphereDataSourceFactory

YamlShardingSphereDataSourceFactory源码如下：

package org.apache.shardingsphere.driver.api.yaml;

@NoArgsConstructor(access = AccessLevel.PRIVATE)
public final class YamlShardingSphereDataSourceFactory {
    
	// Yaml中rules配置转换引擎
    private static final YamlRuleConfigurationSwapperEngine SWAPPER_ENGINE = new YamlRuleConfigurationSwapperEngine();

    // yaml中dataSources配置信息转换器
    private static final YamlDataSourceConfigurationSwapper DATA_SOURCE_SWAPPER = new YamlDataSourceConfigurationSwapper();
    
    /**
     * 创建ShardingSphereDataSource对象
     * @param yamlBytes yaml配置字符串
     */
    public static DataSource createDataSource(final byte[] yamlBytes) throws SQLException, IOException {
        // 解析yaml配置文件，获取YamlRootConfiguration
        YamlRootConfiguration rootConfig = YamlEngine.unmarshal(yamlBytes, YamlRootConfiguration.class);
        // 调用DATA_SOURCE_SWAPPER.swapToDataSources(rootConfig.getDataSources()，
        // 将Yaml中配置的dataSources信息转换为对应的DataSources对象
        return createDataSource(DATA_SOURCE_SWAPPER.swapToDataSources(rootConfig.getDataSources()), rootConfig);
    }
    
    /**
     * 创建ShardingSphereDataSource
     */
    private static DataSource createDataSource(final Map<String, DataSource> dataSourceMap, final YamlRootConfiguration rootConfig) throws SQLException {
        // mode配置转换
        ModeConfiguration modeConfig = null == rootConfig.getMode() ? null : new YamlModeConfigurationSwapper().swapToObject(rootConfig.getMode());
        // rules配置转换
        Collection<RuleConfiguration> ruleConfigs = SWAPPER_ENGINE.swapToRuleConfigurations(rootConfig.getRules());
        // 创建ShardingSphereDataSource
        return ShardingSphereDataSourceFactory.createDataSource(rootConfig.getDatabaseName(), modeConfig, dataSourceMap, ruleConfigs, rootConfig.getProps());
    }

}

通过YamlRuleConfigurationSwapperEngine.swapToRuleConfigurations()方法，将YamlRootConfiguration中的rules配置信息转换为RuleConfiguration集合。

3.2 YamlRuleConfigurationSwapperEngine

YamlRuleConfigurationSwapperEngine的源码如下：

package org.apache.shardingsphere.infra.yaml.config.swapper.rule;

/**
 * Yaml中rules配置转换引擎
 */
public final class YamlRuleConfigurationSwapperEngine {

    /**
     * 将RuleConfiguration集合转换为YamlRuleConfiguration集合
     */
    @SuppressWarnings("unchecked")
    public Collection<YamlRuleConfiguration> swapToYamlRuleConfigurations(final Collection<RuleConfiguration> ruleConfigs) {
        return OrderedSPIRegistry.getRegisteredServices(YamlRuleConfigurationSwapper.class, ruleConfigs).entrySet().stream()
                .map(entry -> (YamlRuleConfiguration) entry.getValue().swapToYamlConfiguration(entry.getKey())).collect(Collectors.toList());
    }
    
    /**
     * 将YamlRuleConfiguration集合转换为RuleConfiguration集合
     */
    @SuppressWarnings("rawtypes")
    public Collection<RuleConfiguration> swapToRuleConfigurations(final Collection<YamlRuleConfiguration> yamlRuleConfigs) {
        Collection<RuleConfiguration> result = new LinkedList<>();
        // 获取配置的规则类型。如数据分片规则YamlShardingRuleConfiguration，对应的类型为ShardingRuleConfiguration
        Collection<Class<?>> ruleConfigTypes = yamlRuleConfigs.stream().map(YamlRuleConfiguration::getRuleConfigurationType).collect(Collectors.toList());
        // 通过SPI，获取能够转换对应规则类型的转换器。如数据分片规则的转换器为YamlShardingRuleConfigurationSwapper，
        // 该类的getTypeClass()方法返回ShardingRuleConfiguration.class
        for (Entry<Class<?>, YamlRuleConfigurationSwapper> entry : OrderedSPIRegistry.getRegisteredServicesByClass(YamlRuleConfigurationSwapper.class, ruleConfigTypes).entrySet()) {
            // 使用转换器，将YamlXxxRuleConfiguration转换为XxxRuleConfiguration，添加到result结合中
            result.addAll(swapToRuleConfigurations(yamlRuleConfigs, entry.getKey(), entry.getValue()));
        }
        return result;
    }

    /**
     * 从yamlRuleConfigs集合中找到YamlRuleConfiguration.getRuleConfigurationType()和ruleConfigType相等的YamlRuleConfiguration，
     * 使用swapper转换器转换为对应的XxxRuleConfiguration
     */
    @SuppressWarnings({"unchecked", "rawtypes"})
    private Collection<RuleConfiguration> swapToRuleConfigurations(final Collection<YamlRuleConfiguration> yamlRuleConfigs,
                                                                   final Class<?> ruleConfigType, final YamlRuleConfigurationSwapper swapper) {
        return yamlRuleConfigs.stream()
                .filter(each -> each.getRuleConfigurationType().equals(ruleConfigType)).map(each -> (RuleConfiguration) swapper.swapToObject(each)).collect(Collectors.toList());
    }
}

在swapToRuleConfigurations(Collection<YamlRuleConfiguration> yamlRuleConfigs)方法中，执行如下：

1）通过YamlRuleConfiguration.getRuleConfigurationType()获取配置的规则类型。如数据分片规则YamlShardingRuleConfiguration，对应的类型为ShardingRuleConfiguration。添加到集合中；

2）通过Java SPI，获取能够转换对应规则类型的转换器。如数据分片规则的转换器为YamlShardingRuleConfigurationSwapper，该类的getTypeClass()方法返回ShardingRuleConfiguration.class；

Java SPI详见：【源码】Sharding-JDBC源码分析之SPI机制-CSDN博客

3）遍历规则类型转换器，从YamlRuleConfiguration集合中通过getRuleConfigurationType()查找匹配的YamlRuleConfiguration，执行转换器swapper.swapToObject(YamlRuleConfiguration)，将YamlRuleConfiguration转换为RuleConfiguration；

3.3 YamlShardingRuleConfigurationSwapper

ShardingSphere有十几种规则，以下以数据分片规则为例，分析YamlRuleConfiguration通过Swapper转换为RuleConfiguration的过程。数据分片规则的转换器为YamlShardingRuleConfigurationSwapper，源码如下：

package org.apache.shardingsphere.sharding.yaml.swapper;

/**
 * yaml中sharding规则配置转换器
 */
public final class YamlShardingRuleConfigurationSwapper implements YamlRuleConfigurationSwapper<YamlShardingRuleConfiguration, ShardingRuleConfiguration> {

    // Yaml中分片表规则配置转换器
    private final YamlShardingTableRuleConfigurationSwapper tableSwapper = new YamlShardingTableRuleConfigurationSwapper();

    // yaml分片算法配置转换器
    private final YamlShardingStrategyConfigurationSwapper shardingStrategySwapper = new YamlShardingStrategyConfigurationSwapper();

    // yaml主键生成器算法配置转换器
    private final YamlKeyGenerateStrategyConfigurationSwapper keyGenerateStrategySwapper = new YamlKeyGenerateStrategyConfigurationSwapper();

    // yaml中算法配置转换器
    private final YamlAlgorithmConfigurationSwapper algorithmSwapper = new YamlAlgorithmConfigurationSwapper();

    // yaml分片审计算法配置转换器
    private final YamlShardingAuditStrategyConfigurationSwapper auditStrategySwapper = new YamlShardingAuditStrategyConfigurationSwapper();

    // yaml中分片autoTable规则配置转换器
    private final YamlShardingAutoTableRuleConfigurationSwapper autoTableYamlSwapper = new YamlShardingAutoTableRuleConfigurationSwapper();

    /**
     * 将ShardingRuleConfiguration转换为YamlShardingRuleConfiguration
     */
    @Override
    public YamlShardingRuleConfiguration swapToYamlConfiguration(final ShardingRuleConfiguration data) {
        YamlShardingRuleConfiguration result = new YamlShardingRuleConfiguration();
        // ShardingTableRuleConfiguration转换为YamlTableRuleConfiguration
        data.getTables().forEach(each -> result.getTables().put(each.getLogicTable(), tableSwapper.swapToYamlConfiguration(each)));
        // ShardingAutoTableRuleConfiguration转换为YamlShardingAutoTableRuleConfiguration
        data.getAutoTables().forEach(each -> result.getAutoTables().put(each.getLogicTable(), autoTableYamlSwapper.swapToYamlConfiguration(each)));
        // 添加关联表
        result.getBindingTables().addAll(data.getBindingTableGroups().stream().map(YamlShardingTableReferenceRuleConfigurationConverter::convertToYamlString).collect(Collectors.toList()));
        // 添加广播表
        result.getBroadcastTables().addAll(data.getBroadcastTables());
        // 默认策略转换
        setYamlStrategies(data, result);
        // 算法转换（分片算法、主键生成算法、审计员生成算法）
        setYamlAlgorithms(data, result);
        // 设置默认分片键
        result.setDefaultShardingColumn(data.getDefaultShardingColumn());
        return result;
    }

    /**
     * 默认策略转换。通过对应策略的转换器，将ShardingStrategyConfiguration转换为YamlShardingStrategyConfiguration
     */
    private void setYamlStrategies(final ShardingRuleConfiguration data, final YamlShardingRuleConfiguration yamlConfig) {
        if (null != data.getDefaultDatabaseShardingStrategy()) {
            yamlConfig.setDefaultDatabaseStrategy(shardingStrategySwapper.swapToYamlConfiguration(data.getDefaultDatabaseShardingStrategy()));
        }
        if (null != data.getDefaultTableShardingStrategy()) {
            yamlConfig.setDefaultTableStrategy(shardingStrategySwapper.swapToYamlConfiguration(data.getDefaultTableShardingStrategy()));
        }
        if (null != data.getDefaultKeyGenerateStrategy()) {
            yamlConfig.setDefaultKeyGenerateStrategy(keyGenerateStrategySwapper.swapToYamlConfiguration(data.getDefaultKeyGenerateStrategy()));
        }
        if (null != data.getDefaultAuditStrategy()) {
            yamlConfig.setDefaultAuditStrategy(auditStrategySwapper.swapToYamlConfiguration(data.getDefaultAuditStrategy()));
        }
    }

    /**
     * 算法深度转换
     */
    private void setYamlAlgorithms(final ShardingRuleConfiguration data, final YamlShardingRuleConfiguration yamlConfig) {
        if (null != data.getShardingAlgorithms()) {
            data.getShardingAlgorithms().forEach((key, value) -> yamlConfig.getShardingAlgorithms().put(key, algorithmSwapper.swapToYamlConfiguration(value)));
        }
        if (null != data.getKeyGenerators()) {
            data.getKeyGenerators().forEach((key, value) -> yamlConfig.getKeyGenerators().put(key, algorithmSwapper.swapToYamlConfiguration(value)));
        }
        if (null != data.getAuditors()) {
            data.getAuditors().forEach((key, value) -> yamlConfig.getAuditors().put(key, algorithmSwapper.swapToYamlConfiguration(value)));
        }
    }

    /**
     * 将YamlShardingRuleConfiguration转换为ShardingRuleConfiguration
     */
    @Override
    public ShardingRuleConfiguration swapToObject(final YamlShardingRuleConfiguration yamlConfig) {
        ShardingRuleConfiguration result = new ShardingRuleConfiguration();
        // 分表转换
        for (Entry<String, YamlTableRuleConfiguration> entry : yamlConfig.getTables().entrySet()) {
            YamlTableRuleConfiguration tableRuleConfig = entry.getValue();
            tableRuleConfig.setLogicTable(entry.getKey());
            result.getTables().add(tableSwapper.swapToObject(tableRuleConfig));
        }
        // 自动表转换
        for (Entry<String, YamlShardingAutoTableRuleConfiguration> entry : yamlConfig.getAutoTables().entrySet()) {
            YamlShardingAutoTableRuleConfiguration tableRuleConfig = entry.getValue();
            tableRuleConfig.setLogicTable(entry.getKey());
            result.getAutoTables().add(autoTableYamlSwapper.swapToObject(tableRuleConfig));
        }
        result.getBindingTableGroups().addAll(yamlConfig.getBindingTables().stream().map(YamlShardingTableReferenceRuleConfigurationConverter::convertToObject).collect(Collectors.toList()));
        result.getBroadcastTables().addAll(yamlConfig.getBroadcastTables());
        setStrategies(yamlConfig, result);
        setAlgorithms(yamlConfig, result);
        result.setDefaultShardingColumn(yamlConfig.getDefaultShardingColumn());
        return result;
    }

    /**
     * 默认策略转换
     */
    private void setStrategies(final YamlShardingRuleConfiguration yamlConfig, final ShardingRuleConfiguration ruleConfig) {
        if (null != yamlConfig.getDefaultDatabaseStrategy()) {
            ruleConfig.setDefaultDatabaseShardingStrategy(shardingStrategySwapper.swapToObject(yamlConfig.getDefaultDatabaseStrategy()));
        }
        if (null != yamlConfig.getDefaultTableStrategy()) {
            ruleConfig.setDefaultTableShardingStrategy(shardingStrategySwapper.swapToObject(yamlConfig.getDefaultTableStrategy()));
        }
        if (null != yamlConfig.getDefaultKeyGenerateStrategy()) {
            ruleConfig.setDefaultKeyGenerateStrategy(keyGenerateStrategySwapper.swapToObject(yamlConfig.getDefaultKeyGenerateStrategy()));
        }
        if (null != yamlConfig.getDefaultAuditStrategy()) {
            ruleConfig.setDefaultAuditStrategy(auditStrategySwapper.swapToObject(yamlConfig.getDefaultAuditStrategy()));
        }
    }

    /**
     * 算法转换。分片算法、主键生成算法、审计员生成算法）
     */
    private void setAlgorithms(final YamlShardingRuleConfiguration yamlConfig, final ShardingRuleConfiguration ruleConfig) {
        if (null != yamlConfig.getShardingAlgorithms()) {
            yamlConfig.getShardingAlgorithms().forEach((key, value) -> ruleConfig.getShardingAlgorithms().put(key, algorithmSwapper.swapToObject(value)));
        }
        if (null != yamlConfig.getKeyGenerators()) {
            yamlConfig.getKeyGenerators().forEach((key, value) -> ruleConfig.getKeyGenerators().put(key, algorithmSwapper.swapToObject(value)));
        }
        if (null != yamlConfig.getAuditors()) {
            yamlConfig.getAuditors().forEach((key, value) -> ruleConfig.getAuditors().put(key, algorithmSwapper.swapToObject(value)));
        }
    }
    
    @Override
    public Class<ShardingRuleConfiguration> getTypeClass() {
        return ShardingRuleConfiguration.class;
    }
    
    @Override
    public String getRuleTagName() {
        return "SHARDING";
    }
    
    @Override
    public int getOrder() {
        return ShardingOrder.ORDER;
    }
}

如果YamlXxxConfiguration类中有其他YamlYyyConfiguration属性对象，在对应的Swapper转换器中，会定义能够转换YamlYyyConfiguration的转换器，逐层将YamlXxxConfiguration转换为XxxConfiguration对象。

如上面的YamlShardingRuleConfiguration，该规则配置包含分表配置、分片策略、分片算法等，对应的类型为YamlTableRuleConfiguration、YamlShardingStrategyConfiguration、YamlAlgorithmConfiguration。

在YamlShardingRuleConfigurationSwapper中进行YamlShardingRuleConfiguration和ShardingRuleConfiguration的互转时，会调用YamlShardingTableRuleConfigurationSwapper，进行YamlTableRuleConfiguration和ShardingTableRuleConfiguration的互转。

mode配置转换

在上面的3.1中YamlShardingSphereDataSourceFactory的createDataSource()方法中，如果配置了mode，则通过YamlModeConfigurationSwapper.swapToObject()方法，将YamlModeConfiguration转换为ModeConfiguration。

4.1 YamlModeConfigurationSwapper

YamlModeConfigurationSwapper的源码如下：

package org.apache.shardingsphere.infra.yaml.config.swapper.mode;

/**
 * Yaml中的mode配置转换器。YamlModeConfiguration和ModeConfiguration的复转
 */
@SuppressWarnings("unchecked")
public final class YamlModeConfigurationSwapper implements YamlConfigurationSwapper<YamlModeConfiguration, ModeConfiguration> {

    /**
     * ModeConfiguration转为YamlModeConfiguration
     */
    @Override
    public YamlModeConfiguration swapToYamlConfiguration(final ModeConfiguration data) {
        YamlModeConfiguration result = new YamlModeConfiguration();
        result.setType(data.getType());
        // 持久化转换
        if (null != data.getRepository()) {
            // 通过Java SPI获取对应配置的type的YamlPersistRepositoryConfigurationSwapper对象。
            // 如Standalone为StandaloneYamlPersistRepositoryConfigurationSwapper
            // Cluster为ClusterYamlPersistRepositoryConfigurationSwapper
            YamlPersistRepositoryConfigurationSwapper<PersistRepositoryConfiguration> swapper = TypedSPIRegistry.getRegisteredService(
                    YamlPersistRepositoryConfigurationSwapper.class, data.getType());
            result.setRepository(swapper.swapToYamlConfiguration(data.getRepository()));
        }
        return result;
    }

    /**
     * YamlModeConfiguration转换为ModeConfiguration
     */
    @Override
    public ModeConfiguration swapToObject(final YamlModeConfiguration yamlConfig) {
        if (null == yamlConfig.getRepository()) {
            return new ModeConfiguration(yamlConfig.getType(), null);
        }
        // 根据配置的mode的类型，找到对应的持久化存储转换器，将YamlPersistRepositoryConfiguration转换为XxxPersistRepositoryConfiguration
        YamlPersistRepositoryConfigurationSwapper<PersistRepositoryConfiguration> swapper = TypedSPIRegistry.getRegisteredService(
                YamlPersistRepositoryConfigurationSwapper.class, yamlConfig.getType());
        return new ModeConfiguration(yamlConfig.getType(), swapper.swapToObject(yamlConfig.getRepository()));
    }
}

如配置的type为Standalone，则对应的持久化存储转换器为StandaloneYamlPersistRepositoryConfigurationSwapper。

4.2 StandaloneYamlPersistRepositoryConfigurationSwapper

StandaloneYamlPersistRepositoryConfigurationSwapper源码如下：

package org.apache.shardingsphere.mode.manager.standalone.yaml;

/**
 * mode类型为Standalone的持久化配置转换器
 */
public final class StandaloneYamlPersistRepositoryConfigurationSwapper implements YamlPersistRepositoryConfigurationSwapper<StandalonePersistRepositoryConfiguration> {
    
    @Override
    public YamlPersistRepositoryConfiguration swapToYamlConfiguration(final StandalonePersistRepositoryConfiguration data) {
        YamlPersistRepositoryConfiguration result = new YamlPersistRepositoryConfiguration();
        result.setType(data.getType());
        result.setProps(data.getProps());
        return result;
    }
    
    @Override
    public StandalonePersistRepositoryConfiguration swapToObject(final YamlPersistRepositoryConfiguration yamlConfig) {
        return new StandalonePersistRepositoryConfiguration(yamlConfig.getType(), yamlConfig.getProps());
    }
    
    @Override
    public String getType() {
        return "Standalone";
    }
}

小结

以上为本篇分析的全部内容，以下做一个小结：

1）Yaml中最核心的配置信息rules和mode，解析后的对象的类名为YamlXxxConfiguration，且对应一个XxxConfiguration类，通过对应的YamlXxxConfigurationSwapper进行互转；

2）dataSources配置，通过YamlDataSourceConfigurationSwapper转换器，采用反射机制，创建对应的DataSource对象；

3）rules配置，通过YamlRuleConfigurationSwapperEngine，进行YamlRuleConfiguration集合和RuleConfiguration集合的互转；

4）mode配置信息通过YamlModeConfigurationSwapper，将YamlModeConfiguraton转换为ModeConfiguration对象；

关于本篇内容你有什么自己的想法或独到见解，欢迎在评论区一起交流探讨下吧。

JingAi_jia917

关注

17
点赞
踩
15

收藏

觉得还不错? 一键收藏
0
评论
【源码】Sharding-JDBC源码分析之Yaml分片配置转换原理

本篇从源码的角度分析了Sharding-JDBC中的rules和mode配置从Yaml配置对象转换为对应的XxxConfiguration的实现原理。重点分析了YamlRuleConfigurationSwapperEngine、YamlShardingRuleConfigurationSwapper、YamlModeConfigurationSwapper、StandaloneYamlPersistRepositoryConfigurationSwapper。
复制链接

扫一扫

专栏目录