Sharding-jdbc-ReadWriteSplit-Sharding总结

最新推荐文章于 2023-07-29 10:04:55 发布

ANDM2

最新推荐文章于 2023-07-29 10:04:55 发布

阅读量533

点赞数

文章标签： 1024程序员节

本文链接：https://blog.csdn.net/u014511571/article/details/127494281

版权

背景

存在某个服务查询压力很大,但是之前只是查询主库,导致部分Session查询超时,导致web监控报警等一些列问题
为了解决这些问题,要求改造成本最小

现状

目前使用了sharding-jdbc-3.0.0.M1作为数据库中间件,由于sharding-jdbc-3.0.0-M1存在内存泄漏和主从延迟的问题,1-2个月内会升级成sharding-jdbc-4.1.1版本
项目中使用了spring bean的方式引入了sharding-jdbc分表
本次改造需要同时支持分表和读写分离

遇到的问题

官方文档读写分离和分片都是单独的,没有融合到一块 https://shardingsphere.apache.org/document/4.1.1/en/manual/sharding-jdbc/usage/read-write-splitting/
网上的教程,对于3.0.0-M1版本经过实验,配置项是错误的 https://www.jianshu.com/p/3d253d9c280c
按照上文的配置项目启动时会出现 " Cannot find data source in sharding rule, invalid actual data node is " 的异常

解决方式

网上查询资料发现没有类似的问题,因此根据报错采用查看源码的方式定位问题

sharding-jdbc引入方式

spring bean
yaml
spring -xml

https://shardingsphere.apache.org/document/4.1.1/en/manual/sharding-jdbc/configuration/

问题定位

应用初始化的时候,会判断数据源是否包含主从配置的数据源,如果不包含会抛出异常

List<MasterSlaveRuleConfiguration> getMasterSlaveRuleConfigs(){
    // 主从配置
    MasterSlaveRuleConfiguration masterSlave01 = new MasterSlaveRuleConfiguration("ds_0","ds_0", Arrays.asList("ds_0_slave_0"));
    MasterSlaveRuleConfiguration masterSlave02 = new MasterSlaveRuleConfiguration("ds_1","ds_1",Arrays.asList("ds_1_slave_0"));
    return Lists.newArrayList(masterSlave01,masterSlave02);
}

将读写分离和分片数据源关联

shardingRuleConfig.setDefaultDatabaseShardingStrategyConfig(new NoneShardingStrategyConfiguration());
        shardingRuleConfig.setDefaultTableShardingStrategyConfig(new NoneShardingStrategyConfiguration());
        shardingRuleConfig.setMasterSlaveRuleConfigs(getMasterSlaveRuleConfigs());
        Map<String, DataSource> createDataSourceMap = makeDynamicDataSource();
        Properties properties = new Properties();
        properties.setProperty(ShardingPropertiesConstant.SQL_SHOW.getKey(), "true");
        shardingRuleConfig.getTableRuleConfigs().add(getCustomerFileTableRuleConfiguration());
        return ShardingDataSourceFactory.createDataSource(createDataSourceMap, shardingRuleConfig, new HashMap(), new Properties());

初始化shariding数据源

import io.shardingsphere.core.api.config.ShardingRuleConfiguration;
import io.shardingsphere.core.jdbc.core.datasource.ShardingDataSource;
import io.shardingsphere.core.rule.ShardingRule;
import lombok.AccessLevel;
import lombok.NoArgsConstructor;

import javax.sql.DataSource;
import java.sql.SQLException;
import java.util.Map;
import java.util.Properties;

/**
 * Sharding data source factory.
 * 
 * @author zhangliang 
 */
@NoArgsConstructor(access = AccessLevel.PRIVATE)
public final class ShardingDataSourceFactory {
    
    /**
     * Create sharding data source.
     *
     * @param dataSourceMap data source map
     * @param shardingRuleConfig rule configuration for databases and tables sharding
     * @param configMap config map
     * @param props properties for data source
     * @return sharding data source
     * @throws SQLException SQL exception
     */
    public static DataSource createDataSource(
            final Map<String, DataSource> dataSourceMap, final ShardingRuleConfiguration shardingRuleConfig, final Map<String, Object> configMap, final Properties props) throws SQLException {
        return new ShardingDataSource(dataSourceMap, new ShardingRule(shardingRuleConfig, dataSourceMap.keySet()), configMap, props);
    }
}