【源码解析】分库分表框架 Shardingsphere 源码解析

于 2023-05-27 09:55:59 发布
阅读量1k 收藏
点赞数
分类专栏： 源码解析 开源项目介绍 文章标签： spring java spring boot
版权声明：本文为博主原创文章，遵循 CC 4.0 BY-SA 版权协议，转载请附上原文出处链接和本声明。
本文链接：https://blog.csdn.net/qq_42985872/article/details/130897705
版权

前言

以前研究过如何使用ShardingJdbc，使用ShardingJdbc进行分库分表，但是原理方面没有细致的深入了解。如果仅仅了解如何使用的话，对于改造和排查问题，其实都是不够的，所以跟踪源码了解其运行原理是很重要的。

Demo

依赖

        <!-- 分库分表  -->
        <dependency>
            <groupId>org.apache.shardingsphere</groupId>
            <artifactId>shardingsphere-jdbc-core-spring-boot-starter</artifactId>
            <version>5.1.2</version>
        </dependency>

配置文件

spring:
  shardingsphere:
    mode:
      type: Memory                                         # 内存模式，元数据保存在当前进程中
    datasource:
      names: test0,test1                                   # 数据源名称，这里有两个
      test0:                                               # 跟上面的数据源对应
        type: com.alibaba.druid.pool.DruidDataSource       # 连接池
        url: jdbc:mysql://127.0.0.1:3306/test0             # 连接url
        username: root
        password: root
      test1:                                               # 跟上面的数据源对应
        type: com.alibaba.druid.pool.DruidDataSource
        url: jdbc:mysql://127.0.0.1:3306/test1
        username: root
        password: root
    rules:
      sharding:
        tables:
          user:                                            # 这个可以随便取，问题不大
            actual-data-nodes: test$->{0..1}.user$->{0..2} # 实际节点名称，格式为 库名$->{0..n1}.表名$->{0..n2}
              # 其中n1、n2分别为库数量-1和表数量-1
            # 也可以使用${0..n1}的形式，但是会与Spring属性文件占位符冲突
            # 所以使用$->{0..n1}的形式

            database-strategy:                             # 分库策略
              standard:                                    # 标准分库策略
                sharding-column: age                       # 分库列名
                sharding-algorithm-name: age-mod           # 分库算法名字
            table-strategy:                                # 分表策略
              standard:                                    # 标准分表策略
                sharding-column: id                        # 分表列名
                sharding-algorithm-name: id-mod            # 分表算法名字
        sharding-algorithms:                               # 配置分库和分表的算法
          age-mod:                                         # 分库算法名字
            type: MOD                                      # 算法类型为取模
            props:                                         # 算法配置的键名，所有算法配置都需要在props下
              sharding-count: 2                            # 分片数量
          id-mod:                                          # 分表算法名字
            type: MOD                                      # 算法类型为取模
            props:                                         # 算法配置的键名，所有算法配置都需要在props下
              sharding-count: 3                            # 分片数量
    props:
      sql-show: true                                       # 打印SQL

创表语句

-- auto-generated definition
create table user
(
    id   bigint       not null
        primary key,
    name varchar(200) null,
    age  int          null
);

实体类

@Data
@Builder
public class User {
    
    @TableId(type = IdType.ASSIGN_ID)
    private Long id;
    private String name;
    private Integer age;
}

Mapper

public interface UserMapper extends BaseMapper<User> {

}

控制器

@RestController
@RequestMapping("/user")
public class UserController {

    @Autowired
    private UserMapper userMapper;

    @GetMapping("/insert")
    public boolean insert() {
        userMapper.insert(User.builder().name("name").age(new Random().nextInt(100) + 1).build());
        return true;
    }

    @GetMapping("/select")
    public List<User> select() {
        return userMapper.selectList(new QueryWrapper<>());
    }
}

源码解析

解析

PreparedStatementHandler#instantiateStatement，创建Statement会用到sql解析。

    protected Statement instantiateStatement(Connection connection) throws SQLException {
        String sql = this.boundSql.getSql();
        if (this.mappedStatement.getKeyGenerator() instanceof Jdbc3KeyGenerator) {
            String[] keyColumnNames = this.mappedStatement.getKeyColumns();
            return keyColumnNames == null ? connection.prepareStatement(sql, 1) : connection.prepareStatement(sql, keyColumnNames);
        } else {
            return this.mappedStatement.getResultSetType() == ResultSetType.DEFAULT ? connection.prepareStatement(sql) : connection.prepareStatement(sql, this.mappedStatement.getResultSetType().getValue(), 1007);
        }
    }

ShardingSpherePreparedStatement

    public ShardingSpherePreparedStatement(final ShardingSphereConnection connection, final String sql) throws SQLException {
        this(connection, sql, ResultSet.TYPE_FORWARD_ONLY, ResultSet.CONCUR_READ_ONLY, ResultSet.HOLD_CURSORS_OVER_COMMIT, false);
    }

    private ShardingSpherePreparedStatement(final ShardingSphereConnection connection, final String sql,
                                            final int resultSetType, final int resultSetConcurrency, final int resultSetHoldability, final boolean returnGeneratedKeys) throws SQLException {
        if (Strings.isNullOrEmpty(sql)) {
            SQLExceptionErrorCode errorCode = SQLExceptionErrorCode.SQL_STRING_NULL_OR_EMPTY;
            throw new SQLException(errorCode.getErrorMessage(), errorCode.getSqlState(), errorCode.getErrorCode());
        }
        this.connection = connection;
        metaDataContexts = connection.getContextManager().getMetaDataContexts();
        this.sql = sql;
        statements = new ArrayList<>();
        parameterSets = new ArrayList<>();
        Optional<SQLParserRule> sqlParserRule = metaDataContexts.getMetaData().getGlobalRuleMetaData().findSingleRule(SQLParserRule.class);
        Preconditions.checkState(sqlParserRule.isPresent());
        ShardingSphereSQLParserEngine sqlParserEngine = sqlParserRule.get().getSQLParserEngine(
                DatabaseTypeEngine.getTrunkDatabaseTypeName(metaDataContexts.getMetaData().getDatabases().get(connection.getDatabaseName()).getResource().getDatabaseType()));
        sqlStatement = sqlParserEngine.parse(sql, true);
        sqlStatementContext = SQLStatementContextFactory.newInstance(metaDataContexts.getMetaData().getDatabases(), sqlStatement, connection.getDatabaseName());
        parameterMetaData = new ShardingSphereParameterMetaData(sqlStatement);
        statementOption = returnGeneratedKeys ? new StatementOption(true) : new StatementOption(resultSetType, resultSetConcurrency, resultSetHoldability);
        executor = new DriverExecutor(connection);
        JDBCExecutor jdbcExecutor = new JDBCExecutor(connection.getContextManager().getExecutorEngine(), connection.isHoldTransaction());
        batchPreparedStatementExecutor = new BatchPreparedStatementExecutor(metaDataContexts, jdbcExecutor, connection.getDatabaseName());
        kernelProcessor = new KernelProcessor();
        statementsCacheable = isStatementsCacheable(metaDataContexts.getMetaData().getDatabases().get(connection.getDatabaseName()).getRuleMetaData().getConfigurations());
        trafficRule = metaDataContexts.getMetaData().getGlobalRuleMetaData().findSingleRule(TrafficRule.class).orElse(null);
        statementManager = new StatementManager();
    }

SQLStatementParserEngine#parse，判断是否有缓存。

    public SQLStatement parse(String sql, boolean useCache) {
        return useCache ? (SQLStatement)this.sqlStatementCache.get(sql) : this.sqlStatementParserExecutor.parse(sql);
    }

SQLParserExecutor#parse，解析sql，处理成多个片段。

    public ParseASTNode parse(String sql) {
        ParseASTNode result = this.twoPhaseParse(sql);
        if (result.getRootNode() instanceof ErrorNode) {
            throw new SQLParsingException("Unsupported SQL of `%s`", new Object[]{sql});
        } else {
            return result;
        }
    }

路由

PreparedStatementHandler#query，Mybatis执行查询获取到的PreparedStatement是ShardingSpherePreparedStatement。

	public <E> List<E> query(Statement statement, ResultHandler resultHandler) throws SQLException {
        PreparedStatement ps = (PreparedStatement)statement;
        ps.execute();
        return this.resultSetHandler.handleResultSets(ps);
    }

ShardingSpherePreparedStatement#execute，获取逻辑SQL，调试获取到的sql是INSERT INTO user ( id,name,age ) VALUES ( ?,?,? )。

    @Override
    public boolean execute() throws SQLException {
        try {
            if (statementsCacheable && !statements.isEmpty()) {
                resetParameters();
                return statements.iterator().next().execute();
            }
            clearPrevious();
            LogicSQL logicSQL = createLogicSQL();
            trafficContext = getTrafficContext(logicSQL);
            if (trafficContext.isMatchTraffic()) {
                JDBCExecutionUnit executionUnit = createTrafficExecutionUnit(trafficContext, logicSQL);
                return executor.getTrafficExecutor().execute(executionUnit, (statement, sql) -> ((PreparedStatement) statement).execute());
            }
            executionContext = createExecutionContext(logicSQL);
            if (hasRawExecutionRule()) {
                // TODO process getStatement
                Collection<ExecuteResult> executeResults = executor.getRawExecutor().execute(createRawExecutionGroupContext(), executionContext.getLogicSQL(), new RawSQLExecutorCallback());
                return executeResults.iterator().next() instanceof QueryResult;
            }
            if (executionContext.getRouteContext().isFederated()) {
                ResultSet resultSet = executeFederationQuery(logicSQL);
                return null != resultSet;
            }
            ExecutionGroupContext<JDBCExecutionUnit> executionGroupContext = createExecutionGroupContext();
            cacheStatements(executionGroupContext.getInputGroups());
            return executor.getRegularExecutor().execute(executionGroupContext,
                    executionContext.getLogicSQL(), executionContext.getRouteContext().getRouteUnits(), createExecuteCallback());
        } catch (SQLException ex) {
            handleExceptionInTransaction(connection, metaDataContexts);
            throw ex;
        } finally {
            clearBatch();
        }
    }

KernelProcessor#generateExecutionContext，处理sql主要是进行路由，改写

    public ExecutionContext generateExecutionContext(LogicSQL logicSQL, ShardingSphereDatabase database, ConfigurationProperties props) {
        RouteContext routeContext = this.route(logicSQL, database, props);
        SQLRewriteResult rewriteResult = this.rewrite(logicSQL, database, props, routeContext);
        ExecutionContext result = this.createExecutionContext(logicSQL, database, routeContext, rewriteResult);
        this.logSQL(logicSQL, props, result);
        return result;
    }

SQLRouteEngine#route，SQLRouteEngine调用SQLRouteExecutor进行路由，获取到的是PartialSQLRouteExecutor。

    public RouteContext route(final LogicSQL logicSQL, final ShardingSphereDatabase database) {
        SQLRouteExecutor executor = isNeedAllSchemas(logicSQL.getSqlStatementContext().getSqlStatement()) ? new AllSQLRouteExecutor() : new PartialSQLRouteExecutor(rules, props);
        return executor.route(logicSQL, database);
    }

PartialSQLRouteExecutor，根据SQLRouterFactory获取routers。

    public PartialSQLRouteExecutor(final Collection<ShardingSphereRule> rules, final ConfigurationProperties props) {
        this.props = props;
        routers = SQLRouterFactory.getInstances(rules);
    }

SQLRouterFactory#getInstances，通过SPI的方式获取到SQLRouter。

    public static Map<ShardingSphereRule, SQLRouter> getInstances(final Collection<ShardingSphereRule> rules) {
        return OrderedSPIRegistry.getRegisteredServices(SQLRouter.class, rules);
    }

PartialSQLRouteExecutor#route，根据ShardingSphereRule和SQLRouter获取结果。

    public RouteContext route(final LogicSQL logicSQL, final ShardingSphereDatabase database) {
        RouteContext result = new RouteContext();
        Optional<String> dataSourceName = findDataSourceByHint(logicSQL.getSqlStatementContext(), database.getResource().getDataSources());
        if (dataSourceName.isPresent()) {
            result.getRouteUnits().add(new RouteUnit(new RouteMapper(dataSourceName.get(), dataSourceName.get()), Collections.emptyList()));
            return result;
        }
        for (Entry<ShardingSphereRule, SQLRouter> entry : routers.entrySet()) {
            if (result.getRouteUnits().isEmpty()) {
                result = entry.getValue().createRouteContext(logicSQL, database, entry.getKey(), props);
            } else {
                entry.getValue().decorateRouteContext(result, logicSQL, database, entry.getKey(), props);
            }
        }
        if (result.getRouteUnits().isEmpty() && 1 == database.getResource().getDataSources().size()) {
            String singleDataSourceName = database.getResource().getDataSources().keySet().iterator().next();
            result.getRouteUnits().add(new RouteUnit(new RouteMapper(singleDataSourceName, singleDataSourceName), Collections.emptyList()));
        }
        return result;
    }

ShardingSQLRouter#createRouteContext，根据对应的分库分表规则计算结果。

    public RouteContext createRouteContext(LogicSQL logicSQL, ShardingSphereDatabase database, ShardingRule rule, ConfigurationProperties props) {
        SQLStatement sqlStatement = logicSQL.getSqlStatementContext().getSqlStatement();
        ShardingConditions shardingConditions = this.createShardingConditions(logicSQL, database, rule);
        Optional<ShardingStatementValidator> validator = ShardingStatementValidatorFactory.newInstance(sqlStatement, shardingConditions);
        validator.ifPresent((optional) -> {
            optional.preValidate(rule, logicSQL.getSqlStatementContext(), logicSQL.getParameters(), database);
        });
        if (sqlStatement instanceof DMLStatement && shardingConditions.isNeedMerge()) {
            shardingConditions.merge();
        }

        RouteContext result = ShardingRouteEngineFactory.newInstance(rule, database, logicSQL.getSqlStatementContext(), shardingConditions, props).route(rule);
        validator.ifPresent((optional) -> {
            optional.postValidate(rule, logicSQL.getSqlStatementContext(), logicSQL.getParameters(), database, props, result);
        });
        return result;
    }

ShardingStandardRoutingEngine#route，获取到DataNode的集合。

    public RouteContext route(ShardingRule shardingRule) {
        RouteContext result = new RouteContext();
        Collection<DataNode> dataNodes = this.getDataNodes(shardingRule, shardingRule.getTableRule(this.logicTableName));
        result.getOriginalDataNodes().addAll(this.originalDataNodes);
        Iterator var4 = dataNodes.iterator();

        while(var4.hasNext()) {
            DataNode each = (DataNode)var4.next();
            result.getRouteUnits().add(new RouteUnit(new RouteMapper(each.getDataSourceName(), each.getDataSourceName()), Collections.singleton(new RouteMapper(this.logicTableName, each.getTableName()))));
        }

        return result;
    }

ShardingStandardRoutingEngine#getDataNodes，根据shardingCondition的数据进行路由，根据分库策略获取库名，再根据库名和分表策略获取表名。

    private Collection<DataNode> getDataNodes(final ShardingRule shardingRule, final TableRule tableRule) {
        ShardingStrategy databaseShardingStrategy = createShardingStrategy(shardingRule.getDatabaseShardingStrategyConfiguration(tableRule),
                shardingRule.getShardingAlgorithms(), shardingRule.getDefaultShardingColumn());
        ShardingStrategy tableShardingStrategy = createShardingStrategy(shardingRule.getTableShardingStrategyConfiguration(tableRule),
                shardingRule.getShardingAlgorithms(), shardingRule.getDefaultShardingColumn());
        if (isRoutingByHint(shardingRule, tableRule)) {
            return routeByHint(tableRule, databaseShardingStrategy, tableShardingStrategy);
        }
        if (isRoutingByShardingConditions(shardingRule, tableRule)) {
            return routeByShardingConditions(shardingRule, tableRule, databaseShardingStrategy, tableShardingStrategy);
        }
        return routeByMixedConditions(shardingRule, tableRule, databaseShardingStrategy, tableShardingStrategy);
    }

    private Collection<DataNode> routeByShardingConditions(final ShardingRule shardingRule, final TableRule tableRule,
                                                           final ShardingStrategy databaseShardingStrategy, final ShardingStrategy tableShardingStrategy) {
        return shardingConditions.getConditions().isEmpty()
                ? route0(tableRule, databaseShardingStrategy, Collections.emptyList(), tableShardingStrategy, Collections.emptyList())
                : routeByShardingConditionsWithCondition(shardingRule, tableRule, databaseShardingStrategy, tableShardingStrategy);
    }

    private Collection<DataNode> routeByShardingConditionsWithCondition(final ShardingRule shardingRule, final TableRule tableRule,
                                                                        final ShardingStrategy databaseShardingStrategy, final ShardingStrategy tableShardingStrategy) {
        Collection<DataNode> result = new LinkedList<>();
        for (ShardingCondition each : shardingConditions.getConditions()) {
            Collection<DataNode> dataNodes = route0(tableRule,
                    databaseShardingStrategy, getShardingValuesFromShardingConditions(shardingRule, databaseShardingStrategy.getShardingColumns(), each),
                    tableShardingStrategy, getShardingValuesFromShardingConditions(shardingRule, tableShardingStrategy.getShardingColumns(), each));
            result.addAll(dataNodes);
            originalDataNodes.add(dataNodes);
        }
        return result;
    }

    private Collection<DataNode> route0(final TableRule tableRule,
                                        final ShardingStrategy databaseShardingStrategy, final List<ShardingConditionValue> databaseShardingValues,
                                        final ShardingStrategy tableShardingStrategy, final List<ShardingConditionValue> tableShardingValues) {
        Collection<String> routedDataSources = routeDataSources(tableRule, databaseShardingStrategy, databaseShardingValues);
        Collection<DataNode> result = new LinkedList<>();
        for (String each : routedDataSources) {
            result.addAll(routeTables(tableRule, each, tableShardingStrategy, tableShardingValues));
        }
        return result;
    }

ShardingStandardRoutingEngine#routeDataSources

    private Collection<String> routeDataSources(final TableRule tableRule, final ShardingStrategy databaseShardingStrategy, final List<ShardingConditionValue> databaseShardingValues) {
        if (databaseShardingValues.isEmpty()) {
            return tableRule.getActualDatasourceNames();
        }
        Collection<String> result = databaseShardingStrategy.doSharding(tableRule.getActualDatasourceNames(), databaseShardingValues, tableRule.getDataSourceDataNode(), properties);
        Preconditions.checkState(!result.isEmpty(), "No database route info");
        Preconditions.checkState(tableRule.getActualDatasourceNames().containsAll(result),
                "Some routed data sources do not belong to configured data sources. routed data sources: `%s`, configured data sources: `%s`", result, tableRule.getActualDatasourceNames());
        return result;
    }

StandardShardingStrategy#doSharding()

    @Override
    public Collection<String> doSharding(final Collection<String> availableTargetNames, final Collection<ShardingConditionValue> shardingConditionValues,
                                         final DataNodeInfo dataNodeInfo, final ConfigurationProperties props) {
        ShardingConditionValue shardingConditionValue = shardingConditionValues.iterator().next();
        Collection<String> shardingResult = shardingConditionValue instanceof ListShardingConditionValue
                ? doSharding(availableTargetNames, (ListShardingConditionValue) shardingConditionValue, dataNodeInfo)
                : doSharding(availableTargetNames, (RangeShardingConditionValue) shardingConditionValue, dataNodeInfo);
        Collection<String> result = new TreeSet<>(String.CASE_INSENSITIVE_ORDER);
        result.addAll(shardingResult);
        return result;
    }

    private Collection<String> doSharding(final Collection<String> availableTargetNames, final ListShardingConditionValue<?> shardingValue, final DataNodeInfo dataNodeInfo) {
        Collection<String> result = new LinkedList<>();
        for (Comparable<?> each : shardingValue.getValues()) {
            String target = shardingAlgorithm.doSharding(availableTargetNames,
                    new PreciseShardingValue(shardingValue.getTableName(), shardingValue.getColumnName(), dataNodeInfo, each));
            if (null != target && availableTargetNames.contains(target)) {
                result.add(target);
            } else if (null != target && !availableTargetNames.contains(target)) {
                throw new ShardingSphereException(String.format("Route table %s does not exist, available actual table: %s", target, availableTargetNames));
            }
        }
        return result;
    }

ModShardingAlgorithm#doSharding()，如果是取余的算法就是获取后缀值。

    @Override
    public String doSharding(final Collection<String> availableTargetNames, final PreciseShardingValue<Comparable<?>> shardingValue) {
        String shardingResultSuffix = getShardingResultSuffix(cutShardingValue(shardingValue.getValue()).mod(new BigInteger(String.valueOf(shardingCount))).toString());
        return findMatchedTargetName(availableTargetNames, shardingResultSuffix, shardingValue.getDataNodeInfo()).orElse(null);
    }

重写

KernelProcessor#rewrite，调用SQLRewriteEntry重写Sql。

    private SQLRewriteResult rewrite(LogicSQL logicSQL, ShardingSphereDatabase database, ConfigurationProperties props, RouteContext routeContext) {
        SQLRewriteEntry sqlRewriteEntry = new SQLRewriteEntry(database, props);
        return sqlRewriteEntry.rewrite(logicSQL.getSql(), logicSQL.getParameters(), logicSQL.getSqlStatementContext(), routeContext);
    }

SQLRewriteEntry#rewrite，调用RouteSQLRewriteEngine重写Sql。

    public SQLRewriteResult rewrite(final String sql, final List<Object> parameters, final SQLStatementContext<?> sqlStatementContext, final RouteContext routeContext) {
        SQLRewriteContext sqlRewriteContext = createSQLRewriteContext(sql, parameters, sqlStatementContext, routeContext);
        SQLTranslatorRule rule = database.getRuleMetaData().findSingleRule(SQLTranslatorRule.class).orElseGet(() -> new SQLTranslatorRule(new SQLTranslatorRuleConfiguration()));
        DatabaseType protocolType = database.getProtocolType();
        DatabaseType storageType = database.getResource().getDatabaseType();
        return routeContext.getRouteUnits().isEmpty()
                ? new GenericSQLRewriteEngine(rule, protocolType, storageType).rewrite(sqlRewriteContext)
                : new RouteSQLRewriteEngine(rule, protocolType, storageType).rewrite(sqlRewriteContext, routeContext);
    }

RouteSQLRewriteEngine#rewrite，重写sql。

    public RouteSQLRewriteResult rewrite(final SQLRewriteContext sqlRewriteContext, final RouteContext routeContext) {
        Map<RouteUnit, SQLRewriteUnit> sqlRewriteUnits = new LinkedHashMap<>(routeContext.getRouteUnits().size(), 1);
        for (Entry<String, Collection<RouteUnit>> entry : aggregateRouteUnitGroups(routeContext.getRouteUnits()).entrySet()) {
            Collection<RouteUnit> routeUnits = entry.getValue();
            if (isNeedAggregateRewrite(sqlRewriteContext.getSqlStatementContext(), routeUnits)) {
                sqlRewriteUnits.put(routeUnits.iterator().next(), createSQLRewriteUnit(sqlRewriteContext, routeContext, routeUnits));
            } else {
                addSQLRewriteUnits(sqlRewriteUnits, sqlRewriteContext, routeContext, routeUnits);
            }
        }
        return new RouteSQLRewriteResult(translate(sqlRewriteContext.getSqlStatementContext().getSqlStatement(), sqlRewriteUnits));
    }

    private void addSQLRewriteUnits(final Map<RouteUnit, SQLRewriteUnit> sqlRewriteUnits, final SQLRewriteContext sqlRewriteContext,
                                    final RouteContext routeContext, final Collection<RouteUnit> routeUnits) {
        for (RouteUnit each : routeUnits) {
            sqlRewriteUnits.put(each, new SQLRewriteUnit(new RouteSQLBuilder(sqlRewriteContext, each).toSQL(), getParameters(sqlRewriteContext.getParameterBuilder(), routeContext, each)));
        }
    }

AbstractSQLBuilder#toSQL，重新生成SQL。

    @Override
    public final String toSQL() {
        if (context.getSqlTokens().isEmpty()) {
            return context.getSql();
        }
        Collections.sort(context.getSqlTokens());
        StringBuilder result = new StringBuilder();
        result.append(context.getSql(), 0, context.getSqlTokens().get(0).getStartIndex());
        for (SQLToken each : context.getSqlTokens()) {
            result.append(each instanceof ComposableSQLToken ? getComposableSQLTokenText((ComposableSQLToken) each) : getSQLTokenText(each));
            result.append(getConjunctionText(each));
        }
        return result.toString();
    }

归并

ShardingSpherePreparedStatement#getResultSet()，归并查询到的结果集。

    @Override
    public ResultSet getResultSet() throws SQLException {
        if (null != currentResultSet) {
            return currentResultSet;
        }
        if (trafficContext.isMatchTraffic()) {
            return executor.getTrafficExecutor().getResultSet();
        }
        if (executionContext.getRouteContext().isFederated()) {
            return executor.getFederationExecutor().getResultSet();
        }
        if (executionContext.getSqlStatementContext() instanceof SelectStatementContext || executionContext.getSqlStatementContext().getSqlStatement() instanceof DALStatement) {
            List<ResultSet> resultSets = getResultSets();
            MergedResult mergedResult = mergeQuery(getQueryResults(resultSets));
            currentResultSet = new ShardingSphereResultSet(resultSets, mergedResult, this, executionContext);
        }
        return currentResultSet;
    }

MergeEngine#merge，执行merge。

    public MergedResult merge(final List<QueryResult> queryResults, final SQLStatementContext<?> sqlStatementContext) throws SQLException {
        Optional<MergedResult> mergedResult = executeMerge(queryResults, sqlStatementContext);
        Optional<MergedResult> result = mergedResult.isPresent() ? Optional.of(decorate(mergedResult.get(), sqlStatementContext)) : decorate(queryResults.get(0), sqlStatementContext);
        return result.orElseGet(() -> new TransparentMergedResult(queryResults.get(0)));
    }
    
    @SuppressWarnings({"unchecked", "rawtypes"})
    private Optional<MergedResult> executeMerge(final List<QueryResult> queryResults, final SQLStatementContext<?> sqlStatementContext) throws SQLException {
        for (Entry<ShardingSphereRule, ResultProcessEngine> entry : engines.entrySet()) {
            if (entry.getValue() instanceof ResultMergerEngine) {
                ResultMerger resultMerger = ((ResultMergerEngine) entry.getValue()).newInstance(
                        database.getName(), database.getResource().getDatabaseType(), entry.getKey(), props, sqlStatementContext);
                return Optional.of(resultMerger.merge(queryResults, sqlStatementContext, database));
            }
        }
        return Optional.empty();
    }

ShardingDQLResultMerger#merge，根据group by、distinct、order by等关键字做不同的归并处理

    public MergedResult merge(final List<QueryResult> queryResults, final SQLStatementContext<?> sqlStatementContext, final ShardingSphereDatabase database) throws SQLException {
        if (1 == queryResults.size() && !isNeedAggregateRewrite(sqlStatementContext)) {
            return new IteratorStreamMergedResult(queryResults);
        }
        Map<String, Integer> columnLabelIndexMap = getColumnLabelIndexMap(queryResults.get(0));
        SelectStatementContext selectStatementContext = (SelectStatementContext) sqlStatementContext;
        selectStatementContext.setIndexes(columnLabelIndexMap);
        MergedResult mergedResult = build(queryResults, selectStatementContext, columnLabelIndexMap, database);
        return decorate(queryResults, selectStatementContext, mergedResult);
    }

    private MergedResult build(final List<QueryResult> queryResults, final SelectStatementContext selectStatementContext,
                               final Map<String, Integer> columnLabelIndexMap, final ShardingSphereDatabase database) throws SQLException {
        String defaultSchemaName = DatabaseTypeEngine.getDefaultSchemaName(selectStatementContext.getDatabaseType(), database.getName());
        ShardingSphereSchema schema = selectStatementContext.getTablesContext().getSchemaName()
                .map(optional -> database.getSchemas().get(optional)).orElseGet(() -> database.getSchemas().get(defaultSchemaName));
        if (isNeedProcessGroupBy(selectStatementContext)) {
            return getGroupByMergedResult(queryResults, selectStatementContext, columnLabelIndexMap, schema);
        }
        if (isNeedProcessDistinctRow(selectStatementContext)) {
            setGroupByForDistinctRow(selectStatementContext);
            return getGroupByMergedResult(queryResults, selectStatementContext, columnLabelIndexMap, schema);
        }
        if (isNeedProcessOrderBy(selectStatementContext)) {
            return new OrderByStreamMergedResult(queryResults, selectStatementContext, schema);
        }
        return new IteratorStreamMergedResult(queryResults);
    }

在这里插入图片描述

秋装什么
关注
  • 0
    点赞
  • 踩
  • 0
    收藏
    觉得还不错? 一键收藏
  • 0
    评论
专栏目录
记录学习 ShardingSphere 遇到的坑
qq_27991253的博客
11-21 2435
记录学习 ShardingSphere 遇到的坑
shardingsphere源码分析（三）-- 路由引擎
可惜没如果的博客
08-31 465
shardingsphere源码分析（二）-- 解析引擎 官方介绍 链接如下： https://shardingsphere.apache.org/document/current/cn/features/sharding/principle/route/ 根据解析上下文匹配数据库和表的分片策略，并生成路由路径。 对于携带分片键的 SQL，根据分片键的不同可以划分为单片路由(分片键的操作符是等号)、多片路由(分片键的操作符是 IN)和范围路由(分片键的操作符是 BETWEEN)。 不携带分片键的 SQL 则
Sharding-jdbc使用干货
x.1000001000的博客
11-06 367
这里： 先来个分表： 引入配置文件： 依赖 <dependency> <groupId>org.apache.shardingsphere</groupId> <artifactId>sharding-jdbc-spring-boot-starter</artifactId&gt...
ShardingSphere笔记（一）： 经验和踩坑总结
MengFly的博客
11-19 9818
这是我在使用和封装框架的过程中一个坑一个坑总结到的最深刻的经验。 利用Springboot spi 提供自动注入。维持原本的SardingSphere-jdbc-core-spring-boot-starter 配置不变，提供下面的功能。 1. 根据配置自动配置历史数据的按月分表 2. 无需配置真实数据表，框架启动后会根据路由判断真实表是否存在，真实表不存在自动创建表。（查询语句不创建表，否则将会导致大量空表） 3. 动态切换数据库
集成 sharding-jdbc 启动慢
我是李白啦~
04-12 1063
sharding-jdbc启动慢
SpringBoot + ShardingSphere分库分表实战(附完整代码)
04-11
【SpringBoot + ShardingSphere分库分表实战】 在应对大数据量的系统中，分库分表是一种常见的解决策略，可以有效地分散数据库压力，提高系统的处理能力。在Java开发领域，ShardingSphere是一个广泛使用的开源...
深入剖析ShardingSphere：探索其内核原理与核心源码，揭秘分库分表技术的奥秘
最新发布
01-02
本文深入探讨了ShardingSphere的内核原理及其核心源码，提供了关于其解析、路由、改写、执行和归并引擎的详细分析。首先，解析引擎部分介绍了...此外，还包括了设置源码环境和具体实现分库分表功能的步骤。
深度学习框架Caffe源码解析
02-01
本文将从整体架构和底层实现的视角，对Caffe源码进行解析。 Caffe框架主要有五个组件，Blob，Solver，Net，Layer，Proto，其结构图如下图1所示。Solver负责深度网络的训练，每个Solver中包含一个训练网络对象和一个...
mycat1.6.5源码，分库分表，分布式
12-22
mycat1.6.5源码，分库分表，分布式
Spring MVC +Spring + Mybatis 构建分库分表源码
03-10
本资源对应博文：http://blog.csdn.net/buchengbugui/article/details/60972176，可以通过博文进行学习，博文有详细教程，以及代码。
shardingsphere-demo
04-06
分片球演示
shardingsphere.core.exception.ShardingException: Unsupported Date type:class java.lang.String
05-09 3521
最近使用sharding做分表, 其中做一个简单的查询时出现以下报错. Cause: org.apache.shardingsphere.core.exception.ShardingException: Unsupported Date type:class java.lang.String at org.apache.ibatis.exceptions.ExceptionFactory.wrapException(ExceptionFactory.java:30) at org.apache.i
Sharding JDBC 实战 —— 分布式事务处理
Java是世界上最好的语言
01-07 5828
在我们使用Sharding JDBC分库分表的时候，会带来另外一个问题，就是分布式事务问题，如下图所示。用户采购商品业务，整个业务包含3个微服务： 库存服务: 扣减给定商品的库存数量。 订单服务: 根据采购请求生成订单。 账户服务: 用户账户金额扣减。 这三个业务操作应该属于同一个事务，但是这些数据却分配在不同的数据库上，所以没办法采用数据库的事务来保证数据一致性。 这个时候，要解决分布式事务问题，就需要引入全局事务。 1. 什么是全局事务呢？ 全局事务是一个DTP模型的事务，所谓DTP模
SpringBoot+ShardingSphereJDBC实现读写分离！
m0_71777195的博客
08-08 555
本文讲述了如何使用 ​​ + ​​ 进行读写分离，以及利用 ​​MySQL​​ 进行一主一从的主从复制。​​MySQL​​ 主从复制环境准备（Docker）搭建++Druid环境测试服务端代理：需要独立部署一个代理服务，该代理服务后面管理多个数据库实例，在应用中通过一个数据源与该代理服务器建立连接，由该代理去操作底层数据库，并返回相应结果。优点是支持多语言，对业务透明，缺点是实现复杂，实现难度大，同时代理需要确保自身高可用。...
SpringBoot整合Sharding-JDBC实现水平分表
桐花思雨
01-12 1314
目录数据库的水平分表需求说明 数据库的水平分表 关于水平分表可以参考文章：https://blog.csdn.net/weixin_38192427/article/details/122441366 需求说明 创建两张表 t_order_1 和 t_order_2，这两张表是订单表水平拆分后的表，通过 Sharding-JDBC 向订单表插入数据， 按照一定的分片规则，主键为偶数的进入 t_order_1，主键为奇数进入 t_order_2，通过 Sharding-JDBC 查询数据，根据 SQL 语句的
ShardingException: Cannot find data source in sharding rule, invalid actual data node is
yygr的博客
05-01 2896
https://blog.csdn.net/qq_39846820/article/details/108016846?spm=1001.2101.3001.6661.1&utm_medium=distribute.pc_relevant_t0.none-task-blog-2%7Edefault%7ECTRLIST%7Edefault-1.pc_relevant_aa&depth_1-utm_source=distribute.pc_relevant_t0.none-task-blog-2
sharding sphere 4.1.1 - “路由装饰器RouteDecorator-分片” 之 “分片路由-引擎”
~~简~~
03-12 477
. .
分库分表利器之Sharding Sphere（深度好文，看过的人都说好）
跟着Mic学架构
10-24 1852
Sharding-Sphere Sharding-JDBC 最早是当当网内部使用的一款分库分表框架，到2017年的时候才开始对外开源，这几年在大量社区贡献者的不断迭代下，功能也逐渐完善，现已更名为 ShardingSphere，2020年4⽉16⽇正式成为 Apache 软件基⾦会的顶级项⽬。 随着版本的不断更迭 ShardingSphere 的核心功能也变得多元化起来。如图7-1，ShardingSphere生态包含三款开源分布式数据库中间件解决方案，Sharding-JDBC、Sharding-Prox
sharding-jdbc-spring-boot-starter最简单使用
weixin_33613947的博客
11-03 4048
前言 网上已经有很多的关于sharding-jdbc的使用，但是很多都是抄来抄去，说的也不是特别的完整，作者本来是闲来无事想跑起来试一下效果，但是找了一些文档都不是说的很明白。而且很多都是使用的是sharding-jdbc-core ，但是我们现在开发应该很少用spring mvc开发吧，大部分都是springboot开发，所以还是想用starter的方式。经历不断试错后搭建起最简单的、配置代码最少的测试案例。如果帮助到了你帮忙点赞啊。 一、引入依赖 mybatis-spring-boot-starter
Java源码解析框架有哪些？
03-29
Java源码解析框架有以下几种： 1. Javaparser：一个用Java编写的解析器，可以解析Java源代码，并提供了API，可以以编程方式访问解析结果。 2. ANTLR：一个强大的语法分析器生成器，可以用于生成用于解析Java源代码...

“相关推荐”对你有帮助么？

  • 非常没帮助
  • 没帮助
  • 一般
  • 有帮助
  • 非常有帮助
提交
评论
添加红包

请填写红包祝福语或标题

个

红包个数最小为10个

元

红包金额最低5元

当前余额3.43元 前往充值 >
需支付：10.00元
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付元
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明：

1.余额是钱包充值的虚拟货币，按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载，可以购买VIP、付费专栏及课程。

余额充值