集成 sharding-jdbc 启动慢

  1. 项目要做数据库水平分表,集成了sharding-jdbc。集成后本地开发项目启动慢要5分钟左右
    在这里插入图片描述
    根据日志找到源码执行位置 org.apache.shardingsphere.core.metadata.ShardingMetaDataLoader#loadShardingSchemaMetaData
  private SchemaMetaData loadShardingSchemaMetaData(DatabaseType databaseType) throws SQLException {
        log.info("Loading {} logic tables' meta data.", this.shardingRule.getTableRules().size());
        Map<String, TableMetaData> tableMetaDataMap = new HashMap(this.shardingRule.getTableRules().size(), 1.0F);
        Iterator var3 = this.shardingRule.getTableRules().iterator();

        while(var3.hasNext()) {
            TableRule each = (TableRule)var3.next();
            tableMetaDataMap.put(each.getLogicTable(), this.load(each.getLogicTable(), databaseType));
        }

        return new SchemaMetaData(tableMetaDataMap);
    }

SchemaMetaDataLoader.load (org.apache.shardingsphere.core.metadata.ShardingMetaDataLoader#loadDefaultSchemaMetaData ) 耗时较长 ,看到第二个参数值 this.maxConnectionsSizePerQuery 默认为1

   private SchemaMetaData loadDefaultSchemaMetaData(DatabaseType databaseType) throws SQLException {
        Optional<String> actualDefaultDataSourceName = this.shardingRule.findActualDefaultDataSourceName();
        return actualDefaultDataSourceName.isPresent() ? SchemaMetaDataLoader.load((DataSource)this.dataSourceMap.get(actualDefaultDataSourceName.get()), this.maxConnectionsSizePerQuery, databaseType.getName()) : new SchemaMetaData(Collections.emptyMap());
    }

官方解释: sharding-jdbc java配置手册
https://shardingsphere.apache.org/document/legacy/4.x/document/cn/manual/sharding-jdbc/configuration/config-java/

增加配置修改 maxConnectionsSizePerQuery 为 30

spring:
    shardingsphere:
        props:
            max:
                connections:
                    size:
                        per:
                            query: 30

修改后重启项目,加载时间由 328302 ms 降至 51977 ms,由此问题解决。

  1. 那么为什么修改连接数后就快了?
    接着追查源码 org.apache.shardingsphere.sql.parser.binder.metadata.schema.SchemaMetaDataLoader#load(javax.sql.DataSource, int, java.lang.String),复制出关键代码,注释含义

    public static SchemaMetaData load(DataSource dataSource, int maxConnectionCount, String databaseType) throws SQLException {
        Connection connection = dataSource.getConnection();
        Throwable var5 = null;

        List tableNames;
        try {
            tableNames = loadAllTableNames(connection);
        } catch (Throwable var14) {
            var5 = var14;
            throw var14;
        } finally {
            if (connection != null) {
                if (var5 != null) {
                    try {
                        connection.close();
                    } catch (Throwable var13) {
                        var5.addSuppressed(var13);
                    }
                } else {
                    connection.close();
                }
            }

        }

        log.info("Loading {} tables' meta data.", tableNames.size());
        if (0 == tableNames.size()) {
            return new SchemaMetaData(Collections.emptyMap());
        } else {
        	// 根据最大连接数切分 tableNames 得出 tableGroups
            List<List<String>> tableGroups = Lists.partition(tableNames, Math.max(tableNames.size() / maxConnectionCount, 1));
            //  如果 tableGroups 不等于1,调用 asyncLoad 方法多线程加载元数据信息
            Map<String, TableMetaData> tableMetaDataMap = 1 == tableGroups.size() ? load(dataSource.getConnection(), (Collection)tableGroups.get(0), databaseType) : asyncLoad(dataSource, maxConnectionCount, tableNames, tableGroups, databaseType);
            return new SchemaMetaData(tableMetaDataMap);
        }
    }




private static Map<String, TableMetaData> asyncLoad(DataSource dataSource, int maxConnectionCount, List<String> tableNames, List<List<String>> tableGroups, String databaseType) throws SQLException {
        Map<String, TableMetaData> result = new ConcurrentHashMap(tableNames.size(), 1.0F);
        ExecutorService executorService = Executors.newFixedThreadPool(Math.min(tableGroups.size(), maxConnectionCount));
        Collection<Future<Map<String, TableMetaData>>> futures = new LinkedList();
        Iterator var8 = tableGroups.iterator();

        while(var8.hasNext()) {
            List<String> each = (List)var8.next();
            futures.add(executorService.submit(() -> {
                return load(dataSource.getConnection(), each, databaseType);
            }));
        }

        var8 = futures.iterator();

        while(var8.hasNext()) {
            Future<Map<String, TableMetaData>> each = (Future)var8.next();

            try {
                result.putAll((Map)each.get());
            } catch (ExecutionException | InterruptedException var11) {
                if (var11.getCause() instanceof SQLException) {
                    throw (SQLException)var11.getCause();
                }

                Thread.currentThread().interrupt();
            }
        }

        return result;
    }
  • 0
    点赞
  • 1
    收藏
    觉得还不错? 一键收藏
  • 0
    评论

“相关推荐”对你有帮助么?

  • 非常没帮助
  • 没帮助
  • 一般
  • 有帮助
  • 非常有帮助
提交
评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值