- 项目要做数据库水平分表,集成了sharding-jdbc。集成后本地开发项目启动慢要5分钟左右
根据日志找到源码执行位置 org.apache.shardingsphere.core.metadata.ShardingMetaDataLoader#loadShardingSchemaMetaData
private SchemaMetaData loadShardingSchemaMetaData(DatabaseType databaseType) throws SQLException {
log.info("Loading {} logic tables' meta data.", this.shardingRule.getTableRules().size());
Map<String, TableMetaData> tableMetaDataMap = new HashMap(this.shardingRule.getTableRules().size(), 1.0F);
Iterator var3 = this.shardingRule.getTableRules().iterator();
while(var3.hasNext()) {
TableRule each = (TableRule)var3.next();
tableMetaDataMap.put(each.getLogicTable(), this.load(each.getLogicTable(), databaseType));
}
return new SchemaMetaData(tableMetaDataMap);
}
SchemaMetaDataLoader.load (org.apache.shardingsphere.core.metadata.ShardingMetaDataLoader#loadDefaultSchemaMetaData ) 耗时较长 ,看到第二个参数值 this.maxConnectionsSizePerQuery 默认为1
private SchemaMetaData loadDefaultSchemaMetaData(DatabaseType databaseType) throws SQLException {
Optional<String> actualDefaultDataSourceName = this.shardingRule.findActualDefaultDataSourceName();
return actualDefaultDataSourceName.isPresent() ? SchemaMetaDataLoader.load((DataSource)this.dataSourceMap.get(actualDefaultDataSourceName.get()), this.maxConnectionsSizePerQuery, databaseType.getName()) : new SchemaMetaData(Collections.emptyMap());
}
官方解释: sharding-jdbc java配置手册
增加配置修改 maxConnectionsSizePerQuery 为 30
spring:
shardingsphere:
props:
max:
connections:
size:
per:
query: 30
修改后重启项目,加载时间由 328302 ms 降至 51977 ms,由此问题解决。
- 那么为什么修改连接数后就快了?
接着追查源码 org.apache.shardingsphere.sql.parser.binder.metadata.schema.SchemaMetaDataLoader#load(javax.sql.DataSource, int, java.lang.String),复制出关键代码,注释含义
public static SchemaMetaData load(DataSource dataSource, int maxConnectionCount, String databaseType) throws SQLException {
Connection connection = dataSource.getConnection();
Throwable var5 = null;
List tableNames;
try {
tableNames = loadAllTableNames(connection);
} catch (Throwable var14) {
var5 = var14;
throw var14;
} finally {
if (connection != null) {
if (var5 != null) {
try {
connection.close();
} catch (Throwable var13) {
var5.addSuppressed(var13);
}
} else {
connection.close();
}
}
}
log.info("Loading {} tables' meta data.", tableNames.size());
if (0 == tableNames.size()) {
return new SchemaMetaData(Collections.emptyMap());
} else {
// 根据最大连接数切分 tableNames 得出 tableGroups
List<List<String>> tableGroups = Lists.partition(tableNames, Math.max(tableNames.size() / maxConnectionCount, 1));
// 如果 tableGroups 不等于1,调用 asyncLoad 方法多线程加载元数据信息
Map<String, TableMetaData> tableMetaDataMap = 1 == tableGroups.size() ? load(dataSource.getConnection(), (Collection)tableGroups.get(0), databaseType) : asyncLoad(dataSource, maxConnectionCount, tableNames, tableGroups, databaseType);
return new SchemaMetaData(tableMetaDataMap);
}
}
private static Map<String, TableMetaData> asyncLoad(DataSource dataSource, int maxConnectionCount, List<String> tableNames, List<List<String>> tableGroups, String databaseType) throws SQLException {
Map<String, TableMetaData> result = new ConcurrentHashMap(tableNames.size(), 1.0F);
ExecutorService executorService = Executors.newFixedThreadPool(Math.min(tableGroups.size(), maxConnectionCount));
Collection<Future<Map<String, TableMetaData>>> futures = new LinkedList();
Iterator var8 = tableGroups.iterator();
while(var8.hasNext()) {
List<String> each = (List)var8.next();
futures.add(executorService.submit(() -> {
return load(dataSource.getConnection(), each, databaseType);
}));
}
var8 = futures.iterator();
while(var8.hasNext()) {
Future<Map<String, TableMetaData>> each = (Future)var8.next();
try {
result.putAll((Map)each.get());
} catch (ExecutionException | InterruptedException var11) {
if (var11.getCause() instanceof SQLException) {
throw (SQLException)var11.getCause();
}
Thread.currentThread().interrupt();
}
}
return result;
}