数据节点是数据分片的最小单元,由数据源名称和真实表组成。
例:ds_0.t_order_0
直接配置
最直接就是枚举数据节点,如:
db0.t_order0, db0.t_order1, db1.t_order0, db1.t_order1
shardingsphere:
mode:
type: Standalone #单机模式 集群模式:Cluster
datasource:
names: ds0
ds0:
type: com.alibaba.druid.pool.DruidDataSource
driver-class-name: com.mysql.cj.jdbc.Driver
url: jdbc:mysql://192.168.34.72:3306/fsn_core_custom?useUnicode=true&characterEncoding=utf-8&allowMultiQueries=true&serverTimezone=Asia/Shanghai
username: root
password: xxxxx
rules:
sharding:
tables:
t_order:
actual-data-nodes: db0.t_order0, db0.t_order1, db1.t_order0, db1.t_order1
table-strategy: #分表策略
standard: #取值范围:standard 单分片键,complex 多分片键,hint 分配策略
sharding-column: id # 分片列名称
sharding-algorithm-name: order_name #分片算法名称
sharding-algorithms:
order_name: #表的分片算法名称
type: INLINE
props:
# id % 3 以便于获取到0、1、2 几个余数,以便于拼接t_order_0,t_order_1,t_order_2
algorithm-expression: t_order_$->{id % 3}
props:
sql-show: true #打印SQL
行表达式
行表达式使用groovy语法,如:
actual-data-nodes: ds0.t_order_${0..3} #行表达式
以上表示表:
db0.t_order0, db0.t_order1, db0.t_order3
行表示式还有很多高阶的用法,但是对groovy语法没有太多研究,这里就不列举了。
方法调用
在yaml文件中可以使用静态方法调用来返回数据节点:
actual-data-nodes: ds.table_name_$->{com.demo.util.LocalShardingDatabasesAndTablesUtils.getActualDataNodes()}
public class LocalShardingDatabasesAndTablesUtils {
public static int offset = 1;
public static List<String> getActualDataNodes() {
int startYear = 2023;
int endYear = LocalDate.now().getYear() + offset;
return LongStream.range(startYear, endYear)
.mapToObj(Long::toString)
.collect(Collectors.toList());
}
}
但是不知道怎么传入参数
在JavaAPI配置的方式中,可以调用方法返回需要配置的节点:
String newActualDataNodes = buildActualDataNodes(dataSourceName, logicTableName);
// 3、设置置分库分表策略
// (1)设置分片规则,logicTable:逻辑表、actualDataNodes:真实的数据节点
ShardingTableRuleConfiguration orderTableRuleConfiguration = new ShardingTableRuleConfiguration(logicTableName, newActualDataNodes);
//设置新表使用的算法 以及 分片策略
String shardingAlgorithmName = logicTableName + "_algorithm";
orderTableRuleConfiguration.setTableShardingStrategy(new StandardShardingStrategyConfiguration(fieldName, shardingAlgorithmName));
newTableRuleConfigList.add(orderTableRuleConfiguration);
newActualDataNodes
就是配置的数据节点,可以通过在数据库中查询现有维护的真实表,返回节点信息。通过Java代码配置,非常的灵活。
private static String buildActualDataNodes(String dataSourceName, String logicTableName) {
List<CustomShardingActualTable> actualTableByTableName = shardingActualTableMapper.getActualTableByTableName(logicTableName);
List<String> collect = actualTableByTableName.stream().map(CustomShardingActualTable::getActualTable).collect(Collectors.toList());
StringBuilder sb = new StringBuilder();
for (int i = 0; i < collect.size(); i++) {
sb.append(dataSourceName).append(".").append(collect.get(i));
if (i != collect.size() - 1) {
sb.append(",");
}
}
return sb.toString();
}
刷新节点
配置好的节点,如果有更新,则必须要重启服务,如果想不重启服务应该怎么做呢?
官方GitHub上有人给出了解答:
https://github.com/apache/shardingsphere/issues/16725
主要就是获取contextManager
,并调用其 alterRuleConfiguration
方法,把新创建分片规则更新一下
@Component
public class InitActualDataNodesAO {
@Resource
private ShardingSphereDataSource shardingSphereDataSource;
// ShardingSphere中默认值是logic_db,可以通过shardingsphere.schema.name在spring的配置文件中修改
private final String schemaName = "logic_db";
public void testSharding() {
this.reloadShardRuleActualDataNodes(shardingSphereDataSource, schemaNameForTOrder);
}
private void reloadShardRuleActualDataNodes(ShardingSphereDataSource dataSource, String schemaName) {
// Context manager.
org.apache.shardingsphere.mode.manager.ContextManager contextManager = dataSource.getContextManager();
// Rule configuration.
Collection<RuleConfiguration> oldRuleConfigList = dataSource.getContextManager()
.getMetaDataContexts()
.getMetaData(schemaName)
.getRuleMetaData()
.getConfigurations();
// update context
Collection<RuleConfiguration> newRuleConfigList = new ArrayList<>();
//newRuleConfigList ..... 这里实现新的 分片规则
contextManager.alterRuleConfiguration(schemaName, newRuleConfigList);
}
}
但是以上是 shardingsphere5.1.1
版本以下的实现方案,在之后的版本中,dataSource.getContextManager()
这个方法就不存在了,所以需要反射来获取ContextManager
@SneakyThrows(ReflectiveOperationException.class)
public static ContextManager getContextManager(final ShardingSphereDataSource dataSource) {
Field field = ShardingSphereDataSource.class.getDeclaredField("contextManager");
field.setAccessible(true);
return (ContextManager) field.get(dataSource);
}
最后整合 3.方法调用
的配置方式,给出完整的配置代码:
public static void refresh(DataSource dataSource, String dataSourceName, List<TableConfigVO> tableConfigVOS) {
// Context manager.
ContextManager contextManager = getContextManager((ShardingSphereDataSource) dataSource);
// Rule configuration.
ShardingRuleConfiguration newShardingRuleConfig = new ShardingRuleConfiguration();
//新的分片表配置
Collection<ShardingTableRuleConfiguration> newTableRuleConfigList = new LinkedList<>();
//新分片算法配置
Map<String, AlgorithmConfiguration> shardingAlgorithms = new HashMap<>(4);
for (TableConfigVO tableConfigVO : tableConfigVOS) {
String logicTableName = tableConfigVO.getTableName();
String fieldName = tableConfigVO.getFieldName();
//结束时间
String dateTimeUpper = tableConfigVO.getDateTimeUpper();
//开始时间
String dateTimeLower = tableConfigVO.getDateTimeLower();
String suffixPattern = tableConfigVO.getSuffixPattern();
String intervalUnit = tableConfigVO.getIntervalUnit();
String dateTimePattern = tableConfigVO.getDatetimePattern();
// 2、设置新的数据节点
String newActualDataNodes = buildActualDataNodes(dataSourceName, logicTableName);
// 3、设置置分库分表策略
// (1)设置分片规则,logicTable:逻辑表、actualDataNodes:真实的数据节点
ShardingTableRuleConfiguration orderTableRuleConfiguration = new ShardingTableRuleConfiguration(logicTableName, newActualDataNodes);
//设置新表使用的算法 以及 分片策略
String shardingAlgorithmName = logicTableName + "_algorithm";
orderTableRuleConfiguration.setTableShardingStrategy(new StandardShardingStrategyConfiguration(fieldName, shardingAlgorithmName));
newTableRuleConfigList.add(orderTableRuleConfiguration);
// 自定义算法 使用SPI
Properties properties = new Properties();
properties.setProperty("dateTimeLower", dateTimeLower);
properties.setProperty("intervalUnit", intervalUnit);
properties.setProperty("suffixPattern", suffixPattern);
shardingAlgorithms.put(shardingAlgorithmName, new AlgorithmConfiguration("HIS_DATA_SPI_BASED", properties));
}
newShardingRuleConfig.setShardingAlgorithms(shardingAlgorithms);
newShardingRuleConfig.setTables(newTableRuleConfigList);
long startTime = System.currentTimeMillis();
log.info("refresh start .....");
//刷新节点
contextManager.alterRuleConfiguration(SCHEMA_NAME, CollectionUtil.newArrayList(newShardingRuleConfig));
long endTime = System.currentTimeMillis();
log.info("refresh end time:" + (endTime - startTime) + "ms");
}