sharding-jdbc的作用就不必做过多的解释了,主要用途是分库分表;首先先了解下分库分表的策略吧。
1 分片键
分片键即在分库分表时根据表中的某一个或者多个字段为依据,以一定的逻辑进行分割,例如根据自增的主键求模的规则进行分片等。
2 分片算法
分片算法结合分片键对具体的数据进行分片;支持通过=、BETWEEN和IN分片。分片算法需要应用方开发者自行实现,可实现的灵活度非常高。分片算法有四种:
(1)精确分片算法
对应PreciseShardingAlgorithm,用于处理使用单一键作为分片键的=与IN进行分片的场景。需要配合standardShardingStrategy使用。
(2)范围分片算法
对应RangeShardingAlgorithm,用于处理使用单一键作为分片键的BETWEEN AND进行分片的场景。需要配合StandardShardingStrategy使用。
(3)复合分片算法
对应ComplexKeysShardingAlgorithm,用于处理使用多键作为分片键进行分片的场景,包含多个分片键的逻辑较复杂,需要应用开发者自行处理其中的复杂度。需要配合ComplexShardingStrategy使用。
(4) Hint分片算法
对应HintShardingAlgorithm,用于处理使用Hint行分片的场景。需要配合HintShardingStrategy使用。
3 分片策略
包含分片键和分片算法,由于分片算法的独立性,将其独立抽离。真正可用于分片操作的是分片键 + 分片算法,也就是分片策略。目前提供5种分片策略。
(1)标准分片策略
对应StandardShardingStrategy。提供对SQL语句中的=, IN和BETWEEN AND的分片操作支持。StandardShardingStrategy只支持单分片键,提供PreciseShardingAlgorithm和RangeShardingAlgorithm两个分片算法。PreciseShardingAlgorithm是必选的,用于处理=和IN的分片。RangeShardingAlgorithm是可选的,用于处理BETWEEN AND分片,如果不配置RangeShardingAlgorithm,SQL中的BETWEEN AND将按照全库路由处理。
public final class StandardShardingStrategy implements ShardingStrategy {
//分片键 支支持单分片键
private final String shardingColumn;
//精确分片算法
private final PreciseShardingAlgorithm preciseShardingAlgorithm;
//范围分片算法
private final RangeShardingAlgorithm rangeShardingAlgorithm;
//
public StandardShardingStrategy(final StandardShardingStrategyConfiguration standardShardingStrategyConfig) {
//校验分片键 不能为空
Preconditions.checkNotNull(standardShardingStrategyConfig.getShardingColumn(), "Sharding column cannot be null.");
//校验精确分片算法,不能为空
Preconditions.checkNotNull(standardShardingStrategyConfig.getPreciseShardingAlgorithm(), "precise sharding algorithm cannot be null.");
//初始化分片策略的属性
shardingColumn = standardShardingStrategyConfig.getShardingColumn();
preciseShardingAlgorithm = standardShardingStrategyConfig.getPreciseShardingAlgorithm();
//范围分片算法可以为空
rangeShardingAlgorithm = standardShardingStrategyConfig.getRangeShardingAlgorithm();
}
//执行分片
@Override
public Collection<String> doSharding(final Collection<String> availableTargetNames, final Collection<ShardingValue> shardingValues) {
//ShardingValue 提供获取逻辑表和分片键方法
ShardingValue shardingValue = shardingValues.iterator().next();
//判断ShardingValue 分片操作是精确分片还是范围分片,从而执行相应的算法 ;ListShardingValue包含逻辑表、分片键、精确匹配值集合
Collection<String> shardingResult = shardingValue instanceof ListShardingValue
? doSharding(availableTargetNames, (ListShardingValue) shardingValue) : doSharding(availableTargetNames, (RangeShardingValue) shardingValue);
Collection<String> result = new TreeSet<>(String.CASE_INSENSITIVE_ORDER);
result.addAll(shardingResult);
return result;
}
//范围分片算法
@SuppressWarnings("unchecked")
private Collection<String> doSharding(final Collection<String> availableTargetNames, final RangeShardingValue<?> shardingValue) {
//如果范围分片算法为空
if (null == rangeShardingAlgorithm) {
throw new UnsupportedOperationException("Cannot find range sharding strategy in sharding rule.");
}
return rangeShardingAlgorithm.doSharding(availableTargetNames, shardingValue);
}
//精确分片算法
@SuppressWarnings("unchecked")
private Collection<String> doSharding(final Collection<String> availableTargetNames, final ListShardingValue<?> shardingValue) {
Collection<String> result = new LinkedList<>();
for (PreciseShardingValue<?> each : transferToPreciseShardingValues(shardingValue)) {
String target = preciseShardingAlgorithm.doSharding(availableTargetNames, each);
if (null != target) {
result.add(target);
}
}
return result;
}
//将精确匹配值集合进行转换为精确匹配值
@SuppressWarnings("unchecked")
private List<PreciseShardingValue> transferToPreciseShardingValues(final ListShardingValue<?> shardingValue) {
List<PreciseShardingValue> result = new ArrayList<>(shardingValue.getValues().size());
for (Comparable<?> each : shardingValue.getValues()) {
result.add(new PreciseShardingValue(shardingValue.getLogicTableName(), shardingValue.getColumnName(), each));
}
return result;
}
//获取分片键
@Override
public Collection<String> getShardingColumns() {
//String.CASE_INSENSITIVE_ORDER String排序与字母大小写无关
Collection<String> result = new TreeSet<>(String.CASE_INSENSITIVE_ORDER);
result.add(shardingColumn);
return result;
}
}
(2)复合分片策略
对应ComplexShardingStrategy。复合分片策略。提供对SQL语句中的=, IN和BETWEEN AND的分片操作支持。ComplexShardingStrategy支持多分片键,由于多分片键之间的关系复杂,因此并未进行过多的封装,而是直接将分片键值组合以及分片操作符透传至分片算法,完全由应用开发者实现,提供最大的灵活度。
public final class ComplexShardingStrategy implements ShardingStrategy {
//支持多分片键 集合存储分片键
@Getter
private final Collection<String> shardingColumns;
//复合分片算法
private final ComplexKeysShardingAlgorithm shardingAlgorithm;
public ComplexShardingStrategy(final ComplexShardingStrategyConfiguration complexShardingStrategyConfig) {
Preconditions.checkNotNull(complexShardingStrategyConfig.getShardingColumns(), "Sharding columns cannot be null.");
Preconditions.checkNotNull(complexShardingStrategyConfig.getShardingAlgorithm(), "Sharding algorithm cannot be null.");
shardingColumns = new TreeSet<>(String.CASE_INSENSITIVE_ORDER);
shardingColumns.addAll(StringUtil.splitWithComma(complexShardingStrategyConfig.getShardingColumns()));
shardingAlgorithm = complexShardingStrategyConfig.getShardingAlgorithm();
}
@Override
public Collection<String> doSharding(final Collection<String> availableTargetNames, final Collection<ShardingValue> shardingValues) {
Collection<String> shardingResult = shardingAlgorithm.doSharding(availableTargetNames, shardingValues);
Collection<String> result = new TreeSet<>(String.CASE_INSENSITIVE_ORDER);
result.addAll(shardingResult);
return result;
}
}
(3)行表达式分片策略
对应InlineShardingStrategy。使用Groovy的表达式,提供对SQL语句中的=和IN的分片操作支持,只支持单分片键。对于简单的分片算法,可以通过简单的配置使用,从而避免繁琐的Java代码开发,如: t_user_$->{u_id % 8} 表示t_user表根据u_id模8,而分成8张表,表名称为t_user_0到t_user_7。
public final class InlineShardingStrategy implements ShardingStrategy {
private final String shardingColumn;
//表示Groovy中的任何闭包对象
private final Closure<?> closure;
public InlineShardingStrategy(final InlineShardingStrategyConfiguration inlineShardingStrategyConfig) {
Preconditions.checkNotNull(inlineShardingStrategyConfig.getShardingColumn(), "Sharding column cannot be null.");
Preconditions.checkNotNull(inlineShardingStrategyConfig.getAlgorithmExpression(), "Sharding algorithm expression cannot be null.");
shardingColumn = inlineShardingStrategyConfig.getShardingColumn();
//分片Groovy表达式
String algorithmExpression = InlineExpressionParser.handlePlaceHolder(inlineShardingStrategyConfig.getAlgorithmExpression().trim());
closure = new InlineExpressionParser(algorithmExpression).evaluateClosure();
}
@Override
public Collection<String> doSharding(final Collection<String> availableTargetNames, final Collection<ShardingValue> shardingValues) {
ShardingValue shardingValue = shardingValues.iterator().next();
//判断是否是范围分片
Preconditions.checkState(shardingValue instanceof ListShardingValue, "Inline strategy cannot support range sharding.");
Collection<String> shardingResult = doSharding((ListShardingValue) shardingValue);
Collection<String> result = new TreeSet<>(String.CASE_INSENSITIVE_ORDER);
result.addAll(shardingResult);
return result;
}
//与标准分片策略中的实现相同,不同的是需要通过Groovy进行解析出最终的结果
private Collection<String> doSharding(final ListShardingValue shardingValue) {
Collection<String> result = new LinkedList<>();
for (PreciseShardingValue<?> each : transferToPreciseShardingValues(shardingValue)) {
result.add(execute(each));
}
return result;
}
@SuppressWarnings("unchecked")
private List<PreciseShardingValue> transferToPreciseShardingValues(final ListShardingValue<?> shardingValue) {
List<PreciseShardingValue> result = new ArrayList<>(shardingValue.getValues().size());
for (Comparable<?> each : shardingValue.getValues()) {
result.add(new PreciseShardingValue(shardingValue.getLogicTableName(), shardingValue.getColumnName(), each));
}
return result;
}
//通过Closure解析
private String execute(final PreciseShardingValue shardingValue) {
Closure<?> result = closure.rehydrate(new Expando(), null, null);
result.setResolveStrategy(Closure.DELEGATE_ONLY);
result.setProperty(shardingValue.getColumnName(), shardingValue.getValue());
return result.call().toString();
}
@Override
public Collection<String> getShardingColumns() {
Collection<String> result = new TreeSet<>(String.CASE_INSENSITIVE_ORDER);
result.add(shardingColumn);
return result;
}
}
(4)Hint分片策略
对应HintShardingStrategy。通过Hint而非SQL解析的方式分片的策略。
public final class HintShardingStrategy implements ShardingStrategy {
//支持多分片键
@Getter
private final Collection<String> shardingColumns;
private final HintShardingAlgorithm shardingAlgorithm;
public HintShardingStrategy(final HintShardingStrategyConfiguration hintShardingStrategyConfig) {
Preconditions.checkNotNull(hintShardingStrategyConfig.getShardingAlgorithm(), "Sharding algorithm cannot be null.");
shardingColumns = new TreeSet<>(String.CASE_INSENSITIVE_ORDER);
shardingAlgorithm = hintShardingStrategyConfig.getShardingAlgorithm();
}
@Override
public Collection<String> doSharding(final Collection<String> availableTargetNames, final Collection<ShardingValue> shardingValues) {
Collection<String> shardingResult = shardingAlgorithm.doSharding(availableTargetNames, shardingValues.iterator().next());
Collection<String> result = new TreeSet<>(String.CASE_INSENSITIVE_ORDER);
result.addAll(shardingResult);
return result;
}
}
(5)不分片策略
对应NoneShardingStrategy。不分片的策略。
4 SQL Hint
对于分片字段非SQL决定,而由其他外置条件决定的场景,可使用SQL Hint灵活的注入分片字段;