基于拦截器的水平分表实现

水平分表的场景

当单表数据激增,比如mysql单表数据量超过千万级别,查询速度会变慢。当数据继续增长,就需要使用到分库分表。
根据业务量增加表的数量,来提升查询效率。这里提到是水平分表,只考虑数据量增长,当字段较长,可以考虑垂直分表,按照字段将一张表拆成多张表进行存储。

水平分表实现原理

在这里插入图片描述
如图,分表的是根据sql请求中参数,或参数组合进行数据标识,我见过流水业务频繁的按照日期分表,他就根据创建时间来分表,分片策略算法,一般按需实现 。在sharding-jdbc中实现分片接口,doSharding()的方法。就通过上个节点中得到的标识进行运算,获取表名。

基于拦截器的分表实现

基于上述流程,通过mybatis的拦截器,便可以实现。在执行sql前,获取到数据标识。在执行sql前进行拦截,
将sql拦截后,替换当中的表。然后再进行重新绑定sql,推进调用链。

定义拦截器

/**
 * 分表拦截器
 *
 * @author machenike
 */
@Intercepts(@Signature(type = StatementHandler.class, method = "prepare", args = {Connection.class, Integer.class}))
public class ShardingInterceptorPlugin implements Interceptor {

    private static final Logger logger = LoggerFactory.getLogger(ShardingInterceptorPlugin.class);

    private static final String META_OBJECT_KEY_BOUND_SQL = "delegate.boundSql.sql";

    private static ThreadLocal<Object> SHARDINGG_SIGN = new ThreadLocal<Object>();

    private Set<String> shardingTables = new HashSet<String>();

    private ShardingStrategy shardingStrategy;

    @Override
    public Object intercept(Invocation invocation) throws Throwable {
        //获取命令handler取得
        StatementHandler statementHandler = (StatementHandler) invocation.getTarget();
        //原来应该执行的sql吧
        String sql = statementHandler.getBoundSql().getSql();


        // 获取statementHandler包装类
        MetaObject MetaObjectHandler = SystemMetaObject.forObject(statementHandler);

        // 分离代理对象链
        while (MetaObjectHandler.hasGetter("h")) {
            Object obj = MetaObjectHandler.getValue("h");
            MetaObjectHandler = SystemMetaObject.forObject(obj);
        }
        while (MetaObjectHandler.hasGetter("target")) {
            Object obj = MetaObjectHandler.getValue("target");
            MetaObjectHandler = SystemMetaObject.forObject(obj);
        }

        //sql不满足分片请求时
        if (!isMatch(sql) || shardingTables.isEmpty()) {
            return invocation.proceed();
        }
        logger.debug("tableName match,start intercept");
        logger.debug(">> originSql:{}", sql);

        String shardingSql = buildSql(sql, SHARDINGG_SIGN.get());
        logger.debug(">> shardingSql:{}", shardingSql);
        //重新分表sql
        MetaObjectHandler.setValue(META_OBJECT_KEY_BOUND_SQL, shardingSql);
        //推进拦截器调用链
        return invocation.proceed();
    }

    private String buildSql(String sql, Object sigin) {
        Set<String> tableSet = getSqlTables(sql);
        for (String table : tableSet) {
            sql = sql.replaceAll(table, shardingStrategy.doSharding(table, SHARDINGG_SIGN.get()));
        }
        return sql;
    }

    private boolean isMatch(String sql) {
        for (String tableName : shardingTables) {
            if (sql.contains(tableName)) {
                return true;
            }
        }
        return false;
    }


    private Set<String> getSqlTables(String sql) {
        Set<String> set = new HashSet<String>();
        for (String table : shardingTables) {
            if (sql.contains(table)) {
                set.add(table);
            }
        }
        return set;
    }

    public static void setCurrentSign(Object sigin) {
        SHARDINGG_SIGN.set(sigin);
    }

    public ShardingInterceptorPlugin() {
    }

    public ShardingInterceptorPlugin(Set<String> shardingTables, ShardingStrategy shardingStrategy) {
        this.shardingTables = shardingTables;
        this.shardingStrategy = shardingStrategy;
    }

    public Set<String> getShardingTables() {
        return shardingTables;
    }

    public void setShardingTables(Set<String> shardingTables) {
        this.shardingTables = shardingTables;
    }

    public ShardingStrategy getShardingStrategy() {
        return shardingStrategy;
    }

    public void setShardingStrategy(ShardingStrategy shardingStrategy) {
        this.shardingStrategy = shardingStrategy;
    }
}

分片策略接口

public interface ShardingStrategy<T> {

    String doSharding(String table,T sign);

}

帮助类(借鉴pageHelper)

public class ShardingHelper {

    public static void setSign(Object o){
        ShardingInterceptorPlugin.setCurrentSign(o);
    }

}

配置类

@Configuration
public class ShardingConfig {
    
    @Bean
    public Interceptor initShardingInterceptor() {
        Set<String> tables = new HashSet<>();
        tables.add("test");
        Interceptor shardingInterceptor = new ShardingInterceptorPlugin(tables, new ShardingStrategy() {
            @Override
            public String doSharding(String o, Object s) {
                int code = s.hashCode();
                code = code > 0 ? code : -code;
                return o + "_" + (code % 3);
            }
        });
        return shardingInterceptor;
    }
}

这里的ShardingStrategy接口可以按需实现
tables 中装的是需要分表的原始表名 建议将配置抽出
此处的3 是表数量

使用方式

    @Autowired
    private TestDao testDao;

    @Test
    public void test(){
        TestEntity testEntity = new TestEntity();
        testEntity.setText("dadadad");
        //设定数据标识
        ShardingHelper.setSign(testEntity.getText());
        testDao.queryByText(testEntity);
    }

源码地址

评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值