SpringBoot+Sharding-jdbc+Maven分库分表

最新推荐文章于 2023-01-27 02:24:17 发布

Joe192

最新推荐文章于 2023-01-27 02:24:17 发布

阅读量2.9k

点赞数 1

文章标签：分库，分表

joe192168

本文链接：https://blog.csdn.net/Joe192/article/details/117824696

版权

准备

一、了解sharding-jdbc概念

官方文档：
sharding-jdbc

ShardingSphere是一套开源的分布式数据库中间件解决方案组成的生态圈，它由Sharding-JDBC、Sharding-Proxy和Sharding-Sidecar（计划中）这3款相互独立的产品组成。他们均提供标准化的数据分片、分布式事务和数据库治理功能，可适用于如Java同构、异构语言、云原生等各种多样化的应用场景。

我这边就不过多介绍了，咱们重点的是sharding-jdbc，是期中的一个插件。其实核心思绪还是分库，分表。这中间在细分，分库：（水平分库，垂直分库），分表：（水平分表，垂直分表）

垂直划分：
在这里插入图片描述
水平划分：

这块分库，分表我上篇mycat有详细介绍，想了解的可以去看看。

二、环境要求

1、springboot：2.3.1.RELEASE
2、maven：3.6.2
3、sharding-jdbc：4.1.0
4、jdk：1.8
5、mysql：8.0

三、数据库结构

在这里插入图片描述

四、项目结构

在这里插入图片描述

pom文件内容：

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>
    <parent>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-parent</artifactId>
        <version>2.3.1.RELEASE</version>
        <relativePath/> <!-- lookup parent from repository -->
    </parent>
    <groupId>org.example</groupId>
    <artifactId>springboot-sparding-jdbc</artifactId>
    <version>1.0-SNAPSHOT</version>
    <name>springboot-sparding-jdbc</name>
    <description>Demo project for Spring Boot</description>

    <properties>
        <java.version>1.8</java.version>
        <sharding-sphere.version>4.1.0</sharding-sphere.version>
    </properties>

    <dependencies>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-test</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-web</artifactId>
        </dependency>
        <!-- mysql -->
        <dependency>
            <groupId>mysql</groupId>
            <artifactId>mysql-connector-java</artifactId>
            <scope>runtime</scope>
        </dependency>
        <!-- https://mvnrepository.com/artifact/com.alibaba/druid-spring-boot-starter -->
        <dependency>
            <groupId>com.alibaba</groupId>
            <artifactId>druid-spring-boot-starter</artifactId>
            <version>1.1.22</version>
        </dependency>
        <!-- lombok -->
        <dependency>
            <groupId>org.projectlombok</groupId>
            <artifactId>lombok</artifactId>
            <optional>true</optional>
        </dependency>
        <!-- mybatis plus 代码生成器 -->
        <dependency>
            <groupId>org.mybatis.spring.boot</groupId>
            <artifactId>mybatis-spring-boot-starter</artifactId>
            <version>2.1.3</version>
        </dependency>
        <dependency>
            <groupId>com.baomidou</groupId>
            <artifactId>mybatis-plus-boot-starter</artifactId>
            <version>3.4.1</version>
        </dependency>
        <dependency>
            <groupId>com.baomidou</groupId>
            <artifactId>mybatis-plus-generator</artifactId>
            <version>3.4.1</version>
        </dependency>
        <dependency>
            <groupId>org.freemarker</groupId>
            <artifactId>freemarker</artifactId>
            <version>2.3.28</version>
        </dependency>
        <!-- fastjson -->
        <dependency>
            <groupId>com.alibaba</groupId>
            <artifactId>fastjson</artifactId>
            <version>1.2.47</version>
        </dependency>
        <!-- ShardingSphere -->
        <dependency>
            <groupId>org.apache.shardingsphere</groupId>
            <artifactId>sharding-jdbc-spring-boot-starter</artifactId>
            <version>4.1.0</version>
        </dependency>
        <dependency>
            <groupId>org.apache.shardingsphere</groupId>
            <artifactId>sharding-core-common</artifactId>
            <version>4.1.0</version>
        </dependency>
        <!-- commons-lang3 -->
        <dependency>
            <groupId>org.apache.commons</groupId>
            <artifactId>commons-lang3</artifactId>
            <version>3.10</version>
        </dependency>
        <!-- https://mvnrepository.com/artifact/cn.hutool/hutool-all -->
        <dependency>
            <groupId>cn.hutool</groupId>
            <artifactId>hutool-all</artifactId>
            <version>5.5.8</version>
        </dependency>
    </dependencies>

    <build>
        <plugins>
            <plugin>
                <groupId>org.springframework.boot</groupId>
                <artifactId>spring-boot-maven-plugin</artifactId>
            </plugin>
        </plugins>
    </build>

</project>

五、利用构造器，生成模块

通过代码自动生成controller，service，repository代码

package com.joe.config;

import com.baomidou.mybatisplus.core.exceptions.MybatisPlusException;
import com.baomidou.mybatisplus.core.toolkit.StringUtils;
import com.baomidou.mybatisplus.generator.AutoGenerator;
import com.baomidou.mybatisplus.generator.config.*;
import com.baomidou.mybatisplus.generator.config.rules.NamingStrategy;
import com.baomidou.mybatisplus.generator.engine.FreemarkerTemplateEngine;

import java.util.Scanner;

public class GeneratorCodeConfig {

    public static String scanner(String tip) {
        Scanner scanner = new Scanner(System.in);
        StringBuilder help = new StringBuilder();
        help.append("请输入" + tip + "：");
        System.out.println(help.toString());
        if (scanner.hasNext()) {
            String ipt = scanner.next();
            if (!StringUtils.isBlank(ipt)) {
                return ipt;
            }
        }
        throw new MybatisPlusException("请输入正确的" + tip + "！");
    }

    public static void main(String[] args) {
        // 代码生成器
        AutoGenerator mpg = new AutoGenerator();

        // 全局配置
        GlobalConfig gc = new GlobalConfig();
        String projectPath = System.getProperty("user.dir");
        gc.setOutputDir(projectPath + "/src/main/java");
        gc.setAuthor("joe");
        gc.setOpen(false);
        //实体属性 Swagger2 注解
        gc.setSwagger2(false);
        mpg.setGlobalConfig(gc);

        // 数据源配置
        DataSourceConfig dsc = new DataSourceConfig();
        dsc.setUrl("jdbc:mysql://192.168.0.233:3306/test?serverTimezone=UTC&useUnicode=true&characterEncoding=utf-8&zeroDateTimeBehavior=convertToNull&useSSL=false&allowPublicKeyRetrieval=true");
        dsc.setDriverName("com.mysql.jdbc.Driver");
        dsc.setUsername("root");
        dsc.setPassword("root");
        mpg.setDataSource(dsc);

        // 包配置
        PackageConfig pc = new PackageConfig();
//        pc.setModuleName(scanner("模块名"));
        pc.setParent("com.joe");
        pc.setEntity("entity");
        pc.setMapper("mapper");
        pc.setService("service");
        pc.setServiceImpl("service.impl");
        mpg.setPackageInfo(pc);

        // 自定义配置
//        InjectionConfig cfg = new InjectionConfig() {
//            @Override
//            public void initMap() {
//                // to do nothing
//            }
//        };

        // 如果模板引擎是 freemarker
//        String templatePath = "/templates/mapper.xml.ftl";
        // 如果模板引擎是 velocity
        // String templatePath = "/templates/mapper.xml.vm";

        // 自定义输出配置
//        List<FileOutConfig> focList = new ArrayList<>();
        // 自定义配置会被优先输出
//        focList.add(new FileOutConfig(templatePath) {
//            @Override
//            public String outputFile(TableInfo tableInfo) {
//                // 自定义输出文件名 ， 如果你 Entity 设置了前后缀、此处注意 xml 的名称会跟着发生变化！！
//                return projectPath + "/src/main/resources/mapper/" + pc.getModuleName()
//                        + "/" + tableInfo.getEntityName() + "Mapper" + StringPool.DOT_XML;
//            }
//        });
        /*
        cfg.setFileCreate(new IFileCreate() {
            @Override
            public boolean isCreate(ConfigBuilder configBuilder, FileType fileType, String filePath) {
                // 判断自定义文件夹是否需要创建
                checkDir("调用默认方法创建的目录");
                return false;
            }
        });
        */
//        cfg.setFileOutConfigList(focList);
//        mpg.setCfg(cfg);

        // 配置模板
        TemplateConfig templateConfig = new TemplateConfig();

        // 配置自定义输出模板
        //指定自定义模板路径，注意不要带上.ftl/.vm, 会根据使用的模板引擎自动识别
        // templateConfig.setEntity("templates/entity2.java");
        // templateConfig.setService();
        // templateConfig.setController();

        templateConfig.setXml(null);
        mpg.setTemplate(templateConfig);

        // 策略配置
        StrategyConfig strategy = new StrategyConfig();
        strategy.setNaming(NamingStrategy.underline_to_camel);
        strategy.setColumnNaming(NamingStrategy.underline_to_camel);
        strategy.setSuperEntityClass("com.baomidou.mybatisplus.extension.activerecord.Model");
        strategy.setEntityLombokModel(true);
        strategy.setRestControllerStyle(true);

        strategy.setEntityLombokModel(true);
        // 公共父类
//        strategy.setSuperControllerClass("com.baomidou.ant.common.BaseController");
        // 写于父类中的公共字段
//        strategy.setSuperEntityColumns("id");
        strategy.setInclude(scanner("表名，多个英文逗号分割").split(","));
        strategy.setControllerMappingHyphenStyle(true);
        strategy.setTablePrefix(pc.getModuleName() + "_");
        mpg.setStrategy(strategy);
        mpg.setTemplateEngine(new FreemarkerTemplateEngine());
        mpg.execute();
    }
}

六、application.properties配置文件

server.port=8081
server.servlet.context-path=/

spring.jackson.date-format=yyyy-MM-dd HH:mm:ss
spring.jackson.time-zone=GMT+8
spring.jackson.serialization.write-dates-as-timestamps=false

spring.main.allow-bean-definition-overriding=true
# 命名数据源  这个是自定义的
spring.shardingsphere.datasource.names=ds-0,ds-1,ds-2
# 配置数据源ds-0
spring.shardingsphere.datasource.ds-0.type=com.alibaba.druid.pool.DruidDataSource
spring.shardingsphere.datasource.ds-0.driverClassName=com.mysql.jdbc.Driver
spring.shardingsphere.datasource.ds-0.url=jdbc:mysql://192.168.0.233:3306/ds-0?serverTimezone=Asia/Shanghai&useUnicode=true&characterEncoding=utf-8&zeroDateTimeBehavior=convertToNull&useSSL=false&allowPublicKeyRetrieval=true
spring.shardingsphere.datasource.ds-0.username=root
spring.shardingsphere.datasource.ds-0.password=root
# 配置数据源ds-1
spring.shardingsphere.datasource.ds-1.type=com.alibaba.druid.pool.DruidDataSource
spring.shardingsphere.datasource.ds-1.driverClassName=com.mysql.jdbc.Driver
spring.shardingsphere.datasource.ds-1.url=jdbc:mysql://192.168.0.233:3306/ds-1?serverTimezone=Asia/Shanghai&useUnicode=true&characterEncoding=utf-8&zeroDateTimeBehavior=convertToNull&useSSL=false&allowPublicKeyRetrieval=true
spring.shardingsphere.datasource.ds-1.username=root
spring.shardingsphere.datasource.ds-1.password=root
# 配置数据源ds-2
spring.shardingsphere.datasource.ds-2.type=com.alibaba.druid.pool.DruidDataSource
spring.shardingsphere.datasource.ds-2.driverClassName=com.mysql.jdbc.Driver
spring.shardingsphere.datasource.ds-2.url=jdbc:mysql://192.168.0.233:3306/ds-user?serverTimezone=Asia/Shanghai&useUnicode=true&characterEncoding=utf-8&zeroDateTimeBehavior=convertToNull&useSSL=false&allowPublicKeyRetrieval=true
spring.shardingsphere.datasource.ds-2.username=root
spring.shardingsphere.datasource.ds-2.password=root
# 配置默认数据源ds-0
spring.shardingsphere.sharding.default-data-source-name=ds-0

# 配置分片表 t_order
# 配置真实数据节点  库：ds-0,ds-1；表：t_order_0,t_order_1,t_order_2
spring.shardingsphere.sharding.tables.t_order.actual-data-nodes=ds-$->{0..1}.t_order_$->{0..2}
# 分库分片建
spring.shardingsphere.sharding.tables.t_order.database-strategy.inline.sharding-column=order_id
# 分库分片算法： 对order_id字段进行取模分库，2代表分片库的个数
spring.shardingsphere.sharding.tables.t_order.database-strategy.inline.algorithm-expression=ds-$->{order_id % 2}
# 分表分片健
spring.shardingsphere.sharding.tables.t_order.table-strategy.inline.sharding-column=order_id
# 分表算法：对order_id字段进行取模分库，3代表分片表的个数
spring.shardingsphere.sharding.tables.t_order.table-strategy.inline.algorithm-expression=t_order_$->{order_id % 3}
# 配置自增主键字段
spring.shardingsphere.sharding.tables.t_order.key-generator.column=order_id
# 自增主键ID 生成方案：雪花算法
spring.shardingsphere.sharding.tables.t_order.key-generator.type=SNOWFLAKE

# 配置分片表 t_order_item
spring.shardingsphere.sharding.tables.t_order_item.actual-data-nodes=ds-$->{0..1}.t_order_item_$->{0..2}
spring.shardingsphere.sharding.tables.t_order_item.database-strategy.inline.sharding-column=order_id
spring.shardingsphere.sharding.tables.t_order_item.database-strategy.inline.algorithm-expression=ds-$->{order_id % 2}
spring.shardingsphere.sharding.tables.t_order_item.table-strategy.inline.sharding-column=order_id
spring.shardingsphere.sharding.tables.t_order_item.table-strategy.inline.algorithm-expression=t_order_item_$->{order_id % 3}


# 配置分片表 t_user_order
spring.shardingsphere.sharding.tables.t_user_order.actual-data-nodes=ds-$->{0..1}.t_user_order_$->{0..2}
spring.shardingsphere.sharding.tables.t_user_order.database-strategy.inline.sharding-column=user_order_id
spring.shardingsphere.sharding.tables.t_user_order.database-strategy.inline.algorithm-expression=ds-$->{user_order_id % 2}
spring.shardingsphere.sharding.tables.t_user_order.table-strategy.inline.sharding-column=user_order_id
spring.shardingsphere.sharding.tables.t_user_order.table-strategy.inline.algorithm-expression=t_user_order_$->{user_order_id % 3}
spring.shardingsphere.sharding.tables.t_user_order.key-generator.column=user_order_id
spring.shardingsphere.sharding.tables.t_user_order.key-generator.type=SNOWFLAKE

#配置专库专表
spring.shardingsphere.sharding.tables.t_user.actual-data-nodes=ds-2.t_user
spring.shardingsphere.sharding.tables.t_user.database-strategy.inline.sharding-column=id
spring.shardingsphere.sharding.tables.t_user.database-strategy.inline.algorithm-expression=ds-2
spring.shardingsphere.sharding.tables.t_user.table-strategy.inline.sharding-column=id
spring.shardingsphere.sharding.tables.t_user.table-strategy.inline.algorithm-expression=t_user
spring.shardingsphere.sharding.tables.t_user.key-generator.column=id
spring.shardingsphere.sharding.tables.t_user.key-generator.type=SNOWFLAKE

# 绑定表关系  不加[0]查询时会导致笛卡尔积
spring.shardingsphere.sharding.binding-tables[0]=t_order,t_order_item
# 配置广播表=公共表(PS:所有数据库中都必须有这张表)
spring.shardingsphere.sharding.broadcast-tables=t_config
# 开启SQL解析日志
spring.shardingsphere.props.sql.show=true

# 配置mybatis-plus
mybatis-plus.configuration.map-underscore-to-camel-case=true
mybatis-plus.configuration.auto-mapping-behavior=full
mybatis-plus.configuration.log-impl=org.apache.ibatis.logging.stdout.StdOutImpl
mybatis-plus.mapper-locations=classpath*:mapper/**/*Mapper.xml
mybatis-plus.global-config.db-config.logic-not-delete-value=0
mybatis-plus.global-config.db-config.logic-delete-value=1
mybatis-plus.global-config.db-config.logic-delete-field=deleted

配置文件里面有详细的注释说明，主要就是数据节点，分库策略和分表策略，以及主键规则，其实也不难

七、测试用例

package com.joe;

import com.alibaba.fastjson.JSON;
import com.joe.entity.*;
import com.joe.mapper.*;
import org.junit.Test;
import org.junit.runner.RunWith;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.test.context.SpringBootTest;
import org.springframework.test.context.junit4.SpringRunner;

import java.math.BigDecimal;
import java.util.List;

@RunWith(SpringRunner.class)
@SpringBootTest
public class SpringBootAppTest {

    @Autowired
    private TOrderMapper orderMapper;
    @Autowired
    private TOrderItemMapper orderItemMapper;
    @Autowired
    private TUserMapper userMapper;
    @Autowired
    private TConfigMapper configMapper;
    @Autowired
    private TUserOderMapper tUserOderMapper;

    private static final Integer num = 100;
    private static final String orderNoPre = "JD-";

    /**
     * 测试专库专表
     */
    @Test
    public void test() {
        for (int i = 1; i <= 3; i++) {
            TUser user = new TUser();
            user.setName("用户"+i);
            user.setAge(18);
            user.setAddress("地址");
            userMapper.insert(user);
        }
        /*TUser user1 = userMapper.selectById(1376363351721992194L);
        System.err.println(user1);*/
    }

    /**
     * 测试分库分表-插入
     */
    @Test
    public void testOrderSave() {
        for (int i = 1; i <= num; i++) {
            String orderNo = orderNoPre + String.format("%04d", i);
            //订单表
            TOrder order = new TOrder();
            order.setOrderNo(orderNo);
            order.setCreateName("订单名称" + i);
            order.setPrice(new BigDecimal("" + i));
            orderMapper.insert(order);

            //订单详细表
            TOrderItem orderItem = new TOrderItem();
            Long orderId = order.getOrderId();
            orderItem.setOrderId(orderId);
            orderItem.setOrderNo(orderNo);
            orderItem.setItemName("商品名称" + i);
            orderItem.setItemDesc("商品描述" + i);
            orderItemMapper.insert(orderItem);

            //用户-订单关系表
            TUserOrder tUserOrder = new TUserOrder();
            tUserOrder.setUserId(1403278739984207873L);
            tUserOrder.setOrderId(orderId);
            tUserOderMapper.insert(tUserOrder);

        }
    }

    /**
     * 测试分库分表-查询
     */
    @Test
    public void testOrderQuery() {
        List<TOrder> orderListByPage = orderMapper.findOrderListByPage();
        System.err.println(JSON.toJSONString(orderListByPage));
    }
    

    /**
     * 测试公共表
     */
    @Test
    public void testConfig() {
        TConfig config = new TConfig();
        config.setRemark("bbbb");
        configMapper.insert(config);
    }

}

相当是造一些数据，测试分库，分表是否起作用了

八、查询

根据用户id查询，用户-订单关联表信息
在这里插入图片描述
总结：利用sharding-jdbc进行垂直分库，垂直分表，只要按照分库，分表策略不是很难，难的地方就是表如何去设计。还有一点sharding-jdbc版本从3.x到4.x，以及最新的5.x差异还是很大的。

传送门：Git代码

Joe192

关注

1
点赞
踩
1

收藏

觉得还不错? 一键收藏
0
评论
SpringBoot+Sharding-jdbc+Maven分库分表

准备一、了解sharding-jdbc概念官方文档：sharding-jdbcShardingSphere是一套开源的分布式数据库中间件解决方案组成的生态圈，它由Sharding-JDBC、Sharding-Proxy和Sharding-Sidecar（计划中）这3款相互独立的产品组成。他们均提供标准化的数据分片、分布式事务和数据库治理功能，可适用于如Java同构、异构语言、云原生等各种多样化的应用场景。我这边就不过多介绍了，咱们重点的是sharding-jdbc，是期中的一个插件。其实核心思绪
复制链接

扫一扫