目录
一、简介
1.1、垂直分表
垂直分片又称为纵向拆分。最简单的就是单库把一个表拆成多个关联的表,通常是一个表中存储的信息类型比较多,比如一二十个字段,但是经常用到的可能又比较少,频繁操作就影响性能,所以就把大表拆分成多个小表,比如 tb_course 拆分成 tb_course 和 tb_course_detail 。这个主要是数据库设计的相关知识,也和我们这里说的 Sharding-JDBC 关系不是那么大,我就不多进行讲解,所以后面也不会讲垂直分表这种了,主要是我们要讲的水平分表。
1.2、水平分表
水平分片又称为横向拆分。 最简单的就是单库水平分表,也就是一个库中有两个一模一样的表,只是名称不一样比如: tb_order_1 和 tb_order_2 ,如下图所示。
二、maven依赖
pom.xml
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<parent>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-parent</artifactId>
<version>2.6.0</version>
<relativePath/> <!-- lookup parent from repository -->
</parent>
<groupId>com.alian</groupId>
<artifactId>sharding-jdbc</artifactId>
<version>0.0.1-SNAPSHOT</version>
<name>sharding-jdbc</name>
<description>sharding-jdbc</description>
<properties>
<java.version>1.8</java.version>
</properties>
<dependencies>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-data-jpa</artifactId>
</dependency>
<dependency>
<groupId>org.apache.shardingsphere</groupId>
<artifactId>sharding-jdbc-spring-boot-starter</artifactId>
<version>4.1.1</version>
</dependency>
<dependency>
<groupId>com.alibaba</groupId>
<artifactId>druid</artifactId>
<version>1.2.15</version>
</dependency>
<dependency>
<groupId>mysql</groupId>
<artifactId>mysql-connector-java</artifactId>
<version>8.0.26</version>
<scope>runtime</scope>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-test</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.projectlombok</groupId>
<artifactId>lombok</artifactId>
<version>1.18.20</version>
</dependency>
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>4.12</version>
<scope>test</scope>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-maven-plugin</artifactId>
</plugin>
</plugins>
</build>
</project>
有些小伙伴的 druid 可能用的是 druid-spring-boot-starter
<dependency>
<groupId>com.alibaba</groupId>
<artifactId>druid-spring-boot-starter</artifactId>
<version>1.2.6</version>
</dependency>
然后出现可能使用不了的各种问题,这个时候你只需要在主类上添加 @SpringBootApplication(exclude = {DruidDataSourceAutoConfigure.class}) 即可
package com.alian.shardingjdbc;
import com.alibaba.druid.spring.boot.autoconfigure.DruidDataSourceAutoConfigure;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
@SpringBootApplication(exclude = {DruidDataSourceAutoConfigure.class})
@SpringBootApplication
public class ShardingJdbcApplication {
public static void main(String[] args) {
SpringApplication.run(ShardingJdbcApplication.class, args);
}
}
三、数据库
3.1、创建数据库
sharding_0
CREATE DATABASE `sharding_0` DEFAULT CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci;
3.2、创建表
tb_order_1
CREATE TABLE `tb_order_1` (
`order_id` bigint(20) NOT NULL COMMENT '主键',
`user_id` int unsigned NOT NULL DEFAULT '0' COMMENT '用户id',
`price` int unsigned NOT NULL DEFAULT '0' COMMENT '价格(单位:分)',
`order_status` tinyint unsigned NOT NULL DEFAULT '1' COMMENT '订单状态(1:待付款,2:已付款,3:已取消)',
`order_time` datetime NOT NULL DEFAULT CURRENT_TIMESTAMP COMMENT '创建时间',
`title` varchar(100) NOT NULL DEFAULT '' COMMENT '订单标题',
PRIMARY KEY (`order_id`),
KEY `idx_user_id` (`user_id`),
KEY `idx_order_time` (`order_time`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COMMENT='订单表';
tb_order_2
CREATE TABLE `tb_order_2` (
`order_id` bigint(20) NOT NULL COMMENT '主键',
`user_id` int unsigned NOT NULL DEFAULT '0' COMMENT '用户id',
`price` int unsigned NOT NULL DEFAULT '0' COMMENT '价格(单位:分)',
`order_status` tinyint unsigned NOT NULL DEFAULT '1' COMMENT '订单状态(1:待付款,2:已付款,3:已取消)',
`order_time` datetime NOT NULL DEFAULT CURRENT_TIMESTAMP COMMENT '创建时间',
`title` varchar(100) NOT NULL DEFAULT '' COMMENT '订单标题',
PRIMARY KEY (`order_id`),
KEY `idx_user_id` (`user_id`),
KEY `idx_order_time` (`order_time`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COMMENT='订单表';
四、配置(二选一)
4.1、properties配置
application.properties
server.port=8899
server.servlet.context-path=/sharding-jdbc
# 允许一个实体映射多个表
#spring.main.allow-bean-definition-overriding=true
# 数据源名称,多数据源以逗号分隔
spring.shardingsphere.datasource.names=ds1
# 数据库连接池类名称
spring.shardingsphere.datasource.ds1.type=com.alibaba.druid.pool.DruidDataSource
# 数据库驱动类名
spring.shardingsphere.datasource.ds1.driver-class-name=com.mysql.cj.jdbc.Driver
# 数据库url连接
spring.shardingsphere.datasource.ds1.url=jdbc:mysql://192.168.19.129:3306/sharding_0?serverTimezone=GMT%2B8&characterEncoding=utf8&useUnicode=true&useSSL=false&zeroDateTimeBehavior=CONVERT_TO_NULL&autoReconnect=true&allowMultiQueries=true&failOverReadOnly=false&connectTimeout=6000&maxReconnects=5
# 数据库用户名
spring.shardingsphere.datasource.ds1.username=alian
# 数据库密码
spring.shardingsphere.datasource.ds1.password=123456
# 指定tb_order表的数据分布情况,配置数据节点,使用Groovy的表达式,逻辑表tb_order对应的节点是:ds1.tb_order_1, ds1.tb_order_2
spring.shardingsphere.sharding.tables.tb_order.actual-data-nodes=ds1.tb_order_$->{1..2}
# 采用行表达式分片策略:InlineShardingStrategy
# 指定tb_order表的分片策略中的分片键
spring.shardingsphere.sharding.tables.tb_order.table-strategy.inline.sharding-column=order_id
# 指定tb_order表的分片策略中的分片算法表达式,使用Groovy的表达式
spring.shardingsphere.sharding.tables.tb_order.table-strategy.inline.algorithm-expression=tb_order_$->{ order_id % 2 == 0 ? 2 : 1 }
# 指定tb_order表的主键为order_id
spring.shardingsphere.sharding.tables.tb_order.key-generator.column=order_id
# 指定tb_order表的主键生成策略为SNOWFLAKE
spring.shardingsphere.sharding.tables.tb_order.key-generator.type=SNOWFLAKE
# 指定雪花算法的worker.id
spring.shardingsphere.sharding.tables.tb_order.key-generator.props.worker.id=100
# 指定雪花算法的max.tolerate.time.difference.milliseconds
spring.shardingsphere.sharding.tables.tb_order.key-generator.props.max.tolerate.time.difference.milliseconds=20
# 打开sql输出日志
spring.shardingsphere.props.sql.show=true
4.2、yml配置
application.yml
server:
port: 8899
servlet:
context-path: /sharding-jdbc
spring:
main:
# 允许定义相同的bean对象去覆盖原有的
allow-bean-definition-overriding: true
shardingsphere:
props:
sql:
# 打开sql输出日志
show: true
datasource:
# 数据源名称,多数据源以逗号分隔
names: ds1
ds1:
# 数据库连接池类名称
type: com.alibaba.druid.pool.DruidDataSource
# 数据库驱动类名
driver-class-name: com.mysql.cj.jdbc.Driver
# 数据库url连接
url: jdbc:mysql://192.168.19.129:3306/sharding_0?serverTimezone=GMT%2B8&characterEncoding=utf8&useUnicode=true&useSSL=false&zeroDateTimeBehavior=CONVERT_TO_NULL&autoReconnect=true&allowMultiQueries=true&failOverReadOnly=false&connectTimeout=6000&maxReconnects=5
# 数据库用户名
username: alian
# 数据库密码
password: 123456
sharding:
# 未配置分片规则的表将通过默认数据源定位
default-data-source-name: ds1
tables:
tb_order:
# 由数据源名 + 表名组成,以小数点分隔。多个表以逗号分隔,支持inline表达式
actual-data-nodes: ds1.tb_order_$->{1..2}
# 分表策略
table-strategy:
# 行表达式分片策略
inline:
# 分片键
sharding-column: order_id
# 算法表达式
algorithm-expression: tb_order_$->{order_id%2==0?2:1}
# key生成器
key-generator:
# 自增列名称,缺省表示不使用自增主键生成器
column: order_id
# 自增列值生成器类型,缺省表示使用默认自增列值生成器(SNOWFLAKE/UUID)
type: SNOWFLAKE
# SnowflakeShardingKeyGenerator
props:
# SNOWFLAKE算法的worker.id
worker:
id: 100
# SNOWFLAKE算法的max.tolerate.time.difference.milliseconds
max:
tolerate:
time:
difference:
milliseconds: 20
- actual-data-nodes :使用Groovy的表达式 ds1.tb_order_$->{1…2},表示逻辑表tb_order对应的物理表是:ds1.tb_order_1和 ds1.tb_order_2
- key-generator :key生成器,需要指定字段和类型,如果是SNOWFLAKE,最好也配置下props中的两个属性: worker.id 与 max.tolerate.time.difference.milliseconds 属性(主要还是yml中配置)
- table-strategy 表的分片策略,这里只是一个简单的奇数偶数,采用的是 行表达式分片策略 ,需要指定分片键和分片算法表达式(算法支持Groovy的表达式)
五、实现
5.1、实体
Order.java
@Data
@Entity
@Table(name = "tb_order")
public class Order implements Serializable {
@Id
@GeneratedValue(strategy = GenerationType.IDENTITY)
@Column(name = "order_id")
private Long orderId;
@Column(name = "user_id")
private Integer userId;
@Column(name = "price")
private Integer price;
@Column(name = "order_status")
private Integer orderStatus;
@Column(name = "title")
private String title;
@Column(name = "order_time")
private LocalDateTime orderTime;
}
5.2、持久层
OrderRepository.java
public interface OrderRepository extends PagingAndSortingRepository<Order, Long> {
/**
* 根据订单id查询订单
* @param orderId
* @return
*/
Order findOrderByOrderId(Long orderId);
}
5.3、服务层
OrderService.java
@Slf4j
@Service
public class OrderService {
@Autowired
private OrderRepository orderRepository;
public void saveOrder(Order order) {
orderRepository.save(order);
}
public Order queryOrder(Long orderId) {
return orderRepository.findOrderByOrderId(orderId);
}
}
5.4、测试类
OrderTests.java
@Slf4j
@RunWith(SpringJUnit4ClassRunner.class)
@SpringBootTest
public class OrderTests {
@Autowired
private OrderService orderService;
@Test
public void saveOrder() {
for (int i = 0; i < 10; i++) {
Order order = new Order();
order.setUserId(1000);
// 随机生成50到100的金额
int price = (int) Math.round(Math.random() * (10000 - 5000) + 5000);
order.setPrice(price);
order.setOrderStatus(2);
order.setOrderTime(LocalDateTime.now());
order.setTitle("");
orderService.saveOrder(order);
}
}
@Test
public void queryOrder() {
Long orderId = 845308485917736960L;
Order order = orderService.queryOrder(orderId);
log.info("查询的结果:{}", order);
}
}
5.4.1、保存数据
结果如下:
14:34:08 180 INFO [main]:Logic SQL: insert into tb_order (order_status, order_time, price, title, user_id) values (?, ?, ?, ?, ?)
14:34:08 181 INFO [main]:Actual SQL: ds1 ::: insert into tb_order_2 (order_status, order_time, price, title, user_id, order_id) values (?, ?, ?, ?, ?, ?) ::: [2, 2023-03-22 14:34:07.717, 7601, , 1000, 845308485917736960]
14:34:08 231 INFO [main]:Logic SQL: insert into tb_order (order_status, order_time, price, title, user_id) values (?, ?, ?, ?, ?)
14:34:08 231 INFO [main]:Actual SQL: ds1 ::: insert into tb_order_1 (order_status, order_time, price, title, user_id, order_id) values (?, ?, ?, ?, ?, ?) ::: [2, 2023-03-22 14:34:08.23, 7175, , 1000, 845308486400081921]
14:34:08 255 INFO [main]:Logic SQL: insert into tb_order (order_status, order_time, price, title, user_id) values (?, ?, ?, ?, ?)
14:34:08 256 INFO [main]:Actual SQL: ds1 ::: insert into tb_order_2 (order_status, order_time, price, title, user_id, order_id) values (?, ?, ?, ?, ?, ?) ::: [2, 2023-03-22 14:34:08.254, 5318, , 1000, 845308486504939520]
14:34:08 277 INFO [main]:Logic SQL: insert into tb_order (order_status, order_time, price, title, user_id) values (?, ?, ?, ?, ?)
14:34:08 278 INFO [main]:Actual SQL: ds1 ::: insert into tb_order_1 (order_status, order_time, price, title, user_id, order_id) values (?, ?, ?, ?, ?, ?) ::: [2, 2023-03-22 14:34:08.276, 5490, , 1000, 845308486597214209]
14:34:08 299 INFO [main]:Logic SQL: insert into tb_order (order_status, order_time, price, title, user_id) values (?, ?, ?, ?, ?)
14:34:08 300 INFO [main]:Actual SQL: ds1 ::: insert into tb_order_2 (order_status, order_time, price, title, user_id, order_id) values (?, ?, ?, ?, ?, ?) ::: [2, 2023-03-22 14:34:08.298, 7559, , 1000, 845308486689488896]
14:34:08 321 INFO [main]:Logic SQL: insert into tb_order (order_status, order_time, price, title, user_id) values (?, ?, ?, ?, ?)
14:34:08 321 INFO [main]:Actual SQL: ds1 ::: insert into tb_order_1 (order_status, order_time, price, title, user_id, order_id) values (?, ?, ?, ?, ?, ?) ::: [2, 2023-03-22 14:34:08.32, 9422, , 1000, 845308486777569281]
14:34:08 342 INFO [main]:Logic SQL: insert into tb_order (order_status, order_time, price, title, user_id) values (?, ?, ?, ?, ?)
14:34:08 343 INFO [main]:Actual SQL: ds1 ::: insert into tb_order_2 (order_status, order_time, price, title, user_id, order_id) values (?, ?, ?, ?, ?, ?) ::: [2, 2023-03-22 14:34:08.341, 7935, , 1000, 845308486869843968]
14:34:08 362 INFO [main]:Logic SQL: insert into tb_order (order_status, order_time, price, title, user_id) values (?, ?, ?, ?, ?)
14:34:08 363 INFO [main]:Actual SQL: ds1 ::: insert into tb_order_1 (order_status, order_time, price, title, user_id, order_id) values (?, ?, ?, ?, ?, ?) ::: [2, 2023-03-22 14:34:08.361, 8939, , 1000, 845308486953730049]
14:34:08 383 INFO [main]:Logic SQL: insert into tb_order (order_status, order_time, price, title, user_id) values (?, ?, ?, ?, ?)
14:34:08 383 INFO [main]:Actual SQL: ds1 ::: insert into tb_order_2 (order_status, order_time, price, title, user_id, order_id) values (?, ?, ?, ?, ?, ?) ::: [2, 2023-03-22 14:34:08.382, 7499, , 1000, 845308487041810432]
14:34:08 408 INFO [main]:Logic SQL: insert into tb_order (order_status, order_time, price, title, user_id) values (?, ?, ?, ?, ?)
14:34:08 408 INFO [main]:Actual SQL: ds1 ::: insert into tb_order_1 (order_status, order_time, price, title, user_id, order_id) values (?, ?, ?, ?, ?, ?) ::: [2, 2023-03-22 14:34:08.407, 9598, , 1000, 845308487146668033]
效果图:
5.4.2、查询数据
从上面的结果我们可以看到order_id为 845308485917736960 的记录在 tb_order_2 表,我们查询时实际的查询也是去这个表中查的,请看下面的 Actual SQL
15:03:09 579 INFO [main]:Logic SQL: select order0_.order_id as order_id1_0_, order0_.order_status as order_st2_0_, order0_.order_time as order_ti3_0_, order0_.price as price4_0_, order0_.title as title5_0_, order0_.user_id as user_id6_0_ from tb_order order0_ where order0_.order_id=?
15:03:09 580 INFO [main]:Actual SQL: ds1 ::: select order0_.order_id as order_id1_0_, order0_.order_status as order_st2_0_, order0_.order_time as order_ti3_0_, order0_.price as price4_0_, order0_.title as title5_0_, order0_.user_id as user_id6_0_ from tb_order_2 order0_ where order0_.order_id=? ::: [845308485917736960]
15:03:09 621 INFO [main]:查询的结果:Order(orderId=845308485917736960, userId=1000, price=7601, orderStatus=2, title=, orderTime=2023-03-22T14:34:08)