springboot执行批量插入_如何使用JpaRepository进行批量(多行)插入?

When calling the saveAll method of my JpaRepository with a long List from the service layer, trace logging of Hibernate shows single SQL statements being issued per entity.

Can I force it to do a bulk insert (i.e. multi-row) without needing to manually fiddle with EntityManger, transactions etc. or even raw SQL statement strings?

With multi-row insert I mean not just transitioning from:

start transaction

INSERT INTO table VALUES (1, 2)

end transaction

start transaction

INSERT INTO table VALUES (3, 4)

end transaction

start transaction

INSERT INTO table VALUES (5, 6)

end transaction

to:

start transaction

INSERT INTO table VALUES (1, 2)

INSERT INTO table VALUES (3, 4)

INSERT INTO table VALUES (5, 6)

end transaction

but instead to:

start transaction

INSERT INTO table VALUES (1, 2), (3, 4), (5, 6)

end transaction

In PROD I'm using CockroachDB, and the difference in performance is significant.

Below is a minimal example that reproduces the problem (H2 for simplicity).

./src/main/kotlin/ThingService.kt:

package things

import org.springframework.boot.autoconfigure.SpringBootApplication

import org.springframework.boot.runApplication

import org.springframework.web.bind.annotation.RestController

import org.springframework.web.bind.annotation.GetMapping

import org.springframework.data.jpa.repository.JpaRepository

import javax.persistence.Entity

import javax.persistence.Id

import javax.persistence.GeneratedValue

interface ThingRepository : JpaRepository {

}

@RestController

class ThingController(private val repository: ThingRepository) {

@GetMapping("/test_trigger")

fun trigger() {

val things: MutableList = mutableListOf()

for (i in 3000..3013) {

things.add(Thing(i))

}

repository.saveAll(things)

}

}

@Entity

data class Thing (

var value: Int,

@Id

@GeneratedValue

var id: Long = -1

)

@SpringBootApplication

class Application {

}

fun main(args: Array) {

runApplication(*args)

}

./src/main/resources/application.properties:

jdbc.driverClassName = org.h2.Driver

jdbc.url = jdbc:h2:mem:db

jdbc.username = sa

jdbc.password = sa

hibernate.dialect=org.hibernate.dialect.H2Dialect

hibernate.hbm2ddl.auto=create

spring.jpa.generate-ddl = true

spring.jpa.show-sql = true

spring.jpa.properties.hibernate.jdbc.batch_size = 10

spring.jpa.properties.hibernate.order_inserts = true

spring.jpa.properties.hibernate.order_updates = true

spring.jpa.properties.hibernate.jdbc.batch_versioned_data = true

./build.gradle.kts:

import org.jetbrains.kotlin.gradle.tasks.KotlinCompile

plugins {

val kotlinVersion = "1.2.30"

id("org.springframework.boot") version "2.0.2.RELEASE"

id("org.jetbrains.kotlin.jvm") version kotlinVersion

id("org.jetbrains.kotlin.plugin.spring") version kotlinVersion

id("org.jetbrains.kotlin.plugin.jpa") version kotlinVersion

id("io.spring.dependency-management") version "1.0.5.RELEASE"

}

version = "1.0.0-SNAPSHOT"

tasks.withType {

kotlinOptions {

jvmTarget = "1.8"

freeCompilerArgs = listOf("-Xjsr305=strict")

}

}

repositories {

mavenCentral()

}

dependencies {

compile("org.springframework.boot:spring-boot-starter-web")

compile("org.springframework.boot:spring-boot-starter-data-jpa")

compile("org.jetbrains.kotlin:kotlin-stdlib-jdk8")

compile("org.jetbrains.kotlin:kotlin-reflect")

compile("org.hibernate:hibernate-core")

compile("com.h2database:h2")

}

Run:

./gradlew bootRun

Trigger DB INSERTs:

curl http://localhost:8080/test_trigger

Log output:

Hibernate: select thing0_.id as id1_0_0_, thing0_.value as value2_0_0_ from thing thing0_ where thing0_.id=?

Hibernate: call next value for hibernate_sequence

Hibernate: select thing0_.id as id1_0_0_, thing0_.value as value2_0_0_ from thing thing0_ where thing0_.id=?

Hibernate: call next value for hibernate_sequence

Hibernate: select thing0_.id as id1_0_0_, thing0_.value as value2_0_0_ from thing thing0_ where thing0_.id=?

Hibernate: call next value for hibernate_sequence

Hibernate: select thing0_.id as id1_0_0_, thing0_.value as value2_0_0_ from thing thing0_ where thing0_.id=?

Hibernate: call next value for hibernate_sequence

Hibernate: select thing0_.id as id1_0_0_, thing0_.value as value2_0_0_ from thing thing0_ where thing0_.id=?

Hibernate: call next value for hibernate_sequence

Hibernate: select thing0_.id as id1_0_0_, thing0_.value as value2_0_0_ from thing thing0_ where thing0_.id=?

Hibernate: call next value for hibernate_sequence

Hibernate: select thing0_.id as id1_0_0_, thing0_.value as value2_0_0_ from thing thing0_ where thing0_.id=?

Hibernate: call next value for hibernate_sequence

Hibernate: select thing0_.id as id1_0_0_, thing0_.value as value2_0_0_ from thing thing0_ where thing0_.id=?

Hibernate: call next value for hibernate_sequence

Hibernate: select thing0_.id as id1_0_0_, thing0_.value as value2_0_0_ from thing thing0_ where thing0_.id=?

Hibernate: call next value for hibernate_sequence

Hibernate: select thing0_.id as id1_0_0_, thing0_.value as value2_0_0_ from thing thing0_ where thing0_.id=?

Hibernate: call next value for hibernate_sequence

Hibernate: select thing0_.id as id1_0_0_, thing0_.value as value2_0_0_ from thing thing0_ where thing0_.id=?

Hibernate: call next value for hibernate_sequence

Hibernate: select thing0_.id as id1_0_0_, thing0_.value as value2_0_0_ from thing thing0_ where thing0_.id=?

Hibernate: call next value for hibernate_sequence

Hibernate: select thing0_.id as id1_0_0_, thing0_.value as value2_0_0_ from thing thing0_ where thing0_.id=?

Hibernate: call next value for hibernate_sequence

Hibernate: select thing0_.id as id1_0_0_, thing0_.value as value2_0_0_ from thing thing0_ where thing0_.id=?

Hibernate: call next value for hibernate_sequence

Hibernate: insert into thing (value, id) values (?, ?)

Hibernate: insert into thing (value, id) values (?, ?)

Hibernate: insert into thing (value, id) values (?, ?)

Hibernate: insert into thing (value, id) values (?, ?)

Hibernate: insert into thing (value, id) values (?, ?)

Hibernate: insert into thing (value, id) values (?, ?)

Hibernate: insert into thing (value, id) values (?, ?)

Hibernate: insert into thing (value, id) values (?, ?)

Hibernate: insert into thing (value, id) values (?, ?)

Hibernate: insert into thing (value, id) values (?, ?)

Hibernate: insert into thing (value, id) values (?, ?)

Hibernate: insert into thing (value, id) values (?, ?)

Hibernate: insert into thing (value, id) values (?, ?)

Hibernate: insert into thing (value, id) values (?, ?)

解决方案

To get a bulk insert with Sring Boot and Spring Data JPA you need only two things:

set the option spring.jpa.properties.hibernate.jdbc.batch_size to appropriate value you need (for example: 20).

use saveAll() method of your repo with the list of entities prepared for inserting.

Working example is here.

Regarding the transformation of the insert statement into something like this:

INSERT INTO table VALUES (1, 2), (3, 4), (5, 6)

the such is available in PostgreSQL: you can set the option reWriteBatchedInserts to true in jdbc connection string:

jdbc:postgresql://localhost:5432/db?reWriteBatchedInserts=true

then jdbc driver will do this transformation.

Additional info about batching you can find here.

UPDATED

UPDATED

  • 0
    点赞
  • 0
    收藏
    觉得还不错? 一键收藏
  • 0
    评论
提供的源码资源涵盖了安卓应用、小程序、Python应用和Java应用等多个领域,每个领域都包含了丰富的实例和项目。这些源码都是基于各自平台的最新技术和标准编写,确保了在对应环境下能够无缝运行。同时,源码中配备了详细的注释和文档,帮助用户快速理解代码结构和实现逻辑。 适用人群: 这些源码资源特别适合大学生群体。无论你是计算机相关专业的学生,还是对其他领域编程感兴趣的学生,这些资源都能为你提供宝贵的学习和实践机会。通过学习和运行这些源码,你可以掌握各平台开发的基础知识,提升编程能力和项目实战经验。 使用场景及目标: 在学习阶段,你可以利用这些源码资源进行课程实践、课外项目或毕业设计。通过分析和运行源码,你将深入了解各平台开发的技术细节和最佳实践,逐步培养起自己的项目开发和问题解决能力。此外,在求职或创业过程中,具备跨平台开发能力的大学生将更具竞争力。 其他说明: 为了确保源码资源的可运行性和易用性,特别注意了以下几点:首先,每份源码都提供了详细的运行环境和依赖说明,确保用户能够轻松搭建起开发环境;其次,源码中的注释和文档都非常完善,方便用户快速上手和理解代码;最后,我会定期更新这些源码资源,以适应各平台技术的最新发展和市场需求。

“相关推荐”对你有帮助么?

  • 非常没帮助
  • 没帮助
  • 一般
  • 有帮助
  • 非常有帮助
提交
评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值