Mybatis upsert 操作

小咖张

已于 2022-12-30 11:10:07 修改

阅读量2.2k

点赞数 2

分类专栏：数据库文章标签： mybatis java sql

于 2022-12-07 11:08:45 首次发布

本文链接：https://blog.csdn.net/u010738038/article/details/128147370

版权

数据库专栏收录该内容

18 篇文章 0 订阅

订阅专栏

一：插入行的数据和表中的数据主键或者唯一索引出现重复就更新，不重复则插入，通过数据库自身唯一键的查找进行数据排重，并决定INSERT或UPDATE，以下是Mapper.xml中具体使用，亲测可用

Mybatis 对Oracle数据库的操作使用merge into：

  <insert id="updateUserConsumptionSummary" parameterType="model.entity.UserConsumptionSummary">
        merge into CONSUMPTION_SUMMARY using dual on (USER_ID = #{userId,jdbcType=VARCHAR})
        when matched then
        update set CONSUM_AMOUNT = NVL(CONSUM_AMOUNT,0)+#{consumAmount,jdbcType=DECIMAL},LAST_TIME = sysdate
        when not matched then
        insert (USER_ID, ACCT_ID, CONSUM_AMOUNT,LAST_TIME)
        values (#{userId,jdbcType=VARCHAR}, #{acctBookId,jdbcType=VARCHAR}, #{consumAmount,jdbcType=DECIMAL},sysdate)
    </insert>

Mybatis 对Mysql数据库的操作使用 on duplicate key update：

 <insert id="saveInto" parameterType="com.example.demo.entity.User">
        insert into user
        <trim prefix="(" suffix=")" suffixOverrides=",">
            <if test="username != null">
                username,
            </if>
            <if test="year != null">
                year,
            </if>
        </trim>
        <trim prefix="values (" suffix=")" suffixOverrides=",">
            <if test="username != null">
                #{username,jdbcType=VARCHAR},
            </if>
            <if test="year != null">
                #{year,jdbcType=VARCHAR},
            </if>
        </trim>
        on duplicate key update
        username=#{username,jdbcType=VARCHAR},
        year=#{year,jdbcType=VARCHAR}
    </insert>

如果Oralce用了on duplicate key update 就会报错：[Err] ORA-00933: SQL command not properly ended

如果Mysql 用了merge into就会报错：

[Err] 1064 - You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'merge into user using dual on (name = '王五') when matched then ' at line 1

Mybatis 关于插入更新的操作，Oracle和Mysql使用不同，切勿用错

二： ON DUPLICATE KEY UPDATE 和原子操作SELECT+INSERT or UPDATE 的方案进行对比

优点：

1.减少网络连接开销，总体效率上也会略高

2.代码上书写简洁、方便。对已有表批量插入新数据时尤其方便

缺点：

1.迁移数据层产品时造成极大的麻烦，需要付出大量成本去改写代码。如MySQL迁移PostgreSQL或Oracle

2.业务逻辑分散在应用逻辑层和数据层，对项目维护留下隐患

3.多个事务并发执行同一条insert…on duplicate key update 语句时，也就是insert的内容相同时，会有发生死锁的概率：MySQL Bugs: #52020: InnoDB can still deadlock on just INSERT...ON DUPLICATE KEY

在MySQL默认隔离级别REPEATABLE READ下，为避免出现"幻读"发生，防止其他会话插入相同键值的记录
对于普通INSERT操作加锁如下：
1. 对于非唯一索引，需要对新记录加排他锁(X)，另外对新记录和新记录的相邻记录的区间加gap锁。
2. 对于唯一索引，仅需要对新记录加排他锁(X)，唯一索引特性保证其他会话无法插入相同键值。

对于INSERT ON DUPLICATE UPDATE操作，当表中未存在重复键值记录时，加锁特点如下：
1. 对于唯一索引和非唯一索引，都需要对新记录加排他锁(X)，另外对新记录和新记录的相邻记录的区间加gap锁

Tips：对INSER操作部分代码加入异常检查查，当INSERT失败时改为UPDATE操作

三： insert…on duplicate key update 发生死锁校验

建表语句：id设置主键非自增主键（业务自行创建），province，city 联合索引

CREATE TABLE `population` (
  `id` int(11) NOT NULL,
  `province` varchar(20) DEFAULT NULL COMMENT '省',
  `city` varchar(20) DEFAULT NULL COMMENT '市',
  `population` int(11) DEFAULT '0' COMMENT '总人口',
  `bnum` int(11) DEFAULT '0' COMMENT '男生人数',
  `gnum` int(11) DEFAULT '0' COMMENT '女生人数',
  `update_time` datetime DEFAULT NULL,
  PRIMARY KEY (`id`),
  UNIQUE KEY `idx_p_c` (`province`,`city`) USING BTREE
) ENGINE=InnoDB DEFAULT CHARSET=utf8

大事务测试保存到数据库：

 @Transactional(rollbackFor = Exception.class)
    public void population1(String province, String city, Integer population, Integer bnum, Integer gnum, Integer id) throws InterruptedException {
        Population population1 = new Population();
        population1.setProvince(province);
        population1.setCity(city);
        population1.setPopulation(population);
        population1.setBnum(bnum);
        population1.setGnum(gnum);
        population1.setUpdateTime(new Date());
        population1.setId(id);
        log.info("population1开数据库操作开始");
        populationMapper.insertInto(population1);
        //假装有大事务
        Thread.sleep(10000);
        log.info("population1开数据库操作结束");
    }

测试数据，使用RequestMapping 接口测试数据即可：

1：测试数据1

http://localhost:8087/population1?province=江苏&city=连云港&population=4&gnum=4&id=68

2：测试数据2

http://localhost:8087/population1?province=江苏&city=连云港&population=4&gnum=4&id=69
通过show engine innodb status; 查询最近一次死锁

=====================================
2022-12-08 11:31:37 0x3198 INNODB MONITOR OUTPUT
=====================================
Per second averages calculated from the last 11 seconds
-----------------
BACKGROUND THREAD
-----------------
srv_master_thread loops: 296 srv_active, 0 srv_shutdown, 121311 srv_idle
srv_master_thread log flush and writes: 121607
----------
SEMAPHORES
----------
OS WAIT ARRAY INFO: reservation count 318
OS WAIT ARRAY INFO: signal count 286
RW-shared spins 0, rounds 667, OS waits 299
RW-excl spins 0, rounds 17, OS waits 0
RW-sx spins 0, rounds 0, OS waits 0
Spin rounds per wait: 667.00 RW-shared, 17.00 RW-excl, 0.00 RW-sx
------------------------
LATEST DETECTED DEADLOCK
------------------------
2022-12-08 11:31:04 0x2370
*** (1) TRANSACTION:
TRANSACTION 46471, ACTIVE 5 sec inserting
mysql tables in use 1, locked 1
LOCK WAIT 3 lock struct(s), heap size 1136, 2 row lock(s)
MySQL thread id 284, OS thread handle 9948, query id 3085 localhost 127.0.0.1 root update
INSERT INTO population(id,province,city,population,bnum,gnum,update_time)
        VALUE(68,'江苏','连云港',4,null,4,'2022-12-08 11:30:59.945')
        on duplicate key update
        update_time='2022-12-08 11:30:59.945'
         
            ,population=population+4
         
         
         
            ,gnum=gnum+4
*** (1) WAITING FOR THIS LOCK TO BE GRANTED:
RECORD LOCKS space id 143 page no 4 n bits 80 index idx_p_c of table `spring_cache`.`population` trx id 46471 lock_mode X waiting
Record lock, heap no 7 PHYSICAL RECORD: n_fields 3; compact format; info bits 0
 0: len 6; hex e6b19fe88b8f; asc       ;;
 1: len 9; hex e8bf9ee4ba91e6b8af; asc          ;;
 2: len 4; hex 80000044; asc    D;;

*** (2) TRANSACTION:
TRANSACTION 46470, ACTIVE 7 sec starting index read
mysql tables in use 1, locked 1
3 lock struct(s), heap size 1136, 2 row lock(s)
MySQL thread id 283, OS thread handle 9072, query id 3081 localhost 127.0.0.1 root update
INSERT INTO population(id,province,city,population,bnum,gnum,update_time)
        VALUE(69,'江苏','连云港',4,null,4,'2022-12-08 11:30:57.853')
        on duplicate key update
        update_time='2022-12-08 11:30:57.853'
         
            ,population=population+4
         
         
         
            ,gnum=gnum+4
*** (2) HOLDS THE LOCK(S):
RECORD LOCKS space id 143 page no 4 n bits 80 index idx_p_c of table `spring_cache`.`population` trx id 46470 lock_mode X
Record lock, heap no 7 PHYSICAL RECORD: n_fields 3; compact format; info bits 0
 0: len 6; hex e6b19fe88b8f; asc       ;;
 1: len 9; hex e8bf9ee4ba91e6b8af; asc          ;;
 2: len 4; hex 80000044; asc    D;;

*** (2) WAITING FOR THIS LOCK TO BE GRANTED:
RECORD LOCKS space id 143 page no 3 n bits 80 index PRIMARY of table `spring_cache`.`population` trx id 46470 lock_mode X locks rec but not gap waiting
Record lock, heap no 8 PHYSICAL RECORD: n_fields 10; compact format; info bits 0
 0: len 4; hex 80000044; asc    D;;
 1: len 6; hex 00000000b585; asc       ;;
 2: len 7; hex 72000001d00110; asc r      ;;
 3: len 6; hex e6b19fe88b8f; asc       ;;
 4: len 9; hex e8bf9ee4ba91e6b8af; asc          ;;
 5: SQL NULL;
 6: len 4; hex 80000010; asc     ;;
 7: SQL NULL;
 8: len 4; hex 80000010; asc     ;;
 9: len 5; hex 99ae90b7b7; asc      ;;

*** WE ROLL BACK TRANSACTION (2)
------------
TRANSACTIONS
------------
Trx id counter 46474
Purge done for trx's n:o < 46474 undo n:o < 0 state: running but idle
History list length 22
LIST OF TRANSACTIONS FOR EACH SESSION:
---TRANSACTION 283786379763464, not started
0 lock struct(s), heap size 1136, 0 row lock(s)
---TRANSACTION 283786379762592, not started
0 lock struct(s), heap size 1136, 0 row lock(s)
---TRANSACTION 283786379761720, not started
0 lock struct(s), heap size 1136, 0 row lock(s)
---TRANSACTION 283786379760848, not started
0 lock struct(s), heap size 1136, 0 row lock(s)
---TRANSACTION 283786379759976, not started
0 lock struct(s), heap size 1136, 0 row lock(s)
---TRANSACTION 283786379759104, not started
0 lock struct(s), heap size 1136, 0 row lock(s)
---TRANSACTION 283786379758232, not started
0 lock struct(s), heap size 1136, 0 row lock(s)
---TRANSACTION 283786379757360, not started
0 lock struct(s), heap size 1136, 0 row lock(s)
--------
FILE I/O
--------
I/O thread 0 state: wait Windows aio (insert buffer thread)
I/O thread 1 state: wait Windows aio (log thread)
I/O thread 2 state: wait Windows aio (read thread)
I/O thread 3 state: wait Windows aio (read thread)
I/O thread 4 state: wait Windows aio (read thread)
I/O thread 5 state: wait Windows aio (read thread)
I/O thread 6 state: wait Windows aio (write thread)
I/O thread 7 state: wait Windows aio (write thread)
I/O thread 8 state: wait Windows aio (write thread)
I/O thread 9 state: wait Windows aio (write thread)
Pending normal aio reads: [0, 0, 0, 0] , aio writes: [0, 0, 0, 0] ,
 ibuf aio reads:, log i/o's:, sync i/o's:
Pending flushes (fsync) log: 0; buffer pool: 0
587 OS file reads, 2495 OS file writes, 1166 OS fsyncs
0.00 reads/s, 0 avg bytes/read, 0.00 writes/s, 0.00 fsyncs/s
-------------------------------------
INSERT BUFFER AND ADAPTIVE HASH INDEX
-------------------------------------
Ibuf: size 1, free list len 0, seg size 2, 0 merges
merged operations:
 insert 0, delete mark 0, delete 0
discarded operations:
 insert 0, delete mark 0, delete 0
Hash table size 34679, node heap has 0 buffer(s)
Hash table size 34679, node heap has 0 buffer(s)
Hash table size 34679, node heap has 0 buffer(s)
Hash table size 34679, node heap has 0 buffer(s)
Hash table size 34679, node heap has 0 buffer(s)
Hash table size 34679, node heap has 0 buffer(s)
Hash table size 34679, node heap has 0 buffer(s)
Hash table size 34679, node heap has 0 buffer(s)
0.00 hash searches/s, 0.00 non-hash searches/s
---
LOG
---
Log sequence number 3789290
Log flushed up to   3789290
Pages flushed up to 3789290
Last checkpoint at  3789281
0 pending log flushes, 0 pending chkp writes
747 log i/o's done, 0.00 log i/o's/second
----------------------
BUFFER POOL AND MEMORY
----------------------
Total large memory allocated 137297920
Dictionary memory allocated 376404
Buffer pool size   8192
Free buffers       7699
Database pages     493
Old database pages 0
Modified db pages  0
Pending reads      0
Pending writes: LRU 0, flush list 0, single page 0
Pages made young 0, not young 0
0.00 youngs/s, 0.00 non-youngs/s
Pages read 388, created 105, written 1585
0.00 reads/s, 0.00 creates/s, 0.00 writes/s
No buffer pool page gets since the last printout
Pages read ahead 0.00/s, evicted without access 0.00/s, Random read ahead 0.00/s
LRU len: 493, unzip_LRU len: 0
I/O sum[0]:cur[0], unzip sum[0]:cur[0]
--------------
ROW OPERATIONS
--------------
0 queries inside InnoDB, 0 queries in queue
0 read views open inside InnoDB
Process ID=4188, Main thread ID=6836, state: sleeping
Number of rows inserted 2456, updated 46, deleted 12, read 2986
0.00 inserts/s, 0.00 updates/s, 0.00 deletes/s, 0.00 reads/s
----------------------------
END OF INNODB MONITOR OUTPUT
============================

分析下死锁日志，可以得到以下信息：

导致死锁的两条SQL语句。
事务1，持有索引PRIMARY的锁，在等待获取idx_p_c的锁。
事务2，持有idx_p_c的锁，在等待获取PRIMARY的锁。
因事务1和事务2之间发生循环等待，故发生死锁。
事务1和事务2当前持有的锁均为：lock_mode X locks rec but not gap
两个事务对记录加的都是X 锁，No Gap锁，即对当行记录加锁(Record Lock)