一:插入行的数据和表中的数据主键或者唯一索引出现重复就更新,不重复则插入,通过数据库自身唯一键的查找进行数据排重,并决定INSERT或UPDATE,以下是Mapper.xml中具体使用,亲测可用
Mybatis 对Oracle数据库的操作使用merge into:
<insert id="updateUserConsumptionSummary" parameterType="model.entity.UserConsumptionSummary">
merge into CONSUMPTION_SUMMARY using dual on (USER_ID = #{userId,jdbcType=VARCHAR})
when matched then
update set CONSUM_AMOUNT = NVL(CONSUM_AMOUNT,0)+#{consumAmount,jdbcType=DECIMAL},LAST_TIME = sysdate
when not matched then
insert (USER_ID, ACCT_ID, CONSUM_AMOUNT,LAST_TIME)
values (#{userId,jdbcType=VARCHAR}, #{acctBookId,jdbcType=VARCHAR}, #{consumAmount,jdbcType=DECIMAL},sysdate)
</insert>
Mybatis 对Mysql数据库的操作使用 on duplicate key update:
<insert id="saveInto" parameterType="com.example.demo.entity.User">
insert into user
<trim prefix="(" suffix=")" suffixOverrides=",">
<if test="username != null">
username,
</if>
<if test="year != null">
year,
</if>
</trim>
<trim prefix="values (" suffix=")" suffixOverrides=",">
<if test="username != null">
#{username,jdbcType=VARCHAR},
</if>
<if test="year != null">
#{year,jdbcType=VARCHAR},
</if>
</trim>
on duplicate key update
username=#{username,jdbcType=VARCHAR},
year=#{year,jdbcType=VARCHAR}
</insert>
如果Oralce用了on duplicate key update 就会报错:[Err] ORA-00933: SQL command not properly ended
如果Mysql 用了merge into就会报错:
[Err] 1064 - You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'merge into user using dual on (name = '王五') when matched then ' at line 1
Mybatis 关于插入更新的操作,Oracle和Mysql使用不同,切勿用错
二: ON DUPLICATE KEY UPDATE 和 原子操作SELECT+INSERT or UPDATE 的方案进行对比
优点:
1.减少网络连接开销,总体效率上也会略高
2.代码上书写简洁、方便。对已有表批量插入新数据时尤其方便
缺点:
1.迁移数据层产品时造成极大的麻烦,需要付出大量成本去改写代码。如MySQL迁移PostgreSQL或Oracle
2.业务逻辑分散在应用逻辑层和数据层,对项目维护留下隐患
3.多个事务并发执行同一条insert…on duplicate key update 语句时,也就是insert的内容相同时,会有发生死锁的概率:MySQL Bugs: #52020: InnoDB can still deadlock on just INSERT...ON DUPLICATE KEY
在MySQL默认隔离级别REPEATABLE READ下,为避免出现"幻读"发生,防止其他会话插入相同键值的记录
对于普通INSERT操作加锁如下:
1. 对于非唯一索引,需要对新记录加排他锁(X),另外对新记录和新记录的相邻记录的区间加gap锁。
2. 对于唯一索引,仅需要对新记录加排他锁(X),唯一索引特性保证其他会话无法插入相同键值。对于INSERT ON DUPLICATE UPDATE操作,当表中未存在重复键值记录时,加锁特点如下:
1. 对于唯一索引和非唯一索引,都需要对新记录加排他锁(X),另外对新记录和新记录的相邻记录的区间加gap锁
Tips:对INSER操作部分代码加入异常检查查,当INSERT失败时改为UPDATE操作
三: insert…on duplicate key update 发生死锁校验
建表语句:id设置主键非自增主键(业务自行创建),province,city 联合索引
CREATE TABLE `population` (
`id` int(11) NOT NULL,
`province` varchar(20) DEFAULT NULL COMMENT '省',
`city` varchar(20) DEFAULT NULL COMMENT '市',
`population` int(11) DEFAULT '0' COMMENT '总人口',
`bnum` int(11) DEFAULT '0' COMMENT '男生人数',
`gnum` int(11) DEFAULT '0' COMMENT '女生人数',
`update_time` datetime DEFAULT NULL,
PRIMARY KEY (`id`),
UNIQUE KEY `idx_p_c` (`province`,`city`) USING BTREE
) ENGINE=InnoDB DEFAULT CHARSET=utf8
大事务测试保存到数据库:
@Transactional(rollbackFor = Exception.class)
public void population1(String province, String city, Integer population, Integer bnum, Integer gnum, Integer id) throws InterruptedException {
Population population1 = new Population();
population1.setProvince(province);
population1.setCity(city);
population1.setPopulation(population);
population1.setBnum(bnum);
population1.setGnum(gnum);
population1.setUpdateTime(new Date());
population1.setId(id);
log.info("population1开数据库操作开始");
populationMapper.insertInto(population1);
//假装有大事务
Thread.sleep(10000);
log.info("population1开数据库操作结束");
}
测试数据,使用RequestMapping 接口测试数据即可:
1:测试数据1
http://localhost:8087/population1?province=江苏&city=连云港&population=4&gnum=4&id=68
2:测试数据2
http://localhost:8087/population1?province=江苏&city=连云港&population=4&gnum=4&id=69
通过show engine innodb status; 查询最近一次死锁
=====================================
2022-12-08 11:31:37 0x3198 INNODB MONITOR OUTPUT
=====================================
Per second averages calculated from the last 11 seconds
-----------------
BACKGROUND THREAD
-----------------
srv_master_thread loops: 296 srv_active, 0 srv_shutdown, 121311 srv_idle
srv_master_thread log flush and writes: 121607
----------
SEMAPHORES
----------
OS WAIT ARRAY INFO: reservation count 318
OS WAIT ARRAY INFO: signal count 286
RW-shared spins 0, rounds 667, OS waits 299
RW-excl spins 0, rounds 17, OS waits 0
RW-sx spins 0, rounds 0, OS waits 0
Spin rounds per wait: 667.00 RW-shared, 17.00 RW-excl, 0.00 RW-sx
------------------------
LATEST DETECTED DEADLOCK
------------------------
2022-12-08 11:31:04 0x2370
*** (1) TRANSACTION:
TRANSACTION 46471, ACTIVE 5 sec inserting
mysql tables in use 1, locked 1
LOCK WAIT 3 lock struct(s), heap size 1136, 2 row lock(s)
MySQL thread id 284, OS thread handle 9948, query id 3085 localhost 127.0.0.1 root update
INSERT INTO population(id,province,city,population,bnum,gnum,update_time)
VALUE(68,'江苏','连云港',4,null,4,'2022-12-08 11:30:59.945')
on duplicate key update
update_time='2022-12-08 11:30:59.945'
,population=population+4
,gnum=gnum+4
*** (1) WAITING FOR THIS LOCK TO BE GRANTED:
RECORD LOCKS space id 143 page no 4 n bits 80 index idx_p_c of table `spring_cache`.`population` trx id 46471 lock_mode X waiting
Record lock, heap no 7 PHYSICAL RECORD: n_fields 3; compact format; info bits 0
0: len 6; hex e6b19fe88b8f; asc ;;
1: len 9; hex e8bf9ee4ba91e6b8af; asc ;;
2: len 4; hex 80000044; asc D;;
*** (2) TRANSACTION:
TRANSACTION 46470, ACTIVE 7 sec starting index read
mysql tables in use 1, locked 1
3 lock struct(s), heap size 1136, 2 row lock(s)
MySQL thread id 283, OS thread handle 9072, query id 3081 localhost 127.0.0.1 root update
INSERT INTO population(id,province,city,population,bnum,gnum,update_time)
VALUE(69,'江苏','连云港',4,null,4,'2022-12-08 11:30:57.853')
on duplicate key update
update_time='2022-12-08 11:30:57.853'
,population=population+4
,gnum=gnum+4
*** (2) HOLDS THE LOCK(S):
RECORD LOCKS space id 143 page no 4 n bits 80 index idx_p_c of table `spring_cache`.`population` trx id 46470 lock_mode X
Record lock, heap no 7 PHYSICAL RECORD: n_fields 3; compact format; info bits 0
0: len 6; hex e6b19fe88b8f; asc ;;
1: len 9; hex e8bf9ee4ba91e6b8af; asc ;;
2: len 4; hex 80000044; asc D;;
*** (2) WAITING FOR THIS LOCK TO BE GRANTED:
RECORD LOCKS space id 143 page no 3 n bits 80 index PRIMARY of table `spring_cache`.`population` trx id 46470 lock_mode X locks rec but not gap waiting
Record lock, heap no 8 PHYSICAL RECORD: n_fields 10; compact format; info bits 0
0: len 4; hex 80000044; asc D;;
1: len 6; hex 00000000b585; asc ;;
2: len 7; hex 72000001d00110; asc r ;;
3: len 6; hex e6b19fe88b8f; asc ;;
4: len 9; hex e8bf9ee4ba91e6b8af; asc ;;
5: SQL NULL;
6: len 4; hex 80000010; asc ;;
7: SQL NULL;
8: len 4; hex 80000010; asc ;;
9: len 5; hex 99ae90b7b7; asc ;;
*** WE ROLL BACK TRANSACTION (2)
------------
TRANSACTIONS
------------
Trx id counter 46474
Purge done for trx's n:o < 46474 undo n:o < 0 state: running but idle
History list length 22
LIST OF TRANSACTIONS FOR EACH SESSION:
---TRANSACTION 283786379763464, not started
0 lock struct(s), heap size 1136, 0 row lock(s)
---TRANSACTION 283786379762592, not started
0 lock struct(s), heap size 1136, 0 row lock(s)
---TRANSACTION 283786379761720, not started
0 lock struct(s), heap size 1136, 0 row lock(s)
---TRANSACTION 283786379760848, not started
0 lock struct(s), heap size 1136, 0 row lock(s)
---TRANSACTION 283786379759976, not started
0 lock struct(s), heap size 1136, 0 row lock(s)
---TRANSACTION 283786379759104, not started
0 lock struct(s), heap size 1136, 0 row lock(s)
---TRANSACTION 283786379758232, not started
0 lock struct(s), heap size 1136, 0 row lock(s)
---TRANSACTION 283786379757360, not started
0 lock struct(s), heap size 1136, 0 row lock(s)
--------
FILE I/O
--------
I/O thread 0 state: wait Windows aio (insert buffer thread)
I/O thread 1 state: wait Windows aio (log thread)
I/O thread 2 state: wait Windows aio (read thread)
I/O thread 3 state: wait Windows aio (read thread)
I/O thread 4 state: wait Windows aio (read thread)
I/O thread 5 state: wait Windows aio (read thread)
I/O thread 6 state: wait Windows aio (write thread)
I/O thread 7 state: wait Windows aio (write thread)
I/O thread 8 state: wait Windows aio (write thread)
I/O thread 9 state: wait Windows aio (write thread)
Pending normal aio reads: [0, 0, 0, 0] , aio writes: [0, 0, 0, 0] ,
ibuf aio reads:, log i/o's:, sync i/o's:
Pending flushes (fsync) log: 0; buffer pool: 0
587 OS file reads, 2495 OS file writes, 1166 OS fsyncs
0.00 reads/s, 0 avg bytes/read, 0.00 writes/s, 0.00 fsyncs/s
-------------------------------------
INSERT BUFFER AND ADAPTIVE HASH INDEX
-------------------------------------
Ibuf: size 1, free list len 0, seg size 2, 0 merges
merged operations:
insert 0, delete mark 0, delete 0
discarded operations:
insert 0, delete mark 0, delete 0
Hash table size 34679, node heap has 0 buffer(s)
Hash table size 34679, node heap has 0 buffer(s)
Hash table size 34679, node heap has 0 buffer(s)
Hash table size 34679, node heap has 0 buffer(s)
Hash table size 34679, node heap has 0 buffer(s)
Hash table size 34679, node heap has 0 buffer(s)
Hash table size 34679, node heap has 0 buffer(s)
Hash table size 34679, node heap has 0 buffer(s)
0.00 hash searches/s, 0.00 non-hash searches/s
---
LOG
---
Log sequence number 3789290
Log flushed up to 3789290
Pages flushed up to 3789290
Last checkpoint at 3789281
0 pending log flushes, 0 pending chkp writes
747 log i/o's done, 0.00 log i/o's/second
----------------------
BUFFER POOL AND MEMORY
----------------------
Total large memory allocated 137297920
Dictionary memory allocated 376404
Buffer pool size 8192
Free buffers 7699
Database pages 493
Old database pages 0
Modified db pages 0
Pending reads 0
Pending writes: LRU 0, flush list 0, single page 0
Pages made young 0, not young 0
0.00 youngs/s, 0.00 non-youngs/s
Pages read 388, created 105, written 1585
0.00 reads/s, 0.00 creates/s, 0.00 writes/s
No buffer pool page gets since the last printout
Pages read ahead 0.00/s, evicted without access 0.00/s, Random read ahead 0.00/s
LRU len: 493, unzip_LRU len: 0
I/O sum[0]:cur[0], unzip sum[0]:cur[0]
--------------
ROW OPERATIONS
--------------
0 queries inside InnoDB, 0 queries in queue
0 read views open inside InnoDB
Process ID=4188, Main thread ID=6836, state: sleeping
Number of rows inserted 2456, updated 46, deleted 12, read 2986
0.00 inserts/s, 0.00 updates/s, 0.00 deletes/s, 0.00 reads/s
----------------------------
END OF INNODB MONITOR OUTPUT
============================
分析下死锁日志,可以得到以下信息:
导致死锁的两条SQL语句。
事务1,持有索引PRIMARY的锁,在等待获取idx_p_c的锁。
事务2,持有idx_p_c的锁,在等待获取PRIMARY的锁。
因事务1和事务2之间发生循环等待,故发生死锁。
事务1和事务2当前持有的锁均为:lock_mode X locks rec but not gap
两个事务对记录加的都是X 锁,No Gap锁,即对当行记录加锁(Record Lock)
结论:INSERT ON DUPLICATE UPDATE,高并发大事务中,存在多个唯一索引时,会造成死锁错误
Tips:此报错索引,除上述情况,还常见于事务中存在for循环更新某条记录的情况