Seata全局事务开启后，Mysql 批量Insert耗时变长的问题

最新推荐文章于 2024-08-26 09:41:16 发布

咖啡煮码

最新推荐文章于 2024-08-26 09:41:16 发布

阅读量639

点赞数 8

分类专栏：杂谈文章标签： mysql 数据库分布式

本文链接：https://blog.csdn.net/weixin_37557894/article/details/135752912

版权

杂谈专栏收录该内容

7 篇文章 0 订阅

订阅专栏

文章讨论了在使用Seata解决分布式系统事务时，遇到的RPCTimeout问题，尤其是在插入大量数据（8万条）时。作者发现去掉分布式事务后性能显著提升，提示在处理大规模数据批量插入时应避免全局事务。

摘要由CSDN通过智能技术生成

Seata解决了分布式系统的事务问题，但是在某些场景下仍然存在一些问题。今天就又遇到了一个。

我的环境 dubbo + seata + mysql。

相关版本：

dubbo：2.7.15
seata：1.6.1
mysql：5.7

场景：有个接口，功能为生成8万条数据 insert 到一张表中，因为除了insert 操作外还有其他更新操作，所以在主调方法上加上了 @GlobalTransactional 注解，其他子分支的事务添加 @Transactional 注解。

下面是伪代码，

@DubboReference
private MyService myService;

// 调用端,dubbo消费者，声明全局事务
@GlobalTransactional
public void operation() {
    // 生成8万条数据，insert到table_a表中（rpc远程调用）
    List<Integer> list = generateData();
    myService.deal(list);

    // 其他操作（rpc远程调用）
    // …………
}

//======================项目分隔线（上下的代码分属两个项目）=====================

// dubbo服务提供者，添加分支事务
@Autowired
TableAMapper tableAMapper;

@Transactional
public void deal(List<Integer> list) {
    List<List<Integer>> splitList = ListUtil.split(list, 1000);
    for (List<Integer> subList : splitList) {
        tableAMapper.batchInsert(subList);
    }
}

在分支事务中，为了防止单次batch insert 的数据过多导致报错，所以将 insert 数据切割成1000条一份，分批执行。

然后，就报错了

ERROR o.a.dubbo.rpc.filter.ExceptionFilter -  [DUBBO] Got unchecked and undeclared exception which called by 10.243.119.10. service: com.hml.user.MyService, method: deal, exception: org.springframework.transaction.TransactionSystemException: Could not commit JDBC transaction; nested exception is java.sql.SQLException: io.seata.core.exception.RmTransactionException: RPC Timeout, dubbo version: 2.7.15, current host: 10.243.119.168
org.springframework.transaction.TransactionSystemException: Could not commit JDBC transaction; nested exception is java.sql.SQLException: io.seata.core.exception.RmTransactionException: RPC Timeout
	at org.springframework.jdbc.datasource.DataSourceTransactionManager.doCommit(DataSourceTransactionManager.java:332)
	at org.springframework.transaction.support.AbstractPlatformTransactionManager.processCommit(AbstractPlatformTransactionManager.java:743)
	at org.springframework.transaction.support.AbstractPlatformTransactionManager.commit(AbstractPlatformTransactionManager.java:711)
	at org.springframework.transaction.interceptor.TransactionAspectSupport.commitTransactionAfterReturning(TransactionAspectSupport.java:632)
	at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:386)
	at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:118)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
	at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749)
	at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:691)
	at org.apache.dubbo.common.bytecode.Wrapper39.invokeMethod(Wrapper39.java)
	at org.apache.dubbo.rpc.proxy.javassist.JavassistProxyFactory$1.doInvoke(JavassistProxyFactory.java:47)
	at org.apache.dubbo.rpc.proxy.AbstractProxyInvoker.invoke(AbstractProxyInvoker.java:84)
	at org.apache.dubbo.config.invoker.DelegateProviderMetaDataInvoker.invoke(DelegateProviderMetaDataInvoker.java:56)
	at org.apache.dubbo.rpc.protocol.InvokerWrapper.invoke(InvokerWrapper.java:56)
	at com.alibaba.dubbo.rpc.Invoker$CompatibleInvoker.invoke(Invoker.java:55)
	at io.seata.integration.dubbo.alibaba.AlibabaDubboTransactionPropagationFilter.invoke(AlibabaDubboTransactionPropagationFilter.java:45)
	at com.alibaba.dubbo.rpc.Filter.invoke(Filter.java:29)
	at org.apache.dubbo.rpc.protocol.FilterNode.invoke(FilterNode.java:61)
	at org.apache.dubbo.rpc.protocol.FilterNode.invoke(FilterNode.java:61)
	at org.apache.dubbo.rpc.protocol.dubbo.filter.TraceFilter.invoke(TraceFilter.java:77)
	at org.apache.dubbo.rpc.protocol.FilterNode.invoke(FilterNode.java:61)
	at org.apache.dubbo.rpc.filter.TimeoutFilter.invoke(TimeoutFilter.java:46)
	at org.apache.dubbo.rpc.protocol.FilterNode.invoke(FilterNode.java:61)
	at org.apache.dubbo.seata.SeataTransactionPropagationProviderFilter.invoke(SeataTransactionPropagationProviderFilter.java:66)
	at org.apache.dubbo.rpc.protocol.FilterNode.invoke(FilterNode.java:61)
	at org.apache.dubbo.monitor.support.MonitorFilter.invoke(MonitorFilter.java:91)
	at org.apache.dubbo.rpc.protocol.FilterNode.invoke(FilterNode.java:61)
	at org.apache.dubbo.rpc.protocol.FilterNode.invoke(FilterNode.java:61)
	at org.apache.dubbo.rpc.filter.ExceptionFilter.invoke(ExceptionFilter.java:52)
	at org.apache.dubbo.rpc.protocol.FilterNode.invoke(FilterNode.java:61)
	at org.apache.dubbo.rpc.protocol.FilterNode.invoke(FilterNode.java:61)
	at org.apache.dubbo.rpc.filter.GenericFilter.invoke(GenericFilter.java:192)
	at org.apache.dubbo.rpc.protocol.FilterNode.invoke(FilterNode.java:61)
	at org.apache.dubbo.rpc.filter.ClassLoaderFilter.invoke(ClassLoaderFilter.java:38)
	at org.apache.dubbo.rpc.protocol.FilterNode.invoke(FilterNode.java:61)
	at org.apache.dubbo.rpc.filter.EchoFilter.invoke(EchoFilter.java:41)
	at org.apache.dubbo.rpc.protocol.FilterNode.invoke(FilterNode.java:61)
	at org.apache.dubbo.rpc.filter.ContextFilter.invoke(ContextFilter.java:129)
	at org.apache.dubbo.rpc.protocol.FilterNode.invoke(FilterNode.java:61)
	at org.apache.dubbo.rpc.protocol.dubbo.DubboProtocol$1.reply(DubboProtocol.java:148)
	at org.apache.dubbo.remoting.exchange.support.header.HeaderExchangeHandler.handleRequest(HeaderExchangeHandler.java:100)
	at org.apache.dubbo.remoting.exchange.support.header.HeaderExchangeHandler.received(HeaderExchangeHandler.java:175)
	at org.apache.dubbo.remoting.transport.DecodeHandler.received(DecodeHandler.java:51)
	at org.apache.dubbo.remoting.transport.dispatcher.ChannelEventRunnable.run(ChannelEventRunnable.java:57)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at org.apache.dubbo.common.threadlocal.InternalRunnable.run(InternalRunnable.java:41)
	at java.lang.Thread.run(Thread.java:745)
Caused by: java.sql.SQLException: io.seata.core.exception.RmTransactionException: RPC Timeout
	at io.seata.rm.datasource.ConnectionProxy.recognizeLockKeyConflictException(ConnectionProxy.java:161)
	at io.seata.rm.datasource.ConnectionProxy.processGlobalTransactionCommit(ConnectionProxy.java:252)
	at io.seata.rm.datasource.ConnectionProxy.doCommit(ConnectionProxy.java:230)
	at io.seata.rm.datasource.ConnectionProxy.lambda$commit$0(ConnectionProxy.java:188)
	at io.seata.rm.datasource.ConnectionProxy$LockRetryPolicy.doRetryOnLockConflict(ConnectionProxy.java:356)
	at io.seata.rm.datasource.ConnectionProxy$LockRetryPolicy.execute(ConnectionProxy.java:348)
	at io.seata.rm.datasource.ConnectionProxy.commit(ConnectionProxy.java:187)
	at org.springframework.jdbc.datasource.DataSourceTransactionManager.doCommit(DataSourceTransactionManager.java:329)
	... 51 common frames omitted
Caused by: io.seata.core.exception.RmTransactionException: RPC Timeout
	at io.seata.rm.AbstractResourceManager.branchRegister(AbstractResourceManager.java:73)
	at io.seata.rm.DefaultResourceManager.branchRegister(DefaultResourceManager.java:96)
	at io.seata.rm.datasource.ConnectionProxy.register(ConnectionProxy.java:273)
	at io.seata.rm.datasource.ConnectionProxy.processGlobalTransactionCommit(ConnectionProxy.java:250)
	... 57 common frames omitted
Caused by: java.util.concurrent.TimeoutException: null ,cost: 15000 ms
	at io.seata.core.protocol.MessageFuture.get(MessageFuture.java:65)
	at io.seata.core.rpc.netty.AbstractNettyRemotingClient.sendSyncRequest(AbstractNettyRemotingClient.java:174)
	at io.seata.rm.AbstractResourceManager.branchRegister(AbstractResourceManager.java:67)
	... 60 common frames omitted

RPC Timeout，调用超时了。

8万条数据，每1千条一次 insert 的话，需要执行 80 次，也不至于15秒还执行不下来吧？

抱着对seata存疑的态度，将 @GlobalTransactional 去掉了，不要分布式事务了。结果整个接口4秒就执行完了。

可见seata肯定干了什么特别耗时的事情。具体哪里耗的时，暂且不跟了，反正记住这个用法就得了：一条 insert 语句插入多条数据的时候，尤其是可能数据量很大，需要在一次事务中多次执行 insert 时，不要加分布式事务。