问题背景
需求特殊哈,需要从不同的数据源迁移数据,因为涉及到迁移的数据量比较大,故想使用下Spring batch来做,听说它的功能非常强大,于是就踩坑了,被按在地上摩擦了好几天哈哈,最后终于找到了好的解决方案。因国内关于此类问题的文章并不多, 故记录一下!
心路历程
- 使用AI chatGPT ,chatGPT给的答案真的只能当做参考,误导性比较大,特别是对于这种不太常用的技术框架;
- 百度搜索,基本上没有可参考的答案;
- 阅读官方参考文档 : 各版本地址 查看自己使用版本的参考文档就行;
- 啃源码;
问题原因
首先介绍下Spring batch的相关组件:
LZ的问题原因在于读取数据时因数据源数据连接不稳定的从而导致数据库连接断掉,默认Spring batch 捕获到该异常时认为该异常是不可恢复的,所以它会抛出一个它自己定义的致命的异常,即为 NonSkippableReadException ,完整的异常如下:
<#9f70370a> com.mysql.cj.exceptions.ConnectionIsClosedException: No operations allowed after connection closed.
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(NativeConstructorAccessorImpl.java)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at com.mysql.cj.exceptions.ExceptionFactory.createException(ExceptionFactory.java:61)
at com.mysql.cj.exceptions.ExceptionFactory.createException(ExceptionFactory.java:105)
at com.mysql.cj.exceptions.ExceptionFactory.createException(ExceptionFactory.java:151)
at com.mysql.cj.NativeSession.checkClosed(NativeSession.java:1171)
at com.mysql.cj.jdbc.ConnectionImpl.checkClosed(ConnectionImpl.java:576)
at com.mysql.cj.jdbc.ConnectionImpl.setNetworkTimeout(ConnectionImpl.java:2486)
... 81 common frames omitted
Wrapped by: <#7b93c319> java.sql.SQLNonTransientConnectionException: No operations allowed after connection closed.
at com.mysql.cj.jdbc.exceptions.SQLError.createSQLException(SQLError.java:110)
at com.mysql.cj.jdbc.exceptions.SQLError.createSQLException(SQLError.java:97)
at com.mysql.cj.jdbc.exceptions.SQLError.createSQLException(SQLError.java:89)
at com.mysql.cj.jdbc.exceptions.SQLError.createSQLException(SQLError.java:63)
at com.mysql.cj.jdbc.exceptions.SQLError.createSQLException(SQLError.java:73)
at com.mysql.cj.jdbc.exceptions.SQLExceptionsMapping.translateException(SQLExceptionsMapping.java:73)
at com.mysql.cj.jdbc.ConnectionImpl.setNetworkTimeout(ConnectionImpl.java:2490)
at com.zaxxer.hikari.pool.PoolBase.setNetworkTimeout(PoolBase.java:550)
at com.zaxxer.hikari.pool.PoolBase.isConnectionAlive(PoolBase.java:165)
at com.zaxxer.hikari.pool.HikariPool.getConnection(HikariPool.java:179)
... 78 common frames omitted
Wrapped by: <#f9ccd589> java.sql.SQLTransientConnectionException: HikariPool-2 - Connection is not available, request timed out after 30039ms.
at com.zaxxer.hikari.pool.HikariPool.createTimeoutException(Hika