- 环境
spring 4.3.13
Druid 链接池1.1.0
mysql 5.1.41
mybatis 3.4.6
1.spring-test简介
1.1spring-test类图
整个spring-test交互流程分为三部分(对应上图三种颜色):
1.测试启动,构建spring容器,并将applicationContext注入到TestContext,构造测试上下文容器
2.TestContextManager从spring容器中获取数据源事务管理器DataSourceTransactionManager(配置多数据源的时候,如果没有特别申明会注入默认的数据源)
3.spring-test手动开启一个事务,执行用户测试用例(事务操作参考Mybatis执行流程),spring-test手动关闭事务(根据TransactionInfo中记录的sql列表对事务中的数据库操作进行回滚,避免单测对数据库造成污染)
1.2简单的流程示意图
2.springTest配置多数据源导致事务无法回滚
在重构大迁移的背景下,我们初步在A工程接入了新老两个数据源(请不要吐槽一个工程里面配多个数据源,手动狗头)。简单的示例如下:
新数据源配置–可略过不看
/**
* 新数据源
@author vincilovfang
*/
@Configuration
@MapperScan(basePackages = "com.spring.test", sqlSessionTemplateRef = "newSqlSessionTemplate")
public class NewDataSourceConfig {
@Value("${newJdbc.url}")
private String url;
@Value("${newJdbc.username}")
private String username;
@Value("${newJdbc.password}")
private String password;
@Bean(name = "newDataSource")
public DataSource buildDataSource() {
DruidDataSource dataSource = new DruidDataSource();
dataSource.setUrl(url);
dataSource.setUsername(username);
dataSource.setPassword(password);
// set其他属性
return dataSource;
}
@Bean(name = "newSqlSessionFactory")
public SqlSessionFactory buildSqlSessionFactory(
@Qualifier("newDataSource") DataSource dataSource) {
SqlSessionFactoryBean bean = new SqlSessionFactoryBean();
bean.setDataSource(dataSource);
//...set其他属性
}
@Bean(name = "newSqlSessionTemplate")
public SqlSessionTemplate buildSqlSessionTemplate(
@Qualifier("newSqlSessionFactory") SqlSessionFactory sqlSessionFactory) {
return new SqlSessionTemplate(sqlSessionFactory);
}
@Bean(name = "newTransactionManager")
public DataSourceTransactionManager buildTransactionManager(
@Qualifier("newDataSource") DataSource dataSource) {
return new DataSourceTransactionManager(dataSource);
}
}
旧数据源配置,这个地方必须将新老数据源中的一个指定为优先项,否则spring启动会报错。为避免影响已有功能,这里暂时将旧数据源设为首选项
No qualifying bean of type 'javax.sql.DataSource' available: expected single matching bean but found 2: newDataSource,oldDataSource
/**
* 旧数据源
@author vincilovfang
*/
@Configuration
@MapperScan(basePackages = "com.spring.test", sqlSessionTemplateRef = "oldSqlSessionTemplate")
public class OldDataSourceConfig {
@Value("${jdbc.url}")
private String url;
@Value("${jdbc.username}")
private String username;
@Value("${jdbc.password}")
private String password;
@Bean(name = "oldDataSource")
@Primary
public DataSource buildDataSource() {
DruidDataSource dataSource = new DruidDataSource();
dataSource.setUrl(url);
dataSource.setUsername(username);
dataSource.setPassword(password);
//... set其他属性
return dataSource;
}
@Bean(name = "oldSqlSessionFactory")
@Primary
public SqlSessionFactory buildSqlSessionFactory(@Qualifier("oldDataSource") DataSource dataSource) {
SqlSessionFactoryBean bean = new SqlSessionFactoryBean();
bean.setDataSource(dataSource);
//...set其他属性
}
@Bean(name = "oldSqlSessionTemplate")
@Primary
public SqlSessionTemplate buildSqlSessionTemplate(
@Qualifier("oldSqlSessionFactory") SqlSessionFactory sqlSessionFactory) {
return new SqlSessionTemplate(sqlSessionFactory);
}
@Bean(name = "oldTransactionManager")
@Primary
public DataSourceTransactionManager buildTransactionManager(
@Qualifier("oldDataSource") DataSource dataSource) {
return new DataSourceTransactionManager(dataSource);
}
}
单测示例–DemoDO对应新数据源里面的数据表
@RunWith(SpringJUnit4ClassRunner.class)
@SpringBootTest(classes = SpringBootStarter.class)
public class NoRollbackDemoTest extends MockitoTimorTestBase {
@Resource
private DemoDOMapper demoDOMapper;
@Test
public void testDemo() {
DemoDO demoDO = createData(DemoDO.class);
demoDO.setCpId(2341233453L);
demoDOMapper.insertDemo(demoDO);
Demo demo = demoRepository.getDemo(2341233453L);
Assert.assertEquals(demoDO.getCpId(), demo.getCpId());
}
}
2.1.springTest默认事物回滚,但数据库里面数据并未回滚
2.2.跟踪日志也显示回滚
[main:TransactionContext.java:139] _am||traceid=||spanid=||Rolled back transaction for test context [DefaultTestContext@143640d5 testClass = NoRollbackDemoTest, testInstance = com.spring.test.xxx.infrastructure.persistence.NoRollbackDemoTest@6d0fe80c, testMethod = testNoRollbackCase@NoRollbackDemoTest, testException = [null], mergedContextConfiguration = [WebMergedContextConfiguration@6295d394 testClass = NoRollbackDemoTest, locations = '{}'...
看到数据库里面的脏数据第一反应是懵逼的🙃,日志不会说谎,数据库脏数据也是存在的。根据日志提示,追踪TransactionContext的源码,在springTest开始之前、之后,分别会执行startTransaction、endTransaction
2.3.开启回滚–TransactionContext
### TransactionContext
void startTransaction() {
if (this.transactionStatus != null) {
throw new IllegalStateException(
"Cannot start a new transaction without ending the existing transaction first.");
}
this.flaggedForRollback = this.defaultRollback;
this.transactionStatus = this.transactionManager.getTransaction(this.transactionDefinition);
++this.transactionsStarted;
if (logger.isInfoEnabled()) {
logger.info(String.format(
"Began transaction (%s) for test context %s; transaction manager [%s]; rollback [%s]",
this.transactionsStarted, this.testContext, this.transactionManager, flaggedForRollback));
}
}
void endTransaction() {
if (logger.isTraceEnabled()) {
logger.trace(String.format(
"Ending transaction for test context %s; transaction status [%s]; rollback [%s]",
this.testContext, this.transactionStatus, this.flaggedForRollback));
}
if (this.transactionStatus == null) {
throw new IllegalStateException(String.format(
"Failed to end transaction for test context %s: transaction does not exist.", this.testContext));
}
try {
if (this.flaggedForRollback) {
this.transactionManager.rollback(this.transactionStatus);
}
else {
this.transactionManager.commit(this.transactionStatus);
}
}
finally {
this.transactionStatus = null;
}
if (logger.isInfoEnabled()) {
logger.info(String.format("%s transaction for test context %s.",
(this.flaggedForRollback ? "Rolled back" : "Committed"), this.testContext));
}
}
继续走查源码类时序图如图4
2.4.执行回滚–DruidPooledConnection
### DruidPooledConnection
public void rollback() throws SQLException {
if (transactionInfo == null) {
return;
}
if (holder == null) {
return;
}
DruidAbstractDataSource dataSource = holder.getDataSource();
dataSource.incrementRollbackCount();
try {
conn.rollback();
} catch (SQLException ex) {
handleException(ex);
} finally {
handleEndTransaction(dataSource, null);
}
}
发现在在DruidPooledConnection中 transactionInfo为空,事务信息为空,所以导致未真实回滚。google了下transactionInfo
为空的case,https://github.com/alibaba/druid/issues/1635,链接是druid论坛小伙伴的一些回答。博主的答案有点概括,看了之后也不是太明白(只能怪自己bug写多了,人变傻了,理解能力也变差了,再次手动狗头)
2.5.transactionInfo
设置transactionInfo
的地方只有一处,即通过connection执行sql的时候会对事务进行记录。
### DruidPooledConnection
protected void transactionRecord(String sql) throws SQLException {
if (transactionInfo == null && (!conn.getAutoCommit())) {
DruidAbstractDataSource dataSource = holder.getDataSource();
dataSource.incrementStartTransactionCount();
transactionInfo = new TransactionInfo(dataSource.createTransactionId());
}
if (transactionInfo != null) {
List<String> sqlList = transactionInfo.getSqlList();
if (sqlList.size() < MAX_RECORD_SQL_COUNT) {
sqlList.add(sql);
}
}
}
代码中conn的autoCommit属性被设置成了true,connection如下。
而在TransactionContext开启事务的时候connection如下:
一个为DruidPooledConnection@12036
,一个为DruidPooledConnection@11838
,两个DruidPooledConnection
不同,所以springTest的环绕切面无法对事务进行回滚。
2.6.connection创建
现在的问题是为什么TransactionContext.startTransaction中的conn和单测执行中的conn不是一个。接下来要做的是确定在TransactionContext和单测中,connection分别是怎么创建的。
TransactionContext.startTransaction获取connection流程如下
单测中,通过代码执行栈信息分析代码逻辑执行的时候是如何获取DruidPooledConnection
,这里的主要执行流程即为Mybatis执行时序图
其中mybatis中mapperProxy中记录了每个sql执行对应的数据源信息,从而找到对应的数据源进行数据库操作。
根据debug信息栈发现,在SqlSessionTemplate中没有Connection信息,但是在SqlSessionInterceptor中已经存在了(debug图中标红圈部分)
根据栈信息能看出connection由SpringManagedTransaction持有,继续跟踪SpringManagedTransaction源码查看connection的创建
### SpringManagedTransaction
private void openConnection() throws SQLException {
this.connection = DataSourceUtils.getConnection(this.dataSource);
this.autoCommit = this.connection.getAutoCommit();
this.isConnectionTransactional = DataSourceUtils.isConnectionTransactional(this.connection, this.dataSource);
if (LOGGER.isDebugEnabled()) {
LOGGER.debug(
"JDBC Connection ["
+ this.connection
+ "] will"
+ (this.isConnectionTransactional ? " " : " not ")
+ "be managed by Spring");
}
}
connetciton是通过dataSource获取的,由于单测的DemoDO在新数据源中,这里的this.dataSource
为新数据源(mybatis的源头mapperProxy会记录每条sql需要的数据源),进一步跟踪源码我们找到是通过TransactionSynchronizationManager里面的resource获取connectionHolder
### TransactionSynchronizationManager
private static final ThreadLocal<Map<Object, Object>> resources = new NamedThreadLocal<>("Transactional resources");
private static Object doGetResource(Object actualKey) {
Map<Object, Object> map = resources.get();
if (map == null) {
return null;
}
Object value = map.get(actualKey);
// Transparently remove ResourceHolder that was marked as void...
if (value instanceof ResourceHolder && ((ResourceHolder) value).isVoid()) {
map.remove(actualKey);
// Remove entire ThreadLocal if empty...
if (map.isEmpty()) {
resources.remove();
}
value = null;
}
return value;
}
debug发现resources这个里面的记录的是旧数据源信息,所以返回connection为空,便新创建了一个Connection。
到这里我们基本清楚了,TransactionContext用的是旧数据源创建的连接(spring依赖注入优先注入了旧数据源),而单测中用的是新数据源创建的连接,所以TransactionContext无法对单测进行回滚。
resources的初次设置代码如下
DataSourceTransactionManager设置了datasource信息,聪明的你可能马上想到,DataSourceTransactionManager是我们自己在代码中配置的。
我们把OldDataSourceTransactionManager的优先级设置成了@Primary这才导致TransactionContext用的是OldDataSourceTransactionManager来管理事务。现在我们只需要把TransactionContext的事务管理器设置成NewDataSourceTransactionManager即可。
2.7.最终的单测代码如下
@RunWith(SpringJUnit4ClassRunner.class)
@SpringBootTest(classes = SpringBootStarter.class)
@Transactional(transactionManager = "newDataSourceTransactionManager")
public class NoRollbackDemoTest extends MockitoTimorTestBase {
@Resource
private DemoDOMapper demoDOMapper;
@Test
public void testDemo() {
DemoDO demoDO = createData(DemoDO.class);
demoDO.setCpId(2341233453L);
demoDOMapper.insertDemo(demoDO);
Demo demo = demoRepository.getDemo(2341233453L);
Assert.assertEquals(demoDO.getCpId(), demo.getCpId());
}
}