背景:
项目上线启动,发生CPU占满的问题
定位问题
- 积累线上日志排查,发现问题并快速定位问题的能力
按照日志显示,发现在项目有依赖使用tk.mybatis二方库中的tk.mybatis.mapper.mapperhelper.MapperInterceptor, 使用的maven依赖如下:
<dependency>
<groupId>org.mybatis</groupId>
<artifactId>mybatis</artifactId>
<version>3.3.0</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>tk.mybatis</groupId>
<artifactId>mapper</artifactId>
<version>3.1.2</version>
<scope>test</scope>
</dependency>
在MapperHelper这个类中,使用了如下定义:
/**
* 缓存skip结果
*/
private final Map<String, Boolean> msIdSkip = new HashMap<String, Boolean>();
其作用是使用map缓存需要拦截处理的mapper方法,但是在并发情况下,存在并发问题,导致cpu占满的情况。
找到tk.mybatis.mapper.mapperhelper.MapperIntorceptor.intercept()方法,定位到在mapperHelper.isMapperMethod()中 使用HashMap,并发条件下导致HashMap死循环。
HashMap死循环的问题
参考网上一篇文章,感兴趣的可以阅读理解 https://coolshell.cn/articles/9606.html
解决方案
github开源代码: https://github.com/abel533/Mapper
- 常规解决方案
- 联系作者,询问是否有针对的fix版本,得到作者将在下一新版本中修复这个并发问题的答复后,查看作者更新记录,最新版本与这一版差别很大,在3.2.x之后,作者在tk中去掉了MapperInterceptor,改用tkxxxConfiguration替代这个功能,不能直接升级到最新版,目前没有能够直接使用的修复版本。
- 在作者的3.1.2开发分支拉取一个开发分支,修复成threadsafe,修复此问题;
存在的问题是限定了例如3.1.2.x这样一个版本,修改版本号之后还存在此问题,同时作者开发的新功能,新版本的功能无法使用,作者的修改对此版本来说是未知的,我们要与作者的新功能保持同步是一件很麻烦的事情。
- 最务实的解决方案
- 查看使用此框架的地方,确认使用的范围,去掉此框架的使用;
存在的问题是需要找到使用次框架的范围,代码改造并替换框架的功能,并需要全面测试,成本比较大且风险较高。
- 比较优雅的改造方法
- 利用反射修改框架代码,学习使用hack精神,遵循开闭原则,尽可能少的修改代码和原有配置,尽可能的降低对框架版本的依赖。
Hack是基于开源的程序的基础,对其代码进行增加、删除或者修改、优化,使之在功能上符合新的需求。
针对此问题,使用hack的方式解决是极好的,是一个很好的切入点。
/**
* <p>An instance of this class is used to fix tk.mybatis.mapper.mapperhelper.
* MapperInterceptor.mapperHelper.msIdSkip(HashMap<String, boolean> threads not threadSafe)
* through reflect to change msIdSkip.HashMap to msIdSkip.ConcurrentHashMap for threadSafe </p>
*
* <p>The purpose of this class is designed to fix the thread unsafe problems through the way of hack,
* framework to modify the source code for as little as possible, and at the same time as little as possible
* to modify the project configuration, better use of the open-closed principle, reduce dependence on tk</p>
*
* <p>if you want to use this function, you need to add an bean instance of this class at your
* spring.xml,{@link <bean class="TkMapperInterceptorHacker" /> }
* and you don't need to dependency tk.mybatis </p>
*
* @author liuyong
* 2017-08-28
*/
此处第一次提交时没有增加注释,后来补上了hack开发增加hack的背景、解决什么问题、原因、方案和使用方法,方便其他人使用和理解,同时增加自己的被知名度。 框架代码使用英文注释,不要出现中英文混用的情况,同时要去掉警告代码
class TkMapperInterceptorHacker extends InstantiationAwareBeanPostProcessorAdapter {
自定义BeanPostProcessor,在bean加载过程中对bean拦截进行操作。 类名使用TkMapperInterceptorHacker比TkMapperInterceptorHack更好一些。
private static final Logger LOGGER = LoggerFactory.getLogger(TkMapperInterceptorHacker.class);
定义Logger,final static变量名需要大写
// interface method is invoked after the bean is instantiated
最开始这里使用的是中文注释,框架代码中不要出现中英文混合的注释
@Override
public Object postProcessAfterInitialization(Object bean, String beanName){
String beanClassName = bean.getClass().getName();
if (beanClassName.contains("tk.mybatis.mapper.mapperhelper.MapperInterceptor")) {
此处判断bean为我们需要拦截处理的bean时,有以下几点收获:
- 使用指定的beanName,eg:if("mapperInterceptor".equles(beanName)){}; 存在的问题是beanName是多变的,如果注册该bean的beanName不是mapperIntercptor则会失效。
- 使用if(bean instanceof MapperInterceptor){}; 存在的问题是需要依赖tk.mybatis二方库,并且需要在该版本中找到MapperInterceptor类,例如这个具体问题,在3.2.x之后没有MapperInterceptor类了,那么升级到3.2.x之后的版本则会导致找不到类的错误,导致不能使用,使用3.2.x之下的版本同样不能使用开发的新功能,对版本依赖严重。
- 使用if("tk.mybatis.mapper.mapperhelper.MapperInterceptor".equles(bean.getClass().getName())){}; 存在的问题是如果bean对应的类是代理类,则其全类名是tk.mybatis.mapper.mapperhelper.MapperInterceptor.$xxx.$x..x的情况,这样会导致匹配失败。
- 使用if (beanClassName.contains("tk.mybatis.mapper.mapperhelper.MapperInterceptor")){},存在的问题是会扩大匹配范围,例如xxx.tk.mybatis.mapper.mapperhelper.MapperInterceptor.* 都会被匹配上。不过此例可以使用
- 使用if (beanClassName.startWith("tk.mybatis.mapper.mapperhelper.MapperInterceptor")){}, 这样能够精确匹配。
try {
LOGGER.info("TkMapperInterceptorHacker invoke postProcessAfterInitialization method reload bean:" + beanName);
Field mapperHelperFiled = bean.getClass().getDeclaredField("mapperHelper");
//使用setAccessible(true)修改private final不可操作的特性
mapperHelperFiled.setAccessible(true);
Field msIdSkipField = mapperHelperFiled.get(bean).getClass().getDeclaredField("msIdSkip");
//此处使用属性object反射获取类,操作类属性。没有必要现获取到具体的对象在反射操作类属性,这样对具体的类还有依赖。
msIdSkipField.setAccessible(true);
//修改MapperHelper.msIdSkip属性的HashMap为ConcurrentHashMap属性
msIdSkipField.set(mapperHelperFiled.get(bean), new ConcurrentHashMap<>());
} catch (Exception e) {
throw new RuntimeException(e);
//此处应该遵循fast-fail原则,快速失败,但是不能System.exit(),属于野蛮的做法。利用spring的异常捕获,抛出异常,让容器启动失败即可。
}
}
//父类的操作,需要根据父类方法的实现判断是否需要调用,避免父类方法做一些我们未知的操作
return super.postProcessAfterInitialization(bean, beanName);
}
}
补充单元测试
/**
* <P>test class methods keys
* every testCase need follow next rules:
* 1、begin check :self check
* 2、 post : do test method
* 3、check again : check result
* </P>
* Created by yehao on 2017/8/25.
*/
public class TkMapperHackTest {
private static final Logger LOGGER = LoggerFactory.getLogger(TkMapperHackTest.class);
//TestCase1: not the bean we won't to deal, it should return the bean no change
@Test
public void testBeanPostProcessorNotTheBean() {
Object bean = new Object();
Field mapperHelper ;
try {
mapperHelper = bean.getClass().getDeclaredField("mapperHelper");
Assert.assertNull(mapperHelper);
} catch (Exception e) {
Assert.assertEquals(NoSuchFieldException.class, e.getClass());
LOGGER.info("bean self check ok, it should not to change");
}
List<Map<String, Object>> list ;
try {
list = compareTwoClass(new Object(), new TkMapperInterceptorHacker().postProcessAfterInitialization(bean, "mapperInterceptor"));
Assert.assertEquals(0, list.size());
LOGGER.info("bean ckeck again ok, fields no change");
} catch (IllegalAccessException e) {
Assert.fail("bean check again error"+ e);
}
}
/**
* test the bean which we want to deal by post
*/
@Test
public void testBeanPostProcessor() {
MapperInterceptor bean = new MapperInterceptor();
try {
Field mapperHelperFiled = bean.getClass().getDeclaredField("mapperHelper");
mapperHelperFiled.setAccessible(true);
Field msIdSkipField = mapperHelperFiled.get(bean).getClass().getDeclaredField("msIdSkip");
msIdSkipField.setAccessible(true);
Assert.assertEquals(HashMap.class, msIdSkipField.get(mapperHelperFiled.get(bean)).getClass());
LOGGER.info("bean self ckeck ok, fields class is we want");
} catch (Exception e) {
Assert.fail("bean self check error"+ e);
}
new TkMapperInterceptorHacker().postProcessAfterInitialization(bean, "mapperInterceptor");
try {
Field mapperHelperFiled = bean.getClass().getDeclaredField("mapperHelper");
mapperHelperFiled.setAccessible(true);
Field msIdSkipField = mapperHelperFiled.get(bean).getClass().getDeclaredField("msIdSkip");
msIdSkipField.setAccessible(true);
Assert.assertEquals(ConcurrentHashMap.class, msIdSkipField.get(mapperHelperFiled.get(bean)).getClass());
LOGGER.info("bean ckeck again ok, fields class is we want after post deal");
} catch (Exception e) {
Assert.fail("bean self check error"+ e);
}
}
}
针对以上的代码框架,可以解决CPU飙升的问题,然而此版本存在另一个问题是如果tk.MapperHelper是使用的代理类,怎会出现并没有修改msIdSkip的类,为了兼容代理类,修改TkMapperInterceptorHacker类代码如下:
package com.helijia.framework.hack.tk;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.beans.factory.config.InstantiationAwareBeanPostProcessorAdapter;
import org.springframework.util.StringUtils;
import java.lang.reflect.Field;
import java.util.concurrent.ConcurrentHashMap;
/**
* <p>An instance of this class is used to fix tk.mybatis.mapper.mapperhelper.
* MapperInterceptor.mapperHelper.msIdSkip(HashMap<String, boolean> threads not threadSafe)
* through reflect to change msIdSkip.HashMap to msIdSkip.ConcurrentHashMap for threadSafe </p>
* <p>
* <p>The purpose of this class is designed to fix the thread unsafe problems through the way of hack,
* framework to modify the source code for as little as possible, and at the same time as little as possible
* to modify the project configuration, better use of the open-closed principle, reduce dependence on tk</p>
* <p>
* <p>if you want to use this function, you need to add an bean instance of this class at your
* spring.xml,{@link <bean class="TkMapperInterceptorHacker" /> }
* and you don't need to dependency tk.mybatis </p>
* <p>
* <p>This class don't fit for {@code JDKDynamic} proxy class, if class is the agent of JDKDynamic, this class won't make effects</p>
*
* @author liuyong
* 2017-08-28
*/
class TkMapperInterceptorHacker extends InstantiationAwareBeanPostProcessorAdapter {
private static final Logger LOGGER = LoggerFactory.getLogger(TkMapperInterceptorHacker.class);
/**
* interface method is invoked after the bean is instantiated
*
* @param bean
* @param beanName
* @return
*/
@Override
public Object postProcessAfterInitialization(Object bean, String beanName) {
String beanClassName = bean.getClass().getName();
if (!StringUtils.isEmpty(beanClassName) && beanClassName.contains("tk.mybatis.mapper.mapperhelper.MapperInterceptor")) {
try {
LOGGER.info("TkMapperInterceptorHacker invoke postProcessAfterInitialization method reload bean:" + beanName);
exceClass(bean, bean.getClass());
} catch (Exception e) {
throw new RuntimeException(e);
}
}
return super.postProcessAfterInitialization(bean, beanName);
}
private void exceClass(Object bean, Class clazz) throws Exception {
if (null == clazz || clazz == Object.class) {
return;
}
boolean isMatch = false;
Field[] fields = clazz.getDeclaredFields();
for (final Field field : fields) {
if ("mapperHelper".equals(field.getName())) {
field.setAccessible(true);
Field msIdSkipField = field.get(bean).getClass().getDeclaredField("msIdSkip");
msIdSkipField.setAccessible(true);
msIdSkipField.set(field.get(bean), new ConcurrentHashMap<>());
isMatch = true;
break;
}
}
if (!isMatch) {
exceClass(bean, clazz.getSuperclass());
}
}
}
github地址
https://github.com/yehaoSource/hack