线上发现一个问题,应用在启动时会卡死,log上并没有什么异常输出,初判应该是死锁问题.
抓现场的thread dump文件, 确实是有两个线程有deadlock问题.
线程一
"HSFBizProcessor-8-thread-13" daemon prio=10 tid=0x00007fc686a83000 nid=0x37128 waiting for monitor entry [0x000000004b7f3000]
java.lang.Thread.State: BLOCKED (on object monitor)
at org.springframework.beans.factory.support.DefaultListableBeanFactory.getBeanDefinitionNames(DefaultListableBeanFactory.java:192)
- waiting to lock <0x00000007707d84c8> (a java.util.concurrent.ConcurrentHashMap)
at org.springframework.beans.factory.support.DefaultListableBeanFactory.getBeanNamesForType(DefaultListableBeanFactory.java:209)
at org.springframework.beans.factory.BeanFactoryUtils.beanNamesForTypeIncludingAncestors(BeanFactoryUtils.java:187)
at org.springframework.beans.factory.support.DefaultListableBeanFactory.findAutowireCandidates(DefaultListableBeanFactory.java:652)
at org.springframework.beans.factory.support.DefaultListableBeanFactory.resolveDependency(DefaultListableBeanFactory.java:610)
at org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor$AutowiredFieldElement.inject(AutowiredAnnotationBeanPostProcessor.java:412)
at org.springframework.beans.factory.annotation.InjectionMetadata.injectFields(InjectionMetadata.java:105)
at
线程二
"main":
at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:180)
- waiting to lock <0x00000007707ae6c0> (a java.util.concurrent.ConcurrentHashMap)
at org.springframework.beans.factory.support.AbstractBeanFactory.isFactoryBean(AbstractBeanFactory.java:747)
at org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:422)
- locked <0x00000007707d84c8> (a java.util.concurrent.ConcurrentHashMap)
at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:728)
at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:380)
- locked <0x00000007707d7fc8> (a java.lang.Object)
at org.springframework.web.context.ContextLoader.createWebApplicationContext(ContextLoader.java:255)
at org.springframework.web.context.ContextLoader.initWebApplicationContext(ContextLoader.java:199)
栈文件的结尾已经指出了两个线程在竞争什么锁,
which is held by "main"
"main":
waiting to lock monitor 0x00007fc681220a08 (object 0x00000007707ae6c0, a java.util.concurrent.ConcurrentHashMap),
which is held by "HSFBizProcessor-8-thread-13"
"HSFBizProcessor-8-thread-13":
waiting to lock monitor 0x00007fc686692438 (object 0x00000007707d84c8, a java.util.concurrent.ConcurrentHashMap),
which is held by "main"
主要是线程HSFBizProcessor的DefaultListableBeanFactory.getBeanDefinitionNames(DefaultListableBeanFactory.java:192)
需要锁对象0x00000007707d84c8, 而这个对象已经被main的DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:422)
锁住了.
spring的版本是2.5.
这个问题的原因是spring在初始化singleton bean的时候,需要锁两个地方,第一个是
DefaultSingletonBeanRegistry中的singletonObjects:
/** Cache of singleton objects: bean name --> bean instance */
private final Map singletonObjects = CollectionFactory.createConcurrentMapIfPossible(16);
...
synchronized (this.singletonObjects) {
Object singletonObject = this.singletonObjects.get(beanName);
if (singletonObject == null) {
...
try {
singletonObject = singletonFactory.getObject();
}
singletonObjects是DefaultSingletonBeanRegistry中的并发Map.
第二步需要锁DefaultListableBeanFactory中的beanDefinitionMap:
/** Map of bean definition objects, keyed by bean name */
private final Map beanDefinitionMap = CollectionFactory.createConcurrentMapIfPossible(16);
...
public String[] getBeanDefinitionNames() {
synchronized (this.beanDefinitionMap) {
if (this.frozenBeanDefinitionNames != null) {
return this.frozenBeanDefinitionNames;
}
...
}
}
如果在应用启动时只有一个线程进入spring初始化bean时是没问题的, 但这里应用代码在spring的容器启动的同时,有另外一个main方法同时开始运行调用spring的DefaultListableBeanFactory.preInstantiateSingletons方法,两个线程两把锁,是有可能造成饥饿竞争的.
在spring容器外自行调用spring的创建bean方法要注意线程问题.
引申阅读:
Spring Bean Creation is Not Thread Safe
A Java Thread deadlock has occured
Performance bottleneck and potential thread deadlock in DefaultSingletonBeanRegistry
Spring deadlocks between DefaultListableBeanFactory and DefaultSingletonBeanRegistry