简介: 最近遇到线上机器连接池满的情况,排查发现tddl里面在连接池获取和释放流程中有些问题
问题现象:
线上报HSF线程池满,CPU100%。
排查过程:
线程日志dump下来进行分析
Hsf等待线程1:大约400个线程等待获取连接池
at sun.misc.Unsafe.park(Native Method) - waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@22630f9b at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:233) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at com.alibaba.druid.pool.DruidDataSource.pollLast(DruidDataSource.java:1609) at com.alibaba.druid.pool.DruidDataSource.getConnectionInternal(DruidDataSource.java:1172) at com.alibaba.druid.pool.DruidDataSource.getConnectionDirect(DruidDataSource.java:1045) at com.alibaba.druid.filter.FilterChainImpl.dataSource_connect(FilterChainImpl.java:4544) at com.alibaba.intl.commons.monitor.druid.ConnectionMonitorDruidFilter.dataSource_getConnection(ConnectionMonitorDruidFilter.java:60) at com.alibaba.druid.filter.FilterChainImpl.dataSource_connect(FilterChainImpl.java:4540) at com.alibaba.druid.filter.stat.StatFilter.dataSource_getConnection(StatFilter.java:662) at com.alibaba.druid.filter.FilterChainImpl.dataSource_connect(FilterChainImpl.java:4540) at com.alibaba.druid.pool.DruidDataSource.getConnection(DruidDataSource.java:1023) at com.alibaba.druid.pool.DruidDataSource.getConnection(DruidDataSource.java:1015) at com.taobao.tddl.atom.jdbc.TDataSourceWrapper.getConnectionByTargetDataSource(TDataSourceWrapper.java:349) at com.taobao.tddl.atom.jdbc.TDataSourceWrapper.getConnectionWrapper(TDataSourceWrapper.java:319) at com.taobao.tddl.atom.jdbc.TDataSourceWrapper.getConnection0(TDataSourceWrapper.java:290) at com.taobao.tddl.atom.jdbc.TDataSourceWrappe