背景
某应用1.0性能测试
服务强依赖于mysql, 许多接口都会请求mysql
对mysql的请求用GenericObjectPool的连接池来进行管理, 设置如下:
79 connectionPool.setMaxActive(maxActive);
80 connectionPool.setTestOnBorrow(false);
81 connectionPool.setTestOnReturn(true);
82 connectionPool.setTestWhileIdle(true);
83 connectionPool
84 .setMinEvictableIdleTimeMillis(minEvictableIdleTimeMillis);
85
86 // Start the Evictor
87 connectionPool
88 .setTimeBetweenEvictionRunsMillis(timeBetweenEvictionRunsMillis);
问题
用100并发打压服务,发现拿到连接并在干活的线程数只有10+, 而其余80+的线程wait在borrowObject的逻辑, 相应的stack如下:
有许多线程在blocked在makeObject相关的逻辑,均在等一把锁
1462 "resin-tcp-connection-*:3231-321" daemon prio=10 tid=0x000000004dc43800 nid=0x65f5 waiting for monitor entry [0x00000000507ff000]
1463 java.lang.Thread.State: BLOCKED (on object monitor)
1464 at org.apache.commons.dbcp.PoolableConnectionFactory.makeObject(PoolableConnectionFactory.java:290)
1465 - waiting to lock <0x00000000b26ee8a8> (a org.apache.commons.dbcp.PoolableConnectionFactory)
1466 at org.apache.commons.pool.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:771)
1467 at org.apache.commons.dbcp.PoolingDataSource.getConnection(PoolingDataSource.java:95)
1468 at outfox.cps.dao.source.CpsDataSource.getConnection(CpsDataSource.java:61)
1469 at org.springframework.jdbc.datasource.DataSourceUtils.doGetConnection(DataSourceUtils.java:111)
1470 at org.springframework.jdbc.datasource.DataSourceUtils.getConnection(DataSourceUtils.java:77)
1471 at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:572)
1472 at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:636)
1473 at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:665)
1474