php 连接池 idletime,聊聊hikari连接池的idleTimeout及minimumIdle属性

序

本文主要研究一个hikari连接池的idleTimeout及minimumIdle属性

idleTimeout

默认是600000毫秒，即10分钟。如果idleTimeout+1秒>maxLifetime 且 maxLifetime>0，则会被重置为0；如果idleTimeout!=0且小于10秒，则会被重置为10秒。如果idleTimeout=0则表示空闲的连接在连接池中永远不被移除。

只有当minimumIdle小于maximumPoolSize时，这个参数才生效，当空闲连接数超过minimumIdle，而且空闲时间超过idleTimeout，则会被移除。

minimumIdle

控制连接池空闲连接的最小数量，当连接池空闲连接少于minimumIdle，而且总共连接数不大于maximumPoolSize时，HikariCP会尽力补充新的连接。为了性能考虑，不建议设置此值，而是让HikariCP把连接池当做固定大小的处理，默认minimumIdle与maximumPoolSize一样。

当minIdle<0或者minIdle>maxPoolSize,则被重置为maxPoolSize，该值默认为10。

HikariPool.HouseKeeper

HikariCP-2.7.6-sources.jar!/com/zaxxer/hikari/pool/HikariPool.java

private final long HOUSEKEEPING_PERIOD_MS = Long.getLong("com.zaxxer.hikari.housekeeping.periodMs", SECONDS.toMillis(30));

this.houseKeeperTask = houseKeepingExecutorService.scheduleWithFixedDelay(new HouseKeeper(), 100L, HOUSEKEEPING_PERIOD_MS, MILLISECONDS);

/**

* The house keeping task to retire and maintain minimum idle connections.

*/

private final class HouseKeeper implements Runnable

{

private volatile long previous = plusMillis(currentTime(), -HOUSEKEEPING_PERIOD_MS);

@Override

public void run()

{

try {

// refresh timeouts in case they changed via MBean

connectionTimeout = config.getConnectionTimeout();

validationTimeout = config.getValidationTimeout();

leakTaskFactory.updateLeakDetectionThreshold(config.getLeakDetectionThreshold());

final long idleTimeout = config.getIdleTimeout();

final long now = currentTime();

// Detect retrograde time, allowing +128ms as per NTP spec.

if (plusMillis(now, 128) < plusMillis(previous, HOUSEKEEPING_PERIOD_MS)) {

LOGGER.warn("{} - Retrograde clock change detected (housekeeper delta={}), soft-evicting connections from pool.",

poolName, elapsedDisplayString(previous, now));

previous = now;

softEvictConnections();

return;

}

else if (now > plusMillis(previous, (3 * HOUSEKEEPING_PERIOD_MS) / 2)) {

// No point evicting for forward clock motion, this merely accelerates connection retirement anyway

LOGGER.warn("{} - Thread starvation or clock leap detected (housekeeper delta={}).", poolName, elapsedDisplayString(previous, now));

}

previous = now;

String afterPrefix = "Pool ";

if (idleTimeout > 0L && config.getMinimumIdle() < config.getMaximumPoolSize()) {

logPoolState("Before cleanup ");

afterPrefix = "After cleanup ";

final List notInUse = connectionBag.values(STATE_NOT_IN_USE);

int toRemove = notInUse.size() - config.getMinimumIdle();

for (PoolEntry entry : notInUse) {

if (toRemove > 0 && elapsedMillis(entry.lastAccessed, now) > idleTimeout && connectionBag.reserve(entry)) {

closeConnection(entry, "(connection has passed idleTimeout)");

toRemove--;

}

}

}

logPoolState(afterPrefix);

fillPool(); // Try to maintain minimum connections

}

catch (Exception e) {

LOGGER.error("Unexpected exception in housekeeping task", e);

}

}

}

这个HouseKeeper是一个定时任务，在HikariPool构造器里头初始化，默认的是初始化后100毫秒执行，之后每执行完一次之后隔HOUSEKEEPING_PERIOD_MS(30秒)时间执行。

这个定时任务的作用就是根据idleTimeout的值，移除掉空闲超时的连接。

首先检测时钟是否倒退，如果倒退了则立即对过期的连接进行标记evict；之后当idleTimeout>0且配置的minimumIdle

取出状态是STATE_NOT_IN_USE的连接数，如果大于minimumIdle，则遍历STATE_NOT_IN_USE的连接的连接，将空闲超时达到idleTimeout的连接从connectionBag移除掉，若移除成功则关闭该连接，然后toRemove--。

在空闲连接移除之后，再调用fillPool，尝试补充空间连接数到minimumIdle值

HikariPool.fillPool

HikariCP-2.7.6-sources.jar!/com/zaxxer/hikari/pool/HikariPool.java

private final PoolEntryCreator POOL_ENTRY_CREATOR = new PoolEntryCreator(null /*logging prefix*/);

private final PoolEntryCreator POST_FILL_POOL_ENTRY_CREATOR = new PoolEntryCreator("After adding ");

LinkedBlockingQueue addConnectionQueue = new LinkedBlockingQueue<>(config.getMaximumPoolSize());

this.addConnectionQueue = unmodifiableCollection(addConnectionQueue);

this.addConnectionExecutor = createThreadPoolExecutor(addConnectionQueue, poolName + " connection adder", threadFactory, new ThreadPoolExecutor.DiscardPolicy());

/**

* Fill pool up from current idle connections (as they are perceived at the point of execution) to minimumIdle connections.

*/

private synchronized void fillPool()

{

final int connectionsToAdd = Math.min(config.getMaximumPoolSize() - getTotalConnections(), config.getMinimumIdle() - getIdleConnections())

- addConnectionQueue.size();

for (int i = 0; i < connectionsToAdd; i++) {

addConnectionExecutor.submit((i < connectionsToAdd - 1) ? POOL_ENTRY_CREATOR : POST_FILL_POOL_ENTRY_CREATOR);

}

}

PoolEntryCreator

/**

* Creating and adding poolEntries (connections) to the pool.

*/

private final class PoolEntryCreator implements Callable

{

private final String loggingPrefix;

PoolEntryCreator(String loggingPrefix)

{

this.loggingPrefix = loggingPrefix;

}

@Override

public Boolean call() throws Exception

{

long sleepBackoff = 250L;

while (poolState == POOL_NORMAL && shouldCreateAnotherConnection()) {

final PoolEntry poolEntry = createPoolEntry();

if (poolEntry != null) {

connectionBag.add(poolEntry);

LOGGER.debug("{} - Added connection {}", poolName, poolEntry.connection);

if (loggingPrefix != null) {

logPoolState(loggingPrefix);

}

return Boolean.TRUE;

}

// failed to get connection from db, sleep and retry

quietlySleep(sleepBackoff);

sleepBackoff = Math.min(SECONDS.toMillis(10), Math.min(connectionTimeout, (long) (sleepBackoff * 1.5)));

}

// Pool is suspended or shutdown or at max size

return Boolean.FALSE;

}

/**

* We only create connections if we need another idle connection or have threads still waiting

* for a new connection. Otherwise we bail out of the request to create.

*

* @return true if we should create a connection, false if the need has disappeared

*/

private boolean shouldCreateAnotherConnection() {

return getTotalConnections() < config.getMaximumPoolSize() &&

(connectionBag.getWaitingThreadCount() > 0 || getIdleConnections() < config.getMinimumIdle());

}

}

shouldCreateAnotherConnection方法决定了是否需要添加新的连接

createPoolEntry

/**

* Creating new poolEntry. If maxLifetime is configured, create a future End-of-life task with 2.5% variance from

* the maxLifetime time to ensure there is no massive die-off of Connections in the pool.

*/

private PoolEntry createPoolEntry()

{

try {

final PoolEntry poolEntry = newPoolEntry();

final long maxLifetime = config.getMaxLifetime();

if (maxLifetime > 0) {

// variance up to 2.5% of the maxlifetime

final long variance = maxLifetime > 10_000 ? ThreadLocalRandom.current().nextLong( maxLifetime / 40 ) : 0;

final long lifetime = maxLifetime - variance;

poolEntry.setFutureEol(houseKeepingExecutorService.schedule(

() -> {

if (softEvictConnection(poolEntry, "(connection has passed maxLifetime)", false /* not owner */)) {

addBagItem(connectionBag.getWaitingThreadCount());

}

},

lifetime, MILLISECONDS));

}

return poolEntry;

}

catch (Exception e) {

if (poolState == POOL_NORMAL) { // we check POOL_NORMAL to avoid a flood of messages if shutdown() is running concurrently

LOGGER.debug("{} - Cannot acquire connection from data source", poolName, (e instanceof ConnectionSetupException ? e.getCause() : e));

}

return null;

}

}

createPoolEntry方法创建一个poolEntry，同时给它的lifetime过期设定了一个延时任务。

小结

HouseKeeper是一个定时任务，在HikariPool构造器里头初始化，默认的是初始化后100毫秒执行，之后每执行完一次之后隔HOUSEKEEPING_PERIOD_MS(30秒)时间执行。

如果发现时钟倒退，则立即标记evict连接，然后退出；否则都会执行fillPool，来试图维持空闲连接到minimumIdle的数值

当idleTimeout>0且配置的minimumIdle

当minIdle<0或者minIdle>maxPoolSize,则minIdle被重置为maxPoolSize，该值默认为10，官方建议设置为一致，当做固定大小的连接池处理提高性能

idleTimeout有点类似tomcat jdbc pool里头的min-evictable-idle-time-millis参数。不同的是tomcat jdbc pool的连接泄露检测以及空闲连接清除的工作都放在一个名为PoolCleaner的timerTask中处理，该任务的执行间隔为timeBetweenEvictionRunsMillis，默认为5秒；而hikari的连接泄露是每次getConnection的时候单独触发一个延时任务来处理，而空闲连接的清除则是使用HouseKeeper定时任务来处理，其运行间隔由com.zaxxer.hikari.housekeeping.periodMs环境变量控制，默认为30秒。

doc