以前在看Druid的摧毁线程初始化的时候简单提了一下shrink方法,主要在代码里面写了比较详细的注释,今天来详细看一下这个方法具体做了什么。
shrink的作用
在看代码实现前,我们先看下这个方法是用来干嘛的?顾名思义,它是对连接池进行收缩(瘦身)的,防止程序持有的连接池连接数量太多,导致数据库连接不够用。它主要根据minIdle,phyTimeoutMillis,minEvictableIdleTimeMillis,maxEvictableIdleTimeMillis,keepAliveBetweenTimeMillis
这几个参数进行收缩。注意当前最小连接数其实是分两块,当前连接池里面的连接数,和借出去在用的连接数量,对借出去的连接是控制不了的,瘦身的目标只是现在连接池里的连接。
入口
框架里面shrink()调用是在摧毁线程池的定时任务执行的,默认是1s执行一次,如果你设置了timeBetweenEvictionRunsMillis,会按照这个周期执行。
shrink整体思路
看着代码来:
public void shrink(boolean checkTime, boolean keepAlive) {
final Lock lock = this.lock;
try {
lock.lockInterruptibly();
} catch (InterruptedException e) {
return;
}
boolean needFill = false;
int evictCount = 0;
int keepAliveCount = 0;
int fatalErrorIncrement = fatalErrorCount - fatalErrorCountLastShrink;
fatalErrorCountLastShrink = fatalErrorCount;
try {
if (!inited) {
return;
}
final int checkCount = poolingCount - minIdle;
final long currentTimeMillis = System.currentTimeMillis();
Arrays.fill(connectionsFlag, 0, poolingCount, false);
for (int i = 0; i < poolingCount; ++i) {
DruidConnectionHolder connection = connections[i];
//连接发生了错误
if ((onFatalError || fatalErrorIncrement > 0) && (lastFatalErrorTimeMillis > connection.connectTimeMillis)) {
keepAliveConnections[keepAliveCount++] = connection;
connectionsFlag[i] = true;
continue;
}
if (checkTime) {
if (phyTimeoutMillis > 0) {
long phyConnectTimeMillis = currentTimeMillis - connection.connectTimeMillis;
if (phyConnectTimeMillis > phyTimeoutMillis) {
evictConnections[evictCount++] = connection;
connectionsFlag[i] = true;
continue;
}
}
long idleMillis = currentTimeMillis - connection.lastActiveTimeMillis;
if (idleMillis < minEvictableIdleTimeMillis
&& idleMillis < keepAliveBetweenTimeMillis
) {
break;
}
if (idleMillis >= minEvictableIdleTimeMillis) {
if (checkTime && i < checkCount) {
evictConnections[evictCount++] = connection;
connectionsFlag[i] = true;
continue;
} else if (idleMillis > maxEvictableIdleTimeMillis) {
evictConnections[evictCount++] = connection;
connectionsFlag[i] = true;
continue;
}
}
if (keepAlive && idleMillis >= keepAliveBetweenTimeMillis) {
keepAliveConnections[keepAliveCount++] = connection;
connectionsFlag[i] = true;
}
} else {
if (i < checkCount) {
evictConnections[evictCount++] = connection;
connectionsFlag[i] = true;
} else {
break;
}
}
}
int removeCount = evictCount + keepAliveCount;
if (removeCount > 0) {
int remaining = 0;
for (int i = 0; i < connections.length; i++) {
if (!connectionsFlag[i]) {
shrinkBuffer[remaining++] = connections[i];
}
}
Arrays.fill(connections, 0, poolingCount, null);
//从shrinkBuffer复制remaining个数到connections里
System.arraycopy(shrinkBuffer, 0, connections, 0, remaining);
Arrays.fill(shrinkBuffer, 0, remaining, null);
poolingCount -= removeCount;
}
keepAliveCheckCount += keepAliveCount;
if (keepAlive && poolingCount + activeCount < minIdle) {
needFill = true;
}
} finally {
lock.unlock();
}
if (evictCount > 0) {
for (int i = 0; i < evictCount; ++i) {
DruidConnectionHolder item = evictConnections[i];
Connection connection = item.getConnection();
JdbcUtils.close(connection);
destroyCountUpdater.incrementAndGet(this);
}
Arrays.fill(evictConnections, null);
}
if (keepAliveCount > 0) {
// keep order
for (int i = keepAliveCount - 1; i >= 0; --i) {
DruidConnectionHolder holder = keepAliveConnections[i];
Connection connection = holder.getConnection();
holder.incrementKeepAliveCheckCount();
boolean validate = false;
try {
this.validateConnection(connection);
validate = true;
} catch (Throwable error) {
keepAliveCheckErrorLast = error;
keepAliveCheckErrorCountUpdater.incrementAndGet(this);
if (LOG.isDebugEnabled()) {
LOG.debug("keepAliveErr", error);
}
}
boolean discard = !validate;
if (validate) {
holder.lastKeepTimeMillis = System.currentTimeMillis();
boolean putOk = put(holder, 0L, true);
if (!putOk) {
discard = true;
}
}
if (discard) {
try {
connection.close();
} catch (Exception error) {
discardErrorLast = error;
discardErrorCountUpdater.incrementAndGet(DruidDataSource.this);
if (LOG.isErrorEnabled()) {
LOG.error("discard connection error", error);
}
}
if (holder.socket != null) {
try {
holder.socket.close();
} catch (Exception error) {
discardErrorLast = error;
discardErrorCountUpdater.incrementAndGet(DruidDataSource.this);
if (LOG.isErrorEnabled()) {
LOG.error("discard connection error", error);
}
}
}
lock.lock();
try {
holder.discard = true;
discardCount++;
if (activeCount + poolingCount <= minIdle) {
emptySignal();
}
} finally {
lock.unlock();
}
}
}
this.getDataSourceStat().addKeepAliveCheckCount(keepAliveCount);
Arrays.fill(keepAliveConnections, null);
}
if (needFill) {
lock.lock();
try {
int fillCount = minIdle - (activeCount + poolingCount + createTaskCount);
for (int i = 0; i < fillCount; ++i) {
emptySignal();
}
} finally {
lock.unlock();
}
} else if (onFatalError || fatalErrorIncrement > 0) {
lock.lock();
try {
emptySignal();
} finally {
lock.unlock();
}
}
}
首先定义一个检查数(这是一个大概的数,没那么准确),checkCount = poolingCount - minIdle
,它基本忽略的借出去的连接数。
然后遍历连接吃里面的所有连接,这里有两个集合,一个evictConnections,这里面的连接是直接会丢弃关闭的,一个是keepAliveConnections,这些是认为可以“抢救”的,会先看看连接是否有效,有效的话会放回连接池,无效才会丢弃关闭连接。对于发生了致命错误的连接放到keepAliveConnections集合里,对于超过phyTimeoutMillis时间的连接会放到evictConnections集合里,超过minEvictableIdleTimeMillis(最小存活时间)的连接,会直接丢掉checkCount个,然后清理掉超过maxEvictableIdleTimeMillis(最大存活时间)的连接,将这些连接放进evictConnections集合里。存活时间超过keepAliveBetweenTimeMillis的连接放到keepAliveConnections集合里。
整个移除的连接数为removeCount = evictCount + keepAliveCount
,先把留下的连接放到shrinkBuffer里,然后清空连接池connections,再将shrinkBuffer里的连接放到连接池connections里面,在清空shrinkBuffer。这块主要是空间换时间的思想,避免数组移动,直接删除然后复制。
shrink将连接池里面连接分两部分池里的连接poolingCount和在使用的连接activeCount,如果这两块加起来小于最小存活连接,那么就需要添加连接了。如果需要添加连接,实际添加的连接数为fillCount = minIdle - (activeCount + poolingCount + createTaskCount)
,除了池里连接和使用连接,还要加上正在创建的连接,少的就是需要在创建的连接。
总结
shrink方法一开始想着实现应该比较简单,它的作用就是去掉连接池里多余的连接,但是当你要细化或者想把它写好,就需要思考很多细节,比如会把去掉的连接进行细分,比如一些去掉的去保活在扔进去,一些连接直接丢掉,然后丢掉之后如果连接小于最小连接数,又需要再补充连接等。
然后整个弄下来发现要写一个好一点的连接池收缩方法考虑的点其实不少,想写好不容易。