在 httpclient 使用中,可以配置 请求的超时时间 connectionRequestTimeout
、connectionTimeout
和 socketTimeout
参数。而这三个参数的配置无法保证我们整个调用是在想要的时间内完成的。比如你想要调用一个数据接口,如果在 3s 内没有完成,则直接返回结束,为了达成这一目的,我们采用了异步线程池的方式。
首先,我们先理解一下这三个参数的意义。
关于 httpclient 的超时参数
上面提到,代码中使用配置的超时时间作为 httpclient 的 connectionRequestTimeout
、connectionTimeout
和 socketTimeout
参数,下面简单介绍一下这三个参数的含义。
connectionRequestTimeout
:指从连接池获取连接的超时时间(当请求并发数量大于连接池中的连接数量时,则获取不到连接的请求会被放入 pending 队列等待,如果超过设定的时间,则抛出ConnectionPoolTimeoutException
)。connectionTimeout
:指客户端和服务器建立连接的超时时间。(当客户端和服务器在建立链接时,如果在指定时间内无法成功建立链接,则抛出ConnectionTimeoutException
)。socketTimeout
:指客户端从服务器读取数据的超时时间,即客户端和服务器 socket 通信的超时时间,其实这个时间是客户端两次读取数据的最长时间,如果客户端在网络抖动的情况下,每次返回部分数据,两次数据包的时间在设定时间之内,也是不会超时的。
问题背景
为了保证计时的准确性,我们采用异步提交线程池,用 Future.get(timeout)
的方式保证任务可以在超过设定时间后,计时的准确性,大致代码如下:
public class Main {
private static final Logger logger = LoggerFactory.getLogger(Main.class);
private static final ThreadPoolExecutor executor = new ThreadPoolExecutor(10, 10, 60,
java.util.concurrent.TimeUnit.SECONDS, new java.util.concurrent.LinkedBlockingQueue<>(10), new ThreadPoolExecutor.CallerRunsPolicy());
public static void main(String[] args) throws IOException, InterruptedException {
for (int i = 0; i < 10; i++) {
// 请求一个阻塞接口,不会返回数据,必定超时
HttpGet httpGet = new HttpGet("*****");
CloseableHttpResponse response = null;
Future<CloseableHttpResponse> future = null;
try {
future = executor.submit(() -> {
try {
return HttpClientUtil.execute(httpGet);
} catch (Exception e) {
logger.error("", e);
return null;
}
});
response = future.get(5, TimeUnit.SECONDS);
System.out.println("response = " + response);
} catch (Exception e) {
if (e instanceof TimeoutException && future != null) {
logger.info(Thread.currentThread().getName() + " start cancel future");
logger.error("", e);
}
} finally {
httpGet.abort();
httpGet.releaseConnection();
if (null != response) {
EntityUtils.consume(response.getEntity());
}
}
}
}
}
在验证过程中,压力比较大时会出现大量超时,导致大量调用返回超时异常,出现异常
org.apache.http.conn.ConnectionPoolTimeoutException: Timeout waiting for connection from pool
at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.leaseConnection(PoolingHttpClientConnectionManager.java:313)
at org.apache.http.impl.conn.PoolingHttpClientConnectionManager$1.get(PoolingHttpClientConnectionManager.java:279)
at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:191)
at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185)
at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89)
at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110)
at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:108)
at org.example.http.SimpleGetRequestExecutor.lambda$execute$0(SimpleGetRequestExecutor.java:56)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
出现大量异常后,服务就无法使用了,即任何接口的调用都是超时状态。此时通过 debug 发现,即使没有调用,连接也依旧处于 lease
状态。
问题排查
首先,出现 Timeout waiting for connection from pool
是由于 httpclient 在从连接池获取连接时,在 connectionRequectTimeout
时间内没有获取到连接,而抛出的异常信息,从连接池获取连接的流程如下。
httpclient 从连接池获取连接
首先,根据请求的路由和 token 构建 ConnectionRequest
对象,此对象保存了获取从连接池获取连接的 get
方法,代码如下:
@Override
public CloseableHttpResponse execute(
final HttpRoute route,
final HttpRequestWrapper request,
final HttpClientContext context,
final HttpExecutionAware execAware) throws IOException, HttpException {
// ......
Object userToken = context.getUserToken();
final ConnectionRequest connRequest = connManager.requestConnection(route, userToken);
if (execAware != null) {
if (execAware.isAborted()) {
connRequest.cancel();
throw new RequestAbortedException("Request aborted");
} else {
execAware.setCancellable(connRequest);
}
}
// .....
}
我们可以看到,此时使用的超时时间就是我们传入配置的 connectionRequestTimeout
,下面我们看下 ConnectionRequest
对象的构建。
@Override
public ConnectionRequest requestConnection(
final HttpRoute route,
final Object state) {
Args.notNull(route, "HTTP route");
if (this.log.isDebugEnabled()) {
this.log.debug("Connection request: " + format(route, state) + formatStats(route));
}
final Future<CPoolEntry> future = this.pool.lease(route, state, null);
return new ConnectionRequest() {
@Override
public boolean cancel() {
return future.cancel(true);
}
@Override
public HttpClientConnection get(
final long timeout,
final TimeUnit tunit) throws InterruptedException, ExecutionException, ConnectionPoolTimeoutException {
final HttpClientConnection conn = leaseConnection(future, timeout, tunit);
if (conn.isOpen()) {
final HttpHost host;
if (route.getProxyHost() != null) {
host = route.getProxyHost();
} else {
host = route.getTargetHost();
}
final SocketConfig socketConfig = resolveSocketConfig(host);
conn.setSocketTimeout(socketConfig.getSoTimeout());
}
return conn;
}
};
}
protected HttpClientConnection leaseConnection(
final Future<CPoolEntry> future,
final long timeout,
final TimeUnit tunit) throws InterruptedException, ExecutionException, ConnectionPoolTimeoutException {
final CPoolEntry entry;
try {
entry = future.get(timeout, tunit);
if (entry == null || future.isCancelled()) {
throw new InterruptedException();
}
Asserts.check(entry.getConnection() != null, "Pool entry with no connection");
if (this.log.isDebugEnabled()) {
this.log.debug("Connection leased: " + format(entry) + formatStats(entry.getRoute()));
}
return CPoolProxy.newProxy(entry);
} catch (final TimeoutException ex) {
throw new ConnectionPoolTimeoutException("Timeout waiting for connection from pool");
}
}
我们可以看到,代码中捕获了 TimeoutException
,并重新构建 ConnectionPoolTimeoutException
,也就是说,future.get
会在超时的时候抛出 TimeoutException
,然后被外层的 catch
捕获,下面我们看 final Future<CPoolEntry> future = this.pool.lease(route, state, null);
中的 Future
是如何实现的:
/**
* {@inheritDoc}
* <p>
* Please note that this class does not maintain its own pool of execution
* {@link Thread}s. Therefore, one <b>must</b> call {@link Future#get()}
* or {@link Future#get(long, TimeUnit)} method on the {@link Future}
* returned by this method in order for the lease operation to complete.
*/
@Override
public Future<E> lease(final T route, final Object state, final FutureCallback<E> callback) {
Args.notNull(route, "Route");
Asserts.check(!