HttpClient 的一次Timeout waiting for connection from pool 之旅

最新推荐文章于 2024-06-12 17:54:11 发布

weixin_34290096

最新推荐文章于 2024-06-12 17:54:11 发布

阅读量472

点赞数

文章标签： java python

原文链接：https://my.oschina.net/newchaos/blog/2994348

版权

2019独角兽企业重金招聘Python工程师标准>>>

httpClient大家用到地方会很多,先简单描述一下几个关键配置的意义

httpClient版本为4.5.1

maxTotal:整个连接池的最大支持连接数

defaultMaxPerRoute:当前主机到目的主机的一个路由，主要作用在通过httpClient转发请求到不同的目的主机的连接数限制,是maxTotal的一个细分;比如：

maxtTotal=400 defaultMaxPerRoute=200

而我只连接到http://www.baidu.com时，到这个主机的并发最多只有200；而不是400；

而我连接到http://www.baidu.com 和 http://www.jd.com时，到每个主机的并发最多只有200；即加起来是400（但不能超过400）；所以起作用的设置是defaultMaxPerRoute。

connectionRequestTimeout:从PoolingHttpClientConnectionManager中获取连接超时时间（必填，如果未配置将一直等待从连接池中获取可用连接，此值不易太大）

connectTimeout:和目的主机建立连接的超时时间

socketTimeout:从目的主机读取数据超时时间

keepAliveDuration:连接存活时间（长连接使用）

事故现场：

  public static final int connTimeout = 5000;
  public static final int readTimeout = 5000;
  public static final int connRequestTimeout = 1000;
 
PoolingHttpClientConnectionManager cm = new PoolingHttpClientConnectionManager();
 cm.setMaxTotal(3000);
 cm.setDefaultMaxPerRoute(20);
 HttpClient client = HttpClients.custom().setConnectionManager(cm).build();

public static String postFormRedirect(String url, Map<String, String> params,
      Map<String, String> headers, Integer connTimeout, Integer readTimeout)
      throws ConnectTimeoutException, SocketTimeoutException, Exception {
    String resultStr = "";
    HttpClient client = null;
    HttpResponse res = null;
    HttpPost post = new HttpPost(url);
    try {
      if (params != null && !params.isEmpty()) {
        List<NameValuePair> formParams = new ArrayList<org.apache.http.NameValuePair>();
        Set<Entry<String, String>> entrySet = params.entrySet();
        for (Entry<String, String> entry : entrySet) {
          formParams.add(new BasicNameValuePair(entry.getKey(), entry.getValue()));
        }
        UrlEncodedFormEntity entity = new UrlEncodedFormEntity(formParams, Consts.UTF_8);
        post.setEntity(entity);
      }

      if (headers != null && !headers.isEmpty()) {
        for (Entry<String, String> entry : headers.entrySet()) {
          post.addHeader(entry.getKey(), entry.getValue());
        }
      }
      // 设置参数
      Builder customReqConf = RequestConfig.custom();
      customReqConf.setConnectTimeout(connTimeout);
      customReqConf.setSocketTimeout(readTimeout);
      customReqConf.setConnectionRequestTimeout(connRequestTimeout);
      post.setConfig(customReqConf.build());
      if (url.startsWith("https")) {
        // 执行 Https 请求.
        client = createSSLInsecureClient();
        res = client.execute(post);
      } else {
        // 执行 Http 请求.
        client = HttpClientUtils.client;
        res = client.execute(post);
      }
      int status = res.getStatusLine().getStatusCode();
      if (status == HttpStatus.SC_OK) {
        resultStr = IOUtils.toString(res.getEntity().getContent(), "UTF-8");
      } else if (status == HttpStatus.SC_MOVED_PERMANENTLY || // 处理转向
          status == HttpStatus.SC_MOVED_TEMPORARILY || status == HttpStatus.SC_CREATED) { // 从头中取出转向地址
        Header locationHeader = res.getFirstHeader("location");
        if (locationHeader != null) {
          java.net.URI uri = new java.net.URI(locationHeader.getValue());
          post.setURI(uri);
          res = client.execute(post);
          resultStr = IOUtils.toString(res.getEntity().getContent(), "UTF-8");
        }
      }
      return resultStr;
    } finally {
      if (res != null) {
        EntityUtils.consume(res.getEntity());
      }
      post.releaseConnection();
      if (url.startsWith("https") && client != null && client instanceof CloseableHttpClient) {
        ((CloseableHttpClient) client).close();
      }
    }
  }

代码逻辑很简单,拿到一个url进行转发请求，如果请求结果需要跳转，则主动请求需要跳转的地址，将跳转地址的返回结果进行返回

defaultMaxPerRoute设置的20，maxTotal设置的3000；

写个测试类循环调用该结果20次

ST-21142-9Na2eefb1nWWdcgSXW56-cas01.example.org,:20
0耗时：944
ST-21143-VFk6eu9gLZ5BrX3h3aLs-cas01.example.org,:20
1耗时：6
ST-21144-jf7cCBv4VPNpqjBsWOUJ-cas01.example.org,:20
2耗时：5
ST-21145-IDIIhlMdTjSDBPcGa0NN-cas01.example.org,:20
3耗时：6
ST-21146-2WGXIdfeQOarOKHhEcw3-cas01.example.org,:20
4耗时：5
ST-21147-tg6r1BGaEfu4GSkEbiX6-cas01.example.org,:20
5耗时：6
ST-21148-eLV2nq2EkW4CK7GCNFDk-cas01.example.org,:20
6耗时：5
ST-21149-xG3bXAdNFr9aNuBGtLRq-cas01.example.org,:20
7耗时：5
ST-21150-yL4ed9qT2fBmjXPbsswr-cas01.example.org,:20
8耗时：5
ST-21151-scWrQqQh25PRs3TX2mrs-cas01.example.org,:20
9耗时：6
ST-21152-KAwooFwbbZUBvF7MWNbK-cas01.example.org,:20
10耗时：5
ST-21153-HPC6gdigioPr1mqgOa4f-cas01.example.org,:20
11耗时：5
ST-21154-9ZgIqgfKUUiglfgymEie-cas01.example.org,:20
12耗时：5
ST-21155-mIDRgL1fAjpPoRuH1Lfg-cas01.example.org,:20
13耗时：5
ST-21156-SjdNbDbVYdC5rMmafMhp-cas01.example.org,:20
14耗时：5
ST-21157-3dFJcfbhffvW5fHryshZ-cas01.example.org,:20
15耗时：6
ST-21158-g3VnDke2UgqSEWHX1e1P-cas01.example.org,:20
16耗时：5
ST-21159-XtiDbc64IUfl0dZLOgD5-cas01.example.org,:20
17耗时：5
ST-21160-mbmUUbiabfvxh7iA2XWc-cas01.example.org,:20
18耗时：4
org.apache.http.conn.ConnectionPoolTimeoutException: Timeout waiting for connection from pool
	at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.leaseConnection(PoolingHttpClientConnectionManager.java:286)
	at org.apache.http.impl.conn.PoolingHttpClientConnectionManager$1.get(PoolingHttpClientConnectionManager.java:263)
	at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:190)
	at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:184)
	at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:88)
	at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110)
	at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:184)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:107)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55)
	at com.fwyun.common.rs.client.HttpClientUtils.postFormRedirect(HttpClientUtils.java:538)
	at com.fwyun.common.rs.client.HttpClientUtils.postFormParametersRedirect(HttpClientUtils.java:109)
	at Main.main(Main.java:64)

看到Timeout waiting for connection from pool这个异常很容易联想到是connectionRequestTimeout超时了,为什么会超时呢？是资源没有释放吗？代码中也明确的写了释放资源了！总共才循环20次，为什么就从连接池中获取不到资源呢？反复检查了下代码发现

if (status == HttpStatus.SC_OK) {
        resultStr = IOUtils.toString(res.getEntity().getContent(), "UTF-8");
      } else if (status == HttpStatus.SC_MOVED_PERMANENTLY || // 处理转向
          status == HttpStatus.SC_MOVED_TEMPORARILY || status == HttpStatus.SC_CREATED) { // 从头中取出转向地址
        Header locationHeader = res.getFirstHeader("location");
        if (locationHeader != null) {
          java.net.URI uri = new java.net.URI(locationHeader.getValue());
          post.setURI(uri);
          res = client.execute(post);
          resultStr = IOUtils.toString(res.getEntity().getContent(), "UTF-8");
        }
      }

这段代码存在问题，如果请求返回httpStatus==200的时候正常逻辑走到finally 对资源进行释放，是不存在问题的，问题在于httpStatus!=200需要处理跳转的时候，采用的依然是第一个请求返回的res对象来接收返回值

 res = client.execute(post);
 resultStr = IOUtils.toString(res.getEntity().getContent(), "UTF-8");

这样写的话就会将原本第一个请求的返回值对象丢失,导致无法释放第一个连接的资源

这样一下就开朗多了稍微修改一下,两个返回结果对象分开进行接收，并在finally中统一进行释放

 public static String postFormRedirect(String url, Map<String, String> params,
      Map<String, String> headers, Integer connTimeout, Integer readTimeout)
      throws ConnectTimeoutException, SocketTimeoutException, Exception {
    String resultStr = "";
    HttpClient client = null;
    HttpResponse res = null;
    HttpResponse redirtRes = null;
    HttpPost post = new HttpPost(url);
    try {
      if (params != null && !params.isEmpty()) {
        List<NameValuePair> formParams = new ArrayList<org.apache.http.NameValuePair>();
        Set<Entry<String, String>> entrySet = params.entrySet();
        for (Entry<String, String> entry : entrySet) {
          formParams.add(new BasicNameValuePair(entry.getKey(), entry.getValue()));
        }
        UrlEncodedFormEntity entity = new UrlEncodedFormEntity(formParams, Consts.UTF_8);
        post.setEntity(entity);
      }

      if (headers != null && !headers.isEmpty()) {
        for (Entry<String, String> entry : headers.entrySet()) {
          post.addHeader(entry.getKey(), entry.getValue());
        }
      }
      // 设置参数
      Builder customReqConf = RequestConfig.custom();
      if (connTimeout != null) {
        customReqConf.setConnectTimeout(connTimeout);
      }
      if (readTimeout != null) {
        customReqConf.setSocketTimeout(readTimeout);
      }
      customReqConf.setConnectionRequestTimeout(connRequestTimeout);
      post.setConfig(customReqConf.build());

      if (url.startsWith("https")) {
        // 执行 Https 请求.
        client = createSSLInsecureClient();
        res = client.execute(post);
      } else {
        // 执行 Http 请求.
        client = HttpClientUtils.client;
        res = client.execute(post);
      }
      int status = res.getStatusLine().getStatusCode();
      if (status == HttpStatus.SC_OK) {
        resultStr = IOUtils.toString(res.getEntity().getContent(), "UTF-8");
      } else if (status == HttpStatus.SC_MOVED_PERMANENTLY || // 处理转向
          status == HttpStatus.SC_MOVED_TEMPORARILY || status == HttpStatus.SC_CREATED) { // 从头中取出转向地址
        Header locationHeader = res.getFirstHeader("location");
        if (locationHeader != null) {
          java.net.URI uri = new java.net.URI(locationHeader.getValue());
          post.setURI(uri);
          redirtRes = client.execute(post);
          resultStr = IOUtils.toString(redirtRes.getEntity().getContent(), "UTF-8");
        }
      }
      return resultStr;
    } finally {
      if (res != null) {
        EntityUtils.consume(res.getEntity());
      }
      if (redirtRes != null) {
        EntityUtils.consume(redirtRes.getEntity());
      }
      post.releaseConnection();
      if (url.startsWith("https") && client != null && client instanceof CloseableHttpClient) {
        ((CloseableHttpClient) client).close();
      }
    }
  }

再次验证一下刚刚的推理是否正确，还是采用刚刚的循环20次调用

ST-21161-6grVVqIgr0Bs4NVGDCIz-cas01.example.org,:20
0耗时：957
ST-21162-vepVQxbdLSisirnY1iqA-cas01.example.org,:20
1耗时：10
ST-21163-Xy1MlSofMwHCCLbv4DTA-cas01.example.org,:20
2耗时：9
ST-21164-jr3p6WdfH0QWYqv33Agm-cas01.example.org,:20
3耗时：9
ST-21165-49i1GzLr3U0edcPZoBTk-cas01.example.org,:20
4耗时：7
ST-21166-OVc7MnZQiLhScFu6RZrb-cas01.example.org,:20
5耗时：6
ST-21167-U9MWZrOlBQeYT9BfMzuY-cas01.example.org,:20
6耗时：5
ST-21168-FHXDdei5LyO9bDCxzh5i-cas01.example.org,:20
7耗时：7
ST-21169-aQuUtUMYUFSh77yNvuj1-cas01.example.org,:20
8耗时：7
ST-21170-PsbSatB6RZzjTxTuVFk4-cas01.example.org,:20
9耗时：4
ST-21171-cLjku1Pd9npdXLG7CRzs-cas01.example.org,:20
10耗时：5
ST-21172-iKAmCyksxCz6WNoaUOtV-cas01.example.org,:20
11耗时：3
ST-21173-gUfa7qimg4TpAMjfyVPf-cas01.example.org,:20
12耗时：4
ST-21174-f49ZWfbdyvMvaLe1Ge3a-cas01.example.org,:20
13耗时：4
ST-21175-LdtWVaRVmQUrhEA3Iceb-cas01.example.org,:20
14耗时：4
ST-21176-5AmcumhwGBpfTYpGccUd-cas01.example.org,:20
15耗时：4
ST-21177-zqVujMoOKz534PcSQI3x-cas01.example.org,:20
16耗时：4
ST-21178-X3RgCYubbmdwE4iXWxaj-cas01.example.org,:20
17耗时：4
ST-21179-dHAkOflr0qU04fgVCBim-cas01.example.org,:20
18耗时：3
ST-21180-wIuYRRQFUabgyQX5bHfN-cas01.example.org,:20
19耗时：4

看样子问题已经解决了

循环加大到50次再次测试

ST-21181-yd6JhdVUw1MsidkDqTc9-cas01.example.org,:50
0耗时：926
ST-21182-IE1gUdi4OlQX2qflXfDo-cas01.example.org,:50
1耗时：5
ST-21183-HUmsb7exvNdI0cRudevt-cas01.example.org,:50
2耗时：3
ST-21184-PwMc3SfVeQbKU0lBwvXq-cas01.example.org,:50
3耗时：4
ST-21185-1G7ruiPXdDkzUcCTWR1d-cas01.example.org,:50
4耗时：4
ST-21186-HDgDMkqyYkWw2IjcGVwK-cas01.example.org,:50
5耗时：5
ST-21187-3td91BvAobMxiZXVyueO-cas01.example.org,:50
6耗时：4
ST-21188-QC1Kvd3CybO24oYXuV0f-cas01.example.org,:50
7耗时：3
ST-21189-n6fqvtLnyovmZX4CuMPM-cas01.example.org,:50
8耗时：4
ST-21190-QNXV56amzenUqycRAkvI-cas01.example.org,:50
9耗时：4
ST-21191-f2pb9zatfZ5y2Fee6aSs-cas01.example.org,:50
10耗时：3
ST-21192-hVgsgtNKKdu2l0ztll4m-cas01.example.org,:50
11耗时：4
ST-21193-tEWmw62cfwAokMJd6I7h-cas01.example.org,:50
12耗时：4
ST-21194-MvACRw7PhGxM6OFGOWvh-cas01.example.org,:50
13耗时：6
ST-21195-4NLHy9Ip9d9yChN6egNz-cas01.example.org,:50
14耗时：8
ST-21196-ok9LLVdTiApd1YHclevd-cas01.example.org,:50
15耗时：4
ST-21197-Qrd0CDTV3LzFwTe6zeY3-cas01.example.org,:50
16耗时：4
ST-21198-YO9luE4nhdSNzJ4Betgg-cas01.example.org,:50
17耗时：3
ST-21199-k5mXx3PuBRkKYeb01Pwo-cas01.example.org,:50
18耗时：4
ST-21200-5WIVROMMvZuVV0npKfl2-cas01.example.org,:50
19耗时：3
ST-21201-X2bCusnlV1CaK7d2E5iO-cas01.example.org,:50
20耗时：4
ST-21202-0RYS4fBczjMfsr6XUHfW-cas01.example.org,:50
21耗时：3
ST-21203-WJVhKsf1DV3Hzv4X00Hw-cas01.example.org,:50
22耗时：4
ST-21204-arQVGHsQHDvUFFAvH33b-cas01.example.org,:50
23耗时：4
ST-21205-LuueDdc3ttzMUhMXkuCg-cas01.example.org,:50
24耗时：4
ST-21206-cvUDZT5GO2u7BXwKu3Gd-cas01.example.org,:50
25耗时：4
ST-21207-5C9vru22Kpmh7XUpuBJU-cas01.example.org,:50
26耗时：3
ST-21208-rD3cxkHNi5WNdQ1D5Ilt-cas01.example.org,:50
27耗时：4
ST-21209-ZnZ9on3CSNfdu6qvqGLF-cas01.example.org,:50
28耗时：4
ST-21210-nGDqq6kpYuSiZ1sd7GVt-cas01.example.org,:50
29耗时：3
ST-21211-YY0ecrlxuV1z6KGsAVil-cas01.example.org,:50
30耗时：4
ST-21212-gDXGHsNPgTg6uBAsVdMg-cas01.example.org,:50
31耗时：3
ST-21213-lVKxdqbLcIZWOiXUzdml-cas01.example.org,:50
32耗时：8
ST-21214-hUVHAY4AW2Hqk0JiaAEh-cas01.example.org,:50
33耗时：5
ST-21215-dkOuVEzshOgHfPkm1Q5x-cas01.example.org,:50
34耗时：3
ST-21216-1IgMpJ4y17dn0ddjwadJ-cas01.example.org,:50
35耗时：4
ST-21217-c6KlWVOBms3DTuVtsMS0-cas01.example.org,:50
36耗时：3
ST-21218-tcdsM1nXNs0o3EZnZK4M-cas01.example.org,:50
37耗时：3
ST-21219-d3b6A7H2zy2DNVXP2u6L-cas01.example.org,:50
38耗时：4
ST-21220-pEFI1bpQ1chk7EGjh9AZ-cas01.example.org,:50
39耗时：4
ST-21221-vet4doDAJsdCnWA01TFO-cas01.example.org,:50
40耗时：4
ST-21222-iscVPPkgtT7j3xf7b3HS-cas01.example.org,:50
41耗时：6
ST-21223-g0a4pfd3qb1hMsSdHOqi-cas01.example.org,:50
42耗时：6
ST-21224-DEzfJa9a7bE0sBrLWQAG-cas01.example.org,:50
43耗时：4
ST-21225-2ldk9S0ZvLXwkQmFLFJb-cas01.example.org,:50
44耗时：4
ST-21226-yiugvLRcsPteqAe31X0t-cas01.example.org,:50
45耗时：3
ST-21227-Gd3HN99vxmx6cAule0vn-cas01.example.org,:50
46耗时：5
ST-21228-YbGi4hILdYN4RaPvJYvy-cas01.example.org,:50
47耗时：3
ST-21229-VfR7iKA3r4WFG74kYid5-cas01.example.org,:50
48耗时：3
ST-21230-j7BUX4cApbcLJQ6ptKqJ-cas01.example.org,:50
49耗时：19

好了到此问题已经修复了！

总结一下问题的原因，必须确保每一个response的资源都释放掉了,否则连接池一直判定对象为正在使用状态，这样累积将达到defaultMaxPerRoute，满了后再有请求需要获取连接，因为已经没有了可用资源，超时后将报出Timeout waiting for connection from pool异常。

转载于:https://my.oschina.net/newchaos/blog/2994348