HttpClient4 遇到的问题1：httpclient4.x 使用连接池发送https请求使用总结

滴水可藏海

于 2019-10-14 11:49:54 发布

阅读量803

点赞数

原文链接：https://www.iteye.com/blog/bsr1983-2211102

版权

由于项目中需要用httpclient进行内部服务请求处理，但之前没有是用httpclient中提供的连接池，而是自己封装的一个类似池的结构，一段时间后产生了性能问题，转而尝试通过是用httpclient内部的池进行处理，而且需要支持https的连接。

所使用的httpclient的版本为4.4.1，一共用到了三个jar：httpclient-4.4.1.jar、httpclient-cache-4.4.1.jar、httpcore-4.4.1.jar。

所使用的连接池管理器的类为PoolingHttpClientConnectionManager，但httpclient部分参数，包括池的大小，最大路由，缓存，超时这些参数还未做深入研究；

public void init()  
 {  
   try {  
   //需要通过以下代码声明对https连接支持  
   SSLContext sslcontext = SSLContexts.custom().loadTrustMaterial(null,  
                   new TrustSelfSignedStrategy())  
           .build();  
   HostnameVerifier hostnameVerifier = SSLConnectionSocketFactory.ALLOW_ALL_HOSTNAME_VERIFIER;  
   SSLConnectionSocketFactory sslsf = new SSLConnectionSocketFactory(  
           sslcontext,hostnameVerifier);  
   Registry<ConnectionSocketFactory> socketFactoryRegistry = RegistryBuilder.<ConnectionSocketFactory>create()  
           .register("http", PlainConnectionSocketFactory.getSocketFactory())  
           .register("https", sslsf)  
           .build();  
   //初始化连接管理器  
   poolConnManager = new PoolingHttpClientConnectionManager(socketFactoryRegistry);  
   // Increase max total connection to 200  
   poolConnManager.setMaxTotal(maxTotalPool);  
   // Increase default max connection per route to 20  
   poolConnManager.setDefaultMaxPerRoute(maxConPerRoute);  
  } catch (KeyManagementException e) {  
   // TODO Auto-generated catch block  
   e.printStackTrace();  
  } catch (NoSuchAlgorithmException e) {  
   // TODO Auto-generated catch block  
   e.printStackTrace();  
  } catch (KeyStoreException e) {  
   // TODO Auto-generated catch block  
   e.printStackTrace();  
  }  
 }  
//获取连接  
 public CloseableHttpClient getConnection()  
 {  
      CloseableHttpClient httpClient = HttpClients.custom()  
.setConnectionManager(poolConnManager).build();   
      return httpClient;  
 }  
//发送请求 url为请求url，jsonstr为请求参数  
public String post(String url, String jsonStr)  
 {  
  String returnStr = null;  
  //参数检测  
  if(url==null||"".equals(url))  
  {  
   return returnStr;  
  }  
  try {  
   HttpPost httpPost = new HttpPost(url);  
   List <NameValuePair> nvps = new ArrayList <NameValuePair>();  
   //设置post参数对  
   nvps.add(new BasicNameValuePair("jsonstr", jsonStr));  
   //设置编码，如果包含中文，一定要进行设置，否则按照系统默认字符集进行转码会出现乱码  
   httpPost.setEntity(new UrlEncodedFormEntity(nvps, "UTF-8"));  
   CloseableHttpResponse response = client.execute(httpPost);  
   //获取响应状态码  
   int status = response.getStatusLine().getStatusCode();  
            if (status >= 200 && status < 300) {  
                HttpEntity entity = response.getEntity();  
                return entity != null ? EntityUtils.toString(entity,"utf-8") : null;  
            } else {  
                throw new ClientProtocolException("Unexpected response status: " + status);  
            }  
  } catch (UnsupportedEncodingException e) {  
   // TODO Auto-generated catch block  
   e.printStackTrace();  
  } catch (ClientProtocolException e) {  
   // TODO Auto-generated catch block  
   e.printStackTrace();  
  } catch (IOException e) {  
   // TODO Auto-generated catch block  
   e.printStackTrace();  
  }  
  //相应返回值  
  return returnStr;  
 }

在使用上述代码一段时间后，出现了tomcat异常后死掉的bug，而且是随机出现，没有找到具体原因，最终在加了一堆日志后，分析得知是因为在返回状态码非200的时候，httpclient的连接不会自动断开，需要手工断开，否则在连接达到pool的最大值后，无可用连接，最终导致tomcat死掉。

最后修改的代码如下：

public void init()  
    {  
         try {  
            SSLContext sslcontext = SSLContexts.custom().loadTrustMaterial(null,  
                            new TrustSelfSignedStrategy())  
                    .build();  
            HostnameVerifier hostnameVerifier = SSLConnectionSocketFactory.ALLOW_ALL_HOSTNAME_VERIFIER;  
            SSLConnectionSocketFactory sslsf = new SSLConnectionSocketFactory(  
                    sslcontext,hostnameVerifier);  
            Registry<ConnectionSocketFactory> socketFactoryRegistry = RegistryBuilder.<ConnectionSocketFactory>create()  
                    .register("http", PlainConnectionSocketFactory.getSocketFactory())  
                    .register("https", sslsf)  
                    .build();  
            poolConnManager = new PoolingHttpClientConnectionManager(socketFactoryRegistry);  
            // Increase max total connection to 200  
            poolConnManager.setMaxTotal(maxTotalPool);  
            // Increase default max connection per route to 20  
            poolConnManager.setDefaultMaxPerRoute(maxConPerRoute);  
            SocketConfig socketConfig = SocketConfig.custom().setSoTimeout(socketTimeout).build();  
            poolConnManager.setDefaultSocketConfig(socketConfig);  
        } catch (Exception e) {  
            log.error("InterfacePhpUtilManager init Exception"+e.toString());  
        }  
    }  
    public CloseableHttpClient getConnection()  
    {  
        RequestConfig requestConfig = RequestConfig.custom().setConnectionRequestTimeout(connectionRequestTimeout)  
                .setConnectTimeout(connectTimeout).setSocketTimeout(socketTimeout).build();  
        CloseableHttpClient httpClient = HttpClients.custom()  
                    .setConnectionManager(poolConnManager).setDefaultRequestConfig(requestConfig).build();  
        if(poolConnManager!=null&&poolConnManager.getTotalStats()!=null)  
        {  
            log.info("now client pool "+poolConnManager.getTotalStats().toString());  
        }  
        return httpClient;  
    }

其中输出的连接一共有四个状态，是需要时时关注的，可以作为调整参数的依据，具体含义为：

leased ：the number of persistent connections tracked by the connection manager currently being used to execute requests.  
  
available ：the number idle persistent connections.  
  
pending : the number of connection requests being blocked awaiting a free connection.  
  
max: the maximum number of allowed persistent connections.

需要特别关注pending和leased的值，如果leased的值特别大，接近max，则需要修改max，如果pending的值也比较大，也需要调整max，并考虑设置timeout，可以设置两个timeout，一个是获取连接的timeout，另外一个是获取socket数据的timeout。代码中已经有相关示例了。

具体获取响应时，要改为如下代码，即在返回的状态码不是200时，要主动关闭连接

public String postPhp(String url, String jsonStr)  
    {  
        String returnStr = null;  
        //参数检测  
        if(url==null||"".equals(url))  
        {  
            return returnStr;  
        }  
        HttpPost httpPost = new HttpPost(url);  
        try {  
              
            long currentTime=System.currentTimeMillis();  
            List <NameValuePair> nvps = new ArrayList <NameValuePair>();  
            nvps.add(new BasicNameValuePair("jsonstr", jsonStr));  
            httpPost.setEntity(new UrlEncodedFormEntity(nvps, "UTF-8"));  
            sysoutLog(currentTime+" 开始发送 请求：url"+url);  
            CloseableHttpResponse response = client.execute(httpPost);  
            int status = response.getStatusLine().getStatusCode();  
            if (status >= 200 && status < 300) {  
                HttpEntity entity = response.getEntity();  
                String resopnse="";  
                if(entity != null)  
                {  
                    resopnse=EntityUtils.toString(entity,"utf-8");  
                }  
                sysoutLog(currentTime+" 接收响应：url"+url+" status="+status);  
                return entity != null ? resopnse : null;  
            } else {  
                HttpEntity entity = response.getEntity();  
                httpPost.abort();  
                sysoutLog(currentTime+" 接收响应：url"+url+" status="+status+" resopnse="+EntityUtils.toString(entity,"utf-8"));  
                throw new ClientProtocolException("Unexpected response status: " + status);  
            }  
        } catch (Exception e) {  
            httpPost.abort();  
            log.error(" Exception"+e.toString()+" url="+url+" jsonstr="+jsonStr);  
        }   
        return returnStr;  
    }