最近公司通过API接口调用数据,本人搞了多线程调用。起初每次调用都单独 new HttpClient(),造成问题是内存越跑占用越大,请求越来越慢,甚至出现 Read Time Out 异常。
之后了解HttpClient可复用,则改为单例请求,加入配置设置问题解决。注:本人用的httpclient-4.3.6.jar
(1)创建HttpClient
CloseableHttpClient client=HttpClientUtil.getHttpClient();
HttpclientUtil 用的连接池的概念,当请求结束并不是直接断开连接,而是返回给连接池方便下次调用。
public class HttpClientUtil { private static Logger LOGGER = LoggerFactory.getLogger(HttpClientUtil.class); private static PoolingHttpClientConnectionManager cm = null; static { LayeredConnectionSocketFactory sslsf = null; try { sslsf = new SSLConnectionSocketFactory(SSLContext.getDefault()); } catch (NoSuchAlgorithmException e) { LOGGER.error("创建SSL连接失败"); } Registry<ConnectionSocketFactory> socketFactoryRegistry = RegistryBuilder.<ConnectionSocketFactory>create() .register("https", sslsf) .register("http", new PlainConnectionSocketFactory()) .build(); cm =new PoolingHttpClientConnectionManager(socketFactoryRegistry); cm.setMaxTotal(200);//多线程调用注意配置,根据线程数设定 cm.setDefaultMaxPerRoute(300);//多线程调用注意配置,根据线程数设定 } public static CloseableHttpClient getHttpClient() { CloseableHttpClient httpClient = HttpClients.custom() .setConnectionManager(cm) .build(); return httpClient; } }此处则创建httpclient 成功,下一注意点为 RequestConfig 参数配置,此处配置直接影响调用接口返回速度及是否 出现 Read time out 异常。
CloseableHttpClient client=HttpClientUtil.getHttpClient();
HttpclientUtil 用的连接池的概念,当请求结束并不是直接断开连接,而是返回给连接池方便下次调用。
public class HttpClientUtil { private static Logger LOGGER = LoggerFactory.getLogger(HttpClientUtil.class); private static PoolingHttpClientConnectionManager cm = null; static { LayeredConnectionSocketFactory sslsf = null; try { sslsf = new SSLConnectionSocketFactory(SSLContext.getDefault()); } catch (NoSuchAlgorithmException e) { LOGGER.error("创建SSL连接失败"); } Registry<ConnectionSocketFactory> socketFactoryRegistry = RegistryBuilder.<ConnectionSocketFactory>create() .register("https", sslsf) .register("http", new PlainConnectionSocketFactory()) .build(); cm =new PoolingHttpClientConnectionManager(socketFactoryRegistry); cm.setMaxTotal(200);//多线程调用注意配置,根据线程数设定 cm.setDefaultMaxPerRoute(300);//多线程调用注意配置,根据线程数设定 } public static CloseableHttpClient getHttpClient() { CloseableHttpClient httpClient = HttpClients.custom() .setConnectionManager(cm) .build(); return httpClient; } }此处则创建httpclient 成功,下一注意点为 RequestConfig 参数配置,此处配置直接影响调用接口返回速度及是否 出现 Read time out 异常。
(2)参数配置
CloseableHttpClient client=HttpClientUtil.getHttpClient(); // 发送get请求 HttpGet request = new HttpGet(url); CloseableHttpResponse response=null; // 设置请求和传输超时时间 RequestConfig requestConfig = RequestConfig.custom() .setSocketTimeout(20000)//数据传输过程中数据包之间间隔的最大时间 .setConnectTimeout(20000)//连接建立时间,三次握手完成时间 .setExpectContinueEnabled(true)//重点参数 .setConnectionRequestTimeout(10000) .setStaleConnectionCheckEnabled(true)//重点参数,在请求之前校验链接是否有效 .build(); request.setConfig(requestConfig); try { response = client.execute(request); if(response.getStatusLine().getStatusCode() != HttpStatus.SC_OK) { System.out.println("请求失败"); return null; } HttpEntity resEntity = response.getEntity(); if(resEntity==null){ return null; } String result=EntityUtils.toString(resEntity, "UTF-8"); return result; } catch (UnsupportedEncodingException e) { System.out.println(e.getMessage()); return null; } catch (ClientProtocolException e) { System.out.println(e.getMessage()); return null; } catch (IOException e) { System.out.println(e.getMessage()); }finally { if(response != null) { try {
//此处调优重点,多线程模式下可提高性能。 EntityUtils.consume(response.getEntity());//此处高能,通过源码分析,由EntityUtils是否回收HttpEntity response.close(); } catch (IOException e) { System.out.println("关闭response失败:"+ e); } } }
此配置,经过2周观察,暂时是最高效的,感觉网上大牛写的不一定是最适合你的,自己多调试,配置出属于自己最高效的。