Web开发 之HttpClient详解

参考网址

HttpClient中文官网
HttpClient Apache官方文档
HttpClient 4.3教程
一个非常好的HttpClient系列教程

Maven依赖

<dependency>
    <groupId>org.apache.httpcomponents</groupId>
    <artifactId>httpclient</artifactId>
    <version>4.5.10</version>
</dependency>
<dependency>
    <groupId>com.alibaba</groupId>
    <artifactId>fastjson</artifactId>
    <version>1.2.62</version>
</dependency>

HttpClient模拟post请求示例

使用HttpClient发送请求、接收响应一般步骤:

  • 创建HttpClient对象。
  • 创建请求方法的实例,并指定请求URL。如果需要发送GET请求,创建HttpGet对象;如果需要发送POST请求,创建HttpPost对象。
  • 如果需要发送请求参数,可调用HttpGet、HttpPost共同的setParams(HttpParams params)方法来添加请求参数;对于HttpPost对象而言,也可调用setEntity(HttpEntity entity)方法来设置请求参数。
  • 调用HttpClient对象的execute(HttpUriRequest request)发送请求,该方法返回一个HttpResponse。
  • 调用HttpResponse的getAllHeaders()、getHeaders(String name)等方法可获取服务器的响应头;调用HttpResponse的getEntity()方法可获取HttpEntity对象,该对象包装了服务器的响应内容。程序可通过该对象获取服务器的响应内容。
  • 释放连接。无论执行方法是否成功,都必须释放连接。
import org.apache.http.HttpEntity;
import org.apache.http.NameValuePair;
import org.apache.http.ParseException;
import org.apache.http.client.entity.UrlEncodedFormEntity;
import org.apache.http.client.methods.CloseableHttpResponse;
import org.apache.http.client.methods.HttpPost;
import org.apache.http.impl.client.CloseableHttpClient;
import org.apache.http.impl.client.HttpClients;
import org.apache.http.message.BasicNameValuePair;
import org.apache.http.util.EntityUtils;

import java.io.IOException;
import java.util.ArrayList;
import java.util.List;
import java.util.Map;

public class SimpleHttpClientDemo {
    /**
     * 模拟请求
     *
     * @param url      资源地址
     * @param map      参数列表
     * @param encoding 编码
     * @return
     * @throws ParseException
     * @throws IOException
     */
    public static String send(String url, Map<String, String> map, String encoding) throws ParseException, IOException {
        String body = "";

        //创建httpclient对象
        CloseableHttpClient client = HttpClients.createDefault();
        //创建post方式请求对象
        HttpPost httpPost = new HttpPost(url);

        //装填参数
        List<NameValuePair> nvps = new ArrayList<NameValuePair>();
        if (map != null) {
            for (Map.Entry<String, String> entry : map.entrySet()) {
                nvps.add(new BasicNameValuePair(entry.getKey(), entry.getValue()));
            }
        }
        //设置参数到请求对象中
        httpPost.setEntity(new UrlEncodedFormEntity(nvps, encoding));

        System.out.println("请求地址:" + url);
        System.out.println("请求参数:" + nvps.toString());

        //设置header信息
        //指定报文头【Content-type】、【User-Agent】
        httpPost.setHeader("Content-type", "application/x-www-form-urlencoded");
        httpPost.setHeader("User-Agent", "Mozilla/4.0 (compatible; MSIE 5.0; Windows NT; DigExt)");

        //执行请求操作,并拿到结果(同步阻塞)
        CloseableHttpResponse response = client.execute(httpPost);
        //获取结果实体
        HttpEntity entity = response.getEntity();
        if (entity != null) {
            //按指定编码转换结果实体为String类型
            body = EntityUtils.toString(entity, encoding);
        }
        EntityUtils.consume(entity);
        //释放链接
        response.close();
        return body;
    }

}

HttpClient连接池

HttpClient连接池设置引发的一次雪崩
httpclient连接池的测试报告(jmeter做并发测试工具)
HttpClient高并发下性能优化
HttpClient连接池的一些思考
httpclient连接池是否能真的提高性能
Http持久连接与HttpClient连接池
HttpClient连接池配置

PoolingHttpClientConnectionManager pool = new PoolingHttpClientConnectionManager();
// 设置连接池的最大连接数
pool.setMaxTotal(200);
// 设置每个路由的最大连接数,路由是对MaxTotal的细分
pool.setDefaultMaxPerRoute(10);
// 在从连接池获取连接时,连接不活跃多长时间后需要进行一次验证,默认2s
pool.setValidateAfterInactivity(5*1000);
// 单独为某个站点设置最大连接个数
pool.setMaxPerRoute(new HttpRoute(new HttpHost("127.0.0.1", 80)), 50);
//默认请求配置
RequestConfig requestConfig = RequestConfig.custom()
        .setConnectTimeout(2*1000)// 设置连接超时时间 2s
        .setSocketTimeout(5*1000)// 设置等待数据超时时间 5s
        .setConnectionRequestTimeout(2000)// 设置从连接池获取连接的等待超时时间
        .build();
CloseableHttpClient httpClient = HttpClients.custom().setConnectionManager(pool)
                .evictIdleConnections(1, TimeUnit.MINUTES) //定期回收空闲连接
                .setDefaultRequestConfig(requestConfig) // 设置默认请求配置
                .evictExpiredConnections()//定期回收过期连接
                .build();

为什么要用Http连接池

1、降低延迟:如果不采用连接池,每次连接发起Http请求的时候都会重新建立TCP连接(经历3次握手),用完就会关闭连接(4次挥手),如果采用连接池则减少了这部分时间损耗。

2、支持更大的并发:如果不采用连接池,每次连接都会打开一个端口,在大并发的情况下系统的端口资源很快就会被用完,导致无法建立新的连接。
如短连接去做压测,可能会报错:java.net.NoRouteToHostException: Cannot assign requested address。

而我们一般用的长连接是通过添加Keep-Alive头参解决: header.add(new NVPair(“Connection”, “Keep-Alive”));

而事实上头参中加了此参数只是保持长连接,与建立TCP连接保持的长连接不一样。

HttpClient connecttimeout与sockettimeout区别

HttpClient 作为 Http 工具包,是基于socket的封装,所以 HttpClient 的connectTimeout、socketTimeout其实都是socket的connectTimeout、socketTimeout。

connectTimeout: 建立链接的最大时间,也就是完成三次握手,建立TCP链接所用的时间。
socketTimeout: InputStream上调用read()方法阻塞的最大时间,即等待数据的时间或者两个包之间的间隔时间。

  • 程序中最好设置connectTimeout、socketTimeout,可以防止阻塞
    • 如果不设置connectTimeout会导致,建立TCP链接时,阻塞,假死。
    • 如果不设置socketTimeout会导致,已经建立了TCP链接,在通信时,发送了请求报文,恰好此时,网络断掉,程序就阻塞,假死在那。
  • 有时,connectTimeout并不像你想的那样一直到最大时间
    • socket建立链接时,如果网络层确定不可达,会直接抛出异常,不会一直到connectTimeout的设定值。
    • socket连接一个不存在的ip,很快就会提示连不上,而不是等到指定时间才报错。推测可能底层网络已经确定目标不可达所以不会一直重试直到connecttimeout。

HttpClient重试

关于HttpClient重试策略的研究

提供两种重试机制:

  • HttpRequestRetryHandler:针对异常的重试(例如:connect timed out、read timed out)
  • ServiceUnavailableRetryStrategy:针对响应错误码的重试(例如:400、500等)

HttpRequestRetryHandler

HttpClient提供针对异常重试的接口,实现该接口就可以编写异常重试方案。

DefaultHttpRequestRetryHandler

HttpClient提供的默认重试方案。

StandardHttpRequestRetryHandler

这是官方提供的一个标准的retry方案,为了保证幂等性约定Resetful接口必须是GET, HEAD, PUT, DELETE, OPTIONS, and TRACE中的一种。
在这里插入图片描述

自定义重试

实现 HttpRequestRetryHandler 接口,在 retryRequest 方法中编写重试逻辑。

public static CloseableHttpClient getHttpClient() {
    HttpRequestRetryHandler httpRequestRetryHandler = new HttpRequestRetryHandler() {
        // exception:异常,executionCount:重试次数
        // 返回true表示重试,返回false表示不需要重试
        public boolean retryRequest(IOException exception, int executionCount, HttpContext context) {
            System.out.println("重试第" + executionCount + "次");
            if (executionCount >= 3) {
                // Do not retry if over max retry count
                return false;
            }
            if (exception instanceof InterruptedIOException) {
                // Timeout
                return false;
            }
            if (exception instanceof UnknownHostException) {
                // Unknown host
                return false;
            }
            if (exception instanceof ConnectTimeoutException) {
                // Connection refused
                return false;
            }
            if (exception instanceof SSLException) {
                // SSL handshake exception
                return false;
            }
            HttpClientContext clientContext = HttpClientContext.adapt(context);
            HttpRequest request = clientContext.getRequest();
            boolean idempotent = !(request instanceof HttpEntityEnclosingRequest);
            if (idempotent) {
                // 如果请求被认为是幂等的,那么就重试
                return true;
            }
            return false;
        }
    };
    return HttpClients.custom().setRetryHandler(httpRequestRetryHandler)
            .setConnectionManager(new PoolingHttpClientConnectionManager()).build();
}

ServiceUnavailableRetryStrategy

针对响应错误码的重试,并且可以自定义重试时间的间隔。


public class MyServiceUnavailableRetryStrategy implements ServiceUnavailableRetryStrategy {
    private int executionCount;
    private long retryInterval;
    MyServiceUnavailableRetryStrategy(Builder builder) {
        this.executionCount = builder.executionCount;
        this.retryInterval = builder.retryInterval;
    }
    // retry逻辑
    @Override
    public boolean retryRequest(HttpResponse response, int executionCount, HttpContext context) {
        if (response.getStatusLine().getStatusCode() != 200 && executionCount < this.executionCount){
            // 表示错误码不是200并且小于最大重试次数进行重试
            return true;
        }
        else{
            return false;
        }
    }
    // retry间隔时间
    @Override
    public long getRetryInterval() {
        return this.retryInterval;
    }
    public static final class Builder {
        private int executionCount;
        private long retryInterval;
        public Builder() {
            executionCount = 3;
            retryInterval = 1000;
        }
        public Builder executionCount(int executionCount) {
            this.executionCount = executionCount;
            return this;
        }
        public Builder retryInterval(long retryInterval) {
            this.retryInterval = retryInterval;
            return this;
        }
        public MyServiceUnavailableRetryStrategy build() {
            return new MyServiceUnavailableRetryStrategy(this);
        }
    }
}
public static CloseableHttpClient getHttpClient() {
    ServiceUnavailableRetryStrategy serviceUnavailableRetryStrategy = new MyServiceUnavailableRetryStrategy.Builder()
            .executionCount(3)
            .retryInterval(1000)
            .build();
    return HttpClients.custom().setServiceUnavailableRetryStrategy(serviceUnavailableRetryStrategy)
            .setConnectionManager(new PoolingHttpClientConnectionManager()).build();
}

HttpClient封装

HttpClient用法–这一篇全了解(内含例子)
HttpClient详细使用示例

httpclient.properties 配置:

http.maxTotal=200
http.defaultMaxPerRoute=50
http.connectTimeout=2000
http.connectionRequestTimeout=500
http.socketTimeout=10000
http.staleConnectionCheckEnabled=true

Bean配置类:

import lombok.Getter;
import lombok.Setter;
import org.apache.http.client.config.RequestConfig;
import org.apache.http.impl.client.CloseableHttpClient;
import org.apache.http.impl.client.HttpClients;
import org.apache.http.impl.conn.PoolingHttpClientConnectionManager;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.context.annotation.PropertySource;

import java.util.concurrent.TimeUnit;

@Configuration
@PropertySource(value = "classpath:httpclient.properties")
@Getter
@Setter
public class HttpClientConfig {
    @Value("${http.maxTotal}")
    private int maxTotal;
    @Value("${http.defaultMaxPerRoute}")
    private int defaultMaxPerRoute;
    @Value("${http.connectTimeout}")
    private int connectTimeout;
    @Value("${http.connectionRequestTimeout}")
    private int connectionRequestTimeout;
    @Value("${http.socketTimeout}")
    private int socketTimeout;
    @Value("${http.staleConnectionCheckEnabled}")
    private boolean staleConnectionCheckEnabled;

    @Bean
    public CloseableHttpClient httpClient(){
        PoolingHttpClientConnectionManager pool = new PoolingHttpClientConnectionManager();
        // 设置连接池的最大连接数
        pool.setMaxTotal(maxTotal);
        // 设置每个路由的最大连接数,路由是对MaxTotal的细分
        pool.setDefaultMaxPerRoute(defaultMaxPerRoute);
        return HttpClients.custom().setConnectionManager(pool)
                // 定期回收空闲连接
                .evictIdleConnections(1, TimeUnit.MINUTES)
                // 定期回收过期连接
                .evictExpiredConnections()
                .build();
    }

    @Bean
    public RequestConfig requestConfig(){
        return RequestConfig.custom()
                // 设置连接超时时间
                .setConnectTimeout(connectTimeout)
                // 设置等待数据超时时间
                .setSocketTimeout(socketTimeout)
                // 设置从连接池获取连接的等待超时时间
                .setConnectionRequestTimeout(connectionRequestTimeout)
                .build();
    }
}

Http响应结果类:

public class HttpResult {
    private Integer status;
    private String content;
    public HttpResult(){}

    public HttpResult(Integer status, String content) {
        this.status = status;
        this.content = content;
    }

    public Integer getStatus() {
        return status;
    }

    public void setStatus(Integer status) {
        this.status = status;
    }

    public String getContent() {
        return content;
    }

    public void setContent(String content) {
        this.content = content;
    }
}

Http请求封装类:

import org.apache.http.NameValuePair;
import org.apache.http.client.ClientProtocolException;
import org.apache.http.client.config.RequestConfig;
import org.apache.http.client.entity.UrlEncodedFormEntity;
import org.apache.http.client.methods.CloseableHttpResponse;
import org.apache.http.client.methods.HttpDelete;
import org.apache.http.client.methods.HttpEntityEnclosingRequestBase;
import org.apache.http.client.methods.HttpGet;
import org.apache.http.client.methods.HttpPost;
import org.apache.http.client.methods.HttpPut;
import org.apache.http.client.methods.HttpRequestBase;
import org.apache.http.client.utils.URIBuilder;
import org.apache.http.entity.ContentType;
import org.apache.http.entity.StringEntity;
import org.apache.http.impl.client.CloseableHttpClient;
import org.apache.http.message.BasicNameValuePair;
import org.apache.http.util.EntityUtils;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.stereotype.Component;

import java.io.IOException;
import java.io.UnsupportedEncodingException;
import java.net.URISyntaxException;
import java.util.ArrayList;
import java.util.HashMap;
import java.util.List;
import java.util.Map;

@Component
public class HttpClientUtil {

    @Autowired(required = false)
    private CloseableHttpClient httpClient;

    @Autowired(required = false)
    private RequestConfig requestConfig;

    /**
     * 执行GET请求,如果请求状态不为200返回null
     *
     * @param url
     * @return
     * @throws ClientProtocolException
     * @throws IOException
     */
    public String doGet(String url) throws ClientProtocolException, IOException {
        // 创建http GET请求
        HttpGet httpGet = new HttpGet(url);
        httpGet.setConfig(requestConfig);

        CloseableHttpResponse response = null;
        try {
            // 执行请求
            response = httpClient.execute(httpGet);
            // 判断返回状态是否为200
            if (response.getStatusLine().getStatusCode() == 200) {
                return EntityUtils.toString(response.getEntity(), "UTF-8");
            }
        } finally {
            if (response != null) {
                response.close();
            }
        }
        return null;
    }

    /**
     * 带参数的GET请求
     *
     * @param url
     * @param params
     * @return
     * @throws URISyntaxException
     * @throws ClientProtocolException
     * @throws IOException
     */
    public String doGet(String url, Map<String, Object> params) throws URISyntaxException,
            ClientProtocolException, IOException {
        // 拼接URL
        URIBuilder uriBuilder = new URIBuilder(url);
        if (null != params) {
            for (Map.Entry<String, Object> entry : params.entrySet()) {
                uriBuilder.setParameter(entry.getKey(), entry.getValue().toString());
            }
        }
        String newUrl = uriBuilder.build().toString();
        return doGet(newUrl);
    }

    /**
     * 执行doPOST请求,请求响应状态不为200返回null
     *
     * @param url
     * @param params
     * @return
     * @throws ClientProtocolException
     * @throws IOException
     */
    public HttpResult doPost(String url, Map<String, Object> params) throws ClientProtocolException,
            IOException {
        // 创建http POST请求
        HttpPost httpPost = new HttpPost(url);
        httpPost.setConfig(requestConfig);
        setFormEntity(httpPost, params);
        return doHttp(httpPost);
    }

    /**
     * post提交json
     *
     * @param url
     * @param json
     * @return
     * @throws ClientProtocolException
     * @throws IOException
     */
    public HttpResult doPostJson(String url, String json) throws ClientProtocolException, IOException {
        // 创建http POST请求
        HttpPost httpPost = new HttpPost(url);
        httpPost.setConfig(requestConfig);
        // 设置参数
        if (null != json) {
            // 构造一个form表单式的实体
            StringEntity stringEntity = new StringEntity(json, ContentType.APPLICATION_JSON);
            // 将请求实体设置到httpPost对象中
            httpPost.setEntity(stringEntity);
        }

        return doHttp(httpPost);
    }

    /**
     * 执行POST请求
     *
     * @param url
     * @return
     * @throws ClientProtocolException
     * @throws IOException
     */
    public HttpResult doPost(String url) throws ClientProtocolException, IOException {
        return doPost(url, null);
    }

    /**
     * 执行doPut请求,请求响应状态不为200返回null
     *
     * @param url
     * @return
     * @throws ClientProtocolException
     * @throws IOException
     */
    public HttpResult doPut(String url) throws ClientProtocolException, IOException {
        return doPut(url, null);
    }

    /**
     * 执行doPut请求,请求响应状态不为200返回null
     *
     * @param url
     * @param params
     * @return
     * @throws ClientProtocolException
     * @throws IOException
     */
    public HttpResult doPut(String url, Map<String, Object> params) throws ClientProtocolException,
            IOException {
        // 创建http PUT请求
        HttpPut httpPut = new HttpPut(url);
        httpPut.setConfig(requestConfig);
        setFormEntity(httpPut, params);
        return doHttp(httpPut);
    }

    /**
     * 执行DELETE请求
     *
     * @param url
     * @return
     * @throws ClientProtocolException
     * @throws IOException
     */
    public HttpResult doDelete(String url) throws ClientProtocolException, IOException {
        // 创建http PUT请求
        HttpDelete httpDelete = new HttpDelete(url);
        httpDelete.setConfig(requestConfig);
        return doHttp(httpDelete);
    }

    /**
     * 执行DELETE请求,使用_method指定DELETE请求,实际发送的POST请求
     *
     * @param url
     * @param params
     * @return
     * @throws ClientProtocolException
     * @throws IOException
     */
    public HttpResult doDelete(String url, Map<String, Object> params) throws ClientProtocolException,
            IOException {
        if (params == null) {
            params = new HashMap<String, Object>();
        }
        params.put("_method", "DELETE");
        return doPost(url, params);
    }

    private void setFormEntity(HttpEntityEnclosingRequestBase requestBase, Map<String, Object> params)
            throws UnsupportedEncodingException {
        // 设置参数
        if (null != params) {
            // 参数列表
            List<NameValuePair> parameters = new ArrayList<NameValuePair>(0);
            for (Map.Entry<String, Object> entry : params.entrySet()) {
                parameters.add(new BasicNameValuePair(entry.getKey(), entry.getValue().toString()));
            }
            // 构造一个form表单式的实体
            UrlEncodedFormEntity formEntity = new UrlEncodedFormEntity(parameters, "UTF-8");
            // 将请求实体设置到httpPost对象中
            requestBase.setEntity(formEntity);
        }
    }

    private HttpResult doHttp(HttpRequestBase requestBase) throws ClientProtocolException, IOException {
        CloseableHttpResponse response = null;
        try {
            // 执行请求
            response = httpClient.execute(requestBase);
            // 判断返回状态是否为200
            if (response.getStatusLine().getStatusCode() == 200) {
                return new HttpResult(200, EntityUtils.toString(response.getEntity(), "UTF-8"));
            } else {
                return new HttpResult(response.getStatusLine().getStatusCode(), null);
            }
        } finally {
            if (response != null) {
                response.close();
            }
        }
    }

}
  • 1
    点赞
  • 4
    收藏
    觉得还不错? 一键收藏
  • 0
    评论
HttpClient 网络不可达可能是由于以下几个原因导致的。首先,可能是网络连接出现了问题,比如网络故障或者服务器宕机。其次,可能是由于网络配置问题,比如防火墙阻止了对服务器的访问。最后,可能是由于使用的 HttpClient 版本过旧或者配置不正确导致的。在处理这种情况时,可以考虑以下几个解决方案。首先,检查网络连接是否正常,确保网络畅通。其次,检查防火墙设置,确保允许对服务器的访问。最后,可以尝试升级 HttpClient 版本或者重新配置 HttpClient 的连接参数,比如设置适当的连接超时时间和读取超时时间,以适应网络环境。\[1\]\[2\]\[3\] #### 引用[.reference_title] - *1* *3* [Web开发HttpClient详解](https://blog.csdn.net/const_/article/details/104424916)[target="_blank" data-report-click={"spm":"1018.2226.3001.9630","extra":{"utm_source":"vip_chatgpt_common_search_pc_result","utm_medium":"distribute.pc_search_result.none-task-cask-2~all~insert_cask~default-1-null.142^v91^control_2,239^v3^insert_chatgpt"}} ] [.reference_item] - *2* [网关使用 Apache HttpClient 连接池出现异常](https://blog.csdn.net/huangjinjin520/article/details/125176293)[target="_blank" data-report-click={"spm":"1018.2226.3001.9630","extra":{"utm_source":"vip_chatgpt_common_search_pc_result","utm_medium":"distribute.pc_search_result.none-task-cask-2~all~insert_cask~default-1-null.142^v91^control_2,239^v3^insert_chatgpt"}} ] [.reference_item] [ .reference_list ]
评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值