httpclient使用系列

最新推荐文章于 2024-03-28 14:03:02 发布

GKTDSR

最新推荐文章于 2024-03-28 14:03:02 发布

阅读量417

点赞数

分类专栏：技术文章标签： Java HTTPclient

本文链接：https://blog.csdn.net/GKTDSR/article/details/79406066

版权

技术专栏收录该内容

27 篇文章 0 订阅

订阅专栏

1.httpclient介绍：

httpclient我理解为就是Java做到模拟访问指定浏览器的以获取指定信息或者制定操作的一种技术，而其实现就是通过自己拼装请求头以及请求参数必要时还可以自己组装cookie来实现程序端访问网站服务器达到自动化的过程。

2.httpclient常用方法：

1. 创建HttpClient 。

2.创建请求实例，指定URL。发送GET请求，创建HttpGet对象；发送POST请求，创建HttpPost对象。

3.传递请求参数，可调用HttpGet、HttpPost共同的setParams(HetpParams params)方法添加请求参数；对于HttpPost，也可调用setEntity(HttpEntity entity)方法来设置请求参数。

4.调用HttpClient对象的execute(HttpUriRequest request)发送请求，返回一个HttpResponse。

5.调用HttpResponse的getAllHeaders()、getHeaders(String name)等方法可获取服务器的响应头；调用HttpResponse的getEntity()方法可获取HttpEntity对象，该对象包装了服务器返回的信息。

6.释放连接。

具体代码示例：

2.1 例(httpclient提交基类)：

import java.io.IOException;
import java.io.UnsupportedEncodingException;
import java.io.OutputStream;
import java.io.BufferedReader;
import java.io.InputStreamReader;
import java.io.DataOutputStream;
import java.net.HttpURLConnection;
import java.net.URL;
import java.security.KeyManagementException;
import java.security.NoSuchAlgorithmException;
import java.security.cert.X509Certificate;
import java.util.Arrays;
import java.util.HashMap;
import java.util.List;
import java.util.Map;

import javax.net.ssl.SSLContext;
import javax.net.ssl.TrustManager;
import javax.net.ssl.X509TrustManager;

import org.apache.commons.lang.StringUtils;
import org.apache.http.Header;
import org.apache.http.HttpEntity;
import org.apache.http.HttpHost;
import org.apache.http.HttpResponse;
import org.apache.http.NameValuePair;
import org.apache.http.StatusLine;
import org.apache.http.client.CookieStore;
import org.apache.http.client.config.CookieSpecs;
import org.apache.http.client.config.RequestConfig;
import org.apache.http.client.entity.UrlEncodedFormEntity;
import org.apache.http.client.methods.CloseableHttpResponse;
import org.apache.http.client.methods.HttpGet;
import org.apache.http.client.methods.HttpPost;
import org.apache.http.client.methods.HttpUriRequest;
import org.apache.http.config.SocketConfig;
import org.apache.http.conn.ssl.SSLConnectionSocketFactory;
import org.apache.http.cookie.Cookie;
import org.apache.http.entity.StringEntity;
import org.apache.http.impl.client.BasicCookieStore;
import org.apache.http.impl.client.CloseableHttpClient;
import org.apache.http.impl.client.HttpClients;
import org.apache.http.impl.cookie.BasicClientCookie;
import org.apache.http.message.BasicNameValuePair;
import org.apache.http.util.EntityUtils;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

import com.google.common.base.Charsets;
import com.google.common.collect.Lists;


/**
 * www 公共的HTTP 服务
 */
public class BrowserHttpClient {

    private final static Logger logger = LoggerFactory.getLogger(BrowserHttpClient.class);

    private final static int SOCKETTIMEOUT = 10000;
    private final static int CONNECTIONTIMEOUT = 5000;
    private final static int CONNREQUESTTIMEOUT = 5000;
    private final static int SOCKETSOTIMEOUT = 15000;

    private final static int PROXY_RECONNECT_CODE = 601;

    private CloseableHttpClient browserHttpClient = null;
    private RequestConfig requestConfig = null;
    private BasicCookieStore cookieStore = null;

    public BrowserHttpClient(String ip, int port, Cookie[] cookies) {
        cookieStore = new BasicCookieStore();
        RequestConfig.Builder builder = RequestConfig.custom().setSocketTimeout(SOCKETTIMEOUT)
                .setConnectionRequestTimeout(CONNREQUESTTIMEOUT)
                .setConnectTimeout(CONNECTIONTIMEOUT).setCookieSpec(CookieSpecs.BROWSER_COMPATIBILITY);
        if (cookies != null && cookies.length > 0) {
            cookieStore.addCookies(cookies);
        }
        if (StringUtils.isNotBlank(ip)) {
            HttpHost proxy = new HttpHost(ip, port);
            requestConfig = builder.setProxy(proxy).build();
        } else {
            requestConfig = builder.build();
        }
        this.browserHttpClient = createHttpsClient();
    }

    /**
     * 默认的常规的
     */
    public BrowserHttpClient(String ip, int port, Cookie[] cookies, int conTimeout, int socketTimeout) {
        cookieStore = new BasicCookieStore();
        RequestConfig.Builder builder = RequestConfig.custom().setSocketTimeout(socketTimeout)
                .setConnectionRequestTimeout(CONNREQUESTTIMEOUT)
                .setConnectTimeout(conTimeout).setCookieSpec(CookieSpecs.BROWSER_COMPATIBILITY);
        if (cookies != null && cookies.length > 0) {
            cookieStore.addCookies(cookies);
        }
        if (StringUtils.isNotBlank(ip)) {
            HttpHost proxy = new HttpHost(ip, port);
            requestConfig = builder.setProxy(proxy).build();
        } else {
            requestConfig = builder.build();
        }
        this.browserHttpClient = createHttpsClient();
    }


    /**
     * 是否允许循环重定向
     */
    public BrowserHttpClient( boolean isRedirect,String ip, int port, Cookie[] cookies, int conTimeout, int socketTimeout) {
        cookieStore = new BasicCookieStore();
        RequestConfig.Builder builder = RequestConfig.custom().setSocketTimeout(socketTimeout)
                .setConnectionRequestTimeout(CONNREQUESTTIMEOUT)
                .setConnectTimeout(conTimeout).setCookieSpec(CookieSpecs.BROWSER_COMPATIBILITY)
                .setCircularRedirectsAllowed(isRedirect).setMaxRedirects(10);

        if (cookies != null && cookies.length > 0) {
            cookieStore.addCookies(cookies);
        }
        if (StringUtils.isNotBlank(ip)) {
            HttpHost proxy = new HttpHost(ip, port);
            requestConfig = builder.setProxy(proxy).build();
        } else {
            requestConfig = builder.build();
        }
        this.browserHttpClient = createHttpsClient();
    }


    /**
     * 是否允许单重定向
     */
    public BrowserHttpClient(String ip, int port, Cookie[] cookies, int conTimeout, int socketTimeout, boolean isRedirect) {
        cookieStore = new BasicCookieStore();
        RequestConfig.Builder builder = RequestConfig.custom().setSocketTimeout(socketTimeout)
                .setConnectionRequestTimeout(CONNREQUESTTIMEOUT)
                .setConnectTimeout(conTimeout).setCookieSpec(CookieSpecs.BROWSER_COMPATIBILITY)
                .setRedirectsEnabled(isRedirect);

        if (cookies != null && cookies.length > 0) {
            cookieStore.addCookies(cookies);
        }
        if (StringUtils.isNotBlank(ip)) {
            HttpHost proxy = new HttpHost(ip, port);
            requestConfig = builder.setProxy(proxy).build();
        } else {
            requestConfig = builder.build();
        }
        this.browserHttpClient = createHttpsClient();
    }

    public void updateCookieStore(Cookie[] cookies) {
        cookieStore.addCookies(cookies);
    }

    public void updateCookieStore(String str) {
        if (StringUtils.isNotBlank(str)) {
            String[] strs = str.split(";");
            for (int i = 0; i < strs.length; i++) {
                int opstion = strs[i].indexOf("=");
                int length = strs[i].length();
                BasicClientCookie cookie = new BasicClientCookie(strs[i].substring(0, opstion),
                        strs[i].substring(opstion + 1, length));
                cookie.setDomain("ch.com");
                cookieStore.clear();
                cookieStore.addCookie(cookie);
            }
        }
    }

    public CookieStore getCookieStore() {
        return cookieStore;
    }


    public String get(String url, Header[] headers, int retryCount) throws IOException {
        for (int i = 0; i < retryCount; i++) {
            HttpGet httpGet = new HttpGet(url);
            httpGet.setConfig(requestConfig);
            if (headers != null && headers.length > 0) {
                httpGet.setHeaders(headers);
            }
            try {
                HttpResponse httpResponse = browserHttpClient.execute(httpGet);
                int code = httpResponse.getStatusLine().getStatusCode();
                logger.info("statusCode:{}", code);
                String response = EntityUtils.toString(httpResponse.getEntity(), Charsets.UTF_8.toString());
                if (code == PROXY_RECONNECT_CODE) {
                    QMonitor.recordOne("packetix_reconnect_stop_trade");
                    response = null;
                }
                if (response != null && response.contains("The requested URL could not be retrieved")
                        && i < retryCount - 1) {
                    logger.warn("可能是代理IP异常，重试:{}", response);
                    try {
                        Thread.sleep(1000);
                    } catch (InterruptedException e) {
                        logger.error("sleep异常", e);
                    }
                    continue;
                }
                return response;
            } catch (IOException e) {
                if (i == retryCount - 1) {
                    throw e;
                } else {
                    logger.warn("请求网络异常重试", e);
                    try {
                        Thread.sleep(1000);
                    } catch (InterruptedException e2) {
                        logger.error("sleep异常", e2);
                    }
                    continue;
                }
            } finally {
                httpGet.releaseConnection();
            }
        }
        return null;
    }

    public byte[] getEntity(String url, Header[] headers) throws IOException {
        HttpGet httpGet = new HttpGet(url);
        httpGet.setConfig(requestConfig);
        if (headers != null && headers.length > 0) {
            httpGet.setHeaders(headers);
        }
        try {
            HttpResponse httpResponse = browserHttpClient.execute(httpGet);
            int code = httpResponse.getStatusLine().getStatusCode();
            if (code == PROXY_RECONNECT_CODE) {
                QMonitor.recordOne("packetix_reconnect_stop_trade");
                return null;
            }
            return EntityUtils.toByteArray(httpResponse.getEntity());
        } finally {
            httpGet.releaseConnection();
        }
    }

    public String get(String url, Header[] headers, List<Header> resHeaders) throws IOException {
        HttpGet httpGet = new HttpGet(url);
        httpGet.setConfig(requestConfig);
        if (headers != null && headers.length > 0) {
            httpGet.setHeaders(headers);
        }

        try {
            HttpResponse httpResponse = browserHttpClient.execute(httpGet);
            int code = httpResponse.getStatusLine().getStatusCode();
            logger.info("statusCode:{}", code);
            resHeaders.addAll(Arrays.asList(httpResponse.getAllHeaders()));
            if (code == PROXY_RECONNECT_CODE) {
                QMonitor.recordOne("packetix_reconnect_stop_trade");
                return "";
            }
            return EntityUtils.toString(httpResponse.getEntity(), Charsets.UTF_8.toString());
        } finally {
            httpGet.releaseConnection();
        }
    }

   public String get(String url, Header[] headers) throws IOException {
      HttpGet httpGet = new HttpGet(url);
      httpGet.setConfig(requestConfig);
      if (headers != null && headers.length > 0) {
         httpGet.setHeaders(headers);
      }

      try {
         HttpResponse httpResponse = browserHttpClient.execute(httpGet);
            int code = httpResponse.getStatusLine().getStatusCode();
         logger.info("statusCode:{}", code);
            if (code == PROXY_RECONNECT_CODE) {
                QMonitor.recordOne("packetix_reconnect_stop_trade");
                return "";
            }
         return EntityUtils.toString(httpResponse.getEntity(), Charsets.UTF_8.toString());
      } finally {
         httpGet.releaseConnection();
      }
   }

    public Map<String,String> getStatusCode(String url, Header[] headers) throws IOException {
        HttpGet httpGet = new HttpGet(url);
        httpGet.setConfig(requestConfig);
        if (headers != null && headers.length > 0) {
            httpGet.setHeaders(headers);
        }
        try {
            HttpResponse httpResponse = browserHttpClient.execute(httpGet);
            logger.info("statusCode:{}", httpResponse.getStatusLine().getStatusCode());
            Map<String,String>res=new HashMap<String, String>();
            int code = httpResponse.getStatusLine().getStatusCode();
            res.put("statusCode", String.valueOf(code));
            String response = EntityUtils.toString(httpResponse.getEntity(), Charsets.UTF_8.toString());
            if (code == PROXY_RECONNECT_CODE) {
                QMonitor.recordOne("packetix_reconnect_stop_trade");
                response = "";
            }
            res.put("result", response);
            return res;
        } finally {
            httpGet.releaseConnection();
        }
    }

   public String post(String url, List<NameValuePair> nvPairs, Header[] headers) throws IOException {
        HttpPost httpPost = null;
        try {
            httpPost = new HttpPost(url);
            httpPost.setConfig(requestConfig);

            UrlEncodedFormEntity urlEncodedFormEntity = new UrlEncodedFormEntity(nvPairs, Charsets.UTF_8.toString());
//            urlEncodedFormEntity.setContentType("multipart/form-data");
            httpPost.setHeaders(headers);
            httpPost.setEntity(urlEncodedFormEntity);
            HttpResponse httpResp = browserHttpClient.execute(httpPost);
            int code = httpResp.getStatusLine().getStatusCode();
            logger.info("statusCode:{}", code);
            if (code == PROXY_RECONNECT_CODE) {
                QMonitor.recordOne("packetix_reconnect_stop_trade");
                return "";
            }
            return EntityUtils.toString(httpResp.getEntity(), Charsets.UTF_8.toString());
        } finally {
            if (httpPost != null) {
                httpPost.releaseConnection();
            }
        }
    }

    public String post(String url, Map<String, String> parameter) throws IOException {
        HttpPost httpPost = null;
        try {
            httpPost = new HttpPost(url);
            httpPost.setConfig(requestConfig);
            httpPost.setEntity(mapToHttpEntity(parameter));

            HttpResponse httpResp = browserHttpClient.execute(httpPost);
            int code = httpResp.getStatusLine().getStatusCode();
            logger.info("statusCode:{}", code);
            if (code == PROXY_RECONNECT_CODE) {
                QMonitor.recordOne("packetix_reconnect_stop_trade");
                return "";
            }
            return EntityUtils.toString(httpResp.getEntity(), Charsets.UTF_8.toString());

        } finally {
            if (httpPost != null) {
                httpPost.releaseConnection();
            }
        }
    }

    public String post(String url, Map<String, String> parameter, Header[] headers) throws IOException {
        HttpPost httpPost = null;
        try {
            httpPost = new HttpPost(url);
            httpPost.setConfig(requestConfig);
            httpPost.setEntity(mapToHttpEntity(parameter));
            httpPost.setHeaders(headers);
            logger.info("httpPost result is " + httpPost);
            HttpResponse httpResp = browserHttpClient.execute(httpPost);
            int code = httpResp.getStatusLine().getStatusCode();
            logger.info("statusCode:{}", code);
            if (code == PROXY_RECONNECT_CODE) {
                QMonitor.recordOne("packetix_reconnect_stop_trade");
                return "";
            }
            return EntityUtils.toString(httpResp.getEntity(), Charsets.UTF_8.toString());
        } finally {
            if (httpPost != null) {
                httpPost.releaseConnection();
            }
        }
    }

    /**
     * post 获取LOCATION
     * @throws IOException
     */
    public Map<String,String>  postForHeader(String url, Map<String, String> content, Header[] headers) throws IOException {
        Map<String,String>  result = new HashMap<String, String>();
        HttpPost httpPost = null;
        try {
            httpPost = new HttpPost(url);
            httpPost.setConfig(requestConfig);
            httpPost.setEntity(mapToHttpEntity(content));
            httpPost.setHeaders(headers);
            HttpResponse httpResp = browserHttpClient.execute(httpPost);
            int code = httpResp.getStatusLine().getStatusCode();
            logger.info("pay postForHeader statusCode:{}", code);
            String response = EntityUtils.toString(httpResp.getEntity(), Charsets.UTF_8.toString());
            //
            Header firstHeader = httpResp.getFirstHeader("Location");
            if (firstHeader!=null && StringUtils.isNotBlank(firstHeader.getValue())){
                result.put("LOCATION",firstHeader.getValue());
            }
            if (code == PROXY_RECONNECT_CODE) {
                QMonitor.recordOne("packetix_reconnect_stop_trade");
                response = "";
            }
            result.put("RESULT",response);
        } finally {
            if (httpPost != null) {
                httpPost.releaseConnection();
            }
        }
        return result;
    }

    /**
     * get 获取LOCATION
     * @throws IOException
     */
    public Map<String,String> getForHeader(String url, Header[] headers) throws IOException {

        Map<String,String>  result = new HashMap<String, String>();
        HttpGet httpGet = new HttpGet(url);
        try {
            httpGet.setConfig(requestConfig);
            if (headers != null && headers.length > 0) {
                httpGet.setHeaders(headers);
            }
            HttpResponse httpResponse = browserHttpClient.execute(httpGet);
            int code = httpResponse.getStatusLine().getStatusCode();
            logger.info("statusCode:{}", code);
            //
            String response = EntityUtils.toString(httpResponse.getEntity(), Charsets.UTF_8.toString());
            if (code == PROXY_RECONNECT_CODE) {
                QMonitor.recordOne("packetix_reconnect_stop_trade");
                response = "";
            }
            //
            Header firstHeader = httpResponse.getFirstHeader("Location");
            if (firstHeader!=null && StringUtils.isNotBlank(firstHeader.getValue())){
                result.put("LOCATION",firstHeader.getValue());
            }
            result.put("RESULT",response);
        } finally {
            httpGet.releaseConnection();
        }
        return result;
    }


    public Map<String,String> postStatusCode(String url, Map<String, String> parameter, Header[] headers) throws IOException {
        HttpPost httpPost = null;
        try {
            httpPost = new HttpPost(url);
            httpPost.setConfig(requestConfig);
            httpPost.setEntity(mapToHttpEntity(parameter));
            httpPost.setHeaders(headers);
            HttpResponse httpResp = browserHttpClient.execute(httpPost);
            logger.info("statusCode:{}", httpResp.getStatusLine().getStatusCode());
            Map<String,String>result=new HashMap<String, String>();
            int code = httpResp.getStatusLine().getStatusCode();
            String response = EntityUtils.toString(httpResp.getEntity(), Charsets.UTF_8.toString());
            if (code == PROXY_RECONNECT_CODE) {
                QMonitor.recordOne("packetix_reconnect_stop_trade");
                response = "";
            }
            result.put("statusCode",String.valueOf(code));
            result.put("result", response);
            return result;
        } finally {
            if (httpPost != null) {
                httpPost.releaseConnection();
            }
        }
    }
    public String post(String url, String content, Header[] headers) throws IOException {
        HttpPost httpPost = null;
        try {
            httpPost = new HttpPost(url);
            httpPost.setConfig(requestConfig);
            httpPost.setEntity(new StringEntity(content,"UTF-8"));
            httpPost.setHeaders(headers);
            HttpResponse httpResp = browserHttpClient.execute(httpPost);
            int code = httpResp.getStatusLine().getStatusCode();
            logger.info("statusCode:{}", code);
            if (code == PROXY_RECONNECT_CODE) {
                QMonitor.recordOne("packetix_reconnect_stop_trade");
                return "";
            }
            return EntityUtils.toString(httpResp.getEntity(), Charsets.UTF_8.toString());
        } finally {
            if (httpPost != null) {
                httpPost.releaseConnection();
            }
        }
    }

    public HttpResponse httpPost(String url, Map<String, String> parameter, Header[] headers) throws IOException {
        HttpPost httpPost = null;
        try {
            httpPost = new HttpPost(url);
            httpPost.setConfig(requestConfig);
            httpPost.setEntity(mapToHttpEntity(parameter));
            httpPost.setHeaders(headers);
            CloseableHttpResponse response = browserHttpClient.execute(httpPost);
            int code = response.getStatusLine().getStatusCode();
            if (code == PROXY_RECONNECT_CODE) {
                QMonitor.recordOne("packetix_reconnect_stop_trade");
                return null;
            }
            return response;
        } finally {
            if (httpPost != null) {
                httpPost.releaseConnection();
            }
        }
    }

    public byte[] httpExecuteBinary(HttpUriRequest request, Header[] headers)
            throws IOException {
        HttpEntity entity = null;
        try {
            request.setHeaders(headers);
            HttpResponse response = browserHttpClient.execute(request);
            StatusLine status = response.getStatusLine();
            entity = response.getEntity();
            int code = status.getStatusCode();
            if (code == PROXY_RECONNECT_CODE) {
                QMonitor.recordOne("packetix_reconnect_stop_trade");
            }
            if (status != null && code == 200) {
                byte[] content = EntityUtils.toByteArray(entity);
                entity = null;
                return content;
            } else {
                logger.warn("can\'t post url: " + request.getURI() + "   " + status);
                return null;
            }
        } finally {
            EntityUtils.consume(entity);
        }
    }

    public String post(String url, Map<String, String> parameter, Header[] headers, int retryCount) throws IOException {
        for (int i = 0; i < retryCount; i++) {
            HttpPost httpPost = null;
            try {
                httpPost = new HttpPost(url);
                httpPost.setConfig(requestConfig);
                httpPost.setEntity(mapToHttpEntity(parameter));
                httpPost.setHeaders(headers);
                HttpResponse httpResp = browserHttpClient.execute(httpPost);
                int code = httpResp.getStatusLine().getStatusCode();
                logger.info("statusCode:{}", code);
                String response = EntityUtils.toString(httpResp.getEntity(), Charsets.UTF_8.toString());
                if (code == PROXY_RECONNECT_CODE) {
                    QMonitor.recordOne("packetix_reconnect_stop_trade");
                    response = null;
                }
                if (response != null && response.contains("The requested URL could not be retrieved")
                        && i < retryCount - 1) {
                    logger.warn("可能是代理IP异常，重试:{}", response);
                    try {
                        Thread.sleep(1000);
                    } catch (InterruptedException e) {
                        logger.error("sleep异常", e);
                    }
                    continue;
                }
                return response;
            } catch (IOException e) {
                if (i == retryCount - 1) {
                    throw e;
                } else {
                    logger.warn("请求网络异常重试", e);
                    try {
                        Thread.sleep(1000);
                    } catch (InterruptedException e2) {
                        logger.error("sleep异常", e2);
                    }
                    continue;
                }
            } finally {
                if (httpPost != null) {
                    httpPost.releaseConnection();
                }
            }
        }
        return null;
    }

    public String post(String url, Map<String, String> parameter, Header[] headers, List<Header> resHeaders)
            throws IOException {
        HttpPost httpPost = null;
        try {
            httpPost = new HttpPost(url);
            httpPost.setConfig(requestConfig);
            httpPost.setEntity(mapToHttpEntity(parameter));
            httpPost.setHeaders(headers);
            HttpResponse httpResp = browserHttpClient.execute(httpPost);
            int code = httpResp.getStatusLine().getStatusCode();
            logger.info("statusCode:{}", code);
            resHeaders.addAll(Arrays.asList(httpResp.getAllHeaders()));
            if (code == PROXY_RECONNECT_CODE) {
                QMonitor.recordOne("packetix_reconnect_stop_trade");
                return "";
            }
            return EntityUtils.toString(httpResp.getEntity());
        } finally {
            if (httpPost != null) {
                httpPost.releaseConnection();
            }
        }
    }

    private HttpEntity mapToHttpEntity(Map<String, String> params) throws UnsupportedEncodingException {
        if (params != null && !params.isEmpty()) {
            List<NameValuePair> nvPairs = Lists.newArrayList();
            for (String key : params.keySet()) {
                nvPairs.add(
                        new BasicNameValuePair(StringUtils.trimToEmpty(key), StringUtils.trimToEmpty(params.get(key))));
            }
            logger.info("mapToHttpEntity result is OK ! and nvPairs result is " + nvPairs);
            return new UrlEncodedFormEntity(nvPairs, Charsets.UTF_8.toString());
        }
        logger.info("mapToHttpEntity result is Faile !");
        return null;
    }

    private CloseableHttpClient createHttpsClient() {
        X509TrustManager x509mgr = new X509TrustManager() {
            @Override
            public void checkClientTrusted(X509Certificate[] xcs, String string) {
            }

            @Override
            public void checkServerTrusted(X509Certificate[] xcs, String string) {
            }

            @Override
            public X509Certificate[] getAcceptedIssuers() {
                return null;
            }
        };

        SSLContext sslContext = null;
        try {
            sslContext = SSLContext.getInstance("TLS");
            sslContext.init(null, new TrustManager[]{x509mgr}, null);
        } catch (NoSuchAlgorithmException e) {
            logger.error("https createHttpsClient NoSuchAlgorithmException:{}", e);
        } catch (KeyManagementException e1) {
            logger.error("https createHttpsClient KeyManagementException:{}", e1);
        }

        SSLConnectionSocketFactory sslConnectionSocketFactory = new SSLConnectionSocketFactory(sslContext,
                SSLConnectionSocketFactory.ALLOW_ALL_HOSTNAME_VERIFIER);

        SocketConfig socketConfig = SocketConfig.custom().setSoKeepAlive(true).setSoLinger(-1).
                setSoReuseAddress(false).setSoTimeout(SOCKETSOTIMEOUT).setTcpNoDelay(true).build();

        return HttpClients.custom().setSSLSocketFactory(sslConnectionSocketFactory).setDefaultCookieStore(cookieStore)
                .setMaxConnTotal(400)
                .setMaxConnPerRoute(50).setDefaultSocketConfig(socketConfig).build();
    }

}

补充：有些参数或者报文会在报文中，通过Postforhead或者getforhead方式

2.2 初始化httpclient，实际使用ThreadLocal再启动一个线程

public static final ThreadLocal<BrowserHttpClient> HTTP_CLIENT_THREAD_LOCAL = new ThreadLocal<BrowserHttpClient>();

public static BrowserHttpClient getHttpClient() {
    return HTTP_CLIENT_THREAD_LOCAL.get();
}
BrowserHttpClient httpClient = Service.getHttpClient();    // 获得一个访问的线程


MUCommonRes<String> commonRes = new MUCommonRes<String>(); //初始化请求返回类型

2.3 组装请求头

//通用请求头
Header userAgent = new BasicHeader("User-Agent","Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/63.0.3239.84 Safari/537.36");
Header accept = new BasicHeader("Accept","text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8");
Header acceptEncoding = new BasicHeader("Accept-Encoding","gzip, deflate");
Header acceptLanguage = new BasicHeader("Accept-Language","zh-cn,zh;q=0.8,en-us;q=0.5,en;q=0.3");
headers.add(userAgent);
headers.add(accept);
headers.add(acceptEncoding);
headers.add(acceptLanguage);

List<Header> headers = MUOrderService.choosePayHeaders();  //通用请求头
Header referer = new BasicHeader("Referer",MUConstant.MU_CHOOSE_CASH);  //每个页面请求referer不同
headers.add(referer);
Header[] reqHeader = headers.toArray(new Header[]{}); //转化请求头类型

2.4 Post提交方式:

Post提交参数方式有三种，可以是String、Map、List，具体的提交方式参考上面httpclient基类,提交的过程中会有两种提交格式:

①application/x-www-form-urlencoded(默认):数据提交方式key1=val1&key2=val2的方式进行编码，key 和 val 都进行了 URL 转码。
②multipart/form-data: 。首先生成了一个 boundary 用于分割不同的字段，为了避免与正文内容重复，boundary 很长很复杂。然后 Content-Type 里指明了数据是以 mutipart/form-data 来编码，本次请求的 boundary 是什么内容。消息主体里按照字段个数又分为多个结构类似的部分，每部分都是以 --boundary 开始，紧接着内容描述信息，然后是回车，最后是字段具体内容（文本或二进制）。如果传输的是文件，还要包含文件名和文件类型信息。消息主体最后以 --boundary-- 标示结束。

//Post提交的参数格式
//1、String：这种方式区分每个参数的方式是在每个参数前加上边界线boundary，在结尾加上固定的结束字符标志。边界线的设置是由
// 头
String boundary = BOUNDARY;
// 传输内容
StringBuffer contentBody = new StringBuffer();
// 尾
String endBoundary ="--" + boundary + "--\r\n";
//将参数和边界线拼接成一个完整的String
for (NameValuePair param : params) {
    contentBody
            .append("--")
            .append(boundary)
            .append("\r\n")
            .append("Content-Disposition: form-data; name=\"")
            .append(param.getName() + "\"")
            .append("\r\n")
            .append("\r\n")
            .append(param.getValue())
            .append("\r\n");
}
contentBody.append(endBoundary);


//组装httpPost相应参数进行发送
httpPost = new HttpPost(url);
httpPost.setConfig(requestConfig);
httpPost.setEntity(new StringEntity(content,"UTF-8"));
httpPost.setHeaders(headers);
HttpResponse httpResp = browserHttpClient.execute(httpPost);


//2、Map
//内部实际将Map的参数转化成List后进行提交，转化成List后提交与下面相同。



//3、List
httpPost = new HttpPost(url);
httpPost.setConfig(requestConfig);
//把输入数据编码成合适的内容，内部已经写死成www格式，转化完后的结果是key1=val1&key2=val2
UrlEncodedFormEntity urlEncodedFormEntity = new UrlEncodedFormEntity(nvPairs, Charsets.UTF_8.toString()); 
httpPost.setHeaders(headers);
httpPost.setEntity(urlEncodedFormEntity);
HttpResponse httpResp = browserHttpClient.execute(httpPost);

2.5 Get提交方式：

Get:将提交内拼接到URL当中，有时候会把传递内容全部加密


HttpGet httpGet = new HttpGet(url);
httpGet.setConfig(requestConfig);
if (headers != null && headers.length > 0) {
   httpGet.setHeaders(headers);
}

try {
   HttpResponse httpResponse = browserHttpClient.execute(httpGet);

3.页面解析：

页面解析主要是讲返回的页面提取固定的信息，返回的页面多以json串的方式来返回，有两种方式可以进行解析，json.toJSONString和json.parseObject两种方式来进行转换，下面实例两种不同方式转换的对比：

 <groupId>com.alibaba</groupId>
 <artifactId>fastjson</artifactId>
 <version>${fastjson_version}</version>

package jsonTest;

import com.alibaba.fastjson.JSON;

public class jsonTest {
    public static void main(String[] args) {
        /**
         * json字符串转化为对象
         */
        String jsonString = "{name:'Antony',age:'12',sex:'male',telephone:'88888'}";
        Staff staff = JSON.parseObject(jsonString, Staff.class);
        System.out.println(staff.toString());

        /**
         * 对象转化为json字符串
         */
        String jsonStr = JSON.toJSONString(staff);
        System.out.println(jsonStr);
    }
}

Staff{name='Antony', age=12, sex='male', birthday=null}

{"age":12,"name":"Antony","sex":"male"}
//如果age是String类型，那么输出结果变为
//{"age":"12","name":"Antony","sex":"male"}

parseObject将会自动匹配名称相同的属性，建议自己拼装对象，如有json中有集合可以用List进行操作。

GKTDSR

关注

0
点赞
踩
0

收藏

觉得还不错? 一键收藏
0
评论
httpclient使用系列

1.httpclient介绍： httpclient我理解为就是Java做到模拟访问指定浏览器的以获取指定信息或者制定操作的一种技术，而其实现就是通过自己拼装请求头以及请求参数必要时还可以自己组装cookie来实现程序端访问网站服务器达到自动化的过程。2.httpclient常用方法：1. 创建HttpClient 。2.创建请求实例，指定URL。发送GET请求，创建HttpGet对象；发...
复制链接

扫一扫