简化Httpclient使用

背景

  工欲善其事，必先利其器。最近笔者的产品又对我提出了很多的非分的需求，数据库没有的字段也非要我导到Excel里。几千条数据，让我一个一个去查，笔者每天加班到死，手里一堆活，于是乎跟产品开始撕X，产品气得说要自己查。虽然表面上产品像我的死党一样，但是毕竟几千条数据，也不好查，况且产品还是个妹纸。表面很冷，但豆腐心肠的我，准备把导出的数据用java导入内存，然后用Httpclient爬取数据，拼装后再导出。

传统的请求方式

发现最新的Httpclient变动还是挺大，于是参考了这篇博客：HttpClient用法–这一篇全了解，该博客的一个示例如下：

public class HttpClientUtils {  
    
    private static PoolingHttpClientConnectionManager connectionManager = null;  
    private static HttpClientBuilder httpBuilder = null;  
    private static RequestConfig requestConfig = null;  
    
    private static int MAXCONNECTION = 10;  
    
    private static int DEFAULTMAXCONNECTION = 5;  
    
    private static String IP = "cnivi.com.cn";  
    private static int PORT = 80;  
    
    static {  
        //设置http的状态参数  
        requestConfig = RequestConfig.custom()  
                .setSocketTimeout(5000)  
                .setConnectTimeout(5000)  
                .setConnectionRequestTimeout(5000)  
                .build();  
    
        HttpHost target = new HttpHost(IP, PORT);  
        connectionManager = new PoolingHttpClientConnectionManager();  
        connectionManager.setMaxTotal(MAXCONNECTION);//客户端总并行链接最大数  
        connectionManager.setDefaultMaxPerRoute(DEFAULTMAXCONNECTION);//每个主机的最大并行链接数  
        connectionManager.setMaxPerRoute(new HttpRoute(target), 20);  
        httpBuilder = HttpClients.custom();  
        httpBuilder.setConnectionManager(connectionManager);  
    }  
    
    public static CloseableHttpClient getConnection() {  
        CloseableHttpClient httpClient = httpBuilder.build();  
        return httpClient;  
    }  
    
    
    public static HttpUriRequest getRequestMethod(Map<String, String> map, String url, String method) {  
        List<NameValuePair> params = new ArrayList<NameValuePair>();  
        Set<Map.Entry<String, String>> entrySet = map.entrySet();  
        for (Map.Entry<String, String> e : entrySet) {  
            String name = e.getKey();  
            String value = e.getValue();  
            NameValuePair pair = new BasicNameValuePair(name, value);  
            params.add(pair);  
        }  
        HttpUriRequest reqMethod = null;  
        if ("post".equals(method)) {  
            reqMethod = RequestBuilder.post().setUri(url)  
                    .addParameters(params.toArray(new BasicNameValuePair[params.size()]))  
                    .setConfig(requestConfig).build();  
        } else if ("get".equals(method)) {  
            reqMethod = RequestBuilder.get().setUri(url)  
                    .addParameters(params.toArray(new BasicNameValuePair[params.size()]))  
                    .setConfig(requestConfig).build();  
        }  
        return reqMethod;  
    }  
    
    public static void main(String args[]) throws IOException {  
        Map<String, String> map = new HashMap<String, String>();  
        map.put("account", "");  
        map.put("password", "");  
    
        HttpClient client = getConnection();  
        HttpUriRequest post = getRequestMethod(map, "http://cnivi.com.cn/login", "post");  
        HttpResponse response = client.execute(post);  
    
        if (response.getStatusLine().getStatusCode() == 200) {  
            HttpEntity entity = response.getEntity();  
            String message = EntityUtils.toString(entity, "utf-8");  
            System.out.println(message);  
        } else {  
            System.out.println("请求失败");  
        }  
    }  
}  

第一感觉是，前端的axios都可以那么简单，我就发送一个Http请求，Java怎么那么麻烦。于是对HttpClient进行了封装。

笔者的请求方式

这里笔者以必应搜索首页抓到的请求为例：https://cn.bing.com/HPImageArchive.aspx?format=js&idx=0&n=1&nc=1563378874917&pid=hp

    @Test
    public void testGet() {
        try {
            String url = "https://cn.bing.com/HPImageArchive.aspx?format=js&idx=0&n=1&nc=1563378874917&pid=hp";
            R<Map<String,Object>> result = Request.url(url).get();
            System.out.println(result.getData());
        } catch (Exception e) {
            e.printStackTrace();
        }
    }

打印结果：

{
    "images": [{
        "startdate": "20190716",
        "fullstartdate": "201907161600",
        "enddate": "20190717",
        "url": "/th?id=OHR.Narrenmuehle_ZH-CN5582540867_1920x1080.jpg&rf=LaDigue_1920x1080.jpg&pid=hp",
        "urlbase": "/th?id=OHR.Narrenmuehle_ZH-CN5582540867",
        "copyright": "Narrenmuehle（愚人磨坊风车），德国Dülken (© dpa picture alliance/Alamy)",
        "copyrightlink": "/search?q=Narrenmuehle&form=hpcapt&mkt=zh-cn",
        "title": "",
        "quiz": "/search?q=Bing+homepage+quiz&filters=WQOskey:%22HPQuiz_20190716_Narrenmuehle%22&FORM=HPQUIZ",
        "wp": true,
        "hsh": "d9b06221f1008b2d9208638a18b9b55c",
        "drk": 1,
        "top": 1,
        "bot": 1,
        "hs": []
    }],
    "tooltips": {
        "loading": "正在加载...",
        "previous": "上一个图像",
        "next": "下一个图像",
        "walle": "此图片不能下载用作壁纸。",
        "walls": "下载今日美图。仅限用作桌面壁纸。"
    }
}

笔者还有几个带body, header, 以及返回结果泛型自动转化的例子等，这里再列举几个：

    @Test
    public void testJsonGet() {
        try {
            String url = "http://local.home:9200/_search";
            String json = "{\n"
                + "  \"query\": {\n"
                + "    \"match_all\": {}\n"
                + "  }\n"
                + "}";
            R result = Request.url(url).body(json).get();
            System.out.println(result.getData());
        } catch (Exception e) {
            e.printStackTrace();
        }
    }

    @Test
    public void testJsonGet2() {
        try {
            String url = "http://local.home:9200/_search";

            HashMap<String, Object> f2 = new HashMap<>();
            f2.put("match_all", new Object());

            HashMap<String, Object> json = new HashMap<>();
            json.put("query", f2);

            R result = Request.url(url).body(json).get();
            System.out.println(result.getData());
        } catch (Exception e) {
            e.printStackTrace();
        }
    }

    @Test
    public void testJsonGet3() {
        try {
            String url = "http://local.home:9200/_search";

            HashMap<String, Object> f2 = new HashMap<>();
            f2.put("match_all", new Object());

            HashMap<String, Object> json = new HashMap<>();
            json.put("query", f2);

            R<Map<String, Object>> result = Request.url(url).body(json).dataType(new TypeReference<Map<String, Object>>() {
            }).get();
            System.out.println(result.getData());
        } catch (Exception e) {
            e.printStackTrace();
        }
    }

封装之后的方法，发送一个请求是不是简单很多了。

使用

maven引用

        <dependency>
            <groupId>com.github.rxyor</groupId>
            <artifactId>carp-common-util</artifactId>
            <version>1.0.4</version>
        </dependency>