java的HttpClient爬虫小工具

最新推荐文章于 2022-05-06 11:59:11 发布

定好

最新推荐文章于 2022-05-06 11:59:11 发布

阅读量143

点赞数

分类专栏：工具文章标签：爬虫 java

本文链接：https://blog.csdn.net/qq_43847263/article/details/111933313

版权

java的HttpClient爬虫小工具

public class Client {
    static CloseableHttpClient httpClient;
    static CloseableHttpResponse response;
    static HttpGet get;
    static HttpPost post;
    static String codeType = "utf-8";                    //网页的编码格式

    public static void after() {
        //6.关闭
        HttpClientUtils.closeQuietly(response);
        HttpClientUtils.closeQuietly(httpClient);
    }

    public static void beforePost() {
        httpClient = HttpClients.createDefault();
        response = null;
        post = new HttpPost();
        //用于进行请求头伪装
        post.setHeader("User-Agent","Mozilla/5.0(Windows NT 6.1) ApplewebKit/537.36(KHTML, like Gecko) Chrome/74.0.3729.169 Safari/537.36");

//        request.setConfig(config);
    }

    public static void beforeGet() {
        //1.生成httpclient，相当于该打开一个浏览器
        httpClient = HttpClients.createDefault();
        response = null;
        //2.创建get请求，相当于在浏览器地址