HttpClient使用代理

一、Maven依赖

<dependency>
    <groupId>org.apache.httpcomponents</groupId>
    <artifactId>httpclient</artifactId>
</dependency>

二、工具方法

/**
     * 构建的HTTP客户端
     * @param useProxy
     * @param proxyIp
     * @param proxyPort
     * @return
     * @author: fuyang.qin
     * @date: 2020/12/16
     */
private CloseableHttpClient buildHttpClient(String useProxy, String proxyIp, String proxyPort) {
        CloseableHttpClient httpClient;
        if (useProxyFlag.equals(useProxy.toLowerCase())) {
            // 代理
            HttpHost pr = new HttpHost(proxyIp, Integer.parseInt(proxyPort), "http");
            RequestConfig defaultRequestConfig = RequestConfig.custom().setProxy(pr).build();
            httpClient = HttpClients.custom().setDefaultRequestConfig(defaultRequestConfig).build();
        } else {
            // 非代理
            httpClient = HttpClients.createDefault();
        }
        return httpClient;
    }

Tip:此处参数useProxy为数据库获取到的配置所以使用String类型,根据需要改为boolean类型可能更合适

三、使用

public static void main(String[] args) {
       	// CloseableHttpClient httpClient = new ProxyTest().buildHttpClient("true", "10.10.122.33", "9000");
        CloseableHttpClient httpClient = new ProxyTest().buildHttpClient("false", null, null);
        try {
            HttpGet httpGet = new HttpGet("http://baidu.com");
            CloseableHttpResponse response = httpClient.execute(httpGet);
            String resp;
            try {
                HttpEntity entity = response.getEntity();
                resp = EntityUtils.toString(entity, "UTF-8");
                EntityUtils.consume(entity);
            } finally {
                response.close();
            }
            System.out.println(resp);
        } catch (Exception e) {
            e.printStackTrace();
        }
    }
  • 0
    点赞
  • 0
    收藏
    觉得还不错? 一键收藏
  • 1
    评论
Apache HttpClient是一个非常流行的Java HTTP客户端库,它可以用来发送HTTP请求并处理响应。如果你需要使用代理服务器来发送HTTP请求,那么你可以使用HttpClient来获取代理列表。下面是一个使用HttpClient获取代理列表的示例代码: ```java import java.io.BufferedReader; import java.io.IOException; import java.io.InputStreamReader; import java.net.URI; import java.util.ArrayList; import java.util.List; import org.apache.http.HttpHost; import org.apache.http.HttpResponse; import org.apache.http.client.HttpClient; import org.apache.http.client.methods.HttpGet; import org.apache.http.client.utils.URIBuilder; import org.apache.http.impl.client.HttpClientBuilder; import org.apache.http.util.EntityUtils; import org.jsoup.Jsoup; import org.jsoup.nodes.Document; import org.jsoup.select.Elements; public class ProxyUtil { public static List<HttpHost> getProxyList() throws Exception { List<HttpHost> proxyList = new ArrayList<>(); URI uri = new URIBuilder().setScheme("http").setHost("www.xicidaili.com").setPath("/nn/").setParameter("wt", "1").build(); HttpGet request = new HttpGet(uri); HttpClient client = HttpClientBuilder.create().build(); HttpResponse response = client.execute(request); String html = EntityUtils.toString(response.getEntity()); Document doc = Jsoup.parse(html); Elements elements = doc.select("table#ip_list tr"); for (int i = 1; i < elements.size(); i++) { String ip = elements.get(i).select("td:eq(1)").text(); String port = elements.get(i).select("td:eq(2)").text(); HttpHost proxy = new HttpHost(ip, Integer.parseInt(port)); proxyList.add(proxy); } return proxyList; } public static void main(String[] args) { try { List<HttpHost> proxyList = getProxyList(); for (HttpHost proxy : proxyList) { System.out.println(proxy); } } catch (Exception e) { e.printStackTrace(); } } } ``` 这个示例代码使用了一个免费的代理IP网站(www.xicidaili.com)来获取代理列表,然后解析HTML文档提取出代理IP和端口号,最后将它们封装成HttpHost对象并添加到代理列表中。你可以修改这个示例代码来适应其他的代理IP网站。

“相关推荐”对你有帮助么?

  • 非常没帮助
  • 没帮助
  • 一般
  • 有帮助
  • 非常有帮助
提交
评论 1
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值