1、为什么使用代理
当爬取网页的时候,有的目标站点有反爬虫机制,对于频繁访问站点以及规则性访问站点的行为,会采用屏蔽IP的措施,这时候代理IP就派上用场了。
2、代理的分类
(一)透明代理(Transparent Proxy)-透明代理虽然可以直接隐藏你的IP地址,但是还是从HTTP_X_FORWARD_FOR 来查到你是谁
REMOTE_ADDR= Proxy IP
HTTP_VIA = Proxy IP
HTTP_X_FORWORAD_FOR= YOUR IP
(二)匿名代理(Anonymous Proxy)-匿名代理比透明代理进步一点,别人只能知道你用了代理,无法知道你是谁
REMOTE_ADDR= proxy Ip
HTTP_VIA = proxy IP
HTTP_X_FORWARD_FOR = proxy_IP
(三)混淆代理(Distorting Proxies)-与匿名代理相同,如果使用了混淆代理,别人还是能知道你在用代理,但是会得到一个假的IP地址,伪装的更逼真
REMOTE_ADDR=PROXY_IP
HTTP_VIA =PROXY IP
HTTP_X_FOREARD_FOR=Random IP ADDRESS
(四)高匿代理(Elite Proxy 或High Anonymity Proxy)-可以看出,高匿代理让别人无法发现你是在用代理,是爬虫最好的选择
REMOTE_ADDR=PROXY_IP
HTTP_VIA = not determined
HTTP_X_FORWARD_FOR= not determined
3、 代理IP的获取
@Test
public void testHttpProxy() throws Exception{
HttpClient httpClient = HttpClients.createDefault();
HttpGet httpGet = new HttpGet("http://www.baidu.com");
//使用代理服务器
HttpHost httpHost = new HttpHost("220.194.55.160",3128);
RequestConfig config = RequestConfig.custom().setProxy(httpHost).build();
httpGet.setConfig(config);
CloseableHttpResponse response = (CloseableHttpResponse) httpClient.execute(httpGet);
HttpEntity entity = response.getEntity();
//输出网页内容
System.out.println("网页内容:");
System.out.println(EntityUtils.toString(entity,"utf-8"));
response.close();
}
4、HttpClient代理配置
HttpClient支持复杂的路由方案和代理链,同样也支持直接或者只通过一跳的连接。
(一)使用代理服务器最简单的方式,执行一个默认的默认的代理。
HttpHost proxy = new HttpHost("someproxy", 8080);
DefaultProxyRoutePlanner routePlanner = new DefaultProxyRoutePlanner(proxy);
CloseableHttpClient httpclient = HttpClients.custom()
.setRoutePlanner(routePlanner)
.build();
(二) HttpClient使用jre代理服务器
SystemDefaultRoutePlanner routePlanner = new SystemDefaultRoutePlanner(
ProxySelector.getDefault());
CloseableHttpClient httpclient = HttpClients.custom()
.setRoutePlanner(routePlanner)
.build();
手动配置RoutePlanner,这样就可以完全控制Http路由的过程
HttpRoutePlanner routePlanner = new HttpRoutePlanner() {
public HttpRoute determineRoute(
HttpHost target,
HttpRequest request,
HttpContext context) throws HttpException {
return new HttpRoute(target, null, new HttpHost("someproxy", 8080),
"https".equalsIgnoreCase(target.getSchemeName()));
}
};
CloseableHttpClient httpclient = HttpClients.custom()
.setRoutePlanner(routePlanner)
.build();
}
}
例子
package com.book.common.utils;
import com.alibaba.fastjson.JSONObject;
import com.book.common.constant.StaticConstant;
import com.book.common.result.Failed;
import com.google.common.collect.Lists;
import com.google.common.collect.Maps;
import lombok.Getter;
import lombok.extern.slf4j.Slf4j;
import org.apache.commons.lang3.StringUtils;
import org.apache.http.HttpHost;
import org.apache.http.NameValuePair;
import org.apache.http.client.config.RequestConfig;
import org.apache.http.client.entity.UrlEncodedFormEntity;
import org.apache.http.client.methods.*;
import org.apache.http.conn.ssl.NoopHostnameVerifier;
import org.apache.http.entity.ByteArrayEntity;
import org.apache.http.entity.StringEntity;
import org.apache.http.impl.client.CloseableHttpClient;
import org.apache.http.impl.client.HttpClients;
import org.apache.http.message.BasicNameValuePair;
import org.apache.http.ssl.SSLContexts;
import org.apache.http.util.EntityUtils;
import org.springframework.util.CollectionUtils;
import javax.net.ssl.SSLContext;
import java.io.*;
import java.net.*;
import java.nio.charset.StandardCharsets;
import java.security.KeyManagementException;
import java.security.KeyStoreException;
import java.security.NoSuchAlgorithmException;
import java.util.ArrayList;
import java.util.List;
import java.util.Map;
import java.util.Random;
/**
* Description:
* HTTP调用;https路径会转化为http,所以需要能使用http调用访问的路径才可以使用
* 用法:HttpUtils.doGet(HttpUtils.create().addHostPort("http://suyun.nevtsp.com/").addProxy("116.30.240.246:80").build());
*
* @Author: leo.xiong
* @CreateDate: 2021/11/30 9:43
* @Email: leo.xiong@suyun360.com
* @Since:
*/
@Slf4j
public final class HttpUtils {
private static final int DOWN_LOAD_READ_BYTE_BUFFER = 1024;
private static final int SO_TIMEOUT_MS = 8000;
private static final int CONNECTION_TIMEOUT_MS = 3000;
private static final int SUCCESS = 200;
private static final int HTTP_PORT = 80;
private static final int HTTPS_PORT = 443;
private static final Random RANDOM = new Random();
/**
* 创建参数对象信息
*
* @return
*/
public static HttpClientParam create() {
return new HttpClientParam();
}
/**
* get
*
* @param httpClientParam
* @return
*/
public static String doGet(final HttpClientParam httpClientParam) {
HttpGet request = new HttpGet(httpClientParam.getUrl());
buildHeaderProxy(request, httpClientParam);
return execute(httpClientParam.getCloseableHttpClient(), request);
}
/**
* Post From
*
* @param httpClientParam
* @return
*/
public static String doPost(final HttpClientParam httpClientParam) {
HttpPost request = new HttpPost(httpClientParam.getUrl());
buildHeaderProxy(request, httpClientParam);
if (!CollectionUtils.isEmpty(httpClientParam.getBodyMap())) {
List<NameValuePair> nameValuePairList = Lists.newArrayList();
for (String key : httpClientParam.getBodyMap().keySet()) {
nameValuePairList.add(new BasicNameValuePair(key, httpClientParam.getBodyMap().get(key)));
}
UrlEncodedFormEntity formEntity = new UrlEncodedFormEntity(nameValuePairList, StandardCharsets.UTF_8);
formEntity.setContentType("application/x-www-form-urlencoded; charset=UTF-8");
request.setEntity(formEntity);
} else if (!StringUtils.isEmpty(httpClientParam.getBodyStr())) {
request.setEntity(new StringEntity(httpClientParam.getBodyStr(), StandardCharsets.UTF_8));
} else if (httpClientParam.getBody() != null) {
request.setEntity(new ByteArrayEntity(httpClientParam.getBody()));
}
return execute(httpClientParam.getCloseableHttpClient(), request);
}
/**
* Put Stream
*
* @param httpClientParam
* @return
*/
public static String doPut(final HttpClientParam httpClientParam) {
HttpPut request = new HttpPut(httpClientParam.getUrl());
buildHeaderProxy(request, httpClientParam);
if (StringUtils.isNotEmpty(httpClientParam.getBodyStr())) {
request.setEntity(new StringEntity(httpClientParam.getBodyStr(), StandardCharsets.UTF_8));
} else if (httpClientParam.getBody() != null) {
request.setEntity(new ByteArrayEntity(httpClientParam.getBody()));
}
return execute(httpClientParam.getCloseableHttpClient(), request);
}
/**
* Delete Stream
*
* @param httpClientParam
* @return
*/
public static String doDelete(final HttpClientParam httpClientParam) {
HttpDelete request = new HttpDelete(httpClientParam.getUrl());
buildHeaderProxy(request, httpClientParam);
return execute(httpClientParam.getCloseableHttpClient(), request);
}
/**
* 执行信息
*
* @param httpClient
* @param request
* @return
*/
private static String execute(CloseableHttpClient httpClient, HttpUriRequest request) {
CloseableHttpResponse response = null;
try {
response = httpClient.execute(request);
if (response == null) {
return JSONObject.toJSONString(new Failed("100000", "返回信息为空"));
} else {
if (response.getStatusLine().getStatusCode() == SUCCESS) {
return EntityUtils.toString(response.getEntity(), StandardCharsets.UTF_8);
} else {
return JSONObject.toJSONString(new Failed(response.getStatusLine().getStatusCode() + "", "返回信息失败"));
}
}
} catch (IOException e) {
log.warn("执行失败", e);
return JSONObject.toJSONString(new Failed("100001", e.getMessage()));
} finally {
try {
if (response != null) {
response.close();
}
if (httpClient != null) {
httpClient.close();
}
} catch (IOException e) {
}
}
}
/**
* 根据URL下载文件信息,保存在指定的目录下面
*
* @param urlStr
* @param fileName
* @param savePath
*/
public static void downLoadFromUrl(String urlStr, String fileName, String savePath) {
downLoadFromUrl(urlStr, fileName, savePath, null, null);
}
/**
* 从网络Url中下载文件
* http://suyun-nevtsp-foreverfile.oss-cn-shenzhen.aliyuncs.com/vehicle_admin/dbc_excel/2020/11/16/L16_V1.1_20201116095257890.xlsx
*
* @param urlStr 下载URL
* @param fileName 下载文件名
* @param savePath 保存的路径
* @param tokeName Token 名称
* @param token 需要的TOKEN新
* @throws IOException
*/
public static void downLoadFromUrl(String urlStr, String fileName, String savePath, String tokeName, String token) {
FileOutputStream fos = null;
InputStream is = null;
try {
URL url = new URL(urlStr);
HttpURLConnection conn = (HttpURLConnection) url.openConnection();
//设置超时间为3秒
conn.setConnectTimeout(CONNECTION_TIMEOUT_MS);
//防止屏蔽程序抓取而返回403错误
conn.setRequestProperty("User-Agent", "Mozilla/4.0 (compatible; MSIE 5.0; Windows NT; DigExt)");
if (StringUtils.isNotEmpty(token)) {
conn.setRequestProperty(tokeName, token);
}
//得到输入流
is = conn.getInputStream();
//获取自己数组
byte[] getData = readInputStream(is);
//文件保存位置
File saveDir = new File(savePath);
if (!saveDir.exists()) {
saveDir.mkdir();
}
File file = new File(saveDir + File.separator + fileName);
fos = new FileOutputStream(file);
fos.write(getData);
} catch (IOException e) {
e.printStackTrace();
} finally {
try {
if (fos != null) {
fos.close();
}
if (is != null) {
is.close();
}
} catch (IOException e) {
}
}
}
/**
* 添加头部信息,判断是否使用代理
*
* @param httpRequestBase
* @param httpClientParam
*/
private static void buildHeaderProxy(HttpRequestBase httpRequestBase, final HttpClientParam httpClientParam) {
addHeader(httpRequestBase, httpClientParam.getHeaderMap());
if (StringUtils.isEmpty(httpClientParam.getProxy()) || !httpClientParam.getProxy().contains(StaticConstant.Common.COLON)) {
httpRequestBase.setConfig(RequestConfig.custom()
.setSocketTimeout(SO_TIMEOUT_MS)
.setConnectTimeout(CONNECTION_TIMEOUT_MS).build());
return;
}
Proxy proxy = null;
if (httpClientParam.getProxy().startsWith(StaticConstant.Common.HTTP)) {
proxy = new Proxy(Proxy.Type.HTTP, new InetSocketAddress(httpClientParam.getProxy().replace(StaticConstant.Common.HTTP, StaticConstant.Common.EMPTY_STR), HTTP_PORT));
} else if (httpClientParam.getProxy().startsWith(StaticConstant.Common.HTTPS)) {
proxy = new Proxy(Proxy.Type.HTTP, new InetSocketAddress(httpClientParam.getProxy().replace(StaticConstant.Common.HTTPS, StaticConstant.Common.EMPTY_STR), HTTPS_PORT));
} else {
String[] ipPort = httpClientParam.getProxy().split(StaticConstant.Common.COLON);
proxy = new Proxy(Proxy.Type.HTTP, new InetSocketAddress(ipPort[0], Integer.parseInt(ipPort[1])));
}
InetSocketAddress socketAddress = (InetSocketAddress) proxy.address();
String ip = socketAddress.getAddress().getHostAddress();
Integer port = socketAddress.getPort();
if (StringUtils.isNotEmpty(ip) && port != null) {
HttpHost httpHost = new HttpHost(ip, port);
httpRequestBase.setConfig(RequestConfig.custom().setProxy(httpHost)
.setSocketTimeout(SO_TIMEOUT_MS)
.setConnectTimeout(CONNECTION_TIMEOUT_MS).build());
} else {
httpRequestBase.setConfig(RequestConfig.custom()
.setSocketTimeout(SO_TIMEOUT_MS)
.setConnectTimeout(CONNECTION_TIMEOUT_MS).build());
}
if (httpRequestBase.getHeaders("X-Forwarded-For") == null || httpRequestBase.getHeaders("X-Forwarded-For").length == 0) {
httpRequestBase.addHeader("X-Forwarded-For", httpClientParam.getForwardIp());
}
}
private static void addHeader(HttpRequestBase httpRequestBase, Map<String, String> headerMap) {
if (!CollectionUtils.isEmpty(headerMap)) {
for (Map.Entry<String, String> header : headerMap.entrySet()) {
httpRequestBase.addHeader(header.getKey(), header.getValue());
}
}
if (headerMap.containsKey("User-Agent")) {
return;
}
httpRequestBase.addHeader("User-Agent", USER_AGENT_ALL[RANDOM.nextInt(USER_AGENT_ALL.length - 1)]);
}
/**
* 从输入流中获取字节数组
*
* @param inputStream
* @return
* @throws IOException
*/
private static byte[] readInputStream(InputStream inputStream) throws IOException {
byte[] buffer = new byte[DOWN_LOAD_READ_BYTE_BUFFER];
int len = 0;
ByteArrayOutputStream bos = new ByteArrayOutputStream();
while ((len = inputStream.read(buffer)) != -1) {
bos.write(buffer, 0, len);
}
bos.close();
return bos.toByteArray();
}
/**
* 访问参数
*/
@Getter
public static class HttpClientParam {
/**
* 拼装之后的URL信息
*/
private String url;
/**
* 访问客户端
*/
private CloseableHttpClient closeableHttpClient;
/**
* 访问协议IP和端口例如:http://10.0.0.5:8080 不能为空
*/
private String hostPort;
/**
* 访问的方法 例如: /a/query 可以为空
*/
private String path;
/**
* URL带的参数信息 key 为name value为值 可以为空 ,host+path+url会拼接成真正的URL
*/
private Map<String, String> urlParamMap = Maps.newHashMap();
/**
* 头部参数 key 为name value为值 可以为空
*/
private Map<String, String> headerMap = Maps.newHashMap();
/**
* FormData 参数信息
*/
private Map<String, String> bodyMap = Maps.newHashMap();
/**
* body字符串
*/
private String bodyStr;
/**
* body byte数组 bodyMap,bodyStr,body三个只能使用一个,优先级 bodyMap>bodyStr>body
*/
private byte[] body;
/**
* 是否使用代理,如果为空,表示不代理 10.0.0.1:8080
*/
private String proxy;
private String forwardIp;
private HttpClientParam() {
}
/**
* 访问协议IP和端口例如:http://10.0.0.5:8080 不能为空
*/
public HttpClientParam addHostPort(String hostPort) {
this.hostPort = hostPort;
return this;
}
/**
* 访问的方法映射 例如: /a/query 可以为空
*/
public HttpClientParam addPath(String path) {
this.path = path;
return this;
}
/**
* URL带的参数信息 key 为name value为值 可以为空 ,host+path+url会拼接成真正的URL
*
* @param key
* @param value
* @return
*/
public HttpClientParam addQuery(String key, String value) {
this.urlParamMap.put(key, value);
return this;
}
/**
* URL带的参数信息 key 为name value为值 可以为空 ,host+path+url会拼接成真正的URL
*
* @param queryMap
* @return
*/
public HttpClientParam addQueryAll(Map<String, String> queryMap) {
this.urlParamMap.putAll(queryMap);
return this;
}
/**
* 头部参数 key 为name value为值 可以为空
*
* @param key
* @param value
* @return
*/
public HttpClientParam addHeader(String key, String value) {
this.headerMap.put(key, value);
return this;
}
/**
* 头部参数 key 为name value为值 可以为空
*
* @param headerMap
* @return
*/
public HttpClientParam addHeaderAll(Map<String, String> headerMap) {
this.headerMap.putAll(headerMap);
return this;
}
/**
* FormData 参数信息 bodyMap,bodyStr,body三个只能使用一个,优先级 bodyMap>bodyStr>body
*
* @param key
* @param value
*/
public void addBodyMap(String key, String value) {
this.bodyMap.put(key, value);
}
/**
* FormData 参数信息 bodyMap,bodyStr,body三个只能使用一个,优先级 bodyMap>bodyStr>body
*
* @param bodyMap
*/
public void addBodyMapAll(Map<String, String> bodyMap) {
this.bodyMap.putAll(bodyMap);
}
/**
* body字符串 bodyMap,bodyStr,body三个只能使用一个,优先级 bodyMap>bodyStr>body
*
* @param bodyStr
* @return
*/
public HttpClientParam addBodyStr(String bodyStr) {
this.bodyStr = bodyStr;
return this;
}
/**
* byte数组 bodyMap,bodyStr,body三个只能使用一个,优先级 bodyMap>bodyStr>body
*
* @param body
* @return
*/
public HttpClientParam addBody(byte[] body) {
this.body = body;
return this;
}
/**
* 是否使用代理,如果为空,表示不代理 10.0.0.1:8080
*
* @param proxy
* @return
*/
public HttpClientParam addProxy(String proxy) {
this.proxy = proxy;
return this;
}
/**
* 创建对象信息
*/
public HttpClientParam build() {
boolean isNoValid = StringUtils.isEmpty(this.hostPort) || (!this.hostPort.startsWith(StaticConstant.Common.HTTP) && !this.hostPort.startsWith(StaticConstant.Common.HTTPS));
if (isNoValid) {
log.warn("路径信息错误 hostPort:{}", this.hostPort);
return null;
}
if (this.hostPort.endsWith(StaticConstant.Common.SLASH)) {
this.hostPort = this.hostPort.substring(0, this.hostPort.lastIndexOf(StaticConstant.Common.COLON));
}
if (StringUtils.isNotEmpty(this.path) && !this.path.startsWith(StaticConstant.Common.SLASH)) {
this.path = StaticConstant.Common.SLASH + this.path;
}
try {
if (this.proxy != null) {
String tempProxy = null;
this.forwardIp = this.proxy.substring(0, this.proxy.lastIndexOf(StaticConstant.Common.COLON));
if (this.hostPort.startsWith(StaticConstant.Common.HTTP)) {
tempProxy = this.hostPort.replace(StaticConstant.Common.HTTP, StaticConstant.Common.EMPTY_STR);
this.hostPort = StaticConstant.Common.HTTP + this.proxy;
} else {
tempProxy = this.hostPort.replace(StaticConstant.Common.HTTPS, StaticConstant.Common.EMPTY_STR);
this.hostPort = StaticConstant.Common.HTTPS + this.proxy;
}
this.proxy = tempProxy;
}
this.url = buildUrl(this.hostPort, this.path, this.urlParamMap);
this.closeableHttpClient = wrapClient(this.hostPort);
} catch (Exception e) {
log.warn("创建访问对象错误", e);
return null;
}
return this;
}
/**
* 拼接URL
*
* @param host
* @param path
* @param queryMap
* @return
* @throws UnsupportedEncodingException
*/
private String buildUrl(String host, String path, Map<String, String> queryMap) {
StringBuilder sbUrl = new StringBuilder();
sbUrl.append(host);
if (!StringUtils.isBlank(path)) {
sbUrl.append(path);
}
if (CollectionUtils.isEmpty(queryMap)) {
return sbUrl.toString();
}
StringBuilder sbQuery = new StringBuilder();
for (Map.Entry<String, String> query : queryMap.entrySet()) {
if (sbQuery.length() > 0) {
sbQuery.append(StaticConstant.Common.AND);
}
if (StringUtils.isBlank(query.getKey()) && !StringUtils.isBlank(query.getValue())) {
sbQuery.append(query.getValue());
}
if (!StringUtils.isBlank(query.getKey())) {
sbQuery.append(query.getKey());
if (!StringUtils.isBlank(query.getValue())) {
sbQuery.append("=");
try {
sbQuery.append(URLEncoder.encode(query.getValue(), StandardCharsets.UTF_8.name()));
} catch (UnsupportedEncodingException e) {
log.warn("编码失败 value: {}", query.getValue());
}
}
}
}
if (sbQuery.length() > 0) {
sbUrl.append(StaticConstant.Common.UN_KNOW).append(sbQuery);
}
return sbUrl.toString();
}
private CloseableHttpClient wrapClient(String host) {
CloseableHttpClient httpClient = HttpClients.createDefault();
if (host.startsWith(StaticConstant.Common.HTTPS)) {
SSLContext sslContext = null;
try {
sslContext = SSLContexts.custom().loadTrustMaterial(null, (x509Certificates, s) -> {
return true;
}).build();
} catch (NoSuchAlgorithmException | KeyManagementException | KeyStoreException e) {
e.printStackTrace();
}
//创建httpClient
return HttpClients.custom().setSSLContext(sslContext).
setSSLHostnameVerifier(new NoopHostnameVerifier()).build();
}
return httpClient;
}
}
private static final String[] USER_AGENT_ALL = {
"Mozilla/5.0 (Windows NT 6.2; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/32.0.1667.0 Safari/537.36",
"Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/66.0.3359.139 Safari/537.36",
"Mozilla/5.0 (Windows NT 5.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2224.3 Safari/537.36",
"Mozilla/5.0 (Windows NT 6.2; Win64; x64; rv:21.0.0) Gecko/20121011 Firefox/21.0.0",
"Mozilla/5.0 (Windows NT 6.1; WOW64; rv:64.0) Gecko/20100101 Firefox/64.0",
"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10; rv:33.0) Gecko/20100101 Firefox/33.0",
"Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.1 (KHTML, like Gecko) Chrome/22.0.1207.1 Safari/537.1",
"Mozilla/5.0 (X11; CrOS i686 2268.111.0) AppleWebKit/536.11 (KHTML, like Gecko) Chrome/20.0.1132.57 Safari/536.11",
"Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/536.6 (KHTML, like Gecko) Chrome/20.0.1092.0 Safari/536.6",
"Mozilla/5.0 (Windows NT 6.2) AppleWebKit/536.6 (KHTML, like Gecko) Chrome/20.0.1090.0 Safari/536.6",
"Mozilla/5.0 (Windows NT 6.2; WOW64) AppleWebKit/537.1 (KHTML, like Gecko) Chrome/19.77.34.5 Safari/537.1",
"Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/536.5 (KHTML, like Gecko) Chrome/19.0.1084.9 Safari/536.5",
"Mozilla/5.0 (Windows NT 6.0) AppleWebKit/536.5 (KHTML, like Gecko) Chrome/19.0.1084.36 Safari/536.5",
"Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/536.3 (KHTML, like Gecko) Chrome/19.0.1063.0 Safari/536.3",
"Mozilla/5.0 (Windows NT 5.1) AppleWebKit/536.3 (KHTML, like Gecko) Chrome/19.0.1063.0 Safari/536.3",
"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_8_0) AppleWebKit/536.3 (KHTML, like Gecko) Chrome/19.0.1063.0 Safari/536.3",
"Mozilla/5.0 (Windows NT 6.2) AppleWebKit/536.3 (KHTML, like Gecko) Chrome/19.0.1062.0 Safari/536.3",
"Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/536.3 (KHTML, like Gecko) Chrome/19.0.1062.0 Safari/536.3",
"Mozilla/5.0 (Windows NT 6.2) AppleWebKit/536.3 (KHTML, like Gecko) Chrome/19.0.1061.1 Safari/536.3",
"Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/536.3 (KHTML, like Gecko) Chrome/19.0.1061.1 Safari/536.3",
"Mozilla/5.0 (Windows NT 6.1) AppleWebKit/536.3 (KHTML, like Gecko) Chrome/19.0.1061.1 Safari/536.3",
"Mozilla/5.0 (Windows NT 6.2) AppleWebKit/536.3 (KHTML, like Gecko) Chrome/19.0.1061.0 Safari/536.3",
"Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/535.24 (KHTML, like Gecko) Chrome/19.0.1055.1 Safari/535.24",
"Mozilla/5.0 (Windows NT 6.2; WOW64) AppleWebKit/535.24 (KHTML, like Gecko) Chrome/19.0.1055.1 Safari/535.24",
"Mozilla/5.0 (Macintosh; U; Mac OS X Mach-O; en-US; rv:2.0a) Gecko/20040614 Firefox/3.0.0 ",
"Mozilla/5.0 (Macintosh; U; PPC Mac OS X 10.5; en-US; rv:1.9.0.3) Gecko/2008092414 Firefox/3.0.3",
"Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.5; en-US; rv:1.9.1) Gecko/20090624 Firefox/3.5",
"Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.6; en-US; rv:1.9.2.14) Gecko/20110218 AlexaToolbar/alxf-2.0 Firefox/3.6.14",
"Mozilla/5.0 (Macintosh; U; PPC Mac OS X 10.5; en-US; rv:1.9.2.15) Gecko/20110303 Firefox/3.6.15",
"Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:2.0.1) Gecko/20100101 Firefox/4.0.1",
"Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/531.21.8 (KHTML, like Gecko) Version/4.0.4 Safari/531.21.10",
"Mozilla/5.0 (Windows; U; Windows NT 5.2; en-US) AppleWebKit/533.17.8 (KHTML, like Gecko) Version/5.0.1 Safari/533.17.8",
"Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US) AppleWebKit/533.19.4 (KHTML, like Gecko) Version/5.0.2 Safari/533.18.5",
"Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; Trident/5.0",
"Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.0; Trident/4.0)",
"Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 6.0)",
"Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)"
};
}
private static void proxy() {
String result = HttpUtils.doGet(HttpUtils.create()
//访问的服务器IP
.addHostPort("http://116.30.240.246:8080")
//访问的方法
.addPath("/a/login")
//代理IP和端口
效果:
使用代理
不用代理
StaticConstant.Common.COLON=“:”;
StaticConstant.Common.HTTP=“http”;
StaticConstant.Common.HTTPS=“https”;
StaticConstant.Common.EMPTY_STR=“”;
StaticConstant.Common.SLASH=“/”;
StaticConstant.Common.AND=“&”;
StaticConstant.Common.UN_KNOW=“?”;
Filed类里面有code errorMsg2属性;