做java的对httpclient应该是不会陌生的吧,应该算比较常见的包了,这两天项目中有同学遇到了httpclient的问题,从团队同学解决问题的思路、对这块知识的了解程度上看,发现很多人都停留在用的层面。其实,对这块我也是一知半解。有一知半解的地方就可以做出,所以浅谈一下,一方面是自己对这块理解更深,另外一方面发现很久没总结,没写文章了,要坚持下去。
httpclient算是神器了,当年大学时候就喜欢用这玩意,给自己的博客刷个访问量,比如:当年的网易博客,不以uid为维度计数,只以页面count计数,当年用的还是VC(因为VC写出来的exe,直接双击就可以运行了,逼格高,话说大学写代码的动力不就是装逼么),用VC自己socket+http协议封装了一个类似c版的httpclient。还做过很多,比如当年创业时候从人人网拉流量,申请一堆大波妹账号,然后遍历留言,出现验证码就请求回来做验证码破解(之前有文章讲过),总之用这类似的玩意做了挺多。
牛皮吹完了,回归正题,用过java的httpclient的应该都大致了解这玩意有俩jar包,并且都是apache的,尼玛好多困惑有么有,是不是每次用这玩意就1.从老项目中拷贝一个前人已经写好的httpclientUtil.java然后自己就用了,能通就对了!2.百度httpclient,拷贝一个例子,写个测试用例访问下百度,返回成功就哦了。但是好像大家都不清楚这俩httpclient之间啥区别。
httpclient:org.apache.httpcomponents » httpclient
commons-httpclient:commons-httpclient » commons-httpclient
apache提供了以上两个jar,都有httpclient,他们之前的关系是这样的。
1. 你可以认为他们都是Httpclient,commons-httpclient是3.x以前版本,httpclient是4.x以后的版本
2. commons-httpclient后续是不会在做维护了,apache给的建议也是用新的httpclient
3. 从功能上来讲,httpclient比commons-httpclient的功能要强大很多,比如:常见的HTTPS的支持、代理等。。。
其实我也是有点不理解的,因为从编码的易用性上来讲commons-httpclient貌似对研发者更友好,httpclient虽然功能很强大,但是使用方法上不是太友好
如下示例:
commons-httpclient
get示例
public static String get(String url, Map<String, String> queryParams) {
if (StringUtils.isBlank(url)) {
return null;
}
HttpClient httpclient = new HttpClient();
GetMethod getMethod = new GetMethod(url);
//组装参数
if (queryParams != null && !queryParams.isEmpty()) {
NameValuePair[] pairs = new NameValuePair[queryParams.size()];
int i = 0;
for (Map.Entry<String, String> entry : queryParams.entrySet()) {
pairs[i] = new NameValuePair(entry.getKey(), entry.getValue());
i++;
}
getMethod.setQueryString(pairs);
}
String resultString = null;
try {
int statusCode = httpclient.executeMethod(getMethod);
if (statusCode == HttpStatus.SC_OK) {
//获取请求响应内容
resultString = getMethod.getResponseBodyAsString();
} else {
throw new EmcooperateException("网络访问异常,错误状态:" + statusCode);
}
} catch (HttpException e) {
throw new EmcooperateException("网络访问异常", e);
} catch (IOException e) {
throw new EmcooperateException("网络访问异常", e);
}
return resultString;
}
post示例
public static String post(String url, Map<String, String> queryParams) {
if (StringUtils.isBlank(url)) {
return null;
}
HttpClient httpclient = new HttpClient();
PostMethod postMethod = new PostMethod(url);
//组装参数
if (queryParams != null && !queryParams.isEmpty()) {
NameValuePair[] pairs = new NameValuePair[queryParams.size()];
int i = 0;
for (Map.Entry<String, String> entry : queryParams.entrySet()) {
pairs[i] = new NameValuePair(entry.getKey(), entry.getValue());
i++;
}
postMethod.setQueryString(pairs);
}
String resultString = null;
try {
int statusCode = httpclient.executeMethod(postMethod);
if (statusCode == HttpStatus.SC_OK) {
//获取请求响应内容
resultString = postMethod.getResponseBodyAsString();
if (logger.isInfoEnabled()) {
logger.info("http返回值 :" + resultString);
}
} else {
if (logger.isInfoEnabled()) {
logger.info("http返回状态值 :" + statusCode);
}
throw new EmcooperateException("网络访问异常,错误状态:" + statusCode);
}
} catch (HttpException e) {
throw new EmcooperateException("网络访问异常", e);
} catch (IOException e) {
throw new EmcooperateException("网络访问异常", e);
}
return resultString;
}
如上:post、get使用都分几步
1. new httpcliet
2. new xxxMethod
3. 构造参数NameValuePair
4. 执行httpclient.executeMethod(xxxMethod)
使用步骤很明确,就是Httpclient、xxxMethod、NameValuePair几个关键的类,并且post和get采用类似的编码步骤,基本上看了就懂,并且现在网上大部分的示例都是这么个用法
接下来看看httpclient的使用:(http://hc.apache.org/httpcomponents-client-4.5.x/quickstart.html)
CloseableHttpClient httpclient = HttpClients.createDefault();
HttpGet httpGet = new HttpGet(url);
CloseableHttpResponse response1 = httpclient.execute(httpGet);
try {
System.out.println(response1.getStatusLine());
HttpEntity entity1 = response1.getEntity();
EntityUtils.consume(entity1);
} finally {
response1.close();
}
HttpPost httpPost = new HttpPost(url);
List <NameValuePair> nvps = new ArrayList <NameValuePair>();
nvps.add(new BasicNameValuePair("username", "vip"));
nvps.add(new BasicNameValuePair("password", "secret"));
httpPost.setEntity(new UrlEncodedFormEntity(nvps));
CloseableHttpResponse response2 = httpclient.execute(httpPost);
try {
HttpEntity entity2 = response2.getEntity();
EntityUtils.consume(entity2);
} finally {
response2.close();
}
URIBuilder builder = new URIBuilder();
URI uri=builder.setScheme("http")
.setHost("www.google.com")
.setPath("/search")
.setParameter("q", "httpclient")
.setParameter("btnG", "Google Search")
.setParameter("aq", "f")
.setParameter("oq", "").build();
HttpGet httpget = new HttpGet(uri);
最后通过setEntity来做数据的传递,这里有各种类型的数据支持 参见:http://hc.apache.org/httpcomponents-client-4.5.x/tutorial/html/fundamentals.html#d5e43 (1.7.1章节)
从上可以看出,httpclient4x开始基本上很少做通用的封装了,更多的是类似http协议的编程实现方式,带来的优势是可以保证各种http协议的支持,缺点就是不理解http协议(head、cookie、response)的同学会觉得用起来很麻烦,毕竟没有像commons-httpclient那么套路了,但是功能确实强大了很多,随便操作。
最后,commons-httpclient做普通的http都没太大问题,但是在做https的访问时候可能就会遇到ssl的签名验证相关问题,这个是因为jdk的
X509TrustManagerImpl会坐checkServerTrusted的证书验证于是就会报错。那么如何能通过httpclient实现https的访问呢。参考下面示例
public class HttpClientUtil2 {
/** logger */
private final static Logger logger = LoggerFactory.getLogger(HttpClientUtil2.class);
public static void ignoreHttps(HttpClient httpClient) {//设置httpclient自己的x509验证,忽略证书
// 创建TrustManager
X509TrustManager xtm = new X509TrustManager() {
public void checkClientTrusted(X509Certificate[] chain,
String authType) throws CertificateException {
System.out.println("1");
}
public void checkServerTrusted(X509Certificate[] chain,
String authType) throws CertificateException {
System.out.println("2");
}
public X509Certificate[] getAcceptedIssuers() {
System.out.println("3");
return new X509Certificate[] {};
}
};
try {
SSLContext ctx = SSLContext.getInstance("SSL");
// 使用TrustManager来初始化该上下文,TrustManager只是被SSL的Socket所使用
ctx.init(null, new TrustManager[] { xtm }, null);
SSLSocketFactory sf = new SSLSocketFactory(
ctx,
SSLSocketFactory.ALLOW_ALL_HOSTNAME_VERIFIER);
Scheme sch = new Scheme("https", 443, sf);
httpClient.getConnectionManager().getSchemeRegistry().register(sch);
} catch (Exception e) {
} finally {
// 关闭连接,释放资源
// httpClient.getConnectionManager().shutdown();
}
}
/**
* http get请求
* @param url 请求的url
* @param queryParams 请求参数
* @return 响应文本
*/
public static String get(String url, Map<String, String> queryParams) {
if (StringUtils.isBlank(url)) {
return null;
}
HttpClient httpclient = new DefaultHttpClient();
//ignoreHttps(httpclient); //这里默认不忽略,那么访问12306就会报错
String params="?";
//组装参数
if (queryParams != null && !queryParams.isEmpty()) {
List <NameValuePair> nvps = new ArrayList <NameValuePair>();
for (Map.Entry<String, String> entry : queryParams.entrySet()) {
nvps.add(new BasicNameValuePair(entry.getKey(), entry.getValue()));
params=params+entry.getKey()+"="+ entry.getValue()+"&";
}
}
HttpGet getMethod = new HttpGet(url+params);
String resultString = null;
try {
HttpResponse statusCode = httpclient.execute(getMethod);
if (statusCode.getStatusLine().getStatusCode() == HttpStatus.SC_OK) {
//获取请求响应内容
resultString = EntityUtils.toString(statusCode.getEntity());
} else {
throw new EmcooperateException("网络访问异常,错误状态:" + statusCode);
}
} catch (Exception e) {
throw new EmcooperateException("网络访问异常", e);
}
return resultString;
}
/**
* http post请求
* @param url 请求的url
* @param queryParams 请求参数
* @return 响应文本
*/
public static String post(String url, Map<String, String> queryParams) {
if (StringUtils.isBlank(url)) {
return null;
}
HttpClient httpclient = new DefaultHttpClient();
buildHttps(httpclient);
HttpPost postMethod = new HttpPost(url);
String resultString = null;
try {
//组装参数
if (queryParams != null && !queryParams.isEmpty()) {
List <NameValuePair> nvps = new ArrayList <NameValuePair>();
for (Map.Entry<String, String> entry : queryParams.entrySet()) {
nvps.add(new BasicNameValuePair(entry.getKey(), entry.getValue()));
}
postMethod.setEntity(new UrlEncodedFormEntity(nvps));
}
HttpResponse statusCode = httpclient.execute(postMethod);
if (statusCode.getStatusLine().getStatusCode() == HttpStatus.SC_OK) {
//获取请求响应内容
resultString = EntityUtils.toString(statusCode.getEntity());
if (logger.isInfoEnabled()) {
logger.info("http返回值 :" + resultString);
}
} else {
if (logger.isInfoEnabled()) {
logger.info("http返回状态值 :" + statusCode);
}
throw new EmcooperateException("网络访问异常,错误状态:" + statusCode);
}
} catch (Exception e) {
throw new EmcooperateException("网络访问异常", e);
}
return resultString;
}
public static void main(String args[]) {
Map<String, String> queryParams =new HashMap<String,String>();
System.out.println(HttpClientUtil2.get("https://www.12306.cn/mormhweb/", queryParams));
}
}