很多情况下,公司的电脑是加入域的,而且上外网是要通过代理上网的,这总情况下使用httpclient访问外网时必须使用代理,并且要通过NTLM验证,所以需要在httpclient中使用NTLM协议。
关于NTLM,可以参考这里。 以下给出解决此问题的代码。
有些时候采集需求会是公司内部人员提出,采集内网bbs
内网权限认证有时会基于NTLM
所以每次请求的时候需要请求头部带上认证信息
public static String getWithNTCredential(String url,boolean needProxy,int msTimeOut){
log.info("--get url:[" + url + "]");
String responseBody = null;
HttpMethod getMethod = new GetMethod(url);
if(msTimeOut>0) {
getMethod.getParams().setSoTimeout(msTimeOut);
}
log.info("--queryString:" + getMethod.getQueryString());
HttpClient httpClient = new HttpClient();
if (needProxy) {
String proxyUrl = "10.xx.xx.xx";//代理机IP
String proxyPort ="8080";//代理机端口
String proxyUser ="xxxx";//代理用户名
String proxyPassword ="xxxx";//代理用户密码
String domain="xxxx.COM";//计算机所在的域
httpClient.getHostConfiguration().setProxy(proxyUrl,Integer.parseInt(proxyPort));
List authPrefs = new ArrayList(2);
authPrefs.add(AuthPolicy.NTLM);
httpClient.getParams().setParameter(HttpMethodParams.USER_AGENT,"Mozilla/5.0 (Windows NT 5.1; rv:14.0) Gecko/20100101 Firefox/14.0.1");
httpClient.getParams().setParameter(AuthPolicy.AUTH_SCHEME_PRIORITY, authPrefs);
// 如果代理需要密码验证,这里设置用户名密码
NTCredentials ntc=new NTCredentials(proxyUser, proxyPassword,proxyUrl,domain);
httpClient.getState().setProxyCredentials(AuthScope.ANY,ntc);
}
try {
int statusCode = httpClient.executeMethod(getMethod);
log.info("http get url:" + getMethod.getURI());
log.info("----http status code:" + statusCode);
if (statusCode == HttpStatus.SC_OK || statusCode==HttpStatus.SC_PARTIAL_CONTENT) {
responseBody = getMethod.getResponseBodyAsString();
}
} catch (HttpException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
} finally {
getMethod.releaseConnection();
}
return responseBody;
}
参考资料:
http://www.cnblogs.com/nbpowerboy/archive/2013/05/19/3086579.html