httpclient访问带cookie限制的网页

  1、未设置请求头的cookie
  String url="http://www.drugstore.com/products/prod.asp? 
  HttpClient client=new HttpClient();  
  GetMethod getMethod = new GetMethod(url);

    int status= client.executeMethod(getMethod); 
  此方式staatus返回301,或者返回的是跳转后的页面结果,无法真正访问url的内容
因此需设置请求头
2、设置请求头的cookie
使用IEHttpHeaders工具在浏览器打入url,查看请求头信息,如下:
浏览器请求信息:
  GET /products/prod.asp?pid=188801&catid=12943 HTTP/1.1
Accept: image/jpeg, application/x-ms-application, image/gif, application/xaml+xml, image/pjpeg, application/x-ms-xbap, application/x-shockwave-flash, application/msword, application/vnd.ms-excel, application/vnd.ms-powerpoint, **
Accept-Language: zh-CN
User-Agent: Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.1; Trident/4.0; QQPinyin 689; SLCC2; .NET CLR 2.0.50727; .NET CLR 3.5.30729; .NET CLR 3.0.30729)
Accept-Encoding: gzip, deflate
Host: www.drugstore.com
Connection: Keep-Alive
Cookie: _br_uid_1=uid%3D6876780775167%3A; STICKY=SEAWEB004P:044086BC26EB4FAE88BFBC53 1C091EDD:5mwwt52kjx2imf55cou5ue55 ; drugstore%2Efish=UserID=728708E24FAA4229A58D61AA 436133E3; s_vi=[CS]v1|25D1666085011D4F-60000109600D3FBA[CE]; foresee.repeatdays=90

代码中设置请求头:
只需根据浏览器发送的请求头,在代码中作相应设置即可
String url="http://www.drugstore.com/products/prod.asp?pid=266446&catid=119689&cmbProdBrandFilter=15867&mp=True&trx=GFI-0-MBS&trxp1=119689&trxp2=266446&trxp3=2&trxp4=2";
HttpClient client=new HttpClient();
//client.getParams().setCookiePolicy(CookiePolicy.BROWSER_COMPATIBILITY);

     GetMethod getMethod = new GetMethod(url);
      
        getMethod.setRequestHeader("Accept"," image/jpeg, application/x-ms-application, image/gif, application/xaml+xml, image/pjpeg, application/x-ms-xbap, application/x-shockwave-flash, application/msword, application/vnd.ms-excel, application/vnd.ms-powerpoint, */*");
      //    getMethod.setRequestHeader("Referer","http://kyxk.net/wForum/disparticle.php?boardName=LifeScience&ID=2114");
        getMethod.setRequestHeader("Accept-Language","en-US");
        //   getMethod.setRequestHeader("Accept-Encoding"," gzip, deflate");
         getMethod.setRequestHeader("If-Modified-Since","Thu, 29 Jul 2004 02:24:49 GMT");
         getMethod.setRequestHeader("If-None-Match","'3014d-1d31-41085ff1'");
         getMethod.setRequestHeader("User-Agent"," Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.1; Trident/4.0; QQPinyin 689; SLCC2; .NET CLR 2.0.50727; .NET CLR 3.5.30729; .NET CLR 3.0.30729)");
         getMethod.setRequestHeader("Host","www.drugstore.com");
         getMethod.setRequestHeader("Connection"," Keep-Alive");
         getMethod.setRequestHeader("Cookie","_br_uid_1=uid%3D6876780775167%3A; BIGipServerdscm_farm=1746184384.0.0000; STICKY=SEAWEB004P:044086BC26EB4FAE88BFBC53 1C091EDD:bdm5j355thye4445yvus2kis ; drugstore%2Efish=UserID=728708E24FAA4229A58D61AA 436133E3; s_vi=[CS]v1|25D1666085011D4F-60000109600D3FBA[CE]; foresee.repeatdays=90");
   
        //getMethod.getParams().setParameter(HttpMethodParams.RETRY_HANDLER,
           //                              new DefaultHttpMethodRetryHa ndler());
 
int status= client.executeMethod(getMethod);

 
以上设置的请求头都是在iehttpheaders看到的请求头
但需把setMethod.setRequestHeader("Accept-Encoding"," gzip, deflate")去掉,否则会乱码
另外setCookie是必须的,值为以上浏览器请求的信息
这样,得到的网页内容就是实际请求的网页内容了
评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值