网页抓取

以易行为例:

使用WPF 专业版,截取数据包.

 

:     !D     隐藏   隐藏   1   发送   !D     隐藏   隐藏   1   接收?  POST /zuyin/basePolicyShowAction.do?method=getTicketBasePolicy&menu=1 HTTP/1.1
Accept: image/jpeg, application/x-ms-application, image/gif, application/xaml+xml, image/pjpeg, application/x-ms-xbap, application/x-shockwave-flash, application/vnd.ms-excel, application/vnd.ms-powerpoint, application/msword, */*
Referer: http://www.zuwin.cn/zuyin/basePolicyShowAction.do?method=getTicketBasePolicy&menu=1
Accept-Language: zh-CN
User-Agent: Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.1; Trident/4.0; Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1) ; SLCC2; .NET CLR 2.0.50727; .NET CLR 3.5.30729; .NET CLR 3.0.30729; Media Center PC 6.0; CIBA)
Content-Type: application/x-www-form-urlencoded
Accept-Encoding: gzip, deflate
Host: www.zuwin.cn
Content-Length: 588
Connection: Keep-Alive
Cache-Control: no-cache
Cookie: userName=zuhtsca; JSESSIONID=E77B3E1056776CBCE17A1058F29E5F0D.tomcat5; JSESSIONID=0B48DE81B651DEB37A076E459C04E0B0.tomcat5

     隐藏   隐藏   964   发送L 
okto=1&xianshi=1&airway=CA&airwayscode=%E4%B8%AD%E5%9B%BD%E5%9B%BD%E9%99%85%E8%88%AA%E7%A9%BA%E5%85%AC%E5%8F%B8&dep=SHA&arrcity=%E4%B8%8A%E6%B5%B7%E8%99%B9%E6%A1%A5&arr=PEK&city=%E5%8C%97%E4%BA%AC%E9%A6%96%E9%83%BD&commany=&flightdatestart=&agioPercent=7&agioPercent=6.3&agioPercent=6.2&agioPercent=6.2&agioPercent=6.1&agioPercent=6&agioPercent=6&agioPercent=5.5&agioPercent=5.1&agioPercent=5&agioPercent=5&agioPercent=5&agioPercent=5&agioPercent=4.7&agioPercent=4.6&agioPercent=4.5&agioPercent=4.5&agioPercent=4.4&agioPercent=4.3&agioPercent=4.2&current_page=1&rows=29&pages=2&gotoPage=1     隐藏   隐藏   588   发送   !D     隐藏   隐藏   1   接收   !D     隐藏   隐藏   1   发送   HTTP/1.1 200 OK
Date: Tue, 02 Feb 2010 09:22:47 GMT

 

其中 Cookie 只能取 JSESSIONID=E77B3E1056776CBCE17A1058F29E5F0D.tomcat5 前面的不能要.

 

okto=1&xianshi=1&airway=CA&airwayscode=%E4%B8%AD%E5%9B%BD%E5%9B%BD%E9%99%85%E8%88%AA%E7%A9%BA%E5%85%AC%E5%8F%B8&dep=SHA&arrcity=%E4%B8%8A%E6%B5%B7%E8%99%B9%E6%A1%A5&arr=PEK&city=%E5%8C%97%E4%BA%AC%E9%A6%96%E9%83%BD&commany=&flightdatestart=&agioPercent=7&agioPercent=6.3&agioPercent=6.2&agioPercent=6.2&agioPercent=6.1&agioPercent=6&agioPercent=6&agioPercent=5.5&agioPercent=5.1&agioPercent=5&agioPercent=5&agioPercent=5&agioPercent=5&agioPercent=4.7&agioPercent=4.6&agioPercent=4.5&agioPercent=4.5&agioPercent=4.4&agioPercent=4.3&agioPercent=4.2&current_page=1&rows=29&pages=2&gotoPage=1

 

这一节全部都要,写到 Request.requestStream 里面.