火车头采集案例 - js源码获取真实地址

比如采集买新鲜.

这个网址源码都是由js生成. 没有html代码.

步骤1

打开fildder. 然后刷新下当前页面,查找真实网址
http://www.my-fresh.com/Store/Ajax/Context/DynamicLinq.ashx

这个POST网站浏览器时不识别的，要转化成GET—就是在这个网址后面加上？和FD抓取的数据包数据组合得出的网址.

http://www.my-fresh.com/Store/Ajax/Context/DynamicLinq.ashx?where=Enable%3D%3Dtrue+%26%26+Visable%3D%3Dtrue+%26%26+Categories!%3Dnull+%26%26+Categories.Any(it+in+(%22ca9094ec-1dbe-45a4-b5b8-24bb0f8aeca4%22))&orderby=Guid.NewGuid()&select=new(Id%2CTitle%2CNumber%2CImage%2CImages%2CBrandId%2CVendorId%2CCartId%2CCategories%2CIntroduce%2CIsStockEnable%2CStock%2CIsVariantEnable%2CVariants%2CCreateOn%2CListPrice%2CPrice%2CIsOnSale%2CSaleStart%2CSaleEnd%2CSalePrice)&take=10&skip=0&path=products

就是你了.