京东商品及价格存入csv文本,只有静态的30个逐页爬,动态的s=30,87,141,206, n=2,4,6,8。
可以再下面在写个函数直接存到文本里,就是这个参数:
把图片往右拖,network,里的XHR的链接规则:
代码:
import requests
from urllib.parse import urlencode
from lxml import etree
import csv
def request(kw,page,s):
headers = {"User-Agent": "Mozilla/5.0 (Windows NT 10.0; WOW64; Trident/7.0; rv:11.0) like Gecko",
"Cookie":"__jdu=965081754; shshshfpa=d8651c76-9914-ed87-bb05-6f3d29a46061-1543231749; shshshfpb=0a7cbd16444b16711e44638105fd14f758419bbc053620b7f5bfbd9064; qrsc=3; __jdc=122270672; __jdv=122270672|direct|-|none|-|1547172752163; PCSYCityID=698; xtest=8541.cf6b6759; ipLoc-djd=1-72-2799-0; rkv=V0800; user-key=9f422950-