页面分析
https://insight.baidu.com/base/search/rank/list?pageSize=20&source=0&toFixed=1&filterType=1&dateType=20200511~20200517&dimensionid=37&rateType=1000
Get请求:
parsed:
页面显示数量
pageSize: 20
source: 0
toFixed: 1
filterType: 1
时间段
dateType: 20200511~20200517
商品类型
dimensionid: 37
rateType: 1000
非必要参数
filterNodes:
callback: _jsonpsutwe93bymi
请求的url:
https://insight.baidu.com/base/search/rank/list?pageSize=20&dateType=20200511~20200517&dimensionid=37
parsed: 页面显示数量,时间段,商品类型
dataType:可单日期:20100510,可时间段:20200511~20200517
代码:
import requests
from fake_useragent import UserAgent
url='https://insight.baidu.com/base/search/rank/list?'
headers={
'User-agent':str(UserAgent(verify_ssl=False).random),
'Cookie':'BAIDUID=14314D9D1182C0F3F18E4D32FB781398:FG=1; BIDUPSID=14314D9D1182C0F3F18E4D32FB781398; PSTM=1589471352; BDUSS=1dPaWxDaFI5akdiSlVpaUN5akR3TmZWRXhNM1V1eGF1V0J3OEhjaklCZ1NqZTFlRVFBQUFBJCQAAAAAABAAAAEAAACfVjf7YXNpbW92MDUxOAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAABIAxl4SAMZeb; ZD_ENTRY=empty; bdindexid=4gqnrgiqkck0gcbrufvjkka5o2; RT="z=1&dm=baidu.com&si=avs4x0yocsr&ss=kapk89x0&sl=5&tt=3pk&bcn=https%3A%2F%2Ffclog.baidu.com%2Flog%2Fweirwood%3Ftype%3Dperf"',
'Accept-Encoding': 'gzip, deflate',
'Accept-Language': 'zh-CN,zh;q=0.9',
}
parsed={
# 每页显示的数量
'pageSize':'20',
# 日期范围 可单日
'dateType':'20200511~20200517',
# 类型id 37:化妆品
'dimensionid':'37',
}
req=requests.get(url,headers=headers,params=parsed).json()
# 类型id_37 化妆品
req_type=req.get('data').get('dimensionid')
# 前20的品牌
req_results=req.get('data').get('results').get('current')
print(req_type)
print(req_results)