(一)scrapy.FormRequest
友情链接:https://blog.csdn.net/qq_33472765/article/details/80958820
普通请求使用scrapy.Request类就可以实现,但是遇到模拟表单或Ajax提交post请求的时候,Request类就不如 子类 FormRequest类方便了,FormRequest类自带 formdata ,专门用来设置表单字段数据,默认method也是POST。
# 获得音乐信息
def parse_music(self, response):
music_id = response.meta['id']
music = response.xpath('//div[@class="tit"]/em[@class="f-ff2"]/text()').extract_first()
artist = response.xpath('//div[@class="cnt"]/p[1]/span/a/text()').extract_first()
album = response.xpath('//div[@class="cnt"]/p[2]/a/text()').extract_first()
data = {
'csrf_token': '',
'params': 'Ak2s0LoP1GRJYqE3XxJUZVYK9uPEXSTttmAS+8uVLnYRoUt/Xgqdrt/13nr6OYhi75QSTlQ9FcZaWElIwE+oz9qXAu87t2DHj6Auu+2yBJDr+arG+irBbjIvKJGfjgBac+kSm2ePwf4rfuHSKVgQu1cYMdqFVnB+ojBsWopHcexbvLylDIMPulPljAWK6MR8',
'encSecKey': '8c85d1b6f53bfebaf5258d171f3526c06980cbcaf490d759eac82145ee27198297c152dd95e7ea0f08cfb7281588cdab305946e01b9d84f0b49700f9c2eb6eeced8624b16ce378bccd24341b1b5ad3d84ebd707dbbd18a4f01c2a007cd47de32f28ca395c9715afa134ed9ee321caa7f28ec82b94307d75144f6b5b134a9ce1a'
}
DEFAULT_REQUEST_HEADERS['Referer'] = self.base_url + '/playlist?id=' + str(music_id)
music_comment = 'http://music.163.com/weapi/v1/resource/comments/R_SO_4_' + str(music_id)
yield FormRequest(music_comment, meta={'id':music_id,'music':music,'artist':artist,'album':album}, \
callback=self.parse_comment, formdata=data)
(二)scrapy.FormRequest.from_response
友情链接:https://www.cnblogs.com/ywjfx/p/11089248.html
FormRequest表单实现自动登陆
FormRequest.from_response模拟自动登陆