scrapy添加cookie

最新推荐文章于 2024-03-19 06:47:18 发布

我把把C

最新推荐文章于 2024-03-19 06:47:18 发布

阅读量889

点赞数 3

文章标签： scrapy 爬虫 python

本文链接：https://blog.csdn.net/qq_62975494/article/details/130143229

版权

1.在DEFAULT_REQUEST_HEADERS中添加

第一步打开settings.py
将COOKIES_ENABLED = False解除注释
然后解除DEFAULT_REQUEST_HEADERS注释
在这里插入图片描述

2.使用scrapy.Request中的cookies参数

首先将COOKIES_ENABLED = False改为true
然后将cookie反序列化

cookie= '_ntes_nuid=8d967fa3f8554d20bd113de35746ad9c; Device-Id=i70Gko1JVoIs7qs6F968; _ga=GA1.2.1134628517.1630412253; _ntes_nnid=8d967fa3f8554d20bd113de35746ad9c,1665043982001; __bid_n=1856cf764a513990824207; FEID=v10-8313c85f7de79a8e2f560cde271fa7b1606b16b3; __xaf_fpstarttimer__=1672570430810; Locale-Supported=zh-Hans; game=csgo; AQ_HD=1; YD_SC_SID=0183795762374AFC8D9C97C77F0869E9; AQ_REQ_FROM=webzj; NTES_YD_SESS=F6QEHmHCD9BObRN.Srsg5rqa.akndo0_1fYhGRWqnnIpkrW0kCD17o36f6h2aVRvEw6vAGPsy9cflHv1WUPteUjmYFJw6kwZbPO2e0J9FDnEHGx.nazr8Lv.OX2bE.Toe68zJoIBYEmEoAGRQR5PyAVdOygO36op.z4M_mX7B772mcNSXqtW6O_oO26gHhDDirSAPjxKNtQ7Miej55zhqhc0hbKCriMn_ivsxYf1UZApV; S_INFO=1681284174|0|0&60##|19196482127; P_INFO=19196482127|1681284174|1|netease_buff|00&99|sic&1681282167&netease_buff#sic&510100#10#0#0|&0|null|19196482127; remember_me=U1106914556|dVZ7ZfYFVatOpmSamrbKsDobKKjVsfKl; session=1-G0v5cQhu-08b_gOO6OAHPFQTBfxXJh5_Mzb6DuINue6h2033496996; csrf_token=IjA2NzQ1MGRjNTcyYjAzOTMxNTMxNWUzOWVkYTMzNGY5ZDZkNzQ0MDki.Fxft2Q.NstXpDi0ofj2at6k-3sOsCfLugY'
cookies={ data.split('=')[0] : data.split('=')[1] for data in cookie.split(';')}
print(cookies)
yield scrapy.Request(url='https://buff.163.com/api/market/goods/price_history/buff?',callback=self.prices,cookies=cookies)