Python实现考拉海购数据采集

最新推荐文章于 2023-01-04 10:16:56 发布

终究不过路人

最新推荐文章于 2023-01-04 10:16:56 发布

阅读量1.2k

点赞数

分类专栏： Python案例文章标签： python pycharm 开发语言

本文链接：https://blog.csdn.net/python_ku/article/details/123408972

版权

本文介绍了使用Python3.8和PyCharm专业版进行数据采集的过程，详细阐述了从发送请求、获取响应、解析网页数据到最终保存数据的步骤，并涉及到requests和parsel库的使用。

摘要由CSDN通过智能技术生成

环境介绍

python 3.8
pycharm 2021专业版
requests >>> pip install requests
parsel >>> pip install parsel

代码实现步骤:

发送请求 >>> 获取数据 >>> 解析数据 >>> 保存数据

代码

发送请求

headers = {
   
    'cookie': 'kaola_user_key=b640efcb-cc0c-4892-9e58-c506543c3b83; JSESSIONID-WKL-8IO=pOH3hziWLSemIOWCMgbpwlepWeb7nVy7uhD%2BoduXLE9%5C%2FnMD7uu%2F%2B9as9RQVWdcPbOe%2FDQfxKTvAV5j0IdlOU7HNtia8TVLdgNfm2PJUlkztL5xj3h0bvWHSx%2FDiq%5COpmVmWLNbrW3MsNkZU85%5CJ%2BaAB5bpCK92UJyxapJIg1jTmW1O4%3A1646288721219; _klhtxd_=31; cna=diVbGqmG/CYCAa8APXP4VXR9; __da_ntes_utma=2525167.670118404.1646202322.1646202322.1646202322.1; davisit=1; __da_ntes_utmz=2525167.1646202322.1.1.utmcsr%3D(direct)%7Cutmccn%3D(direct)%7Cutmcmd%3D(none); __da_ntes_utmfc=utmcsr%3D(direct)%7Cutmccn%3D(direct)%7Cutmcmd%3D(none); xlly_s=1; _samesite_flag_=true; cookie2=1ad0e474c299ec6653e7c5bdbdb00778; t=5da53c986013272dd516019bc902aeec; _tb_token_=eb68509f646be; csg=8c843e87; NTES_OSESS=140147a12e6547538e725cc81b4a63d8; KAOLA_USER_ID=109999078967764758; KAOLA_MAIN_ACCOUNT=16458731261947577@pvkaola.163.com; unb=2213306950380; kaola_csg=287e094b; kaola-user-beta-traffic=15818434647; firstLogin=0; ucn=center; KAOLA_USER_ID.sig=JApGPboS22_xHs24DTHRstXn6LVy3Y0c5tc7qcINN_o; KAOLA_NEW_USER_COOKIE=yes; hb_MA-AE38-1FCC6CD7201B_source=search.kaola.com; __da_ntes_utmb=2525167.1.10.1646205828; NTES_KAOLA_RV=10719774; x5sec=7b227761676272696467652d616c69626162612d67726f75703b32223a226639386134393365663538623963316130636139626433633365363465613165434f4f342f4a4147454c5879774f4c436b5a544b43786f504d6a49784d7a4d774e6a6b