Scrapy框架中Downloader Middleware核心方法:process_request,process_response,process_exception个人学习笔记
说明
最近在学习python Scrapy框架的使用,在看Downloader Middleware的核心方法时候,发现可以用visio画个有关三个核心方法(process_request(request,spider),process_request(request, response, spider),process_exception(request, exception, spider))的图片方便理解,现在拿来与大家分享
更新:Spider Middleware,Item Pipeline
参考书目及网站
用书《Python3网络爬虫开发实战》崔庆才著. --北京:人民邮电出版社,2018.4(2018.7重印)(图灵原创)
本节涉及内容请参考作者博客(Downloader Middleware): https://cloud.tencent.com/developer/article/1151841.
用书作者博客: https://cuiqingcai.com/.
更新,
参考博客(Spider Middleware): https://cloud.tencent.com/developer/article/1151839.
参考博客(Item Pipeline):https://cloud.tencent.com/developer/article/1151838.
学习笔记
个人随意画的,仅供理解参考,不当之处欢迎指正
Downloader Middleware:
更新:
Spider Middleware
Item Pipeline