上一节有几个类还没具体分析,如Scheduler和Scraper,这一节先分析Scheduler的源码。
scrapy/core/scheduler.py:
在分析engine的open_spider函数时,我们讲过scheduler对象是通过类的from_cralwer方法生成的,我们先看下这个方法的实现:
@classmethod def from_crawler(cls, crawler): settings = crawler.settings dupefilter_cls = load_object(settings['DUPEFILTER_CLASS']) dupefilter = dupefilter_cls.from_settings(settings) pqclass = load_object(settings[