今天用Scrapy的ImagesPipeline爬虫个的时候,开启了管道,但是get_media_requests一直没有执行,代码如下
Pipeline代码
class MimeImagePipeline(ImagesPipeline):
def get_media_requests(self, item, info):
print('xxxxxxxxxx')
ims=item['name']
yield scrapy.Request(ims)
def file_path(self, request, response=None, info=None, *, item=None):
print('进入了')
return 'haha.jpg'
def item_completed(self, results, item, info):
return item
Setting配置
ITEM_PIPELINES = {
# "scrapy_demo01.pipelines.ScrapyDemo01Pipeline": 300,
"scrapy_demo01.pipelines.MimeImagePipeline": 300,
}
调试了很久都没有发现问题,但是打开日志的时候,提示 Enabled item pipelines:[] ,意思就是没有开启管道类,最后再日志中发现了这一句话
Disabled mediaPileline: ImagesPipeline requires installing Pillow 4.0.0 or later
大概意思就是要安装Pillow 4.0.0之后的版本,于是我安装了pillow,就可以正常启用管道类了
pip install pillow -i https://pypi.douban.com/simple