scrapy Missing scheme in request url

最新推荐文章于 2021-11-21 01:13:44 发布

小丫小屁孩

最新推荐文章于 2021-11-21 01:13:44 发布

阅读量268

点赞数

分类专栏： python爬虫

本文链接：https://blog.csdn.net/qq_42636010/article/details/90237143

版权

python爬虫专栏收录该内容

8 篇文章 0 订阅

订阅专栏

今天在写爬虫的时候出现这个问题，报错为：

Traceback (most recent call last):
  File "E:\anaconda\Anaconda3\lib\site-packages\twisted\internet\defer.py", line 654, in _runCallbacks
    current.result = callback(current.result, *args, **kw)
  File "E:\anaconda\Anaconda3\lib\site-packages\scrapy\pipelines\media.py", line 80, in process_item
    dlist = [self._process_request(r, info) for r in requests]
  File "E:\anaconda\Anaconda3\lib\site-packages\scrapy\pipelines\media.py", line 80, in <listcomp>
    dlist = [self._process_request(r, info) for r in requests]
  File "e:\pycharmProject\MoocSpider\MoocSpider\MoocImgPipelines.py", line 15, in get_media_requests
    yield scrapy.Request(item['imageUrl'])
  File "E:\anaconda\Anaconda3\lib\site-packages\scrapy\http\request\__init__.py", line 25, in __init__
    self._set_url(url)
  File "E:\anaconda\Anaconda3\lib\site-packages\scrapy\http\request\__init__.py", line 62, in _set_url
    raise ValueError('Missing scheme in request url: %s' % self._url)
ValueError: Missing scheme in request url: //img1.mukewang.com/5b14e8310001197706000338-240-135.jpg

这是出现在我的imagepipeline中的错误，原因是我get_media_requests（）方法中传入request的图片的url没有http，就这种//img1.mukewang.com/5b14e8310001197706000338-240-135.jpg，没有带http，在前边加上http:就行了。