![](https://img-blog.csdnimg.cn/20201014180756738.png?x-oss-process=image/resize,m_fixed,h_64,w_64)
scrapy
Tools-lqp
这个作者很懒,什么都没留下…
展开
-
scrapy xpath css 经典使用
test() 函数 from scrapy import Selector doc = “”” … “”” sel = Selector(text=doc, type=”html”) sel.xpath(‘//li//@href’).extract() [u’link1.html’, u’link...原创 2018-04-06 18:43:27 · 570 阅读 · 0 评论 -
scrapy调用JsonItemPipline类 写入json文件中
调用JsonItemPipline类from scrapy.exporters import JsonItemExporterclass JsonExporterPipline(object):def __init__(self): self.file = open('article.json', 'wb') self.expore = JsonItemExporter...原创 2018-05-03 13:31:50 · 643 阅读 · 1 评论 -
scrapy重写下载img方法 记录存储位置
重写下载img方法 记录存储位置from scrapy.pipelines.images import ImagesPipelineclass download_img(ImagesPipeline):def item_completed(self, results, item, info): # 判断有URL过来 if 'image_urls' in item: ...原创 2018-05-03 20:25:44 · 482 阅读 · 1 评论 -
scrapy twisted.python.failure.Failure OpenSSL.SSL.Error
scrapy twisted.python.failure.Failure OpenSSL.SSL.Errorfrom OpenSSL import SSLfrom scrapy.core.downloader.contextfactory import ScrapyClientContextFactoryclass CustomContextFactory(ScrapyClientCo...原创 2019-09-29 19:41:29 · 3621 阅读 · 11 评论