![](https://img-blog.csdnimg.cn/20201014180756780.png?x-oss-process=image/resize,m_fixed,h_64,w_64)
爬虫
城市的柏油路太硬
城市泊油路太硬
展开
-
scrapy自定义重试
1.通过响应的 状态码或者异常来进行重试class MyselfSpiderMiddleware(RetryMiddleware): def process_response(self, request, response, spider): if request.meta.get('dont_retry', False): return response if response.status in self.retry_http...原创 2020-10-19 14:48:54 · 699 阅读 · 0 评论 -
获取网页中的charset
m = re.compile('<meta .*(http-equiv="?Content-Type"?.*)?charset="?([a-zA-Z0-9_-]+)"?', re.I).search(response_text)if m and m.lastindex == 2: charset = m.group(2).lower()转载 2020-08-19 19:32:05 · 293 阅读 · 0 评论