天善智能爬虫实践

在运行scrapy代码时报错;
Traceback (most recent call last):
File “D:\Python\Anaconda\lib\site-packages\twisted\internet\defer.py”, line 654, in _runCallbacks
current.result = callback(current.result, *args, **kw)
TypeError: close_spider() takes 1 positional argument but 2 were given

发现此代码段报错:
把pipelines.py里
def process_item(self, item, spider):
item_json=json.dumps(item,ensure_ascii=False)
self.fp.write(item_json+’\n’)
return item
改成
def close_spider(self,spider):
self.fp.close()
print(‘爬虫结束’)

这里是覆盖方法，spider参数没用到也必须加。