方法一:通过在cmd中输命令来实现
scrapy crawl novel -o novel.json -s FEED_EXPORT_ENCIDING=UTF-8
novel为爬虫名称(name)
scrapy将数据保存为csv文件: scrapy crawl novel -o novel.csv -s FEED_EXPORT_ENCIDING=UTF-8
方法二:通过pipelines实现
1.自定义自己的pipeline
pipelines.py
import json
class JsonPipeline(object):
def __init__(self):
self.file = open('novel.json', 'wb')
def process_item(self, item, spider):
line = json.dumps(dict(item)) + "\n"
self.file.write(line.encode('utf-8&