预先准备
在开始前,我们先按scrapy官方文档提供的方法新建一个示例项目:
在安装好了scrapy后,在命令行scrapy项目部署的目录中运行以下命令,新建名为tutorial的项目:
scrapy startproject tutorial
接着,我们就应该会获得一个有如下目录结构的scrapy项目:
tutorial/
scrapy.cfg # deploy configuration file
tutorial/ # project's Python module, you'll import your code from here
__init__.py
items.py # project items definition file
pipelines.py # project pipelines file
settings.py # project settings file
spiders/ # a directory where you'll later put your spiders
__init__.py
接着在项目的spiders目录下新建一个python文件,命名为demo_spider.py,并写入如下内容:
import scrapy
class QuotesSpider(scrapy.Spider):
name = "quotes"
def