1、命令:workon article_spider后去到D盘主目录
2、命令:scrapy startproject ArticleSpider
3、命令:scrapy genspider jobbole blog.jobbole.com
(win系统下需要pip install -i https://pypi.douban.com/simple pypiwin32)
4、导入ArticleSpider
(导入后如果出现This inspection detects names that should resolve but don't. Due to dynamic dispatch and duck typing, this is possible in a limited but useful number of cases. Top-level and class-level items are supported better than instance items.)修改如下即可解决。即是对应的虚拟环境
5、新建main.py
内容:
from scrapy.cmdline import execute import sys import os sys.path.append(os.path.dirname(os.path.abspath(__file__))) execute(["scrapy","crawl","jobbole"])
6、打点debug调试