一、适用条件
可以对有规律或者无规律的网站进行自动爬取
二、代码讲解
(1)创健scrapy项目
E:myweb>scrapy startproject mycwpjt
New Scrapy project 'mycwpjt', using template directory 'd:\\python35\\lib\\site-packages\\scrapy\\templates\\project', created in:
D:\Python35\myweb\part16\mycwpjt
You can start your first spider with:
cd mycwpjt
scrapy genspider example example.com
(2) 创健爬虫
E:\myweb>scrapy genspider -t crawl weisuen sohu.com
Created spider 'weisuen' using template 'crawl' in module:
Mycwpjt.spiders.weisuen
(3)item编写