访问的网站是:http://www.imooc.com/course/list?sort=pop
首先我们创建一个Scrapy项目
$ scrapy startproject mooc_subjects
New Scrapy project 'mooc_subjects', using template directory '/home/pit-yk/anaconda3/lib/python3.6/site-packages/scrapy/templates/project', created in:
/media/pit-yk/办公/python/codes/知乎专栏---Ehco/Scrapy/mooc_subjects
You can start your first spider with:
cd mooc_subjects
scrapy genspider example example.com
$ tree
.
├── mooc_subjects
│ ├── __init__.py
│ ├── items.py
│ ├── middlewares.py
│ ├── pipelines.py
│ ├── __pycache__
│ │ ├── __init__.cpython-36.pyc
│ │ ├── items.cpython-36.pyc
│ │ ├── pipelines.cpython-36.pyc
│ │ └── settings.cpython-36.pyc
│ ├── settings.py
│ └── spiders
│ ├── __init__.py
│ ├── MySpider.py
│ └── __pycache__
│ ├── __init__.cpython-36.pyc
│ └── MySpider.cpython-36.pyc
├── mooc_subjects.txt
└── scrapy.cfg
4 directories, 15 files
<