MacOS中创建Scrapy项目

创建Scrapy项目

$ scrapy startproject ZhipinSpider

New Scrapy project 'ZhipinSpider', using template directory '/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/scrapy/templates/project', created in:
    /Users/xxxxxxx/PycharmProjects/ZhipinSpider

You can start your first spider with:
    cd ZhipinSpider
    scrapy genspider example example.com

Scrapy项目分析

1. scrapy.cfg:项目的总配置文件，通常无需修改
2. ZhipinSpider/:项目Python模块，程序将从此处导入Python代码

Scrapy概览

Scrapy核心组件

调度器：该组件由Scrapy框架实现
下载器：该组件由Scrapy框架实现
蜘蛛：该组件由开发者实现
Pipeline：该组件有开发者实现