scrapyd和spiderkeeper实现scarpy项目的可视化操作

scrapy可视化管理工具

scrapyd

• 什么是scrapyd？

  Scrapyd是一个服务，用来运行scrapy爬虫的

  它允许你部署你的scrapy项目以及通过HTTP JSON的方式控制你的爬虫

• scrapyd官方文档

  https://scrapyd.readthedocs.io/en/stable/
  

• 安装scrapyd服务

$ pip install scrapyd

• 如何部署爬虫项目到scrapyd中

# 官方文档
Deploying your project involves eggifying it and uploading the egg to Scrapyd via the addversion.json endpoint. You can do this manually, but the easiest way is to use the scrapyd-deploy tool provided by scrapyd-client which will do it all for you.

# 即使用scrapyd-client 中提供的scrapyd-deploy tool来进行部署

• scrapyd-client

Scrapyd-client is a client for Scrapyd. I