首先,有参考这边文章:
scrapyd spiderkeeper docker部署 by QY_大师兄
我这边只是一个很简单的scrapy项目,需要同时启动 scrapyd服务和spiderkeeper服务,启动完了之后,还要上传egg文件部署。
1.先修改scrapy.cfg文件,需要添加 [scrapyd]
# Automatically created by: scrapy startproject
#
# For more information about the [deploy] section see:
# https://scrapyd.readthedocs.io/en/latest/deploy.html
[settings]
default = newsSpider.settings
[deploy]
url = http://0.0.0.0:6800/
# url = http://localhost:6800/
project = newsSpider
[scrapyd