从安装supervisor到管理pyspider进程

http://www.shsay.com

 

原文链接http://blog.csdn.net/xyang81/article/details/51555473

pyspider配置https://www.cnblogs.com/microman/p/6138082.html

安装

pip install supervisor

 

运行echo_supervisord_conf程序生成supervisor的初始化配置文件

echo_supervisord_conf > /etc/supervisord.conf

 

mkdir supervisor_conf

修改/etc/supervisord.conf中的include参数，将/etc/supervisor_conf目录添加到include中

vim /etc/supervisord.conf

前面的分号表示注释，记得去掉

 

[include]
files = /etc/supervisor_conf/*.ini

 

cd /etc/supervisord.conf/

vim pyspider.ini

内容放到文章最后

 

cd /home/c_pyspider/

mkdir logs

cd logs

vim webui.log

vim scheduler.log

vim processor.log

vim worker.log

vim fetcher.log

vim phantomjs.log

 

启动服务

supervisord -c /etc/supervisord.conf

 

注意：在执行第一条命令出现以下提示信息时：

Error: Another program is already listening on a port that one of our HTTP servers is configured to use.  
Shut this program down first before starting supervisord.
For help, use /usr/bin/supervisord –h

是因为有一个使用supervisor配置的应用程序正在运行，需要在supervisorctl交互终端执行 shutdown命令终止它，或者执行reload 来重新启动配置中的所有程序，或者在xhell命令行直接执行

 

find / -name supervisor.sock

unlink /tmp/supervisor.sock

常用命令：                                                                                                                                 

 

 

service supervisord stop

service supervisord start

 

二、更新新的配置到supervisord    

[vb] view plain copy

1. supervisorctl update  

三、重新启动配置中的所有程序

[vb] view plain copy

1. supervisorctl reload  

四、启动某个进程(program_name=你配置中写的程序名称)

[vb] view plain copy

1. supervisorctl start program_name  

五、查看正在守候的进程

[vb] view plain copy

1. supervisorctl  

六、停止某一进程 (program_name=你配置中写的程序名称)

[vb] view plain copy

1. pervisorctl stop program_name  

七、重启某一进程 (program_name=你配置中写的程序名称)

[vb] view plain copy

1. supervisorctl restart program_name  

八、停止全部进程

[vb] view plain copy

1. supervisorctl stop all  

注意：显示用stop停止掉的进程，用reload或者update都不会自动重启。

 

 

 

其它：开启web界面和配置开机启动

 

[group:pyspider]
program=pyspider-webui,pyspider-scheduler,pyspider-processor,pyspider-result_worker,pyspider-fetcher,pyspider-phantomjs
priority=999

[program:pyspider-webui]
directory=/home/c_pyspider
command=pyspider -c config.json webui
autostart=true
autorestart=true
priority=905
user=root
stdout_logfile_maxbytes = 20MB
stdout_logfile_backups = 20
stdout_logfile = /home/c_pyspider/logs/webui.log


[program:pyspider-scheduler]
directory=/home/c_pyspider
command=pyspider -c config.json scheduler
autostart=true
autorestart=true
priority=900
user=root
stdout_logfile_maxbytes = 20MB
stdout_logfile_backups = 20
stdout_logfile = /home/c_pyspider/logs/scheduler.log

[program:pyspider-processor]
command=pyspider -c config.json  processor
directory=/home/c_pyspider
autostart=true
autorestart=true
priority=903
user=root
stdout_logfile_maxbytes = 20MB
stdout_logfile_backups = 20
stdout_logfile = /home/c_pyspider/logs/processor.log

[program:pyspider-result_worker]
command=pyspider -c config.json result_worker
directory=/home/c_pyspider
autostart=true
autorestart=true
priority=904
user=root
stdout_logfile_maxbytes = 20MB
stdout_logfile_backups = 20
stdout_logfile = /home/c_pyspider/logs/worker.log

[program:pyspider-fetcher]
command=pyspider -c config.json  --phantomjs-proxy="localhost:25555" fetcher
directory=/home/c_pyspider
autostart=true
autorestart=true
priority=902
user=root
stdout_logfile_maxbytes = 20MB
stdout_logfile_backups = 20
stdout_logfile = /home/c_pyspider/logs/fetcher.log

[program:pyspider-phantomjs]
command=pyspider -c config.json phantomjs
directory=/home/c_pyspider
autostart=true
autorestart=true
stopasgroup=true
stopsignal=QUIT
;上面两条针对supervisor停止phantomjs进程
priority=901
user=root
stdout_logfile_maxbytes = 20MB
stdout_logfile_backups = 20
stdout_logfile = /home/c_pyspider/logs/phantomjs.log