Supervisor配合容器使用

最新推荐文章于 2024-08-27 16:28:21 发布

凯尔kyle

最新推荐文章于 2024-08-27 16:28:21 发布

阅读量760

点赞数

分类专栏： docker Linux

本文链接：https://blog.csdn.net/xys2015/article/details/109410032

版权

Linux 同时被 2 个专栏收录

91 篇文章 1 订阅

订阅专栏

docker

23 篇文章 0 订阅

订阅专栏

关键词：服务管理容器内启动多个进程多进程管理 Supervisor Docker

需求分析：平常我们管理服务，最方便的就系统工具直接安装，比如yum安装，systemctl管理，但有些脚本或服务是我们自己写，在额外配上xx.service 管理脚本还有点麻烦，并且要要想让容器支持systemd功能就不太方便

解决方案：引入supervisor管理工具帮我们解决此类问题

快速体验

以CentOS7为例

开启EPEL仓库

参考 https://blog.csdn.net/xys2015/article/details/109378741

安装

yum info supervisor
yum install supervisor

配置

cat << 'EOF' > /etc/supervisord.d/p1.ini
[program:p1]
command=/bin/cat
EOF

cat << 'EOF' > /etc/supervisord.d/p2.ini
[program:p2]
command=/bin/cat
EOF

cat << 'EOF' > /etc/supervisord.d/p3.ini
[program:p3]
command=/bin/cat
EOF

p1、p2、p3 是自定义的程序名
command填写启动命令，这里用 /bin/cat 纯粹是测试用

启动

systemctl restart supervisord.service

查看

# supervisorctl status
p1                               RUNNING   pid 1150, uptime 0:04:21
p2                               RUNNING   pid 1148, uptime 0:04:21
p3                               RUNNING   pid 1149, uptime 0:04:21
# supervisorctl stop p1
# supervisorctl status p1
# supervisorctl start p1

开机自启(可选)

systemctl enable supervisord.service

supervisord 启动之后，其管理的服务会自动起来，除非额外配置 (见下文)

启用图形界面

配置改动如下图:

在这里插入图片描述

配置改动diff对比

[root@10-222-32-10 ~]# cp /etc/supervisord.conf /etc/supervisord.conf.bak
[root@10-222-32-10 ~]# diff /etc/supervisord.conf /etc/supervisord.conf.bak
10,11c10,11
< [inet_http_server]         ; inet (TCP) server disabled by default
< port=0.0.0.0:9001        ; (ip_address:port specifier, *:port for all iface)
---
> ;[inet_http_server]         ; inet (TCP) server disabled by default
> ;port=127.0.0.1:9001        ; (ip_address:port specifier, *:port for all iface)
40,41c40,41
< ;serverurl=unix:///var/run/supervisor/supervisor.sock ; use a unix:// URL  for a unix socket
< serverurl=http://0.0.0.0:9001 ; use an http:// url to specify an inet socket
---
> serverurl=unix:///var/run/supervisor/supervisor.sock ; use a unix:// URL  for a unix socket
> ;serverurl=http://127.0.0.1:9001 ; use an http:// url to specify an inet socket

重启 supervisord

systemctl restart supervisord.service
netstat -lntup | grep 9001

访问 9001 端口

http://10.222.32.10:9001/

在这里插入图片描述

启动顺序的控制 (常用配置文件介绍)

如果我们的服务是几个程序配套提供服务，并且它们之间有先后启动顺序(比如MySQL先启动，PHP后启动)，那么我们可能需要控制启动顺序

最好能把服务启动顺序交给程序自行管理，比如设计PHP先启动自己去尝试重连MySQL若干次等等

此时，我们可以自己写脚本控制启动顺序，或者利用supervisor自己的配置文件来控制

[program:myname]
command=sh /tmp/echo_time.sh
priority=999
numprocs=1
autostart=true
autorestart=true
startsecs=10
startretries=3 
exitcodes=0,2
stopsignal=QUIT
stopwaitsecs=10
user=root
log_stdout=true
log_stderr=true
logfile=/tmp/echo_time.log
logfile_maxbytes=1MB
logfile_backups=10 
stdout_logfile_maxbytes=20MB 
stdout_logfile_backups=20 
stdout_logfile=/tmp/echo_time.stdout.log


command：要执行的命令
priority：执行优先级，值越高就越晚启动，越早关闭
autostart：是否与 supervisord 一起启动
autorestart： 程序发生未知错误时自动重启
startsecs：延时启动时间，默认为 10 秒
startretries：启动重试次数，默认为 3 次
user：以哪个用户执行

更多配置细节介绍，参见官网 http://supervisord.org/configuration.html

yum 安装好后默认的配置

[root@10-222-32-10 ~]# cat /etc/supervisord.conf
; Sample supervisor config file.

[unix_http_server]
file=/var/run/supervisor/supervisor.sock   ; (the path to the socket file)
;chmod=0700                 ; sockef file mode (default 0700)
;chown=nobody:nogroup       ; socket file uid:gid owner
;username=user              ; (default is no username (open server))
;password=123               ; (default is no password (open server))

;[inet_http_server]         ; inet (TCP) server disabled by default
;port=127.0.0.1:9001        ; (ip_address:port specifier, *:port for all iface)
;username=user              ; (default is no username (open server))
;password=123               ; (default is no password (open server))

[supervisord]
logfile=/var/log/supervisor/supervisord.log  ; (main log file;default $CWD/supervisord.log)
logfile_maxbytes=50MB       ; (max main logfile bytes b4 rotation;default 50MB)
logfile_backups=10          ; (num of main logfile rotation backups;default 10)
loglevel=info               ; (log level;default info; others: debug,warn,trace)
pidfile=/var/run/supervisord.pid ; (supervisord pidfile;default supervisord.pid)
nodaemon=false              ; (start in foreground if true;default false)
minfds=1024                 ; (min. avail startup file descriptors;default 1024)
minprocs=200                ; (min. avail process descriptors;default 200)
;umask=022                  ; (process file creation umask;default 022)
;user=chrism                 ; (default is current user, required if root)
;identifier=supervisor       ; (supervisord identifier, default is 'supervisor')
;directory=/tmp              ; (default is not to cd during start)
;nocleanup=true              ; (don't clean up tempfiles at start;default false)
;childlogdir=/tmp            ; ('AUTO' child log dir, default $TEMP)
;environment=KEY=value       ; (key value pairs to add to environment)
;strip_ansi=false            ; (strip ansi escape codes in logs; def. false)

; the below section must remain in the config file for RPC
; (supervisorctl/web interface) to work, additional interfaces may be
; added by defining them in separate rpcinterface: sections
[rpcinterface:supervisor]
supervisor.rpcinterface_factory = supervisor.rpcinterface:make_main_rpcinterface

[supervisorctl]
serverurl=unix:///var/run/supervisor/supervisor.sock ; use a unix:// URL  for a unix socket
;serverurl=http://127.0.0.1:9001 ; use an http:// url to specify an inet socket
;username=chris              ; should be same as http_username if set
;password=123                ; should be same as http_password if set
;prompt=mysupervisor         ; cmd line prompt (default "supervisor")
;history_file=~/.sc_history  ; use readline history if available

; The below sample program section shows all possible program subsection values,
; create one or more 'real' program: sections to be able to control them under
; supervisor.

;[program:theprogramname]
;command=/bin/cat              ; the program (relative uses PATH, can take args)
;process_name=%(program_name)s ; process_name expr (default %(program_name)s)
;numprocs=1                    ; number of processes copies to start (def 1)
;directory=/tmp                ; directory to cwd to before exec (def no cwd)
;umask=022                     ; umask for process (default None)
;priority=999                  ; the relative start priority (default 999)
;autostart=true                ; start at supervisord start (default: true)
;autorestart=true              ; retstart at unexpected quit (default: true)
;startsecs=10                  ; number of secs prog must stay running (def. 1)
;startretries=3                ; max # of serial start failures (default 3)
;exitcodes=0,2                 ; 'expected' exit codes for process (default 0,2)
;stopsignal=QUIT               ; signal used to kill process (default TERM)
;stopwaitsecs=10               ; max num secs to wait b4 SIGKILL (default 10)
;user=chrism                   ; setuid to this UNIX account to run the program
;redirect_stderr=true          ; redirect proc stderr to stdout (default false)
;stdout_logfile=/a/path        ; stdout log path, NONE for none; default AUTO
;stdout_logfile_maxbytes=1MB   ; max # logfile bytes b4 rotation (default 50MB)
;stdout_logfile_backups=10     ; # of stdout logfile backups (default 10)
;stdout_capture_maxbytes=1MB   ; number of bytes in 'capturemode' (default 0)
;stdout_events_enabled=false   ; emit events on stdout writes (default false)
;stderr_logfile=/a/path        ; stderr log path, NONE for none; default AUTO
;stderr_logfile_maxbytes=1MB   ; max # logfile bytes b4 rotation (default 50MB)
;stderr_logfile_backups=10     ; # of stderr logfile backups (default 10)
;stderr_capture_maxbytes=1MB   ; number of bytes in 'capturemode' (default 0)
;stderr_events_enabled=false   ; emit events on stderr writes (default false)
;environment=A=1,B=2           ; process environment additions (def no adds)
;serverurl=AUTO                ; override serverurl computation (childutils)

; The below sample eventlistener section shows all possible
; eventlistener subsection values, create one or more 'real'
; eventlistener: sections to be able to handle event notifications
; sent by supervisor.

;[eventlistener:theeventlistenername]
;command=/bin/eventlistener    ; the program (relative uses PATH, can take args)
;process_name=%(program_name)s ; process_name expr (default %(program_name)s)
;numprocs=1                    ; number of processes copies to start (def 1)
;events=EVENT                  ; event notif. types to subscribe to (req'd)
;buffer_size=10                ; event buffer queue size (default 10)
;directory=/tmp                ; directory to cwd to before exec (def no cwd)
;umask=022                     ; umask for process (default None)
;priority=-1                   ; the relative start priority (default -1)
;autostart=true                ; start at supervisord start (default: true)
;autorestart=unexpected        ; restart at unexpected quit (default: unexpected)
;startsecs=10                  ; number of secs prog must stay running (def. 1)
;startretries=3                ; max # of serial start failures (default 3)
;exitcodes=0,2                 ; 'expected' exit codes for process (default 0,2)
;stopsignal=QUIT               ; signal used to kill process (default TERM)
;stopwaitsecs=10               ; max num secs to wait b4 SIGKILL (default 10)
;user=chrism                   ; setuid to this UNIX account to run the program
;redirect_stderr=true          ; redirect proc stderr to stdout (default false)
;stdout_logfile=/a/path        ; stdout log path, NONE for none; default AUTO
;stdout_logfile_maxbytes=1MB   ; max # logfile bytes b4 rotation (default 50MB)
;stdout_logfile_backups=10     ; # of stdout logfile backups (default 10)
;stdout_events_enabled=false   ; emit events on stdout writes (default false)
;stderr_logfile=/a/path        ; stderr log path, NONE for none; default AUTO
;stderr_logfile_maxbytes=1MB   ; max # logfile bytes b4 rotation (default 50MB)
;stderr_logfile_backups        ; # of stderr logfile backups (default 10)
;stderr_events_enabled=false   ; emit events on stderr writes (default false)
;environment=A=1,B=2           ; process environment additions
;serverurl=AUTO                ; override serverurl computation (childutils)

; The below sample group section shows all possible group values,
; create one or more 'real' group: sections to create "heterogeneous"
; process groups.

;[group:thegroupname]
;programs=progname1,progname2  ; each refers to 'x' in [program:x] definitions
;priority=999                  ; the relative start priority (default 999)

; The [include] section can just contain the "files" setting.  This
; setting can list multiple files (separated by whitespace or
; newlines).  It can also contain wildcards.  The filenames are
; interpreted as relative to this file.  Included files *cannot*
; include files themselves.

[include]
files = supervisord.d/*.ini

注意事项

启动命令不能把程序放在后台，否则supervisor无法管理，会出现被管理的进程反复自动重启的情况
启动命令最好全部是绝对路径，否则有可能报错

实战-容器内使用supervisord

下面以CentOS7容器为例，演示在容器中安装ssh、nginx服务，并通过supervisor管理

安装docker

参见拙文 https://blog.csdn.net/xys2015/article/details/109370082

启动容器

docker run --name stest -d \
-p 30022:22 -p 30080:80 -p 39001:9001 \
--env TZ='Asia/Shanghai' \
--env LANG="en_US.UTF-8" \
--env LC_ALL="en_US.UTF-8" \
daocloud.io/library/centos:7.8.2003 tail -F /tmp/tmp.txt

docker exec -it stest bash

更换阿里云基础仓库

参见拙文 https://blog.csdn.net/xys2015/article/details/109378741

配置ssh

yum install passwd openssl openssh-server openssh-clients net-tools -y
mkdir -p /var/run/ssh
ssh-keygen -q -t rsa -b 2048 -f /etc/ssh/ssh_host_rsa_key -N ''
ssh-keygen -q -t ecdsa -f /etc/ssh/ssh_host_ecdsa_key -N ''
ssh-keygen -t dsa -f /etc/ssh/ssh_host_ed25519_key -N ''

sed -i "s/#UsePrivilegeSeparation.*/UsePrivilegeSeparation no/g" /etc/ssh/sshd_config
sed -i "s/UsePAM.*/UsePAM no/g" /etc/ssh/sshd_config
echo 123456 | passwd --stdin root

配置nginx

yum install nginx -y
rpm -ql nginx | grep service

放一个busybox小工具进去 (可选)

cd /usr/bin
curl -O http://as4k.top:7000/chfs/shared/linux-pkg/busybox
chmod +x busybox
busybox netstat -lntup

配置supervisor

[root@a573002bdf05 bin]# rpm -ql supervisor | grep service
/usr/lib/systemd/system/supervisord.service

cat << 'EOF' > /etc/supervisord.d/nginx.ini
[program:nginx] 
command=/usr/sbin/nginx -c /etc/nginx/nginx.conf -g 'daemon off;'
EOF

cat << 'EOF' > /etc/supervisord.d/sshd.ini
[program:sshd]
command=/usr/sbin/sshd -D
EOF

#容器内默认没有文档那块内容，我们手动创建个测试
mkdir -p /usr/share/nginx/html
rm -f /usr/share/nginx/html/index.html
echo "hello nginx" > /usr/share/nginx/html/index.html

yum install vim -y   (可选)
vimdiff /etc/supervisord.conf /etc/supervisord.conf.bak (可选)

supervisor 配置文件修改参见下图，更多配置文件细节含义，参见上文和官网
在这里插入图片描述

启动supervisor

/usr/bin/supervisord -c /etc/supervisord.conf

验证测试

[root@a573002bdf05 /]# supervisorctl status
nginx                            RUNNING   pid 38, uptime 0:00:21
sshd                             RUNNING   pid 39, uptime 0:00:21
[root@a573002bdf05 /]# curl localhost:80
hello nginx
[root@a573002bdf05 /]# curl 10.222.32.10:30080  #这是物理机内网IP，一开始启动容器我们映射了端口
hello nginx

封装镜像

此时我们的镜像已经制作完毕，可以把容器固化成新的镜像供以后使用

docker commit [选项] <容器ID或容器名> [<仓库名>[:<标签>]]`

[root@10-222-32-10 ~]# docker ps
CONTAINER ID        IMAGE                                 COMMAND                  CREATED             STATUS              PORTS                                                                   NAMES
a573002bdf05        daocloud.io/library/centos:7.8.2003   "tail -F /tmp/tmp.txt"   27 hours ago        Up 7 minutes        0.0.0.0:30022->22/tcp, 0.0.0.0:30080->80/tcp, 0.0.0.0:39001->9001/tcp   stest
[root@10-222-32-10 ~]# yum clean all  #该步骤可减小容器尺寸
[root@10-222-32-10 ~]# docker commit --author "xingyongsheng.top" --message "first commit" stest stest:v2
[root@10-222-32-10 ~]# docker images | grep stest
stest                        v2                  7fab7987bfca        11 seconds ago      722MB

直接启动容器，手动执行命令做镜像，比较方便但是难以维护，适合测试环境，生产环境一般使用Dockfile来制作镜像

正常使用

docker rm -fv stest

docker run --name stest -d \
-p 30022:22 -p 30080:80 -p 39001:9001 \
--env TZ='Asia/Shanghai' \
--env LANG="en_US.UTF-8" \
--env LC_ALL="en_US.UTF-8" \
stest:v2 bash -c '/usr/bin/supervisord --nodaemon -c /etc/supervisord.conf'

docker logs -f stest

ssh、nginx、supervisor ui 都可以正常访问，这里我就不贴截图了，注意docker的启动命令一定要能够挂起在前台，对于supervisor来说，即加上--nodaemon，或者修改其配置文件

启动报错排查

有时候被supervisor管理的服务启动报错，或者无法停止，我们可以使用

supervisorctl tail programname stdout

来查看其详细输出信息

相关冗余信息

echo_supervisord_conf  #这个命令可以直接在终端打印处默认配置

[root@node9 yum.repos.d]# rpm -ql supervisor | grep -v 'site-packages'
/etc/logrotate.d/supervisor
/etc/supervisord.conf
/etc/supervisord.d
/etc/tmpfiles.d/supervisor.conf
/usr/bin/echo_supervisord_conf
/usr/bin/pidproxy
/usr/bin/supervisorctl
/usr/bin/supervisord
/usr/lib/systemd/system/supervisord.service
/usr/share/doc/supervisor-3.4.0
/usr/share/doc/supervisor-3.4.0/CHANGES.txt
/usr/share/doc/supervisor-3.4.0/COPYRIGHT.txt
/usr/share/doc/supervisor-3.4.0/LICENSES.txt
/usr/share/doc/supervisor-3.4.0/README.rst
/var/log/supervisor
/var/run/supervisor

cp /etc/supervisord.conf /etc/supervisord.conf.bak

[root@10-222-32-10 ~]# cat /usr/lib/systemd/system/supervisord.service | grep ExecStart
ExecStart=/usr/bin/supervisord -c /etc/supervisord.conf

[root@10-222-32-10 ~]# systemctl status supervisord.service
● supervisord.service - Process Monitoring and Control Daemon
   Loaded: loaded (/usr/lib/systemd/system/supervisord.service; disabled; vendor preset: disabled)
   Active: active (running) since Thu 2020-10-29 10:01:53 CST; 10h ago
  Process: 1432 ExecStart=/usr/bin/supervisord -c /etc/supervisord.conf (code=exited, status=0/SUCCESS)
 Main PID: 1435 (supervisord)
   CGroup: /system.slice/supervisord.service
           ├─1435 /usr/bin/python /usr/bin/supervisord -c /etc/supervisord.conf
           ├─1436 /bin/cat
           ├─1437 /bin/cat
           └─1469 /bin/cat

Oct 29 10:01:53 10-222-32-10 systemd[1]: Stopped Process Monitoring and Control Daemon.
Oct 29 10:01:53 10-222-32-10 systemd[1]: Starting Process Monitoring and Control Daemon...
Oct 29 10:01:53 10-222-32-10 systemd[1]: Started Process Monitoring and Control Daemon.
[root@10-222-32-10 ~]# cat /usr/lib/systemd/system/supervisord.service
[Unit]
Description=Process Monitoring and Control Daemon
After=rc-local.service nss-user-lookup.target

[Service]
Type=forking
ExecStart=/usr/bin/supervisord -c /etc/supervisord.conf

[Install]
WantedBy=multi-user.target


[root@a573002bdf05 /]# /usr/sbin/nginx -h
nginx version: nginx/1.16.1
Usage: nginx [-?hvVtTq] [-s signal] [-c filename] [-p prefix] [-g directives]

Options:
  -?,-h         : this help
  -v            : show version and exit
  -V            : show version and configure options then exit
  -t            : test configuration and exit
  -T            : test configuration, dump it and exit
  -q            : suppress non-error messages during configuration testing
  -s signal     : send signal to a master process: stop, quit, reopen, reload
  -p prefix     : set prefix path (default: /usr/share/nginx/)
  -c filename   : set configuration file (default: /etc/nginx/nginx.conf)
  -g directives : set global directives out of configuration file


[root@a573002bdf05 /]# ps aux 
USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
root         1  0.0  0.0   7808   688 ?        Ss   23:24   0:00 tail -F /tmp/tmp.txt
root         6  0.0  0.2  15396  2104 pts/0    Ss   23:24   0:00 bash
root        37  0.0  1.2 142892 12820 ?        Ss   23:25   0:00 /usr/bin/python /usr/bin/supervisord -c /etc/supervisord.conf
root        38  0.0  0.7 120900  7448 ?        S    23:25   0:00 nginx: master process /usr/sbin/nginx -c /etc/nginx/nginx.conf -g daemon off;
root        39  0.0  0.4 112924  4324 ?        S    23:25   0:00 /usr/sbin/sshd -D
nginx       40  0.0  0.3 121296  3488 ?        S    23:25   0:00 nginx: worker process
root        49  0.0  0.1  55192  1868 pts/0    R+   23:27   0:00 ps aux

[root@10-222-32-10 ~]# docker logs -f stest
2020-10-31 23:57:15,631 CRIT Supervisor is running as root.  Privileges were not dropped because no user is specified in the config file.  If you intend to run as root, you can set user=root in the config file to avoid this message.
2020-10-31 23:57:15,631 INFO Included extra file "/etc/supervisord.d/nginx.ini" during parsing
2020-10-31 23:57:15,631 INFO Included extra file "/etc/supervisord.d/sshd.ini" during parsing
2020-10-31 23:57:15,652 INFO RPC interface 'supervisor' initialized
2020-10-31 23:57:15,652 CRIT Server 'inet_http_server' running without any HTTP authentication checking
2020-10-31 23:57:15,652 INFO RPC interface 'supervisor' initialized
2020-10-31 23:57:15,652 CRIT Server 'unix_http_server' running without any HTTP authentication checking
2020-10-31 23:57:15,652 INFO supervisord started with pid 1
2020-10-31 23:57:16,658 INFO spawned: 'nginx' with pid 8
2020-10-31 23:57:16,661 INFO spawned: 'sshd' with pid 9
2020-10-31 23:57:17,786 INFO success: nginx entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2020-10-31 23:57:17,787 INFO success: sshd entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)

[root@a9c44d4f249c /]# busybox netstat -lntup
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name    
tcp        0      0 0.0.0.0:9001            0.0.0.0:*               LISTEN      1/python
tcp        0      0 0.0.0.0:80              0.0.0.0:*               LISTEN      8/nginx.conf -g dae
tcp        0      0 0.0.0.0:22              0.0.0.0:*               LISTEN      9/sshd
tcp        0      0 :::80                   :::*                    LISTEN      8/nginx.conf -g dae
tcp        0      0 :::22                   :::*                    LISTEN      9/sshd

参考资料

https://github.com/Supervisor/supervisor
http://supervisord.org/
http://supervisord.org/configuration.html
https://www.cnblogs.com/pingyeaa/p/12782404.html#gen_conf
https://blog.csdn.net/genglei1022/article/details/81239900    supervisor配置 nginx

本文地址：https://blog.csdn.net/xys2015/article/details/109410032