背景:现从事一个容器云产品线的测试工作
场景,假设现在有一个黑客,对产品线进行攻击,在容器云平台上创建了一个容器,容器的启动进程为循环创建zombie进程。
测试点1,容器云产品是否支持僵死进程的stop/ kill
测试点2,容器云产品容器间的隔离性是否足够强,该制造僵死进程的容器是否影响同宿主机上的其他正常用户容器进程的运行。
准备二进制程序
src file:
// zombie_pod.c
// /* create a zombie process*/
#include <sys/types.h>
//#include <signal.h>
#include <unistd.h>
#include <stdlib.h>
#include <stdio.h>
#include <signal.h>
static void sig_child(int signo)
{
pid_t pid;
int stat;
//处理僵尸进程
while ((pid = waitpid(-1, &stat, WNOHANG)) >0)
printf("child %d terminated.\n", pid);
}
int main()
{
pid_t pid;
//signal(SIGCHLD,sig_child);
pid= fork();
if(pid < 0){
printf("create child process error!\n");
}
else{
if(pid==0){
printf("child process %d, father process %d\n",getpid(),getppid());
exit(0);
}else{
printf("current father process, child process:%d, father process:%d\n",getpid(),getppid());
system("ps -o pid,ppid,state,tty,command\n");
while(1){
sleep(1);
}
}
}
return 0;
}
gcc编译成可执行程序 zombie
创建dockerfile
1 FROM ubuntu
2 ADD ./zombie /
3 CMD ["/zombie"]
注意⚠️: 1)ADD 指令的第一个参数是Dockerfile文件的相对路径
build 成image
docker build --tag=cs-kirk/zombie:demo /Users/shaofangma/code/test/zombie_test/
hh:zombie_test shaofangma$ docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
cs-kirk/zombie demo 073718459496 8 seconds ago 130 MB
注意⚠️:build的最后一个参数, 是 Dockerfile所在路径
push到regestry
hh:zombie_test shaofangma$ export QCOSINDEXHOST=https://index-dev.qiniu.io
hh:zombie_test shaofangma$ kirk images push cs-kirk/zombie:demo
启动执行zombie
hh:zombie_test shaofangma$ kirk images
IMAGE DIGEST ACCESS SIZE CREATED
cs-kirk/helloworld-go:demo sha256:774d6d987e855914491ede583ecf55ff34828458ef699.. READ/WRITE 50 MB 10 weeks
cs-kirk/mongo:demo sha256:aff0c497cff4f116583b99b21775a8844a17bcf5c69f7.. READ/WRITE 143 MB 10 weeks
cs-kirk/mongo-express:demo sha256:4c67301f80f256c59770ca9fd721d560fd810e5a9ac1f.. READ/WRITE 96 MB 10 weeks
cs-kirk/nginx:latest sha256:c7c1149150a8f7536bd19b70ea34748bf9dfbc93e5dee.. READ/WRITE 47 MB 7 months
cs-kirk/zombie:demo sha256:73b811e8360ccd7a72adc2b5a0446d5681c7a8324aa61.. READ/WRITE 48 MB About an hour
hh:zombie_test shaofangma$ kirk services run msf/zombie -i cs-kirk/zombie:demo
kirk 0.5.6 is now available (current: 0.5.5). Use 'kirk upgrade' to upgrade to latest version.
hh:zombie_test shaofangma$ kirk ps
CONTAINER SERVICE REV STATUS ERROR
10.128.62.88 msf/java 1 Up About an hour
10.128.62.57 msf/zombie 1 Up Less than a second
kirk 0.5.6 is now available (current: 0.5.5). Use 'kirk upgrade' to upgrade to latest version.
hh:zombie_test shaofangma$ kirk services
SERVICE IMAGE REV UNIT_TYPE N STATEFUL STATE STATUS MODIFIED
msf/java library/java:7 1 1U1G 1 N DEPLOYED RUNNING About an hour ago
msf/zombie cs-kirk/zombie:demo 1 1U1G 1 N DEPLOYED NOT-RUNNING 9 seconds ago
kirk 0.5.6 is now available (current: 0.5.5). Use 'kirk upgrade' to upgrade to latest version.
hh:zombie_test shaofangma$ kirk ps
CONTAINER SERVICE REV STATUS ERROR
10.128.62.88 msf/java 1 Up About an hour
10.128.62.57 msf/zombie 1 EXITED
kirk 0.5.6 is now available (current: 0.5.5). Use 'kirk upgrade' to upgrade to latest version.
service 没有正常启动,短暂up 然后切回EXITED
如何排查问题??
1。更新service
hh:zombie_test shaofangma$ kirk services update msf/zombie --cmd 'sleep 9999'
kirk 0.5.6 is now available (current: 0.5.5). Use 'kirk upgrade' to upgrade to latest version.
hh:zombie_test shaofangma$ kirk ps
CONTAINER SERVICE REV STATUS ERROR
10.128.62.88 msf/java 1 Up About an hour
10.128.62.57 msf/zombie 2 Up 2 seconds
2。然后进入容器,执行启动程序
hh:zombie_test shaofangma$ kirk ssh msf/zombie
root@10-128-62-57:/# ls
bin boot dev etc home lib lib64 media mnt opt proc root run sbin srv sys tmp usr var zombie
root@10-128-62-57:/# ./zombie
bash: ./zombie: cannot execute binary file: Exec format error
原因:发现执行格式问题
反思,二进制程序是在mac上build的 。现在在ubuntu运行 ,当然fail。
重新修改二进制程序
balabala。。。
重新打包images
hh:zombie_test shaofangma$ docker build --tag=cs-kirk/zombie:v1 /Users/shaofangma/code/test/zombie_test/
Sending build context to Docker daemon 20.48 kB
Step 1 : FROM ubuntu
---> 0ef2e08ed3fa
Step 2 : ADD ./zombie /
---> 72bfdbbce835
Removing intermediate container a06e470c78d9
Step 3 : CMD /zombie
---> Running in a7a73f30fa98
---> 70a8068f57e9
Removing intermediate container a7a73f30fa98
Successfully built 70a8068f57e9
hh:zombie_test shaofangma$ docker build --tag=cs-kirk/zombie:demo /Users/shaofangma/code/test/zombie_test/
Sending build context to Docker daemon 20.48 kB
Step 1 : FROM ubuntu
---> 0ef2e08ed3fa
Step 2 : ADD ./zombie /
---> Using cache
---> 72bfdbbce835
Step 3 : CMD /zombie
---> Using cache
---> 70a8068f57e9
Successfully built 70a8068f57e9
hh:zombie_test shaofangma$ docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
cs-kirk/zombie demo 70a8068f57e9 47 seconds ago 130 MB
cs-kirk/zombie v1 70a8068f57e9 47 seconds ago 130 MB
此处 ,我build了两个tag(demo &v1) ,其实只是相当于重命名,实际工程中,应该是build成同名镜像,这样后续升级容器的时候才能直接update。
后续,再push到kirk regestry 或者docker 官方 registry。
升级原来部署的不能运行的容器
hh:zombie_test shaofangma$ kirk services update msf/zombie
kirk 0.5.6 is now available (current: 0.5.5). Use 'kirk upgrade' to upgrade to latest version.
hh:zombie_test shaofangma$ kirk services
SERVICE IMAGE REV UNIT_TYPE N STATEFUL STATE STATUS MODIFIED
msf/java library/java:7 1 1U1G 1 N DEPLOYED RUNNING 2 hours ago
msf/zombie cs-kirk/zombie:demo 3 1U1G 1 N DEPLOYED RUNNING 2 seconds ago
ssh到容器进行检验:发现ready了,一个zombie进程的容器起来了
hh:zombie_test shaofangma$ kirk ssh msf/zombie
root@10-128-62-57:/# ls
bin boot dev etc home lib lib64 media mnt opt proc root run sbin srv sys tmp usr var zombie
root@10-128-62-57:/# ./zombie
current father process, child process:27, father process:16
child process 28, father process 27
PID PPID S TT COMMAND
16 0 S ? /bin/bash
27 16 S ? ./zombie
28 27 Z ? [zombie] <defunct>
29 27 S ? sh -c ps -o pid,ppid,state,tty,command
30 29 R ? ps -o pid,ppid,state,tty,command
奇怪的现象出现了,第二天到公司,发现容器又处于“EXITED“
此时,该查看容器详情 内部运行的command,为“sleep 9999“,
然后执行更新操作:
hh:zombie_test shaofangma$ kirk services update msf/zombie -c '/zombie'
kirk 0.5.6 is now available (current: 0.5.5). Use 'kirk upgrade' to upgrade to latest version.
再次查看容器状态,变成正常”UP”,服务状态变成“running“
进入容器内部,查看是否存在僵死进程,
hh:zombie_test shaofangma$ kirk ssh msf/zombie
root@10-128-62-57:/# ps auwx
USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
root 1 0.0 0.0 1060 4 ? Ss 02:00 0:00 /dev/init /zombie
root 7 0.0 0.0 4364 624 ? S 02:00 0:00 /zombie
root 8 0.0 0.0 0 0 ? Z 02:00 0:00 [zombie] <defunct>
root 39 0.2 0.0 18252 3232 ? Ss 02:15 0:00 /bin/bash
root 49 0.0 0.0 34428 2836 ? R+ 02:15 0:00 ps auwx
图中state 为“Z“ 即为僵尸进程。