问题展示
错误信息
部署成功创建,但在使用 kubectl get pod 查看pod的运行状态时出现异常。
root@k8s-mst:~/web_sample# kubectl apply -f nginx-deployment.yaml
deployment “nginx-deployment” created
root@k8s-mst:~/web_sample# kubectl get pod
NAME READY STATUS RESTARTS AGE
nginx-deployment-4087004473-4m0br 0/1 RunContainerError 0 7s
nginx-deployment-4087004473-80rqd 0/1 RunContainerError 0 7s
nginx-deployment-4087004473-qfdmw 0/1 RunContainerError 0 7s
使用kubectl describe pod 查看详细的错误信息,如下:
Events:
FirstSeen LastSeen Count From SubObjectPath Type Reason Message
34s 34s 1 {default-scheduler } Normal Scheduled Successfully assigned nginx-deployment-4087004473-qfdmw to k8s-nod2
31s 31s 1 {kubelet k8s-nod2} spec.containers{nginx} Normal Created Created container with docker id 7fc4f63e0bd6; Security:[seccomp=unconfined]
30s 30s 1 {kubelet k8s-nod2} spec.containers{nginx} Warning Failed Failed to start container with docker id 7fc4f63e0bd6 with error: Error response from daemon: {“message”:“can’t join IPC of container 2dfd13510b3f816ed0437ea70a8a57daeaa7a38eec0035972044b25dd102f1cd: non-shareable IPC (hint: use IpcMode:shareable for the donor container)”}
28s 28s 1 {kubelet k8s-nod2} spec.containers{nginx} Normal Created Created container with docker id 3c61a9aaa6a4; Security:[seccomp=unconfined]
28s 28s 1 {kubelet k8s-nod2} spec.containers{nginx} Warning Failed Failed to start container with docker id 3c61a9aaa6a4 with error: Error response from daemon: {“message”:“can’t join IPC of container 2dfd13510b3f816ed0437ea70a8a57daeaa7a38eec0035972044b25dd102f1cd: non-shareable IPC (hint: use IpcMode:shareable for the donor container)”}
18s 18s 1 {kubelet k8s-nod2} spec.containers{nginx} Normal Created Created container with docker id 42970a39a9b4; Security:[seccomp=unconfined]
17s 17s 1 {kubelet k8s-nod2} spec.containers{nginx} Warning Failed Failed to start container with docker id 42970a39a9b4 with error: Error response from daemon: {“message”:“can’t join IPC of container 2dfd13510b3f816ed0437ea70a8a57daeaa7a38eec0035972044b25dd102f1cd: non-shareable IPC (hint: use IpcMode:shareable for the donor container)”}
32s 3s 4 {kubelet k8s-nod2} spec.containers{nginx} Normal Pulled Container image “nginx:1.7.9” already present on machine
30s 2s 4 {kubelet k8s-nod2} Warning FailedSync Error syncing pod, skipping: failed to “StartContainer” for “nginx” with RunContainerError: “runContainer: Error response from daemon: {“message”:“can’t join IPC of container 2dfd13510b3f816ed0437ea70a8a57daeaa7a38eec0035972044b25dd102f1cd: non-shareable IPC (hint: use IpcMode:shareable for the donor container)”}”
2s 2s 1 {kubelet k8s-nod2} spec.containers{nginx} Normal Created Created container with docker id 20ca9e358257; Security:[seccomp=unconfined]
2s 2s 1 {kubelet k8s-nod2} spec.containers{nginx} Warning Failed Failed to start container with docker id 20ca9e358257 with error: Error response from daemon: {“message”:“can’t join IPC of container 2dfd13510b3f816ed0437ea70a8a57daeaa7a38eec0035972044b25dd102f1cd: non-shareable IPC (hint: use IpcMode:shareable for the donor container)”}
根据日志信息可知,错误是由IPC mode导致的,并且也给出了解决方案:使用shareable模式
“message”:“can’t join IPC of container 2dfd13510b3f816ed0437ea70a8a57daeaa7a38eec0035972044b25dd102f1cd: non-shareable IPC (hint: use IpcMode:shareable for the donor container)”
解决方案
首先,通过搜索docker ipc,在docker run reference中找到了一些蛛丝马迹,初步确认是由于ipc的模式被默认设置为private导致的
通过在docker docs中搜索docker default ipc,在docker 19.03 release note中提到了原因,看来都是版本惹的祸(
daemon: Now use ‘private’ ipc mode by default. moby/moby#35621
通过链接来到github的pr提交记录上,找到了解决方法
Old (bad, but backward-compatible) behavior (i.e. “shareable” containers by default) can be enabled by either using --default-ipc-mode shareable daemon command line option, or by adding a “default-ipc-mode”: shareable" line in docker.json configuration file.
根据提示,修改/etc/docker/daemon.json(没有的话新建一个),添加如下内容
{
“default-ipc-mode”: “shareable”
}
重启docker
systemctl restart docker
完结撒花
重新部署deployment
root@k8s-mst:~/web_sample# kubectl apply -f nginx-deployment.yaml
deployment “nginx-deployment” created
root@k8s-mst:~/web_sample# kubectl get pods
NAME READY STATUS RESTARTS AGE
nginx-deployment-4087004473-9wjbr 1/1 Running 0 29s
nginx-deployment-4087004473-g9qpr 1/1 Running 0 29s
nginx-deployment-4087004473-z6jkm 1/1 Running 0 29s