K8s-临时容器 Ephemeral Containers

临时容器 Ephemeral Containers

1. 当由于容器崩溃或容器镜像不包含调试工具而导致 kubectl exec 无用时， 临时容器对于交互式故障排查很有用。
2. 尤其是，Distroless 镜像 允许用户部署最小的容器镜像，从而减少攻击面并减少故障和漏洞的暴露。 由于 distroless
   镜像不包含 Shell 或任何的调试工具，因此很难单独使用 kubectl exec 命令进行故障排查。
3. 使用临时容器时，启用 进程名字空间共享 很有帮助，可以查看其他容器中的进程。

开启临时容器功能

开启特性

1. master 节点配置 APIServer 组件
[root@vms120 ~]# cat /etc/kubernetes/manifests/kube-apiserver.yaml
- --feature-gates=EphemeralContainers=true
...

2. master 节点配置 controller-manager
[root@vms120 ~]# vim /etc/kubernetes/manifests/kube-controller-manager.yaml
spec:
  containers:
  - command:
    - --feature-gates=EphemeralContainers=true            # 增加
...

3. master 节点配置 kube-scheduler
[root@vms120 ~]# vim /etc/kubernetes/manifests/kube-scheduler.yaml
spec:
  containers:
  - command:
    - --feature-gates=EphemeralContainers=true            # 增加
# 重启服务 
[root@vms120 ~]# systemctl restart kubelet.service

4. 所有 node 节点配置 kubelet 参数
添加 --feature-gates=EphemeralContainers=true
[root@vms121 kubernetes]# cat /var/lib/kubelet/kubeadm-flags.env
KUBELET_KUBEADM_ARGS="--network-plugin=cni --pod-infra-container-image=registry.aliyuncs.com/google_containers/pause:3.6 --feature-gates=EphemeralContainers=true"
# 重启 node kubelet 服务
[root@vms121 kubernetes]# systemctl daemon-reload
[root@vms121 kubernetes]# systemctl restart kubelet

测试

1. 创建 pod
[root@vms120 ~]# kubectl run ephemeral-demo --image=registry.aliyuncs.com/google_containers/pause:3.2 --restart=Never
pod/ephemeral-demo created

[root@vms120 ~]# kubectl  exec -it ephemeral-demo  -- sh
OCI runtime exec failed: exec failed: unable to start container process: exec: "sh": executable file not found in $PATH: unknown
command terminated with exit code 126
# 无法 kubectl exec

解决无法 exec ，我们创建一个临时容器添加到这个pod里

加上-i参数将直接进入添加的临时容器的控制台界面，因为是使用kubectl run 创建的pod ,所以需要-target 参数指定另一个容器的进程命名空间。因为 kubectl run 不能在它创建的pod中启用 共享进程命名空间

[root@vms120 ~]# kubectl debug -it ephemeral-demo --image=busybox --target=ephemeral-demo
Targeting container "ephemeral-demo". If you don't see processes from this container it may be because the container runtime doesn't support this feature.
Defaulting debug container name to debugger-bljnj.
If you don't see a command prompt, try pressing enter.
/ # ls
bin   dev   etc   home  proc  root  sys   tmp   usr   var
/ # 

此时再去看pod 的信息会发现已经被添加了一个类型为ephemeralContainers的容器

[root@vms120 ~]# kubectl  get pod ephemeral-demo   -o json|jq .spec
{
  "containers": [
    {
      "image": "registry.aliyuncs.com/google_containers/pause:3.2",
      "imagePullPolicy": "IfNotPresent",
      "name": "ephemeral-demo",
      "resources": {},
      "terminationMessagePath": "/dev/termination-log",
      "terminationMessagePolicy": "File",
      "volumeMounts": [
        {
          "mountPath": "/var/run/secrets/kubernetes.io/serviceaccount",
          "name": "kube-api-access-sqmzl",
          "readOnly": true
        }
      ]
    }
  ],
  "dnsPolicy": "ClusterFirst",
  "enableServiceLinks": true,
  "ephemeralContainers": [
    {
      "image": "busbox",
      "imagePullPolicy": "Always",
      "name": "debugger-9l8mw",
      "resources": {},
      "stdin": true,
      "targetContainerName": "ephemeral-demo",
      "terminationMessagePath": "/dev/termination-log",
      "terminationMessagePolicy": "File",
      "tty": true
    },
    {
      "image": "busybox",
      "imagePullPolicy": "Always",
      "name": "debugger-slx6g",
      "resources": {},
      "stdin": true,
      "targetContainerName": "ephemeral-demo",
      "terminationMessagePath": "/dev/termination-log",
      "terminationMessagePolicy": "File",
      "tty": true
    },
    {
      "image": "busybox",
      "imagePullPolicy": "Always",
      "name": "debugger-gw6zt",
      "resources": {},
      "stdin": true,
      "terminationMessagePath": "/dev/termination-log",
      "terminationMessagePolicy": "File"
    },
    {
      "image": "busybox",
      "imagePullPolicy": "Always",
      "name": "debugger-cxc8b",
      "resources": {},
      "stdin": true,
      "targetContainerName": "ephemeral-demo",
      "terminationMessagePath": "/dev/termination-log",
      "terminationMessagePolicy": "File",
      "tty": true
    },
    {
      "image": "busybox",
      "imagePullPolicy": "Always",
      "name": "debugger-bljnj",
      "resources": {},
      "stdin": true,
      "targetContainerName": "ephemeral-demo",
      "terminationMessagePath": "/dev/termination-log",
      "terminationMessagePolicy": "File",
      "tty": true
    }
  ],
  "nodeName": "vms121.rhce.cc",
  "preemptionPolicy": "PreemptLowerPriority",
  "priority": 0,
  "restartPolicy": "Never",
  "schedulerName": "default-scheduler",
  "securityContext": {},
  "serviceAccount": "default",
  "serviceAccountName": "default",
  "terminationGracePeriodSeconds": 30,
  "tolerations": [
    {
      "effect": "NoExecute",
      "key": "node.kubernetes.io/not-ready",
      "operator": "Exists",
      "tolerationSeconds": 300
    },
    {
      "effect": "NoExecute",
      "key": "node.kubernetes.io/unreachable",
      "operator": "Exists",
      "tolerationSeconds": 300
    }
  ],
  "volumes": [
    {
      "name": "kube-api-access-sqmzl",
      "projected": {
        "defaultMode": 420,
        "sources": [
          {
            "serviceAccountToken": {
              "expirationSeconds": 3607,
              "path": "token"
            }
          },
          {
            "configMap": {
              "items": [
                {
                  "key": "ca.crt",
                  "path": "ca.crt"
                }
              ],
              "name": "kube-root-ca.crt"
            }
          },
          {
            "downwardAPI": {
              "items": [
                {
                  "fieldRef": {
                    "apiVersion": "v1",
                    "fieldPath": "metadata.namespace"
                  },
                  "path": "namespace"
                }
              ]
            }
          }
        ]
      }
    }
  ]
}

有些时候 Pod 的配置参数使得在某些情况下很难执行故障排查。例如，在容器镜像中不包含 shell 或者你的应用程序在启动时崩溃的情况下， 就不能通过运行 kubectl exec 来排查容器故障。在这些情况下，你可以使用 kubectl debug 来创建 Pod 的副本，通过更改配置帮助调试。

报错

error: ephemeral containers are disabled for this cluster (error from server: "the server could not find the requested resource").

未成功开启ephemeralContainers特性