【容器】Pod 生命周期

概述

Pod的生命周期包含从Pod创建事件的触发到Pod被停止的整个流程。了解Pod的生命周期方便日常排障，并能帮助较深入了解K8s。

在Pod生命周期中有两个重要的标识：Pod Condition 和 Pod Phase。Pod Phase提供了一个Pod当前状况的概览，可以帮助我们快速定位Pod当前在生命周期的哪个阶段，但是Pod Phase无法提供关于Pod状态更详细的信息。在Debug过程中，我们需要更多的准确反应Pod状态的指标，此时Pod Condition可以提供帮助。Pod Condition反应Pod是否已经到达了一个特定的阶段，并且为什么。不同于Pod Phase，Pod Condition有多个同时存在的标识，这些标识反映了Pod不同纬度的状态。

Pod Phase

Pod Phase 包含 Pending, Running, Succeeded, Failed, Unknow 5个阶段，我们重点关注从启动到运行的两个阶段: Pending & Running。

Pending指一个Pod已经被K8s确认接受，容器镜像被拉取到节点机成功，到所有Pod内容器成功启动之前的阶段。每个新启动的Pod一定会经历这个阶段，Pod处于Pending阶段是很正常的现象。但是一个或多个Pod长时间处于Pending状态，无法转换到Running状态表明系统是有问题的。几个可能导致Pod长时间处于Pending状态的原因：
1）Pod scheduler 工作异常。
2）集群内可分配资源不足。
3）镜像拉取异常。
上述三个原因可能有多个子原因，产生问题时需俱体分析。

Running指一个Pod中所有的Container都已启动，在此之后会一直维持Running。Running后Pod会被kubelet开始进行就绪探测（readiness probe）。

Pod Conditions

Pod Conditions 包含PodScheduled， Ready，Initialized，ContainersReady，DisruptionTarget，我们重点关注PodScheduled, Ready, ContainersReady这三个状态。与Pod Phase不同的是，多个Pod Conditions会同时存在，通过True & False 标识该Pod是否满足了某个Condition。
PodScheduled表示该Pod已经被成功调度，被分配了可用节点。PodScheduled为False的原因和调度相关，包含：
1）Pod scheduler 工作异常
2）集群内可分配资源不足

ContainersReady表示容器中的所有containers都已经通过了readiness probe的探测，准备好开始服务。ContainersReady为False的原因主要包含：
1）Container中1号进程启动失败
2）Container的Readiness Probe探测失败

Ready表示除ContainersReady外，Pod对于存储，网络的需求已经满足，Init Container的工作也执行完毕。当Pod处于Ready的状态时，表明该Pod已经完全准备好接收请求提供服务了。此时Pod的IP会被加入到对应Endpoint资源的Pod对象列表中，endpoint的信息会被同步到kube-proxy，此时Pod就真正要接收请求了。
值得注意的是，ContainersReady和Ready这两个状态通常会同时达到。Ready为False的原因多出了：
1）存储，网络需求未满足
2）init container未完成工作

实验

Pending + PodScheduled:False

由于没有合适的宿主节点导致Pod无法被调度

apiVersion: apps/v1
kind: Deployment
metadata:
  name: redis-deployment
  labels:
    app: redis
spec:
  replicas: 1
  selector:
    matchLabels:
      app: redis
  template:
    metadata:
      labels:
        app: redis
    spec:
      containers:
      - name: redis
        image: redis:latest
        ports:
        - containerPort: 6379
        livenessProbe:
          tcpSocket:
            port: 8080
          initialDelaySeconds: 15
          periodSeconds: 10
      nodeSelector:
        disktype: ssd

Pending + PodScheduled:True&ContainersReady:False

由于镜像服务不可用导致Pod虽然被调度但是无法下载镜像

apiVersion: apps/v1
kind: Deployment
metadata:
  name: redis-deployment
  labels:
    app: redis
spec:
  replicas: 1
  selector:
    matchLabels:
      app: redis
  template:
    metadata:
      labels:
        app: redis
    spec:
      containers:
      - name: redis
        image: myprivateregistry.com/redis:latest
        ports:
        - containerPort: 6379
        livenessProbe:
          tcpSocket:
            port: 8080
          initialDelaySeconds: 15
          periodSeconds: 10

Running + PodScheduled:True&ContainersReady:False

Pod内容器启动，由于错误配置就绪探针导致ContainersReady一直错误。注意此处我们故意配置了错误的就绪探针，在真实场景中

apiVersion: apps/v1
kind: Deployment
metadata:
  name: redis-deployment
  labels:
    app: redis
spec:
  replicas: 1
  selector:
    matchLabels:
      app: redis
  template:
    metadata:
      labels:
        app: redis
    spec:
      containers:
      - name: redis
        image: redis:latest
        ports:
        - containerPort: 6379
        readinessProbe:
          tcpSocket:
            port: 8080
          initialDelaySeconds: 150
          periodSeconds: 10

Running + PodScheduled:True&ContainersReady:True&Ready:True

以下是一个能够正常启动并就绪的Pod yaml描述文件

apiVersion: apps/v1
kind: Deployment
metadata:
  name: redis-deployment
  labels:
    app: redis
spec:
  replicas: 1
  selector:
    matchLabels:
      app: redis
  template:
    metadata:
      labels:
        app: redis
    spec:
      containers:
      - name: redis
        image: redis:latest
        ports:
        - containerPort: 6379
        readinessProbe:
          tcpSocket:
            port: 6379
          initialDelaySeconds: 10
          periodSeconds: 10

总结

在 Kubernetes 中，理解 Pod Phase 和 Pod Conditions 对于有效管理和监控 Pod 的生命周期至关重要。Pod Phase 描述了 Pod 的主要生命周期阶段，而 Pod Conditions 提供了更具体的健康状况信息，帮助我们诊断和解决问题。这些状态信息不仅对于日常的故障排查非常有用，例如帮助我们快速定位为什么大量 Pod 处于 Pending 状态，而且对于开发相关的 Kubernetes 操作也非常关键，如自定义的负载均衡器需要根据 Pod 的就绪状态来更新其配置。此外，理解包括那些较不常见的如 Evicted 在内的状态，可以让我们更全面地掌握和应对各种集群状况。这种知识的深入不仅能提升我们对系统的掌控，还能优化我们的应用性能和可靠性。

Appendix

Pod Phases 在K8s代码中的定义

const (
	// PodPending means the pod has been accepted by the system, but one or more of the containers
	// has not been started. This includes time before being bound to a node, as well as time spent
	// pulling images onto the host.
	PodPending PodPhase = "Pending"
	// PodRunning means the pod has been bound to a node and all of the containers have been started.
	// At least one container is still running or is in the process of being restarted.
	PodRunning PodPhase = "Running"
	// PodSucceeded means that all containers in the pod have voluntarily terminated
	// with a container exit code of 0, and the system is not going to restart any of these containers.
	PodSucceeded PodPhase = "Succeeded"
	// PodFailed means that all containers in the pod have terminated, and at least one container has
	// terminated in a failure (exited with a non-zero exit code or was stopped by the system).
	PodFailed PodPhase = "Failed"
	// PodUnknown means that for some reason the state of the pod could not be obtained, typically due
	// to an error in communicating with the host of the pod.
	// Deprecated in v1.21: It isn't being set since 2015 (74da3b14b0c0f658b3bb8d2def5094686d0e9095)
	PodUnknown PodPhase = "Unknown"
)

Pod Conditions 在K8s代码中的定义

// These are valid conditions of pod.
const (
	// PodScheduled represents status of the scheduling process for this pod.
	PodScheduled PodConditionType = "PodScheduled"
	// PodReady means the pod is able to service requests and should be added to the
	// load balancing pools of all matching services.
	PodReady PodConditionType = "Ready"
	// PodInitialized means that all init containers in the pod have started successfully.
	PodInitialized PodConditionType = "Initialized"
	// ContainersReady indicates whether all containers in the pod are ready.
	ContainersReady PodConditionType = "ContainersReady"
	// DisruptionTarget indicates the pod is about to be terminated due to a
	// disruption (such as preemption, eviction API or garbage-collection).
	DisruptionTarget PodConditionType = "DisruptionTarget"
)