详细描述(摘自stackoverflow,类似的情况)
I updated a StatefulSet with a non-existent Docker image. As expected, a pod of the statefulset is destroyed and can’t be recreated(ErrImagePull). However, when I change back the StatefulSet with an existing image, the StatefulSet doesn’t try to remove the broken pod to replace it by a good one. It keeps trying to pull the non-existing image.
You have to delete the broken pod manually to unblock the situation.
解决方法
手动删除pod,之后statefulset会建立新的pod
问题
官方的人说这个场景不支持,想知道为啥不支持?可能是我只知道这个现象,不知道它具体的应用场景所以现在还不理解。
deployment好像没有这个问题?难道现实场景中deployment比statefulset用得多的多?
https://github.com/kubernetes/kubernetes/issues/67250#issuecomment-414754525
参考
-
https://github.com/kubernetes/kubernetes/issues/67250
-
https://kubernetes.io/docs/concepts/workloads/controllers/statefulset/#forced-rollback
Forced rollback
When using Rolling Updates with the default Pod Management Policy (OrderedReady), it’s possible to get into a broken state that requires manual intervention to repair.
If you update the Pod template to a configuration that never becomes Running and Ready (for example, due to a bad binary or application-level configuration error), StatefulSet will stop the rollout and wait.
In this state, it’s not enough to revert the Pod template to a good configuration. Due to a known issue, StatefulSet will continue to wait for the broken Pod to become Ready (which never happens) before it will attempt to revert it back to the working configuration.
After reverting the template, you must also delete any Pods that StatefulSet had already attempted to run with the bad configuration. StatefulSet will then begin to recreate the Pods using the reverted template.