k8s node NotReady 报错解决
环境信息
- 系统版本:Ubuntu 18.04
- k8s版本:1.14.0
- 集群仅一个节点,master参与调度
排除与解决
查看node状态
k8s@master:~/shiyu$ kubectl get node
NAME STATUS ROLES AGE VERSION
master NotReady master 20d v1.18.2
查看NotReady node详细信息
k8s@master:~$ kubectl describe node master
...
error updating CSINode annotation: timed out waiting for the condition; caused by: the server could not find the requested resource
...
查看版本信息kubectl version
k8s@master:~/shiyu$ kubectl version
Client Version: version.Info{Major:"1", Minor:"18", GitVersion:"v1.18.2", GitCommit:"52c56ce7a8272c798dbc29846288d7cd9fbae032", GitTreeState:"clean", BuildDate:"2020-04-16T11:56:40Z", GoVersion:"go1.13.9", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"14", GitVersion:"v1.14.0", GitCommit:"641856db18352033a0d96dbc99153fa3b27298e5", GitTreeState:"clean", BuildDate:"2019-03-25T15:45:25Z", GoVersion:"go1.12.1", Compiler:"gc", Platform:"linux/amd64"}
node NotReady原因:
系统自动将kubeadm kubelet kubectl更新到最新版本(当前为1.18.2)。新版本中默认启动了一个新特性,导致master节点处于NotReady状态
解决办法有以下两个:
- 降级kubelet,重启kubelet
- sudo apt-get install kubelet=1.14.0-00
- sudo apt-mark hold kubelet
- sudo apt-mark hold kubelet kubeadm kubectl
- 禁用CSIMigration属性
-
在/var/lib/kubelet/config.yaml配置文件中添加以下配置
featureGates: CSIMigration: false
-
systemctl daemon-reload
-
systemctl restart kubelet
-
参考https://stackoverflow.com/questions/59291108/worker-start-to-fail-csinodeifo-error-updating-csinode-annotation
参考github上的issue链接https://github.com/kubernetes/kubernetes/issues/86094