k8s node NotReady-报错 error updating CSINode annotation: timed out waiting for the condition解决

k8s node NotReady 报错解决

环境信息

• 系统版本：Ubuntu 18.04
• k8s版本：1.14.0
• 集群仅一个节点，master参与调度

排除与解决

查看node状态

k8s@master:~/shiyu$ kubectl get node
NAME     STATUS      ROLES    AGE   VERSION
master   NotReady    master   20d   v1.18.2

查看NotReady node详细信息

k8s@master:~$ kubectl describe node master
...
error updating CSINode annotation: timed out waiting for the condition; caused by: the server could not find the requested resource
...

查看版本信息kubectl version

k8s@master:~/shiyu$ kubectl version
Client Version: version.Info{Major:"1", Minor:"18", GitVersion:"v1.18.2", GitCommit:"52c56ce7a8272c798dbc29846288d7cd9fbae032", GitTreeState:"clean", BuildDate:"2020-04-16T11:56:40Z", GoVersion:"go1.13.9", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"14", GitVersion:"v1.14.0", GitCommit:"641856db18352033a0d96dbc99153fa3b27298e5", GitTreeState:"clean", BuildDate:"2019-03-25T15:45:25Z", GoVersion:"go1.12.1", Compiler:"gc", Platform:"linux/amd64"}

node NotReady原因：

系统自动将kubeadm kubelet kubectl更新到最新版本(当前为1.18.2)。新版本中默认启动了一个新特性，导致master节点处于NotReady状态

解决办法有以下两个：

• 降级kubelet，重启kubelet
  • sudo apt-get install kubelet=1.14.0-00
  • sudo apt-mark hold kubelet
  • sudo apt-mark hold kubelet kubeadm kubectl
• 禁用CSIMigration属性

  • 在/var/lib/kubelet/config.yaml配置文件中添加以下配置

    featureGates:
      CSIMigration: false
    

  • systemctl daemon-reload

  • systemctl restart kubelet

参考https://stackoverflow.com/questions/59291108/worker-start-to-fail-csinodeifo-error-updating-csinode-annotation
参考github上的issue链接https://github.com/kubernetes/kubernetes/issues/86094