kubeadm init 后master一直处于notready状态

kubeadm安装Kubernetes，集群状态检测时，master一直处于notready状态

[root@master ~]# kubectl get nodes
NAME     STATUS     ROLES    AGE    VERSION
master   NotReady   master   32m    v1.16.0
node1    NotReady   <none>   8m2s   v1.16.0

找问题，先查看pods状态

[root@master ~]# kubectl get pods --all-namespaces
NAMESPACE     NAME                             READY   STATUS    RESTARTS   AGE
kube-system   coredns-58cc8c89f4-pkx82         0/1     Pending   0          34m
kube-system   coredns-58cc8c89f4-sddpq         0/1     Pending   0          34m
kube-system   etcd-master                      1/1     Running   0          34m
kube-system   kube-apiserver-master            1/1     Running   0          34m
kube-system   kube-controller-manager-master   1/1     Running   0          33m
kube-system   kube-proxy-4fj8z                 1/1     Running   0          10m
kube-system   kube-proxy-v54nh                 1/1     Running   0          34m
kube-system   kube-scheduler-master            1/1     Running   0          34m

发现coredns一直处于pending状态，再进一步看kuberctl.services日志

[root@master ~]# journalctl -f -u kubelet.service
-- Logs begin at Mon 2019-09-23 22:55:58 CST. --
Sep 24 01:33:44 master kubelet[6213]: E0924 01:33:44.107180    6213 kubelet.go:2187] Container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:docker: network plugin is not ready: cni config uninitialized
Sep 24 01:33:46 master kubelet[6213]: W0924 01:33:46.416805    6213 cni.go:237] Unable to update cni config: no networks found in /etc/cni/net.d

看到是网络的问题，应该是fu

[root@master ~]# kubectl apply -f https://raw.githubusercontent.com/coreos/flannel/a70459be0084506e4ec919aa1c114638878db11b/Documentation/kube-flannel.yml
clusterrole.rbac.authorization.k8s.io/flannel unchanged
clusterrolebinding.rbac.authorization.k8s.io/flannel unchanged
serviceaccount/flannel unchanged
configmap/kube-flannel-cfg unchanged
unable to recognize "https://raw.githubusercontent.com/coreos/flannel/a70459be0084506e4ec919aa1c114638878db11b/Documentation/kube-flannel.yml": no matches for kind "DaemonSet" in version "extensions/v1beta1"
unable to recognize "https://raw.githubusercontent.com/coreos/flannel/a70459be0084506e4ec919aa1c114638878db11b/Documentation/kube-flannel.yml": no matches for kind "DaemonSet" in version "extensions/v1beta1"
unable to recognize "https://raw.githubusercontent.com/coreos/flannel/a70459be0084506e4ec919aa1c114638878db11b/Documentation/kube-flannel.yml": no matches for kind "DaemonSet" in version "extensions/v1beta1"
unable to recognize "https://raw.githubusercontent.com/coreos/flannel/a70459be0084506e4ec919aa1c114638878db11b/Documentation/kube-flannel.yml": no matches for kind "DaemonSet" in version "extensions/v1beta1"
unable to recognize "https://raw.githubusercontent.com/coreos/flannel/a70459be0084506e4ec919aa1c114638878db11b/Documentation/kube-flannel.yml": no matches for kind "DaemonSet" in version "extensions/v1beta1"

解决办法，换个链接

[root@master ~]# kubectl apply -f https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml
podsecuritypolicy.policy/psp.flannel.unprivileged created
clusterrole.rbac.authorization.k8s.io/flannel configured
clusterrolebinding.rbac.authorization.k8s.io/flannel unchanged
serviceaccount/flannel unchanged
configmap/kube-flannel-cfg unchanged
daemonset.apps/kube-flannel-ds-amd64 created
daemonset.apps/kube-flannel-ds-arm64 created
daemonset.apps/kube-flannel-ds-arm created
daemonset.apps/kube-flannel-ds-ppc64le created
daemonset.apps/kube-flannel-ds-s390x created

查看node状态，发现还是notready。查看日志

journalctl -f -u kubelet.service 
-- Logs begin at Mon 2019-09-23 22:55:58 CST. --
Sep 24 14:36:17 master kubelet[6213]: E0924 14:36:17.023967    6213 kubelet.go:2187] Container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:docker: network plugin is not ready: cni config uninitialized
Sep 24 14:36:19 master kubelet[6213]: W0924 14:36:19.214442    6213 cni.go:202] Error validating CNI config &{cbr0  false [0xc0006a22a0 0xc0006a2360] [123 10 32 32 34 110 97 109 101 34 58 32 34 99 98 114 48 34 44 10 32 32 34 112 108 117 103 105 110 115 34 58 32 91 10 32 32 32 32 123 10 32 32 32 32 32 32 34 116 121 112 101 34 58 32 34 102 108 97 110 110 101 108 34 44 10 32 32 32 32 32 32 34 100 101 108 101 103 97 116 101 34 58 32 123 10 32 32 32 32 32 32 32 32 34 104 97 105 114 112 105 110 77 111 100 101 34 58 32 116 114 117 101 44 10 32 32 32 32 32 32 32 32 34 105 115 68 101 102 97 117 108 116 71 97 116 101 119 97 121 34 58 32 116 114 117 101 10 32 32 32 32 32 32 125 10 32 32 32 32 125 44 10 32 32 32 32 123 10 32 32 32 32 32 32 34 116 121 112 101 34 58 32 34 112 111 114 116 109 97 112 34 44 10 32 32 32 32 32 32 34 99 97 112 97 98 105 108 105 116 105 101 115 34 58 32 123 10 32 32 32 32 32 32 32 32 34 112 111 114 116 77 97 112 112 105 110 103 115 34 58 32 116 114 117 101 10 32 32 32 32 32 32 125 10 32 32 32 32 125 10 32 32 93 10 125 10]}: [plugin flannel does not support config version ""]
Sep 24 14:36:19 master kubelet[6213]: W0924 14:36:19.214617    6213 cni.go:237] Unable to update cni config: no valid networks found in /etc/cni/net.d

发现报错plugin flannel does not support config version ，修改配置文件

vim /etc/cni/net.d/10-flannel.conflist
//加上cni的版本号
//文件内容如下
{
  "name": "cbr0",
  "cniVersion": "0.2.0",
  "plugins": [
    {
      "type": "flannel",
      "delegate": {
        "hairpinMode": true,
        "isDefaultGateway": true
      }
    },
    {
      "type": "portmap",
      "capabilities": {
        "portMappings": true
      }
    }
  ]
}

修改后，运行

systemctl daemon-reload 

再查看集群状态，发现master正常了，处于ready状态；但是node
1节点还是处于notready状态

[root@master ~]# kubectl get nodes
NAME     STATUS     ROLES    AGE   VERSION
master   Ready      master   14h   v1.16.0
node1    NotReady   <none>   13h   v1.16.0

解决办法：去到node1，查看kubectl日志，发现还是报错no valid networks found in /etc/cni/net.d
将node1中也加上cni的版本号，重新启动，即可看到集群状态变为正常

[root@master ~]# kubectl get nodes
NAME     STATUS   ROLES    AGE   VERSION
master   Ready    master   14h   v1.16.0
node1    Ready    <none>   14h   v1.16.0

参考资料：
kubeadm安装Kubernetes 1.14最佳https://www.kubernetes.org.cn/5462.html
kubernetes安装过程中错误（kube-dns 状态一直是Pending，master节点是NotReady）
https://blog.csdn.net/u013355826/article/details/82786649
How to fix Flannel CNI plugin. Error: [plugin flannel does not support config version “”]
https://stackoverflow.com/questions/58037620/how-to-fix-flannel-cni-plugin-error-plugin-flannel-does-not-support-config-ve
kubernetes nodes notready 解决思路
https://blog.csdn.net/qq_21816375/article/details/80222689