k8s报错 loadFlannelSubnetEnv failed: open /run/flannel/subnet.env: no such file or directory

k8spod一直处于ContainerCreating或者Error状态

用kubectl describe查看pod详细

报错如下：

Events:
  Type     Reason                  Age                     From     Message
  ----     ------                  ----                    ----     -------
  Normal   SandboxChanged          16m (x5451 over 116m)   kubelet  Pod sandbox changed, it will be killed and re-created.
  Warning  FailedCreatePodSandBox  101s (x6255 over 116m)  kubelet  (combined from similar events): Failed to create pod sandbox: rpc error: code = Unknown desc = failed to set up sandbox container "1cc1bdb7462d5fd60cf843195fa4bc5377f60b8ce00f2010ff859ff751350222" network for pod "nginx-deployment-6cb95469b7-tbchv": networkPlugin cni failed to set up pod "nginx-deployment-6cb95469b7-tbchv_default" network: loadFlannelSubnetEnv failed: open /run/flannel/subnet.env: no such file or directory

查看flannel的pod是否正常

用kubectl describe查看pod详细

[root@master ~]# kubectl logs kube-flannel-ds-5t2dg -n kube-system -c kube-flannel
I0410 13:16:50.721034       1 main.go:520] Determining IP address of default interface
I0410 13:16:50.721750       1 main.go:533] Using interface with name ens33 and address 192.168.39.128
I0410 13:16:50.721792       1 main.go:550] Defaulting external address to interface address (192.168.39.128)
W0410 13:16:50.721819       1 client_config.go:608] Neither --kubeconfig nor --master was specified.  Using the inClusterConfig.  This might not work.
E0410 13:16:50.726407       1 main.go:251] Failed to create SubnetManager: error retrieving pod spec for 'kube-system/kube-flannel-ds-5t2dg': Get "https://10.1.0.1:443/api/v1/namespaces/kube-system/pods/kube-flannel-ds-5t2dg": x509: certificate is valid for 192.1.0.1, 192.168.39.128, not 10.1.0.1

是因为初始化集群写错地址把192写成10，需要kubeadm reset重新加入集群就可以了

kubeadm init  --apiserver-advertise-address=192.168.39.128  --image-repository registry.aliyuncs.com/google_containers  --kubernetes-version v1.19.3  --service-cidr=192.1.0.0/16  --pod-network-cidr=192.2.0.0/16  --service-dns-domain=cluster.local  --ignore-preflight-errors=Swap  --ignore-preflight-errors=NumCPU

重新加入集群之后发现coredns虽然处于running状态但是一直是0/1

报错还是这个

error: code = Unknown desc = failed to set up sandbox container "67a84ccf6c53a117d1d0353cb4ed1a43d484572004bef6a5b417d75bad78c180" network for pod "coredns-6d56c8448f-jfxtx": networkPlugin cni failed to set up pod "coredns-6d56c8448f-jfxtx_kube-system" network: loadFlannelSubnetEnv failed: open /run/flannel/subnet.env: no such file or directory

但是flannel是正常的

是因为flannel配置文件没有改（81行这个网络的ip一定要跟k8s初始化集群的一样）

修改完删除flannel的pod与资源重新部署

kubectl delete daemonset kube-flannel-ds -n kube-system  
kubectl delete configmap kube-flannel-cfg -n kube-system
kubectl apply -f flannel.yml

还有一种情况是也是报错 open /run/flannel/subnet.env: no such file or directory

一直处于ContainerCreating 

ls   /run/flannel/subnet.env 发现没有这个文件

所有节点新建文件 vim  /run/flannel/subnet.env

FLANNEL_NETWORK=10.2.0.0/16
FLANNEL_SUBNET=10.1.0.1/24
FLANNEL_MTU=1450
FLANNEL_IPMASQ=true

最后 kubectl get pods  -n kube-system发现全部正常了