k8s network: stat /var/lib/calico/nodename: no such file or directory

(combined from similar events): Failed to create pod sandbox: rpc error: code = Unknown desc = failed to set up sandbox container "0a454776e2cc2f4aebbdba9117a99e41aeca6edd417d77e99fb28b39719e63cf" network for pod "svc-sfaf-0": networkPlugin cni failed to set up pod "svc-sfaf-0_testmongo" network: stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/

这个问题卡了好几天终于解决了，这个应该是之前安装k8s 的时候残留的文件导致的问题

解决思路：

这个问题一直没解决然后我就想到了要换个网络插件试试，然后我就开始安装flannel

kubectl apply -f https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.ymlkube-flannel.yml

如果flannel 镜像下载有问题可以下载国内的镜像

docker pull registry.cn-shanghai.aliyuncs.com/leozhanggg/flannel:v0.12.0-amd64
docker pull registry.cn-shanghai.aliyuncs.com/leozhanggg/flannel:v0.12.0-arm64

docker pull registry.cn-shanghai.aliyuncs.com/leozhanggg/flannel:v0.12.0-arm
docker pull registry.cn-shanghai.aliyuncs.com/leozhanggg/flannel:v0.12.0-ppc64le

docker pull registry.cn-shanghai.aliyuncs.com/leozhanggg/flannel:v0.12.0-s390x

然后再改名

docker tag registry.cn-shanghai.aliyuncs.com/leozhanggg/flannel:v0.12.0-amd64 quay.io/coreos/flannel:v0.12.0-amd64

docker tag registry.cn-shanghai.aliyuncs.com/leozhanggg/flannel:v0.12.0-arm64 quay.io/coreos/flannel:v0.12.0-arm64

docker tag registry.cn-shanghai.aliyuncs.com/leozhanggg/flannel:v0.12.0-arm quay.io/coreos/flannel:v0.12.0-arm

docker tag registry.cn-shanghai.aliyuncs.com/leozhanggg/flannel:v0.12.0-ppc64le quay.io/coreos/flannel:v0.12.0-ppc64le

docker tag registry.cn-shanghai.aliyuncs.com/leozhanggg/flannel:v0.12.0-s390x quay.io/coreos/flannel:v0.12.0-s390x

这些处理好之后我发现还是出现network: stat /var/lib/calico/nodename: no such file or directory 这个错误，这个时候就发现了

是calico 配置残留的问题，然后找到相关的calico 文件删除掉问题就解决了

需要删除   /var/lib/calico 这个目录 和 /etc/cni/net.d/ 这个目录下的calico 文件就行了 

到这里问题就解决了

 

另外还有个问题就是calico/node is not ready: BIRD is not ready: BGP not established

这个问题在换成flannel 后就没有了，而且之前部署的数据库也能正常访问了

 

还有一个问题就是不知道为什么arm64  的节点如果跟主节点不在同一个网段内网络也会出问题，而且报错：

加载失败

Error: Request failed with status code 404

具体原因目前不清楚，但是暂时部署的时候尽量在一个网段内