kubernetes集群节点not ready

在上次学习kubernetes关闭虚拟机后，今天启动虚拟机后node1以及node2为notready状态，在三个虚拟机上重启docker，kubelet，daemon-reload无果后，查看kubelet状态，显示node1、node2 not found。没办法我只好在两个节点上kubeadm reset ，然后重新join。

在join途中，总是卡在preflight，然后就超时。

[root@node1 ~]# kubeadm join 192.168.120.131:6443 --token ff49wp.0p8e3npuzzws1kcv --discovery-token-ca-cert-hash sha256:d74f3be47ebde5c821ff95b03d1047a9bd575c28f938577b18b0bf2d150aa8a0 --ignore-preflight-errors=Swap

[preflight] Running pre-flight checks

error execution phase preflight: couldn't validate the identity of the API Server: Get "https://192.168.120.131:6443/api/v1/namespaces/kube-public/configmaps/cluster-info?timeout=10s": dial tcp 192.168.120.131:6443: connect: no route to host

To see the stack trace of this error execute with --v=5 or higher

我一度认为是token过期的问题，折腾了半天也不行。然后我用--v=5追踪

[root@node1 ~]# kubeadm join 192.168.120.131:6443 --token 7r3l16.5yzfksso5ty2zzie --discovery-token-ca-cert-hash sha256:d74f3be47ebde5c821ff95b03d1047a9bd575c28f938577b18b0bf2d150aa8a0 --ignore-preflight-errors=Swap --v=5

I0124 14:43:36.303201 28959 join.go:413] [preflight] found NodeName empty; using OS hostname as NodeName

I0124 14:43:36.305285 28959 initconfiguration.go:117] detected and using CRI socket: /var/run/dockershim.sock

[preflight] Running pre-flight checks

I0124 14:43:36.306038 28959 preflight.go:92] [preflight] Running general checks

I0124 14:43:36.306856 28959 checks.go:283] validating the existence of file /etc/kubernetes/kubelet.conf

I0124 14:43:36.306900 28959 checks.go:283] validating the existence of file /etc/kubernetes/bootstrap-kubelet.conf

I0124 14:43:36.306911 28959 checks.go:107] validating the container runtime

I0124 14:43:36.435877 28959 checks.go:133] validating if the "docker" service is enabled and active

I0124 14:43:36.449550 28959 checks.go:332] validating the contents of file /proc/sys/net/bridge/bridge-nf-call-iptables

I0124 14:43:36.449633 28959 checks.go:332] validating the contents of file /proc/sys/net/ipv4/ip_forward

I0124 14:43:36.449801 28959 checks.go:654] validating whether swap is enabled or not

I0124 14:43:36.450643 28959 checks.go:373] validating the presence of executable conntrack

I0124 14:43:36.450839 28959 checks.go:373] validating the presence of executable ip

I0124 14:43:36.450877 28959 checks.go:373] validating the presence of executable iptables

I0124 14:43:36.450930 28959 checks.go:373] validating the presence of executable mount

I0124 14:43:36.450983 28959 checks.go:373] validating the presence of executable nsenter

I0124 14:43:36.451014 28959 checks.go:373] validating the presence of executable ebtables

I0124 14:43:36.451034 28959 checks.go:373] validating the presence of executable ethtool

I0124 14:43:36.451056 28959 checks.go:373] validating the presence of executable socat

I0124 14:43:36.451074 28959 checks.go:373] validating the presence of executable tc

I0124 14:43:36.451099 28959 checks.go:373] validating the presence of executable touch

I0124 14:43:36.451147 28959 checks.go:521] running all checks

I0124 14:43:36.570457 28959 checks.go:404] checking whether the given node name is valid and reachable using net.LookupHost

I0124 14:43:36.570604 28959 checks.go:620] validating kubelet version

I0124 14:43:36.628491 28959 checks.go:133] validating if the "kubelet" service is enabled and active

I0124 14:43:36.636018 28959 checks.go:206] validating availability of port 10250

I0124 14:43:36.636316 28959 checks.go:283] validating the existence of file /etc/kubernetes/pki/ca.crt

I0124 14:43:36.637026 28959 checks.go:433] validating if the connectivity type is via proxy or direct

I0124 14:43:36.637096 28959 join.go:530] [preflight] Discovering cluster-info

I0124 14:43:36.637390 28959 token.go:80] [discovery] Created cluster-info discovery client, requesting info from "192.168.120.131:6443"

I0124 14:43:36.640328 28959 token.go:217] [discovery] Failed to request cluster-info, will try again: Get "https://192.168.120.131:6443/api/v1/namespaces/kube-public/configmaps/cluster-info?timeout=10s": dial tcp 192.168.120.131:6443: connect: no route to host

显示出了上述加粗的问题。针对这个问题搜索答案，我在github的一个回答上找到了可能的原因，是因为master节点上的firewalld没有关。说来也是巧，node1，node2上的firewalld我都记得关闭了，还特地查看了状态是loaded，把swap也关闭了，就是忘记了master。

 

kubernetes cluster, in kubeadm join: [discovery] Failed to request cluster info, will try again getsockopt: connection refused] · Issue #58876 · kubernetes/kubernetes · GitHub

关闭防火墙后我顺手重启了一下才join的，不知道有没有这个必要

[root@node1 ~]# systemctl restart docker
[root@node1 ~]# systemctl restart kubelet
[root@node1 ~]# systemctl daemon-reload