记录一个小教训。切记: k8s安装完成后不要轻易修改hostname。
问题:
k8s的master节点变为notReady,另外一个worker节点正常。
症状:master上的flannel pod显示为pending状态,但是describe pod没有有用信息,最后运行journalctl -f -u kubelet.service 日志中,发现节点名称的问题, 想起来刚才修改过主机名,然后问题就清楚了。
journalctl -f -u kubelet.service显示的日志信息
Mar 09 05:04:49 localhost.localdomain kubelet[5724]: E0309 05:04:49.636853 5724 kubelet.go:2248] node "localhost.localdomain" not found
Mar 09 05:04:49 localhost.localdomain kubelet[5724]: E0309 05:04:49.693945 5724 reflector.go:125] k8s.io/kubernetes/pkg/kubelet/config/apiserve
Mar 09 05:04:49 localhost.localdomain kubelet[5724]: E0309 05:04:49.696794 5724 reflector.go:125] k8s.io/kubernetes/pkg/kubelet/kubelet.go:453:
Mar 09 05:04:49 localhost.localdomain kubelet[5724]: E0309 05:04:49.699268 5724 reflector.go:125] k8s.io/kubernetes/pkg/kubelet/kubelet.go:444:
Mar 09 05:04:49 localhost.localdomain kubelet[5724]: I0309 05:04:49.732622 5724 kubelet_node_status.go:286] Setting node annotation to enable v
Mar 09 05:04:49 localhost.localdomain kubelet[5724]: E0309 05:04:49.736983 5724 kubelet.go:2248] node "localhost.localdomain" not found
Mar 09 05:04:49 localhost.localdomain kubelet[5724]: I0309 05:04:49.782223 5724 kubelet_node_status.go:72] Attempting to register node localhos
Mar 09 05:04:49 localhost.localdomain kubelet[5724]: E0309 05:04:49.782581 5724 kubelet_node_status.go:94] Unable to register node "localhost.l
Mar 09 05:04:49 localhost.localdomain kubelet[5724]: I0309 05:04:49.806613 5724 kubelet.go:1822] skipping pod synchronization - container runti
Mar 09 05:04:49 localhost.localdomain kubelet[5724]: I0309 05:04:49.807165 5724 plugin_manager.go:116] Starting Kubelet Plugin Manager
Mar 09 05:04:49 localhost.localdomain kubelet[5724]: E0309 05:04:49.807373 5724 eviction_manager.go:247] eviction manager: failed to get summar
Mar 09 05:04:49 localhost.localdomain kubelet[5724]: E0309 05:04:49.844272 5724 kubelet.go:2248] node "localhost.localdomain" not found
关键的一句就是node “localhost.localdomain” not found
解决办法:
最后将master机器的主机名修改为原先的名称, 重启master机器,
hostnamectl set-hostname localhost.localdomain
重启master机器后,一切正常了
[root@localhostxxxxx]# kubectl get node
NAME STATUS ROLES AGE VERSION
localhost.localdomain Ready master 7d21h v1.15.0
node2 Ready <none> 7d20h v1.15.0