k8s-问题:[root@master log]# kubectl get node The connection to the server 192.168.0.105:6443 was refus

记一次测试虚拟机异常关机导致的问题
关于[root@master log]# kubectl get node
The connection to the server 192.168.0.105:6443 was refused - did you specify the right host or port?
这个问题网上的大部分解决方案是kubeadm init
直接这么搞我相信肯定能恢复,毕竟是重新初始化,但是很少有问题确认或排查的结果

这边的思路是
1、首先看一下kubelet的状态

[root@master log]# systemctl status kubelet
● kubelet.service - kubelet: The Kubernetes Node Agent
   Loaded: loaded (/usr/lib/systemd/system/kubelet.service; enabled; vendor preset: disabled)
  Drop-In: /usr/lib/systemd/system/kubelet.service.d
           └─10-kubeadm.conf
   Active: active (running) since Sun 2022-08-07 21:53:47 CST; 7min ago
     Docs: https://kubernetes.io/docs/
 Main PID: 896 (kubelet)
   CGroup: /system.slice/kubelet.service
           └─896 /usr/bin/kubelet --bootstrap-kubeconfig=/etc/kubernetes/bootstrap-kubelet.conf --kubeconfig=/etc/kubernetes/kubelet.conf --config=/var/lib/kubelet/config.yaml --network-plugin=cni --pod-infra-container-image=k8...

Aug 07 22:01:31 master kubelet[896]: E0807 22:01:31.099750     896 kubelet.go:2466] "Error getting node" err="node \"master\" not found"
Aug 07 22:01:31 master kubelet[896]: E0807 22:01:31.200690     896 kubelet.go:2466] "Error getting node" err="node \"master\" not found"
Aug 07 22:01:31 master kubelet[896]: E0807 22:01:31.302463     896 kubelet.go:2466] "Error getting node" err="node \"master\" not found"

确认问题确实和标题一样,链接6443端口失败
2、确认端口和服务的问题

[root@master log]# netstat -an |grep 443
tcp        0      0 192.168.0.105:45700     101.6.15.130:443        TIME_WAIT  
tcp        0      0 192.168.0.105:53188     54.230.61.3:443         TIME_WAIT  
[root@master log]# netstat -an |grep 6443

netstat查看端口确实没有
想查服务进程,这里想到6443端口是apiserver监听的端口,服务进程肯定是没有了因为是容器启动的,所以需要查看一下容器进程的情况
3、确认apiserver容器进程

[root@master log]# docker ps
CONTAINER ID   IMAGE                  COMMAND                  CREATED          STATUS          PORTS     NAMES
288e14904ca7   5bc0062e9555           "kube-apiserver --ad…"   13 seconds ago   Up 12 seconds             k8s_kube-apiserver_kube-apiserver-master_kube-system_400ffc84fa5a476e6e92ca8f68ba4b4f_17
123224d20e8f   0198979b7707           "kube-scheduler --au…"   4 minutes ago    Up 4 minutes              k8s_kube-scheduler_kube-scheduler-master_kube-system_979e4fa280c61a85414b1eeb67f9a06c_4
86026c6ec248   f6bc1b780606           "kube-controller-man…"   4 minutes ago    Up 4 minutes              k8s_kube-controller-manager_kube-controller-manager-master_kube-system_66054ac4c4bc791e96e1b8b3f0a88fda_4
e042794651b6   k8s.gcr.io/pause:3.6   "/pause"                 4 minutes ago    Up 4 minutes              k8s_POD_etcd-master_kube-system_a7d996a74340d0ec859869d3f547fd5e_5
6ca6069627c1   k8s.gcr.io/pause:3.6   "/pause"                 4 minutes ago    Up 4 minutes              k8s_POD_kube-scheduler-master_kube-system_979e4fa280c61a85414b1eeb67f9a06c_5
96890c46a350   k8s.gcr.io/pause:3.6   "/pause"                 4 minutes ago    Up 4 minutes              k8s_POD_kube-controller-manager-master_kube-system_66054ac4c4bc791e96e1b8b3f0a88fda_5
b16ae5e49636   k8s.gcr.io/pause:3.6   "/pause"                 4 minutes ago    Up 4 minutes              k8s_POD_kube-apiserver-master_kube-system_400ffc84fa5a476e6e92ca8f68ba4b4f_5

apiserver运行12秒???其他进程运行的是4分钟???
好像发现问题所在了,大概率是apiserver的问题
4、查看容器日志
查看apiserver日志

[root@master log]# docker logs 288e14904ca7
I0807 13:58:23.892217       1 server.go:565] external host was not specified, using 192.168.0.105
I0807 13:58:23.892642       1 server.go:172] Version: v1.23.9
I0807 13:58:24.284096       1 shared_informer.go:240] Waiting for caches to sync for node_authorizer
I0807 13:58:24.284850       1 plugins.go:158] Loaded 12 mutating admission controller(s) successfully in the following order: NamespaceLifecycle,LimitRanger,ServiceAccount,NodeRestriction,TaintNodesByCondition,Priority,DefaultTolerationSeconds,DefaultStorageClass,StorageObjectInUseProtection,RuntimeClass,DefaultIngressClass,MutatingAdmissionWebhook.
I0807 13:58:24.284859       1 plugins.go:161] Loaded 11 validating admission controller(s) successfully in the following order: LimitRanger,ServiceAccount,PodSecurity,Priority,PersistentVolumeClaimResize,RuntimeClass,CertificateApproval,CertificateSigning,CertificateSubjectRestriction,ValidatingAdmissionWebhook,ResourceQuota.
I0807 13:58:24.285508       1 plugins.go:158] Loaded 12 mutating admission controller(s) successfully in the following order: NamespaceLifecycle,LimitRanger,ServiceAccount,NodeRestriction,TaintNodesByCondition,Priority,DefaultTolerationSeconds,DefaultStorageClass,StorageObjectInUseProtection,RuntimeClass,DefaultIngressClass,MutatingAdmissionWebhook.
I0807 13:58:24
  • 0
    点赞
  • 5
    收藏
    觉得还不错? 一键收藏
  • 0
    评论

“相关推荐”对你有帮助么?

  • 非常没帮助
  • 没帮助
  • 一般
  • 有帮助
  • 非常有帮助
提交
评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值