k8s解决 kubectl get pod 查看后No resources found问题

1. 创建一个资源

# kubectl create -f nginx-ds.yaml

2. 查看pod显示no resources found

# kubectl get pod
No resources found

3. 查看日志

3.1 查看kube-apiserver日志,无异常

3.2 查看kube-kubelet日志,无异常

3.3 查看kube-scheduler日志如下

查看链接127.0.0.1:8000被拒绝,后来在本地查看并没有8000端口,我的apiserver的端口设置的是8080,可能是kube-scheduler启动脚本端口写错

# tail -f /data/logs/kubernetes/kube-scheduler/scheduler.stdout.log
E0429 09:44:11.299768   48887 reflector.go:125] k8s.io/client-go/informers/factory.go:133: Failed to list *v1beta1.PodDisruptionBudget: Get http://127.0.0.1:8000/apis/policy/v1beta1/poddisruptionbudgets?limit=500&resourceVersion=0: dial tcp 127.0.0.1:8000: connect: connection refused
E0429 09:44:11.304570   48887 reflector.go:125] k8s.io/client-go/informers/factory.go:133: Failed to list *v1.PersistentVolume: Get http://127.0.0.1:8000/api/v1/persistentvolumes?limit=500&resourceVersion=0: dial tcp 127.0.0.1:8000: connect: connection refused
E0429 09:44:11.308370   48887 reflector.go:125] k8s.io/client-go/informers/factory.go:133: Failed to list *v1.Node: Get http://127.0.0.1:8000/api/v1/nodes?limit=500&resourceVersion=0: dial tcp 127.0.0.1:8000: connect: connection refused
E0429 09:44:11.308745   48887 reflector.go:125] k8s.io/client-go/informers/factory.go:133: Failed to list *v1.StorageClass: Get http://127.0.0.1:8000/apis/storage.k8s.io/v1/storageclasses?limit=500&resourceVersion=0: dial tcp 127.0.0.1:8000: connect: connection refused
E0429 09:44:11.310308   48887 reflector.go:125] k8s.io/client-go/informers/factory.go:133: Failed to list *v1.ReplicationController: Get http://127.0.0.1:8000/api/v1/replicationcontrollers?limit=500&resourceVersion=0: dial tcp 127.0.0.1:8000: connect: connection refused
E0429 09:44:11.311481   48887 reflector.go:125] k8s.io/client-go/informers/factory.go:133: Failed to list *v1.StatefulSet: Get http://127.0.0.1:8000/apis/apps/v1/statefulsets?limit=500&resourceVersion=0: dial tcp 127.0.0.1:8000: connect: connection refused
E0429 09:44:12.293511   48887 reflector.go:125] k8s.io/client-go/informers/factory.go:133: Failed to list *v1.Service: Get http://127.0.0.1:8000/api/v1/services?limit=500&resourceVersion=0: dial tcp 127.0.0.1:8000: connect: connection refused
E0429 09:44:12.297952   48887 reflector.go:125] k8s.io/kubernetes/cmd/kube-scheduler/app/server.go:226: Failed to list *v1.Pod: Get http://127.0.0.1:8000/api/v1/pods?fieldSelector=status.phase%21%3DFailed%2Cstatus.phase%21%3DSucceeded&limit=500&resourceVersion=0: dial tcp 127.0.0.1:8000: connect: connection refused
E0429 09:44:12.299568   48887 reflector.go:125] k8s.io/client-go/informers/factory.go:133: Failed to list *v1.PersistentVolumeClaim: Get http://127.0.0.1:8000/api/v1/persistentvolumeclaims?limit=500&resourceVersion=0: dial tcp 127.0.0.1:8000: connect: connection refused
E0429 09:44:12.300620   48887 reflector.go:125] k8s.io/client-go/informers/factory.go:133: Failed to list *v1.ReplicaSet: Get http://127.0.0.1:8000/apis/apps/v1/replicasets?limit=500&resourceVersion=0: dial tcp 127.0.0.1:8000: connect: connection refused
E0429 09:44:12.301799   48887 reflector.go:125] k8s.io/client-go/informers/factory.go:133: Failed to list *v1beta1.PodDisruptionBudget: Get http://127.0.0.1:8000/apis/policy/v1beta1/poddisruptionbudgets?limit=500&resourceVersion=0: dial tcp 127.0.0.1:8000: connect: connection refused
E0429 09:44:12.305372   48887 reflector.go:125] k8s.io/client-go/informers/factory.go:133: Failed to list *v1.PersistentVolume: Get http://127.0.0.1:8000/api/v1/persistentvolumes?limit=500&resourceVersion=0: dial tcp 127.0.0.1:8000: connect: connection refused
E0429 09:44:12.308955   48887 reflector.go:125] k8s.io/client-go/informers/factory.go:133: Failed to list *v1.Node: Get http://127.0.0.1:8000/api/v1/nodes?limit=500&resourceVersion=0: dial tcp 127.0.0.1:8000: connect: connection refused
E0429 09:44:12.310113   48887 reflector.go:125] k8s.io/client-go/informers/factory.go:133: Failed to list *v1.StorageClass: Get http://127.0.0.1:8000/apis/storage.k8s.io/v1/storageclasses?limit=500&resourceVersion=0: dial tcp 127.0.0.1:8000: connect: connection refused
E0429 09:44:12.311407   48887 reflector.go:125] k8s.io/client-go/informers/factory.go:133: Failed to list *v1.ReplicationController: Get http://127.0.0.1:8000/api/v1/replicationcontrollers?limit=500&resourceVersion=0: dial tcp 127.0.0.1:8000: connect: connection refused
E0429 09:44:12.312201   48887 reflector.go:125] k8s.io/client-go/informers/factory.go:133: Failed to list *v1.StatefulSet: Get http://127.0.0.1:8000/apis/apps/v1/statefulsets?limit=500&resourceVersion=0: dial tcp 127.0.0.1:8000: connect: connection refused

3.4 查看controller-manager日志如下

链接8000端口也是被拒绝了,这里我的apiserver的端口是8080,可能是controller-manage端口写错

# tail -f /data/logs/kubernetes/kube-controller-manager/controller.stdout.log 
E0429 09:45:25.841916   48885 leaderelection.go:324] error retrieving resource lock kube-system/kube-controller-manager: Get http://127.0.0.1:8000/api/v1/namespaces/kube-system/endpoints/kube-controller-manager?timeout=10s: dial tcp 127.0.0.1:8000: connect: connection refused
E0429 09:45:28.981869   48885 leaderelection.go:324] error retrieving resource lock kube-system/kube-controller-manager: Get http://127.0.0.1:8000/api/v1/namespaces/kube-system/endpoints/kube-controller-manager?timeout=10s: dial tcp 127.0.0.1:8000: connect: connection refused
E0429 09:45:32.367088   48885 leaderelection.go:324] error retrieving resource lock kube-system/kube-controller-manager: Get http://127.0.0.1:8000/api/v1/namespaces/kube-system/endpoints/kube-controller-manager?timeout=10s: dial tcp 127.0.0.1:8000: connect: connection refused
E0429 09:45:35.969067   48885 leaderelection.go:324] error retrieving resource lock kube-system/kube-controller-manager: Get http://127.0.0.1:8000/api/v1/namespaces/kube-system/endpoints/kube-controller-manager?timeout=10s: dial tcp 127.0.0.1:8000: connect: connection refused
E0429 09:45:38.060622   48885 leaderelection.go:324] error retrieving resource lock kube-system/kube-controller-manager: Get http://127.0.0.1:8000/api/v1/namespaces/kube-system/endpoints/kube-controller-manager?timeout=10s: dial tcp 127.0.0.1:8000: connect: connection refused
E0429 09:45:40.605084   48885 leaderelection.go:324] error retrieving resource lock kube-system/kube-controller-manager: Get http://127.0.0.1:8000/api/v1/namespaces/kube-system/endpoints/kube-controller-manager?timeout=10s: dial tcp 127.0.0.1:8000: connect: connection refused
E0429 09:45:42.638763   48885 leaderelection.go:324] error retrieving resource lock kube-system/kube-controller-manager: Get http://127.0.0.1:8000/api/v1/namespaces/kube-system/endpoints/kube-controller-manager?timeout=10s: dial tcp 127.0.0.1:8000: connect: connection refused
E0429 09:45:45.856238   48885 leaderelection.go:324] error retrieving resource lock kube-system/kube-controller-manager: Get http://127.0.0.1:8000/api/v1/namespaces/kube-system/endpoints/kube-controller-manager?timeout=10s: dial tcp 127.0.0.1:8000: connect: connection refused
E0429 09:45:48.169061   48885 leaderelection.go:324] error retrieving resource lock kube-system/kube-controller-manager: Get http://127.0.0.1:8000/api/v1/namespaces/kube-system/endpoints/kube-controller-manager?timeout=10s: dial tcp 127.0.0.1:8000: connect: connection refused
E0429 09:45:50.974880   48885 leaderelection.go:324] error retrieving resource lock kube-system/kube-controller-manager: Get http://127.0.0.1:8000/api/v1/namespaces/kube-system/endpoints/kube-controller-manager?timeout=10s: dial tcp 127.0.0.1:8000: connect: connection refused
E0429 09:45:54.336228   48885 leaderelection.go:324] error retrieving resource lock kube-system/kube-controller-manager: Get http://127.0.0.1:8000/api/v1/namespaces/kube-system/endpoints/kube-controller-manager?timeout=10s: dial tcp 127.0.0.1:8000: connect: connection refused
E0429 09:45:57.195092   48885 leaderelection.go:324] error retrieving resource lock kube-system/kube-controller-manager: Get http://127.0.0.1:8000/api/v1/namespaces/kube-system/endpoints/kube-controller-manager?timeout=10s: dial tcp 127.0.0.1:8000: connect: connection refused

3.5 查看kube-proxy日志

此处判断可能是没有链接到apiserver导致

# tail -f /data/logs/kubernetes/kube-proxy/proxy.stdout.log 
E0429 09:51:56.124383   48886 reflector.go:125] k8s.io/client-go/informers/factory.go:133: Failed to list *v1.Service: the server rejected our request for an unknown reason (get services)
E0429 09:51:56.125239   48886 reflector.go:125] k8s.io/client-go/informers/factory.go:133: Failed to list *v1.Endpoints: the server rejected our request for an unknown reason (get endpoints)
E0429 09:51:57.126052   48886 reflector.go:125] k8s.io/client-go/informers/factory.go:133: Failed to list *v1.Service: the server rejected our request for an unknown reason (get services)
E0429 09:51:57.126586   48886 reflector.go:125] k8s.io/client-go/informers/factory.go:133: Failed to list *v1.Endpoints: the server rejected our request for an unknown reason (get endpoints)
E0429 09:51:58.128520   48886 reflector.go:125] k8s.io/client-go/informers/factory.go:133: Failed to list *v1.Service: the server rejected our request for an unknown reason (get services)
E0429 09:51:58.129030   48886 reflector.go:125] k8s.io/client-go/informers/factory.go:133: Failed to list *v1.Endpoints: the server rejected our request for an unknown reason (get endpoints)
E0429 09:51:59.130244   48886 reflector.go:125] k8s.io/client-go/informers/factory.go:133: Failed to list *v1.Service: the server rejected our request for an unknown reason (get services)
E0429 09:51:59.130979   48886 reflector.go:125] k8s.io/client-go/informers/factory.go:133: Failed to list *v1.Endpoints: the server rejected our request for an unknown reason (get endpoints)
E0429 09:52:00.132002   48886 reflector.go:125] k8s.io/client-go/informers/factory.go:133: Failed to list *v1.Service: the server rejected our request for an unknown reason (get services)
E0429 09:52:00.134413   48886 reflector.go:125] k8s.io/client-go/informers/factory.go:133: Failed to list *v1.Endpoints: the server rejected our request for an unknown reason (get endpoints)
E0429 09:52:01.134364   48886 reflector.go:125] k8s.io/client-go/informers/factory.go:133: Failed to list *v1.Service: the server rejected our request for an unknown reason (get services)
E0429 09:52:01.135871   48886 reflector.go:125] k8s.io/client-go/informers/factory.go:133: Failed to list *v1.Endpoints: the server rejected our request for an unknown reason (get endpoints)

4. 更改

4.1 更改kube-scheduler启动脚本

# 将8000端口改成8080,我的apiserver端口设置的是8080
# cat kube-scheduler.sh
#!/bin/bash
./kube-scheduler \
  --leader-elect \
  --log-dir /data/logs/kubernetes/kube-scheduler \
  --master http://127.0.0.1:8080 \
  --v 2

4.2 更改controller-manager启动脚本

# 此处将8000端口改成8080
# cat kube-controller-manager.sh
#!/bin/bash
./kube-controller-manager \
  --cluster-cidr 172.7.0.0/16 \
  --leader-elect true \
  --log-dir /data/logs/kubernetes/kube-controller-manager \
  --master http://127.0.0.1:8080 \
  --service-account-private-key-file ./cert/ca-key.pem \
  --service-cluster-ip-range 192.168.0.0/16 \
  --root-ca-file ./cert/ca.pem \
  --v 2

4.3 停止supervisord后在start

 # systemctl stop supervisord.service
 # systemctl start supervisord.service
 # supervisorctl status
etcd-server-7-21                 RUNNING   pid 66496, uptime 0:03:22
kube-apiserver-7-21              RUNNING   pid 66497, uptime 0:03:22
kube-controller-manager-7-21     RUNNING   pid 66499, uptime 0:03:22
kube-kubelet-7-21                RUNNING   pid 66706, uptime 0:03:17
kube-proxy-7-21                  RUNNING   pid 66500, uptime 0:03:22
kube-scheduler-7-21              RUNNING   pid 66501, uptime 0:03:22

5. 查看pod

# kubectl get pod
NAME             READY   STATUS              RESTARTS   AGE
nginx-ds-5q6mh   0/1     ContainerCreating   0          4m13s
nginx-ds-8bpc6   0/1     ContainerCreating   0          4m13s

可以能看到pod资源了,但是查看kub-proxy日志还是3.5描述的错误,继续排查。。。。。。

  • 0
    点赞
  • 4
    收藏
    觉得还不错? 一键收藏
  • 打赏
    打赏
  • 0
    评论

“相关推荐”对你有帮助么?

  • 非常没帮助
  • 没帮助
  • 一般
  • 有帮助
  • 非常有帮助
提交
评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包

打赏作者

河 静

你的鼓励将是我创作的最大动力

¥1 ¥2 ¥4 ¥6 ¥10 ¥20
扫码支付:¥1
获取中
扫码支付

您的余额不足,请更换扫码支付或充值

打赏作者

实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值