k8s常用的排查命令

1. 查看组件状态#kubectl get cs

NAME                 STATUS    MESSAGE             ERROR
scheduler            Healthy   ok
controller-manager   Healthy   ok
etcd-1               Healthy   {"health":"true"}
etcd-2               Healthy   {"health":"true"}
etcd-0               Healthy   {"health":"true"}

2. 查看集群状态#kubectl cluster-info

Kubernetes master is running at https://10.45.1.1:6443
metrics-server is running at https://10.45.1.1:6443/api/v1/namespaces/kube-system/services/heapster/proxy
KubeDNS is running at https://10.45.1.1:6443/api/v1/namespaces/kube-system/services/kube-dns:dns/proxy

To further debug and diagnose cluster problems, use 'kubectl cluster-info dump'.

导出集群信息供排查
#kubectl cluster-info dump --namespace kube-system --output-directory=/path/to/cluster-state

3. 查看node状态#kubectl get node

NAME                            STATUS   ROLES                                           AGE     VERSION
c43k090101.cloud.k09    Ready    master,minio-0,ops1,prometheus,redis-0,worker   6d22h   v1.14.8-aliyun.1
c43k090108.cloud.k09    Ready    worker                                          6d20h   v1.14.8-aliyun.1
c43k090109.cloud.k09    Ready    worker                                          6d20h   v1.14.8-aliyun.1
c43k090110.cloud.k09    Ready    worker                                          6d20h   v1.14.8-aliyun.1
c43k090111.cloud.k09    Ready    worker                                          6d20h   v1.14.8-aliyun.1
c43k090112.cloud.k09    Ready    worker                                          6d20h   v1.14.8-aliyun.1
c43k090113.cloud.k09    Ready    worker                                          6d20h   v1.14.8-aliyun.1
c43k090114.cloud.k09    Ready    worker                                          6d20h   v1.14.8-aliyun.1
c43k090115.cloud.k09    Ready    worker                                          6d20h   v1.14.8-aliyun.1
c43k09012.cloud.k09     Ready    master,minio-1,prometheus,redis-1,worker        6d22h   v1.14.8-aliyun.1
c43k09108.cloud.k10.    Ready    master,minio-2,redis-2,worker                   6d22h   v1.14.8-aliyun.1

#kubectl describe node <node-name>

4. 查看pod#kubectl get pod -owide -A 查看所有命名空间的pod

#kubectl get pod -owide -n <namespace> 查看指定命名空间的pod
#kubectl get pod <pod-name> -oyaml -n <namespace> 查看pod定义及状态
#kubectl describe pod <pod-name> -n <namespace> 查看pod信息及event事件
#kubectl logs -f <pod-name> -n <namespace> -c <container-name> 查看pod日志

5. 查看svc

#kubectl get svc -owide -A 查看所有命名空间的svc
#kubectl get svc -owide -n <namespace> 查看指定命名空间的svc
#kubectl get svc <svc-name> -oyaml -n <namespace> 查看svc定义及状态
#kubectl describe svc <svc-name> -n <namespace> 查看svc信息,event及后端endpoint

6. 查看组件日志获取核心组件：

#kubectl get pod -l component=xxx -n kube-system (component:kube-apiserver/kube-controller-manager/kube-scheduler)

获取addon组件：
#kubectl get pod -l k8s-app=xxx -n kube-system (k8s-app:kube-dns/kube-proxy-master/kube-proxy-worker)

查看日志
#kubectl logs <pod-name> -n kube-system

查看上一次的log
#kubectl logs <pod-name> -n kube-system --previous

获取集群日志
#kubectl cluster-info dump --namespace default,kube-system,... --output-directory=/path/to/dump

7. kubelet日志#cd /cloud/app/kube-base/NodeDaemon#/kubelet/current/

#vim log/tianji/stderr
#vim log/tianji/stdout

8. 其他命令请参考

https://kubernetes.io/zh/docs/reference/kubectl/cheatsheet/
https://jimmysong.io/kubernetes-handbook/guide/kubectl-cheatsheet.html​