k8s注册节点出现kube-flannel-ds服务状态pending

目录

一.排错思路

1.查看报错

2.查看系统环境版本

1)系统版本

2)K8S版本

3)Docker版本

 4)查看hosts文件配置

5) 主机通讯检测

6)配置网桥监控主机流量,主机路由转发

7)指定私有镜像仓库文件

8)查看节点容器运行状态

二.问题解决

1.删除node2节点及配置

1)删除node2节点

2)删除所有容器(node2)

3)删除旧的配置文件

4)删除旧的ca文件

5)重启k8s及docker服务(node2)

6)手动导入镜像

 2.重新注册,加入集群

 1)生成注册命令(master)

 2)加入集群(node)

 3) 查看状态(master)

3.解决另一种方法


一.排错思路

1.查看报错

# 节点状态
~]# kubectl get nodes
NAME              STATUS   ROLES                  AGE     VERSION
master            Ready    control-plane,master   2d17h   v1.23.2
vm-0-114-centos   Ready    <none>                 23h     v1.23.2
vm-0-142-centos   NoReady  <none>                 23s     v1.23.2

# 组件状态
~]# kubectl get pods -A
NAMESPACE     NAME                             READY   STATUS    RESTARTS       AGE
kube-system   coredns-557689b88f-g9pwl         1/1     Running   1 (102m ago)   2d18h
kube-system   coredns-557689b88f-rj84r         1/1     Running   1 (102m ago)   2d18h
kube-system   etcd-master                      1/1     Running   3 (102m ago)   2d18h
kube-system   kube-apiserver-master            1/1     Running   3 (102m ago)   2d18h
kube-system   kube-controller-manager-master   1/1     Running   3 (102m ago)   2d18h
kube-system   kube-flannel-ds-jxwf5            0/1     pending   0              81m
kube-system   kube-flannel-ds-qk97b            1/1     Running   1 (102m ago)   11h
kube-system   kube-flannel-ds-qzhqt            1/1     Running   0              11h
kube-system   kube-proxy-flzrz                 1/1     Running   2 (102m ago)   2d18h
kube-system   kube-proxy-sv5n8                 1/1     Running   0              81m
kube-system   kube-proxy-tkctb                 1/1     Running   0              24h
kube-system   kube-scheduler-master            1/1     Running   4 (102m ago)   2d18h

注;可以再精确一点kubectl get  pods -A -o wide查看是哪节点报错

2.查看系统环境版本

1)系统版本

Master节点

 ~]# cat /etc/os-release
NAME="CentOS Linux"
VERSION="7 (Core)"
ID="centos"
ID_LIKE="rhel fedora"
VERSION_ID="7"
PRETTY_NAME="CentOS Linux 7 (Core)"
ANSI_COLOR="0;31"
CPE_NAME="cpe:/o:centos:centos:7"
HOME_URL="https://www.centos.org/"
BUG_REPORT_URL="https://bugs.centos.org/"

CENTOS_MANTISBT_PROJECT="CentOS-7"
CENTOS_MANTISBT_PROJECT_VERSION="7"
REDHAT_SUPPORT_PRODUCT="centos"
REDHAT_SUPPORT_PRODUCT_VERSION="7"

node节点

~]# cat /etc/os-release 
NAME="CentOS Linux"
VERSION="7 (Core)"
ID="centos"
ID_LIKE="rhel fedora"
VERSION_ID="7"
PRETTY_NAME="CentOS Linux 7 (Core)"
ANSI_COLOR="0;31"
CPE_NAME="cpe:/o:centos:centos:7"
HOME_URL="https://www.centos.org/"
BUG_REPORT_URL="https://bugs.centos.org/"

CENTOS_MANTISBT_PROJECT="CentOS-7"
CENTOS_MANTISBT_PROJECT_VERSION="7"
REDHAT_SUPPORT_PRODUCT="centos"
REDHAT_SUPPORT_PRODUCT_VERSION="7"

2)K8S版本

master节点

~]# kubeadm version
kubeadm version: &version.Info{Major:"1", Minor:"23", GitVersion:"v1.23.2", GitCommit:"9d142434e3af351a628bffee3939e64c681afa4d", GitTreeState:"clean", BuildDate:"2022-01-19T17:34:34Z", GoVersion:"go1.17.5", Compiler:"gc", Platform:"linux/amd64"}

node节点

~]# kubeadm version
kubeadm version: &version.Info{Major:"1", Minor:"23", GitVersion:"v1.23.2", GitCommit:"9d142434e3af351a628bffee3939e64c681afa4d", GitTreeState:"clean", BuildDate:"2022-01-19T17:34:34Z", GoVersion:"go1.17.5", Compiler:"gc", Platform:"linux/amd64"}

3)Docker版本

master节点

~]#  docker version
Client: Docker Engine - Community
 Version:           20.10.12
 API version:       1.41
 Go version:        go1.16.12
 Git commit:        e91ed57
 Built:             Mon Dec 13 11:45:41 2021
 OS/Arch:           linux/amd64
 Context:           default
 Experimental:      true

Server: Docker Engine - Community
 Engine:
  Version:          20.10.12
  API version:      1.41 (minimum version 1.12)
  Go version:       go1.16.12
  Git commit:       459d0df
  Built:            Mon Dec 13 11:44:05 2021
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          1.4.12
  GitCommit:        7b11cfaabd73bb80907dd23182b9347b4245eb5d
 runc:
  Version:          1.0.2
  GitCommit:        v1.0.2-0-g52b36a2
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0

node节点

~]#  docker version
Client: Docker Engine - Community
 Version:           20.10.12
 API version:       1.41
 Go version:        go1.16.12
 Git commit:        e91ed57
 Built:             Mon Dec 13 11:45:41 2021
 OS/Arch:           linux/amd64
 Context:           default
 Experimental:      true

Server: Docker Engine - Community
 Engine:
  Version:          20.10.12
  API version:      1.41 (minimum version 1.12)
  Go version:       go1.16.12
  Git commit:       459d0df
  Built:            Mon Dec 13 11:44:05 2021
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          1.4.12
  GitCommit:        7b11cfaabd73bb80907dd23182b9347b4245eb5d
 runc:
  Version:          1.0.2
  GitCommit:        v1.0.2-0-g52b36a2
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0

 4)查看hosts文件配置

master节点

~]# cat /etc/hosts
.....

172.17.0.51 master
172.17.0.114 node1
172.17.0.142 node2
172.17.0.98 registry

node节点

~]# cat /etc/hosts
.....

172.17.0.51 master
172.17.0.114 node1
172.17.0.142 node2
172.17.0.98 registry

5) 主机通讯检测

master节点

~]#  ping node2 -c 5
PING node2 (172.17.0.142) 56(84) bytes of data.
64 bytes from node2 (172.17.0.142): icmp_seq=1 ttl=64 time=0.231 ms
64 bytes from node2 (172.17.0.142): icmp_seq=2 ttl=64 time=0.263 ms
64 bytes from node2 (172.17.0.142): icmp_seq=3 ttl=64 time=0.188 ms
64 bytes from node2 (172.17.0.142): icmp_seq=4 ttl=64 time=0.220 ms
64 bytes from node2 (172.17.0.142): icmp_seq=5 ttl=64 time=0.182 ms

--- node2 ping statistics ---
5 packets transmitted, 5 received, 0% packet loss, time 3999ms
rtt min/avg/max/mdev = 0.182/0.216/0.263/0.034 ms

node节点

~]#  ping master -c 5
PING master (172.17.0.51) 56(84) bytes of data.
64 bytes from master (172.17.0.51): icmp_seq=1 ttl=64 time=0.188 ms
64 bytes from master (172.17.0.51): icmp_seq=2 ttl=64 time=0.212 ms
64 bytes from master (172.17.0.51): icmp_seq=3 ttl=64 time=0.216 ms
64 bytes from master (172.17.0.51): icmp_seq=4 ttl=64 time=0.222 ms
64 bytes from master (172.17.0.51): icmp_seq=5 ttl=64 time=0.215 ms

--- master ping statistics ---
5 packets transmitted, 5 received, 0% packet loss, time 3999ms
rtt min/avg/max/mdev = 0.188/0.210/0.222/0.019 ms

6)配置网桥监控主机流量,主机路由转发

master

~]# sysctl --system
.....
* Applying /etc/sysctl.d/k8s.conf ...
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1
net.ipv4.ip_forward = 1

node

~]# sysctl --system
.....
* Applying /etc/sysctl.d/k8s.conf ...
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1
net.ipv4.ip_forward = 1

7)指定私有镜像仓库文件

master

~]# cat /etc/docker/daemon.json
{
    "exec-opts": ["native.cgroupdriver=systemd"],
    "registry-mirrors": ["https://hub-mirror.c.163.com"],
    "insecure-registries":["172.17.0.98:5000", "registry:5000"]
}

node

~]# cat /etc/docker/daemon.json
{
    "exec-opts": ["native.cgroupdriver=systemd"],
    "registry-mirrors": ["https://hub-mirror.c.163.com"],
    "insecure-registries":["172.17.0.98:5000", "registry:5000"]
}

8)查看节点容器运行状态

node1

 ~]# docker ps -a
CONTAINER ID   IMAGE                        COMMAND                  CREATED       STATUS                   PORTS     NAMES
~]# docker images
REPOSITORY                                       TAG       IMAGE ID       CREATED        SIZE
172.17.0.98:5000/kube-apiserver                  v1.23.2   8a0228dd6a68   5 days ago     135MB
172.17.0.98:5000/kube-controller-manager         v1.23.2   4783639ba7e0   5 days ago     125MB
172.17.0.98:5000/kube-scheduler                  v1.23.2   6114d758d6d1   5 days ago     53.5MB
172.17.0.98:5000/kube-proxy                      v1.23.2   d922ca3da64b   5 days ago     112MB
172.17.0.98:5000/etcd                            3.5.1-0   25f8c7f3da61   2 months ago   293MB
172.17.0.98:5000/coredns                         v1.8.6    a4ca41631cc7   3 months ago   46.8MB
172.17.0.98:5000/pause                           3.6       6270bb605e12   5 months ago   683kB

node2

 ~]# docker ps -a
CONTAINER ID   IMAGE                        COMMAND                  CREATED       STATUS                   PORTS     NAMES
c67993ac008d   404fc3ab6749                 "/opt/bin/flanneld -…"   2 hours ago   Up 2 hours                         k8s_kube-flannel_kube-flannel-ds-jxwf5_kube-system_e8b3608e-9059-49a7-b5fd-d6244e3bd06c_0
79925c61f378   404fc3ab6749                 "cp -f /etc/kube-fla…"   2 hours ago   Exited (0) 2 hours ago             k8s_install-cni_kube-flannel-ds-jxwf5_kube-system_e8b3608e-9059-49a7-b5fd-d6244e3bd06c_0
13251ae462b1   cd5235cd7dc2                 "cp -f /flannel /opt…"   2 hours ago   Exited (0) 2 hours ago             k8s_install-cni-plugin_kube-flannel-ds-jxwf5_kube-system_e8b3608e-9059-49a7-b5fd-d6244e3bd06c_0
1bb3845bfa4a   d922ca3da64b                 "/usr/local/bin/kube…"   2 hours ago   Up 2 hours                         k8s_kube-proxy_kube-proxy-sv5n8_kube-system_df8c76f4-89ef-41e6-83c0-a678ce3fd986_0
a024c30cd119   172.17.0.98:5000/pause:3.6   "/pause"                 2 hours ago   Up 2 hours                         k8s_POD_kube-proxy-sv5n8_kube-system_df8c76f4-89ef-41e6-83c0-a678ce3fd986_0
c8a4190d3582   172.17.0.98:5000/pause:3.6   "/pause"                 2 hours ago   Up 2 hours                         k8s_POD_kube-flannel-ds-jxwf5_kube-system_e8b3608e-9059-49a7-b5fd-d6244e3bd06c_0
~]# docker images
REPOSITORY                                       TAG       IMAGE ID       CREATED        SIZE
172.17.0.98:5000/kube-apiserver                  v1.23.2   8a0228dd6a68   5 days ago     135MB
172.17.0.98:5000/kube-controller-manager         v1.23.2   4783639ba7e0   5 days ago     125MB
172.17.0.98:5000/kube-scheduler                  v1.23.2   6114d758d6d1   5 days ago     53.5MB
172.17.0.98:5000/kube-proxy                      v1.23.2   d922ca3da64b   5 days ago     112MB
rancher/mirrored-flannelcni-flannel              v0.16.1   404fc3ab6749   2 weeks ago    69.9MB
172.17.0.98:5000/etcd                            3.5.1-0   25f8c7f3da61   2 months ago   293MB
rancher/mirrored-flannelcni-flannel-cni-plugin   v1.0.0    cd5235cd7dc2   2 months ago   9.03MB
172.17.0.98:5000/coredns                         v1.8.6    a4ca41631cc7   3 months ago   46.8MB
172.17.0.98:5000/pause                           3.6       6270bb605e12   5 months ago   683kB

注:此时就可以查看到是容器镜像没有导致容器未启动

二.问题解决

1.删除node2节点及配置

1)删除node2节点

# 先查看节点名称
~]# kubectl get nodes
NAME              STATUS   ROLES                  AGE     VERSION
master            Ready    control-plane,master   2d19h   v1.23.2
vm-0-114-centos   Ready    <none>                 25h     v1.23.2
vm-0-142-centos   NoReady  <none>                 114m    v1.23.2

# 删除节点
~]# kubectl delete node vm-0-142-centos
node "vm-0-142-centos" deleted

# 查看删除结果
~]# kubectl get nodes
NAME              STATUS   ROLES                  AGE     VERSION
master            Ready    control-plane,master   2d17h   v1.23.2
vm-0-114-centos   Ready    <none>                 22h     v1.23.2

2)删除所有容器(node2)

~]# docker ps -qa | xargs  docker rm -f

3)删除旧的配置文件

~]# rm -f /etc/kubernetes/kubelet.conf

4)删除旧的ca文件

~]# rm -f /etc/kubernetes/pki/ca.crt

5)重启k8s及docker服务(node2)

~]# systemctl restart docker kubelet

6)手动导入镜像

~]# docker pull rancher/mirrored-flannelcni-flannel:v0.16.1
~]# docker pull rancher/mirrored-flannelcni-flannel-cni-plugin:v1.0.0

 2.重新注册,加入集群

 1)生成注册命令(master)

~]# kubeadm token create --print-join-command
kubeadm join 172.17.0.51:6443 --token 3mucph.ml5h85umn1flgmir --discovery-token-ca-cert-hash sha256:72db7edc40a9581a8e9f96853a4e3fb03

2)加入集群(node)

~]# kubeadm join 172.17.0.51:6443 --token 3mucph.ml5h85umn1flgmir --discovery-token-ca-cert-hash sha256:72db7edc40a9581a8e9f96853a4e3fb03
[preflight] Running pre-flight checks
	[WARNING Hostname]: hostname "vm-0-142-centos" could not be reached
	[WARNING Hostname]: hostname "vm-0-142-centos": lookup vm-0-142-centos on 183.60.83.19:53: no such host
[preflight] Reading configuration from the cluster...
[preflight] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -o yaml'
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Starting the kubelet
[kubelet-start] Waiting for the kubelet to perform the TLS Bootstrap...

This node has joined the cluster:
* Certificate signing request was sent to apiserver and a response was received.
* The Kubelet was informed of the new secure connection details.

Run 'kubectl get nodes' on the control-plane to see this node join the cluster.

 3) 查看状态(master)

# 查看node节点状态
~]# kubectl get nodes
NAME              STATUS   ROLES                  AGE     VERSION
master            Ready    control-plane,master   2d17h   v1.23.2
vm-0-114-centos   Ready    <none>                 23h     v1.23.2
vm-0-142-centos   Ready    <none>                 25s     v1.23.2
# 查看pod状态
~]# kubectl get pods -A
NAMESPACE     NAME                             READY   STATUS    RESTARTS       AGE
kube-system   coredns-557689b88f-g9pwl         1/1     Running   1 (102m ago)   2d18h
kube-system   coredns-557689b88f-rj84r         1/1     Running   1 (102m ago)   2d18h
kube-system   etcd-master                      1/1     Running   3 (102m ago)   2d18h
kube-system   kube-apiserver-master            1/1     Running   3 (102m ago)   2d18h
kube-system   kube-controller-manager-master   1/1     Running   3 (102m ago)   2d18h
kube-system   kube-flannel-ds-jxwf5            1/1     Running   0              81m
kube-system   kube-flannel-ds-qk97b            1/1     Running   1 (102m ago)   11h
kube-system   kube-flannel-ds-qzhqt            1/1     Running   0              11h
kube-system   kube-proxy-flzrz                 1/1     Running   2 (102m ago)   2d18h
kube-system   kube-proxy-sv5n8                 1/1     Running   0              81m
kube-system   kube-proxy-tkctb                 1/1     Running   0              24h
kube-system   kube-scheduler-master            1/1     Running   4 (102m ago)   2d18h

至此,问题解决!所以导致这个问题的主要原因就是前期导入镜像导致 ,但是此方法过与复杂,

3.解决另一种方法

注:这里就不需要删除节点了

## 因为我们只是镜像没有,所以只需以下几步
# 第一步到node节点下传镜像
~]# docker pull rancher/mirrored-flannelcni-flannel:v0.16.1
~]# docker pull rancher/mirrored-flannelcni-flannel-cni-plugin:v1.0.0

# 第二部重启flannel
~]# kubectl delete -f kube-flannel.yml
~]# kubectl apply -f kube-flannel.yml

# 查看
# 查看node节点状态
~]# kubectl get nodes
NAME              STATUS   ROLES                  AGE     VERSION
master            Ready    control-plane,master   2d17h   v1.23.2
vm-0-114-centos   Ready    <none>                 23h     v1.23.2
vm-0-142-centos   Ready    <none>                 25s     v1.23.2
# 查看pod状态
~]# kubectl get pods -A
NAMESPACE     NAME                             READY   STATUS    RESTARTS       AGE
kube-system   coredns-557689b88f-g9pwl         1/1     Running   1 (102m ago)   2d18h
kube-system   coredns-557689b88f-rj84r         1/1     Running   1 (102m ago)   2d18h
kube-system   etcd-master                      1/1     Running   3 (102m ago)   2d18h
kube-system   kube-apiserver-master            1/1     Running   3 (102m ago)   2d18h
kube-system   kube-controller-manager-master   1/1     Running   3 (102m ago)   2d18h
kube-system   kube-flannel-ds-jxwf5            1/1     Running   0              81m
kube-system   kube-flannel-ds-qk97b            1/1     Running   1 (102m ago)   11h
kube-system   kube-flannel-ds-qzhqt            1/1     Running   0              11h
kube-system   kube-proxy-flzrz                 1/1     Running   2 (102m ago)   2d18h
kube-system   kube-proxy-sv5n8                 1/1     Running   0              81m
kube-system   kube-proxy-tkctb                 1/1     Running   0              24h
kube-system   kube-scheduler-master            1/1     Running   4 (102m ago)   2d18h

  • 2
    点赞
  • 4
    收藏
    觉得还不错? 一键收藏
  • 0
    评论

“相关推荐”对你有帮助么?

  • 非常没帮助
  • 没帮助
  • 一般
  • 有帮助
  • 非常有帮助
提交
评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值