- 原因及处理方法
===================================================================
-
算是记录一个坑吧,花了我挺多时间解决这个破问题的。
-
今天给集群的node节点配置一个–insecure-registry的IP,配置内容如下
[root@node1 ~]# cat /usr/lib/systemd/system/docker.service | grep ExecStart
#ExecStart=/usr/bin/dockerd -H fd:// --containerd=/run/containerd/containerd.sock
ExecStart=/usr/bin/dockerd --insecure-registry=192.168.59.151:5000 -H fd:// --containerd=/run/containerd/containerd.sock
[root@node1 ~]#
- 然后启动就报错了,docker状态和报错内容如下
#重启报错
[root@node1 ~]# systemctl restart docker
Job for docker.service failed because the control process exited with error code. See “systemctl status docker.service” and “journalctl -xe” for details.
[root@node1 ~]#
状态下的日志
[root@node1 ~]# systemctl status docker
● docker.service - Docker Application Container Engine
Loaded: loaded (/usr/lib/systemd/system/docker.service; enabled; vendor preset: disabled)
Active: failed (Result: start-limit) since Tue 2021-11-09 15:47:44 CST; 13s ago
Docs: https://docs.docker.com
Process: 38530 ExecStart=/usr/bin/dockerd --insecure-registry=192.168.59.142:5000 -H fd:// --containerd=/run/containerd/containerd.sock (code=exited, status=1/FAILURE)
Main PID: 38530 (code=exited, status=1/FAILURE)
Nov 09 15:47:42 node1 systemd[1]: docker.service failed.
Nov 09 15:47:44 node1 systemd[1]: docker.service holdoff time over, scheduling restart.
Nov 09 15:47:44 node1 systemd[1]: Stopped Docker Application Container Engine.
Nov 09 15:47:44 node1 systemd[1]: start request repeated too quickly for docker.service
Nov 09 15:47:44 node1 systemd[1]: Failed to start Docker Application Container Engine.
Nov 09 15:47:44 node1 systemd[1]: Unit docker.service entered failed state.
Nov 09 15:47:44 node1 systemd[1]: docker.service failed.
Nov 09 15:47:44 node1 systemd[1]: start request repeated too quickly for docker.service
Nov 09 15:47:44 node1 systemd[1]: Failed to start Docker Application Container Engine.
Nov 09 15:47:44 node1 systemd[1]: docker.service failed.
[root@node1 ~]#
#我们再来看messages
[root@node1 ~]# tail -n100 /var/log/messages
Nov 9 15:49:01 node1 kubelet: Flag --network-plugin has been deprecated, will be removed along with dockershim.
Nov 9 15:49:01 node1 kubelet: Flag --network-plugin has been deprecated, will be removed along with dockershim.
Nov 9 15:49:01 node1 systemd: Started Kubernetes systemd probe.
Nov 9 15:49:01 node1 kubelet: I1109 15:49:01.272480 38654 server.go:440] “Kubelet version” kubeletVersion=“v1.21.0”
Nov 9 15:49:01 node1 kubelet: I1109 15:49:01.272947 38654 server.go:851] “Client rotation is on, will bootstrap in background”
Nov 9 15:49:01 node1 kubelet: I1109 15:49:01.291828 38654 certificate_store.go:130] Loading cert/key pair from “/var/lib/kubelet/pki/kubelet-client-current.pem”.
Nov 9 15:49:01 node1 kubelet: I1109 15:49:01.295753 38654 dynamic_cafile_content.go:167] Starting client-ca-bundle::/etc/kubernetes/pki/ca.crt
Nov 9 15:49:06 node1 kubelet: I1109 15:49:06.338147 38654 server.go:660] “–cgroups-per-qos enabled, but --cgroup-root was not specified. defaulting to /”
Nov 9 15:49:06 node1 kubelet: I1109 15:49:06.338751 38654 container_manager_linux.go:278] “Container manager verified user specified cgroup-root exists” cgroupRoot=[]
Nov 9 15:49:06 node1 kubelet: I1109 15:49:06.338938 38654 container_manager_linux.go:283] “Creating Container Manager object based on Node Config” nodeConfig={RuntimeCgroupsName: SystemCgroupsName: KubeletCgroupsName: ContainerRuntime:docker CgroupsPerQOS:true CgroupRoot:/ CgroupDriver:cgroupfs KubeletRootDir:/var/lib/kubelet ProtectKernelDefaults:false NodeAllocatableConfig:{KubeReservedCgroupName: SystemReservedCgroupName: ReservedSystemCPUs: EnforceNodeAllocatable:map[pods:{}] KubeReserved:map[] SystemReserved:map[] HardEvictionThresholds:[{Signal:nodefs.available Operator:LessThan Value:{Quantity: Percentage:0.1} GracePeriod:0s MinReclaim:} {Signal:nodefs.inodesFree Operator:LessThan Value:{Quantity: Percentage:0.05} GracePeriod:0s MinReclaim:} {Signal:imagefs.available Operator:LessThan Value:{Quantity: Percentage:0.15} GracePeriod:0s MinReclaim:} {Signal:memory.available Operator:LessThan Value:{Quantity:100Mi Percentage:0} GracePeriod:0s MinReclaim:}]} QOSReserved:map[] ExperimentalCPUManagerPolicy:none ExperimentalTopologyManagerScope:container ExperimentalCPUManagerReconcilePeriod:10s ExperimentalMemoryManagerPolicy:None ExperimentalMemoryManagerReservedMemory:[] ExperimentalPodPidsLimit:-1 EnforceCPULimits:true CPUCFSQuotaPeriod:100ms ExperimentalTopologyManagerPolicy:none}
Nov 9 15:49:06 node1 kubelet: I1109 15:49:06.338984 38654 topology_manager.go:120] “Creating topology manager with policy per scope” topologyPolicyName=“none” topologyScopeName=“container”
Nov 9 15:49:06 node1 kubelet: I1109 15:49:06.339007 38654 container_manager_linux.go:314] “Initializing Topology Manager” policy=“none” scope=“container”
Nov 9 15:49:06 node1 kubelet: I1109 15:49:06.339018 38654 container_manager_linux.go:319] “Creating device plugin manager” devicePluginEnabled=true
Nov 9 15:49:06 node1 kubelet: I1109 15:49:06.339254 38654 kubelet.go:310] “Using dockershim is deprecated, please consider using a full-fledged CRI implementation”
Nov 9 15:49:06 node1 kubelet: I1109 15:49:06.339308 38654 client.go:78] “Connecting to docker on the dockerEndpoint” endpoint=“unix:///var/run/docker.sock”
Nov 9 15:49:06 node1 kubelet: I1109 15:49:06.339330 38654 client.go:97] “Start docker client with request timeout” timeout=“2m0s”
Nov 9 15:49:06 node1 kubelet: E1109 15:49:06.339673 38654 server.go:292] “Failed to run kubelet” err=“failed to run Kubelet: failed to get docker version: Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?”
Nov 9 15:49:06 node1 systemd: kubelet.service: main process exited, code=exited, status=1/FAILURE
Nov 9 15:49:06 node1 systemd: Unit kubelet.service entered failed state.
Nov 9 15:49:06 node1 systemd: kubelet.service failed.
Nov 9 15:49:16 node1 systemd: kubelet.service holdoff time over, scheduling restart.
Nov 9 15:49:16 node1 systemd: Stopped kubelet: The Kubernetes Node Agent.
Nov 9 15:49:16 node1 systemd: Started kubelet: The Kubernetes Node Agent.
Nov 9 15:49:16 node1 kubelet: Flag --network-plugin has been deprecated, will be removed along with dockershim.
Nov 9 15:49:16 node1 kubelet: Flag --network-plugin has been deprecated, will be removed along with dockershim.
Nov 9 15:49:16 node1 systemd: Started Kubernetes systemd probe.
Nov 9 15:49:16 node1 kubelet: I1109 15:49:16.678405 38676 server.go:440] “Kubelet version” kubeletVersion=“v1.21.0”
Nov 9 15:49:16 node1 kubelet: I1109 15:49:16.678871 38676 server.go:851] “Client rotation is on, will bootstrap in background”
Nov 9 15:49:16 node1 kubelet: I1109 15:49:16.698382 38676 certificate_store.go:130] Loading cert/key pair from “/var/lib/kubelet/pki/kubelet-client-current.pem”.
Nov 9 15:49:16 node1 kubelet: I1109 15:49:16.700415 38676 dynamic_cafile_content.go:167] Starting client-ca-bundle::/etc/kubernetes/pki/ca.crt
Nov 9 15:49:21 node1 kubelet: I1109 15:49:21.738824 38676 server.go:660] “–cgroups-per-qos enabled, but --cgroup-root was not specified. defaulting to /”
Nov 9 15:49:21 node1 kubelet: I1109 15:49:21.739317 38676 container_manager_linux.go:278] “Container manager verified user specified cgroup-root exists” cgroupRoot=[]
Nov 9 15:49:21 node1 kubelet: I1109 15:49:21.739470 38676 container_manager_linux.go:283] “Creating Container Manager object based on Node Config” nodeConfig={RuntimeCgroupsName: SystemCgroupsName: KubeletCgroupsName: ContainerRuntime:docker CgroupsPerQOS:true CgroupRoot:/ CgroupDriver:cgroupfs KubeletRootDir:/var/lib/kubelet ProtectKernelDefaults:false NodeAllocatableConfig:{KubeReservedCgroupName: SystemReservedCgroupName: ReservedSystemCPUs: EnforceNodeAllocatable:map[pods:{}] KubeReserved:map[] SystemReserved:map[] HardEvictionThresholds:[{Signal:nodefs.inodesFree Operator:LessThan Value:{Quantity: Percentage:0.05} GracePeriod:0s MinReclaim:} {Signal:imagefs.available Operator:LessThan Value:{Quantity: Percentage:0.15} GracePeriod:0s MinReclaim:} {Signal:memory.available Operator:LessThan Value:{Quantity:100Mi Percentage:0} GracePeriod:0s MinReclaim:} {Signal:nodefs.available Operator:LessThan Value:{Quantity: Percentage:0.1} GracePeriod:0s MinReclaim:}]} QOSReserved:map[] ExperimentalCPUManagerPolicy:none ExperimentalTopologyManagerScope:container ExperimentalCPUManagerReconcilePeriod:10s ExperimentalMemoryManagerPolicy:None ExperimentalMemoryManagerReservedMemory:[] ExperimentalPodPidsLimit:-1 EnforceCPULimits:true CPUCFSQuotaPeriod:100ms ExperimentalTopologyManagerPolicy:none}
Nov 9 15:49:21 node1 kubelet: I1109 15:49:21.739516 38676 topology_manager.go:120] “Creating topology manager with policy per scope” topologyPolicyName=“none” topologyScopeName=“container”
Nov 9 15:49:21 node1 kubelet: I1109 15:49:21.739535 38676 container_manager_linux.go:314] “Initializing Topology Manager” policy=“none” scope=“container”
Nov 9 15:49:21 node1 kubelet: I1109 15:49:21.739545 38676 container_manager_linux.go:319] “Creating device plugin manager” devicePluginEnabled=true
Nov 9 15:49:21 node1 kubelet: I1109 15:49:21.739782 38676 kubelet.go:310] “Using dockershim is deprecated, please consider using a full-fledged CRI implementation”
Nov 9 15:49:21 node1 kubelet: I1109 15:49:21.739834 38676 client.go:78] “Connecting to docker on the dockerEndpoint” endpoint=“unix:///var/run/docker.sock”
Nov 9 15:49:21 node1 kubelet: I1109 15:49:21.739860 38676 client.go:97] “Start docker client with request timeout” timeout=“2m0s”
Nov 9 15:49:21 node1 kubelet: E1109 15:49:21.740326 38676 server.go:292] “Failed to run kubelet” err=“failed to run Kubelet: failed to get docker version: Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?”
Nov 9 15:49:21 node1 systemd: kubelet.service: main process exited, code=exited, status=1/FAILURE
Nov 9 15:49:21 node1 systemd: Unit kubelet.service entered failed state.
Nov 9 15:49:21 node1 systemd: kubelet.service failed.
Nov 9 15:49:31 node1 systemd: kubelet.service holdoff time over, scheduling restart.
Nov 9 15:49:31 node1 systemd: Stopped kubelet: The Kubernetes Node Agent.
Nov 9 15:49:31 node1 systemd: Started kubelet: The Kubernetes Node Agent.
Nov 9 15:49:32 node1 kubelet: Flag --network-plugin has been deprecated, will be removed along with dockershim.
Nov 9 15:49:32 node1 kubelet: Flag --network-plugin has been deprecated, will be removed along with dockershim.
Nov 9 15:49:32 node1 systemd: Started Kubernetes systemd probe.
Nov 9 15:49:32 node1 kubelet: I1109 15:49:32.378876 38706 server.go:440] “Kubelet version” kubeletVersion=“v1.21.0”
Nov 9 15:49:32 node1 kubelet: I1109 15:49:32.379407 38706 server.go:851] “Client rotation is on, will bootstrap in background”
Nov 9 15:49:32 node1 kubelet: I1109 15:49:32.398293 38706 certificate_store.go:130] Loading cert/key pair from “/var/lib/kubelet/pki/kubelet-client-current.pem”.
Nov 9 15:49:32 node1 kubelet: I1109 15:49:32.466688 38706 dynamic_cafile_content.go:167] Starting client-ca-bundle::/etc/kubernetes/pki/ca.crt
Nov 9 15:49:37 node1 kubelet: I1109 15:49:37.502634 38706 server.go:660] “–cgroups-per-qos enabled, but --cgroup-root was not specified. defaulting to /”
Nov 9 15:49:37 node1 kubelet: I1109 15:49:37.503144 38706 container_manager_linux.go:278] “Container manager verified user specified cgroup-root exists” cgroupRoot=[]
Nov 9 15:49:37 node1 kubelet: I1109 15:49:37.503303 38706 container_manager_linux.go:283] “Creating Container Manager object based on Node Config” nodeConfig={RuntimeCgroupsName: SystemCgroupsName: KubeletCgroupsName: ContainerRuntime:docker CgroupsPerQOS:true CgroupRoot:/ CgroupDriver:cgroupfs KubeletRootDir:/var/lib/kubelet ProtectKernelDefaults:false NodeAllocatableConfig:{KubeReservedCgroupName: SystemReservedCgroupName: ReservedSystemCPUs: EnforceNodeAllocatable:map[pods:{}] KubeReserved:map[] SystemReserved:map[] HardEvictionThresholds:[{Signal:memory.available Operator:LessThan Value:{Quantity:100Mi Percentage:0} GracePeriod:0s MinReclaim:} {Signal:nodefs.available Operator:LessThan Value:{Quantity: Percentage:0.1} GracePeriod:0s MinReclaim:} {Signal:nodefs.inodesFree Operator:LessThan Value:{Quantity: Percentage:0.05} GracePeriod:0s MinReclaim:} {Signal:imagefs.available Operator:LessThan Value:{Quantity: Percentage:0.15} GracePeriod:0s MinReclaim:}]} QOSReserved:map[] ExperimentalCPUManagerPolicy:none ExperimentalTopologyManagerScope:container ExperimentalCPUManagerReconcilePeriod:10s ExperimentalMemoryManagerPolicy:None ExperimentalMemoryManagerReservedMemory:[] ExperimentalPodPidsLimit:-1 EnforceCPULimits:true CPUCFSQuotaPeriod:100ms ExperimentalTopologyManagerPolicy:none}
Nov 9 15:49:37 node1 kubelet: I1109 15:49:37.503348 38706 topology_manager.go:120] “Creating topology manager with policy per scope” topologyPolicyName=“none” topologyScopeName=“container”
Nov 9 15:49:37 node1 kubelet: I1109 15:49:37.503367 38706 container_manager_linux.go:314] “Initializing Topology Manager” policy=“none” scope=“container”
Nov 9 15:49:37 node1 kubelet: I1109 15:49:37.503377 38706 container_manager_linux.go:319] “Creating device plugin manager” devicePluginEnabled=true
Nov 9 15:49:37 node1 kubelet: I1109 15:49:37.503602 38706 kubelet.go:310] “Using dockershim is deprecated, please consider using a full-fledged CRI implementation”
Nov 9 15:49:37 node1 kubelet: I1109 15:49:37.503654 38706 client.go:78] “Connecting to docker on the dockerEndpoint” endpoint=“unix:///var/run/docker.sock”
Nov 9 15:49:37 node1 systemd: kubelet.service: main process exited, code=exited, status=1/FAILURE
Nov 9 15:49:37 node1 kubelet: I1109 15:49:37.503679 38706 client.go:97] “Start docker client with request timeout” timeout=“2m0s”
Nov 9 15:49:37 node1 kubelet: E1109 15:49:37.504111 38706 server.go:292] “Failed to run kubelet” err=“failed to run Kubelet: failed to get docker version: Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?”
Nov 9 15:49:37 node1 systemd: Unit kubelet.service entered failed state.
Nov 9 15:49:37 node1 systemd: kubelet.service failed.
Nov 9 15:49:47 node1 systemd: kubelet.service holdoff time over, scheduling restart.
Nov 9 15:49:47 node1 systemd: Stopped kubelet: The Kubernetes Node Agent.
Nov 9 15:49:47 node1 systemd: Started kubelet: The Kubernetes Node Agent.
Nov 9 15:49:47 node1 kubelet: Flag --network-plugin has been deprecated, will be removed along with dockershim.
Nov 9 15:49:47 node1 kubelet: Flag --network-plugin has been deprecated, will be removed along with dockershim.
Nov 9 15:49:47 node1 systemd: Started Kubernetes systemd probe.
Nov 9 15:49:47 node1 kubelet: I1109 15:49:47.820327 38730 server.go:440] “Kubelet version” kubeletVersion=“v1.21.0”
Nov 9 15:49:47 node1 kubelet: I1109 15:49:47.820782 38730 server.go:851] “Client rotation is on, will bootstrap in background”
Nov 9 15:49:47 node1 kubelet: I1109 15:49:47.839802 38730 certificate_store.go:130] Loading cert/key pair from “/var/lib/kubelet/pki/kubelet-client-current.pem”.
Nov 9 15:49:47 node1 kubelet: I1109 15:49:47.912499 38730 dynamic_cafile_content.go:167] Starting client-ca-bundle::/etc/kubernetes/pki/ca.crt
Nov 9 15:49:52 node1 kubelet: I1109 15:49:52.950362 38730 server.go:660] “–cgroups-per-qos enabled, but --cgroup-root was not specified. defaulting to /”
Nov 9 15:49:52 node1 kubelet: I1109 15:49:52.950843 38730 container_manager_linux.go:278] “Container manager verified user specified cgroup-root exists” cgroupRoot=[]
Nov 9 15:49:52 node1 kubelet: I1109 15:49:52.950989 38730 container_manager_linux.go:283] “Creating Container Manager object based on Node Config” nodeConfig={RuntimeCgroupsName: SystemCgroupsName: KubeletCgroupsName: ContainerRuntime:docker CgroupsPerQOS:true CgroupRoot:/ CgroupDriver:cgroupfs KubeletRootDir:/var/lib/kubelet ProtectKernelDefaults:false NodeAllocatableConfig:{KubeReservedCgroupName: SystemReservedCgroupName: ReservedSystemCPUs: EnforceNodeAllocatable:map[pods:{}] KubeReserved:map[] SystemReserved:map[] HardEvictionThresholds:[{Signal:nodefs.inodesFree Operator:LessThan Value:{Quantity: Percentage:0.05} GracePeriod:0s MinReclaim:} {Signal:imagefs.available Operator:LessThan Value:{Quantity: Percentage:0.15} GracePeriod:0s MinReclaim:} {Signal:memory.available Operator:LessThan Value:{Quantity:100Mi Percentage:0} GracePeriod:0s MinReclaim:} {Signal:nodefs.available Operator:LessThan Value:{Quantity: Percentage:0.1} GracePeriod:0s MinReclaim:}]} QOSReserved:map[] ExperimentalCPUManagerPolicy:none ExperimentalTopologyManagerScope:container ExperimentalCPUManagerReconcilePeriod:10s ExperimentalMemoryManagerPolicy:None ExperimentalMemoryManagerReservedMemory:[] ExperimentalPodPidsLimit:-1 EnforceCPULimits:true CPUCFSQuotaPeriod:100ms ExperimentalTopologyManagerPolicy:none}
Nov 9 15:49:52 node1 kubelet: I1109 15:49:52.951032 38730 topology_manager.go:120] “Creating topology manager with policy per scope” topologyPolicyName=“none” topologyScopeName=“container”
Nov 9 15:49:52 node1 kubelet: I1109 15:49:52.951049 38730 container_manager_linux.go:314] “Initializing Topology Manager” policy=“none” scope=“container”
Nov 9 15:49:52 node1 kubelet: I1109 15:49:52.951060 38730 container_manager_linux.go:319] “Creating device plugin manager” devicePluginEnabled=true
Nov 9 15:49:52 node1 kubelet: I1109 15:49:52.951271 38730 kubelet.go:310] “Using dockershim is deprecated, please consider using a full-fledged CRI implementation”
Nov 9 15:49:52 node1 kubelet: I1109 15:49:52.951320 38730 client.go:78] “Connecting to docker on the dockerEndpoint” endpoint=“unix:///var/run/docker.sock”
Docker步步实践
目录文档:
①Docker简介
②基本概念
③安装Docker
④使用镜像:
⑤操作容器:
⑥访问仓库:
⑦数据管理:
⑧使用网络:
⑨高级网络配置:
⑩安全:
⑪底层实现:
⑫其他项目:
![](https://upload-images.
必看视频!获取2024年最新Java开发全套学习资料 备注Java
jianshu.io/upload_images/22932333-92b78c314ef233b2.image?imageMogr2/auto-orient/strip%7CimageView2/2/w/1240)
1447)(type-bilibili)(url-https://player.bilibili.com/player.html?aid=1804892953)(image-https://img-blog.csdnimg.cn/img_convert/d03a705143646f4f3f6ea879d59b2788.png)(title-必看视频!获取2024年最新Java开发全套学习资料 备注Java)]
jianshu.io/upload_images/22932333-92b78c314ef233b2.image?imageMogr2/auto-orient/strip%7CimageView2/2/w/1240)