解决minikube无法下载容器镜像的问题(202408)

问题描述

书接上回基于Vagrant和minikube创建Kubernetes运行环境.在测试运行容器时发现容器镜像拉取不下来,运行kubectl get pods状态为ImagePullBackOff：

vagrant@ubuntu-jammy:~$ kubectl run ngx --image=nginx:alpine
vagrant@ubuntu-jammy:~$ kubectl get pods
NAME   READY   STATUS             RESTARTS   AGE
ngx    0/1     ImagePullBackOff   0          7m48s

运行kubectl describe pods确定应该是容器镜像源配置的问题：

尝试了重启minikube并指定--image-mirror-country和--image-repository配置但并没有效果:

vagrant@ubuntu-jammy:~$ minikube stop
vagrant@ubuntu-jammy:~$ minikube start --image-mirror-country='cn' --image-repository='registry.cn-hangzhou.aliyuncs.com/google_containers' --kubernetes-version=v1.23.3

通过查阅文章，发现可通过minikube ssh进入minikube内部，更改docker镜像源配置，然后手动下载需要的镜像.

方案一：直接更改daemon.json

通过minikube ssh进入minikube内部，在其/etc/docker/daemon.json文件追加registry-mirrors配置

vagrant@ubuntu-jammy:~$ minikube ssh
docker@minikube:~$ sudo vim /etc/docker/daemon.json

对/etc/docker/daemon.json追加registry-mirrors配置:

{
        "exec-opts":["native.cgroupdriver=systemd"],
        "registry-mirrors": [
                "https://docker.m.daocloud.io", #2024年8月可用
                "https://hub-mirror.c.163.com",
                "https://docker.mirrors.ustc.edu.cn"

        ],
        "log-driver":"json-file",
        "log-opts":{"max-size":"100m"},
        "storage-driver":"overlay2"
}

重启docker并在minikube内部拉取需要的容器,退回宿主机发现容器已经启动成功：

docker@minikube:~$ sudo systemctl daemon-reload
docker@minikube:~$ sudo systemctl restart docker
docker@minikube:/etc/docker$ docker pull nginx:alpine
alpine: Pulling from library/nginx
Digest: sha256:208b70eefac13ee9be00e486f79c695b15cef861c680527171a27d253d834be9
Status: Image is up to date for nginx:alpine
docker.io/library/nginx:alpine
docker@minikube:~$ exit
logout
vagrant@ubuntu-jammy:~$ kubectl get pods
NAME   READY   STATUS    RESTARTS   AGE
ngx    1/1     Running   0          20m

此方案存在的问题是再次重启minikube时，/etc/docker/daemon.json会被覆盖，追加的registry-mirrors配置会失效，即每次重启minikube都需要重新配置一次.

方案二：通过Trick解决daemon.json被覆写的问题

本方案参考了这个github issue,所谓的Trick就是不用daemon.json,曲线救国采用可以在minikube里固化的配置文件。

同样minikube ssh,只不过这次重新创建一个/etc/docker/alternate-daemon.json文件:

vagrant@ubuntu-jammy:~$ minikube ssh
docker@minikube:~$ vim /etc/docker/alternate-daemon.json

对alternate-daemon.json追加方案一daemon.json一样的配置，然后修改docker服务启动配置文件：

docker@minikube:~$ sudo mkdir -p /etc/systemd/system/docker.service.d
docker@minikube:~$ sudo vim /etc/systemd/system/docker.service.d/docker.conf

加入以下配置：

[Service]
ExecStart=
ExecStart=/usr/bin/dockerd --config-file=/etc/docker/alternate-daemon.json
TimeoutStartSec=5min

然后退出ssh，重启minikube即可：

docker@minikube:~$ exit
vagrant@ubuntu-jammy:~$ minikube stop
vagrant@ubuntu-jammy:~$ minikube start
# 运行另一个版本的nginx
vagrant@ubuntu-jammy:~$ kubectl run nginx --image=nginx:latest 
vagrant@ubuntu-jammy:~$ kubectl get pods
NAME    READY   STATUS    RESTARTS      AGE
nginx   1/1     Running   0             18m
ngx     1/1     Running   1 (57m ago)   110m
redis   1/1     Running   1 (57m ago)   84m
# 通过kubectl describe可查看到拉取运行过程
vagrant@ubuntu-jammy:~$ kubectl describe pods nginx
Name:         nginx
Namespace:    default
Priority:     0
Node:         minikube/192.168.49.2
Start Time:   Tue, 13 Aug 2024 07:43:07 +0000
Labels:       run=nginx
Annotations:  <none>
Status:       Running
IP:           172.17.0.6
IPs:
  IP:  172.17.0.6
Containers:
  nginx:
    Container ID:   docker://a109378ac93828567533c647ef1f9ff104f110dc2be2e42f242d9e1cd13abff3
    Image:          nginx:latest
    Image ID:       docker-pullable://nginx@sha256:81dbf3ecf1120d90ace4593096bb650a2d11d4a11d24c3aaea5bef7c9bbb2c88
    Port:           <none>
    Host Port:      <none>
    State:          Running
      Started:      Tue, 13 Aug 2024 07:44:32 +0000
    Ready:          True
    Restart Count:  0
    Environment:    <none>
    Mounts:
      /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-6bl57 (ro)
Conditions:
  Type              Status
  Initialized       True
  Ready             True
  ContainersReady   True
  PodScheduled      True
Volumes:
  kube-api-access-6bl57:
    Type:                    Projected (a volume that contains injected data from multiple sources)
    TokenExpirationSeconds:  3607
    ConfigMapName:           kube-root-ca.crt
    ConfigMapOptional:       <nil>
    DownwardAPI:             true
QoS Class:                   BestEffort
Node-Selectors:              <none>
Tolerations:                 node.kubernetes.io/not-ready:NoExecute op=Exists for 300s
                             node.kubernetes.io/unreachable:NoExecute op=Exists for 300s
Events:
  Type    Reason     Age   From               Message
  ----    ------     ----  ----               -------
  Normal  Scheduled  19m   default-scheduler  Successfully assigned default/nginx to minikube
  Normal  Pulling    19m   kubelet            Pulling image "nginx:latest"
  Normal  Pulled     18m   kubelet            Successfully pulled image "nginx:latest" in 1m23.386634371s
  Normal  Created    18m   kubelet            Created container nginx
  Normal  Started    18m   kubelet            Started container nginx

Tips:也可以在minikube内部通过doker info查看镜像源配置：

vagrant@ubuntu-jammy:~$ minikube ssh
docker@minikube:~$ docker info
Client: Docker Engine - Community
 Version:    26.1.1
 Context:    default
 Debug Mode: false
 Plugins:
  buildx: Docker Buildx (Docker Inc.)
    Version:  v0.14.0
    Path:     /usr/libexec/docker/cli-plugins/docker-buildx

Server:
 Containers: 41
  Running: 22
  Paused: 0
  Stopped: 19
 Images: 12
 Server Version: 26.1.1
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Using metacopy: false
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: systemd
 Cgroup Version: 2
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local splunk syslog
 Swarm: inactive
 Runtimes: runc io.containerd.runc.v2
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: e377cd56a71523140ca6ae87e30244719194a521
 runc version: v1.1.12-0-g51d5e94
 init version: de40ad0
 Security Options:
  seccomp
   Profile: builtin
  cgroupns
 Kernel Version: 5.15.0-118-generic
 Operating System: Ubuntu 22.04.4 LTS
 OSType: linux
 Architecture: x86_64
 CPUs: 2
 Total Memory: 7.752GiB
 Name: minikube
 ID: c3027c53-e0f7-437c-9d94-4fd01d161dc3
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 No Proxy: control-plane.minikube.internal
 Experimental: false
 Insecure Registries:
  127.0.0.0/8
 # 这里可以看到镜像源配置成功
 #-----------------------------------------
 Registry Mirrors:
  https://docker.m.daocloud.io/
  https://hub-mirror.c.163.com/
  https://docker.mirrors.ustc.edu.cn/
 # ----------------------------------------
 Live Restore Enabled: false