Kubernetes(minikube) 私有 registry 使用详解

最新推荐文章于 2024-02-18 17:31:58 发布

netkiller-

最新推荐文章于 2024-02-18 17:31:58 发布

阅读量766

点赞数

分类专栏：多维度架构文章标签： kubernetes docker

本文链接：https://blog.csdn.net/u010604770/article/details/108960061

版权

多维度架构专栏收录该内容

5 篇文章 4 订阅

订阅专栏

Kubernetes(minikube) 私有 registry 使用详解

Minikube 访问私有库有几种形式：

registry 安装在宿主主机Docker下
registry 安装在minikube下。

Minikube 启动可以选择多种驱动，有些驱动会启动一个虚拟机。因为127.0.0.1是虚拟机的本地地址，所以宿主主机的 127.0.0.1:5000 无法给 minikube 提供 registry 服务。

docker run -d -p 5000:5000 --name registry registry:latest

iMac:~ neo$ minikube start --memory 2048mb --cpus 2 \
--cache-images=true \
--driver=docker \
--image-mirror-country=cn \
--insecure-registry='127.0.0.1:5000' \
--registry-mirror="https://registry.docker-cn.com,https://docker.mirrors.ustc.edu.cn" \
--service-cluster-ip-range='10.10.0.0/24'

当 --driver=docker 的时候，Kubernetes 将会使用宿主主机的 Docker 安装镜像，这时 Kubernetes 与 registry 共同使用一个 Docker，他们就能相互访问。

当 --driver=hyperkit 时就复杂了，无法访问宿主主机的Docker，可以访问宿主主机IP地址。

docker run -d -p 0.0.0.0:5000:5000 --name registry registry:latest

iMac:~ neo$ minikube ssh
                         _             _            
            _         _ ( )           ( )           
  ___ ___  (_)  ___  (_)| |/')  _   _ | |_      __  
/' _ ` _ `\| |/' _ `\| || , <  ( ) ( )| '_`\  /'__`\
| ( ) ( ) || || ( ) || || |\`\ | (_) || |_) )(  ___/
(_) (_) (_)(_)(_) (_)(_)(_) (_)`\___/'(_,__/'`\____)

(failed reverse-i-search)`curl': ^C
$ exit
logout
ssh: Process exited with status 130
iMac:~ neo$ minikube ssh
                         _             _            
            _         _ ( )           ( )           
  ___ ___  (_)  ___  (_)| |/')  _   _ | |_      __  
/' _ ` _ `\| |/' _ `\| || , <  ( ) ( )| '_`\  /'__`\
| ( ) ( ) || || ( ) || || |\`\ | (_) || |_) )(  ___/
(_) (_) (_)(_)(_) (_)(_)(_) (_)`\___/'(_,__/'`\____)

$ curl http://192.168.3.85:5000/v2/
{}

返回 {} 标示可以连接，但是 kubernetes 会提示必须使用 https，重启 minikube 加入 --insecure-registry='192.168.3.85:5000' 配置项。

iMac:~ neo$ minikube start --memory 2048mb --cpus 2 \
--cache-images=true \
--driver=hyperkit \
--image-mirror-country=cn \
--insecure-registry='192.168.3.85:5000' \
--registry-mirror="https://registry.docker-cn.com,https://docker.mirrors.ustc.edu.cn" \
--service-cluster-ip-range='10.10.0.0/24'

注意：修改 --insecure-registry= 配置项目，需要 minikube delete 否责不生效。

iMac:~ neo$ minikube ssh
                         _             _            
            _         _ ( )           ( )           
  ___ ___  (_)  ___  (_)| |/')  _   _ | |_      __  
/' _ ` _ `\| |/' _ `\| || , <  ( ) ( )| '_`\  /'__`\
| ( ) ( ) || || ( ) || || |\`\ | (_) || |_) )(  ___/
(_) (_) (_)(_)(_) (_)(_)(_) (_)`\___/'(_,__/'`\____)

$ ps axww | grep dockerd
 1914 ?        Ssl    0:29 /usr/bin/dockerd -H tcp://0.0.0.0:2376 -H unix:///var/run/docker.sock --default-ulimit=nofile=1048576:1048576 --tlsverify --tlscacert /etc/docker/ca.pem --tlscert /etc/docker/server.pem --tlskey /etc/docker/server-key.pem --label provider=hyperkit --insecure-registry 10.96.0.0/12 --insecure-registry 192.168.3.85:5000 --registry-mirror https://registry.docker-cn.com --registry-mirror https://docker.mirrors.ustc.edu.cn
 3922 pts/0    S+     0:00 grep dockerd

$ docker pull 192.168.3.85:5000/netkiller/welcome
Using default tag: latest
latest: Pulling from netkiller/welcome
79701ada7495: Pull complete 
47708145bbc5: Pull complete 
4594d040b84d: Pull complete 
c42c2fb542d9: Pull complete 
Digest: sha256:9ce915df06c6fc1737e17e0ec4a56dd424f54bf2900f5a8dbac2c4b58b25835e
Status: Downloaded newer image for 192.168.3.85:5000/netkiller/welcome:latest
192.168.3.85:5000/netkiller/welcome:latest

设置Docker环境

iMac:kubernetes neo$ eval $(minikube docker-env) 

iMac:kubernetes neo$ docker pull 192.168.3.85:5000/netkiller/welcome
Using default tag: latest
latest: Pulling from netkiller/welcome
Digest: sha256:9ce915df06c6fc1737e17e0ec4a56dd424f54bf2900f5a8dbac2c4b58b25835e
Status: Image is up to date for 192.168.3.85:5000/netkiller/welcome:latest
192.168.3.85:5000/netkiller/welcome:latest

测试

iMac:kubernetes neo$ kubectl create deployment welcome --image=192.168.3.85:5000/netkiller/welcome:latest
deployment.apps/welcome created

iMac:kubernetes neo$ kubectl expose deployment welcome --port=8080 --target-port=8080 --type=NodePort
service/welcome exposed

iMac:kubernetes neo$ minikube service welcome --url
http://192.168.64.7:31194

iMac:kubernetes neo$ curl http://192.168.64.7:31194
Hello world!

另一种情况是使用 kubernetes 部署 registry 例如

kubectl create deployment registry --image=registry:latest
kubectl expose deployment registry --port=5000 --target-port=5000 --type=NodePort

部署后

iMac:~ neo$ minikube service registry --url
http://192.168.64.6:32070

测试

iMac:~ neo$ curl http://192.168.64.6:32070/v2/
{}

重启 minikube

iMac:~ neo$ minikube stop
✋  Stopping node "minikube"  ...
   1 nodes stopped.


iMac:~ neo$ minikube start --memory 2048mb --cpus 2 --cache-images=true --driver=hyperkit --insecure-registry='127.0.0.1:5000,192.168.64.6:32070' --registry-mirror="https://registry.docker-cn.com,https://docker.mirrors.ustc.edu.cn" --service-cluster-ip-range='10.10.0.0/24'
   Darwin 10.13.6 上的 minikube v1.13.1
✨  根据现有的配置文件使用 hyperkit 驱动程序
   Starting control plane node minikube in cluster minikube
   Restarting existing hyperkit VM for "minikube" ...
   正在 Docker 19.03.12 中准备 Kubernetes v1.19.2…
   Verifying Kubernetes components...
   Enabled addons: dashboard, default-storageclass, storage-provisioner
   Done! kubectl is now configured to use "minikube" by default

推送镜像

iMac:kubernetes neo$ docker push 192.168.64.6:32070/busybox:latest
The push refers to repository [192.168.64.6:32070/busybox]
Get https://192.168.64.6:32070/v2/: http: server gave HTTP response to HTTPS client

悲剧了由于 --insecure-registry='192.168.64.6:32070' 必须 minikube delete 所以这种方式失败了。我又突发奇想将IP地址启动的时候加入进去，但是端口号是Service 随机产生，所以我用 CIDR

iMac:~ neo$ minikube start --memory 2048mb --cpus 2 --cache-images=true --driver=hyperkit --image-mirror-country=cn --insecure-registry="127.0.0.1:5000,192.168.3.85:5000,192.168.64.0/24" --registry-mirror="https://registry.docker-cn.com,https://docker.mirrors.ustc.edu.cn" --service-cluster-ip-range='10.10.0.0/24'
   Darwin 10.13.6 上的 minikube v1.13.1
✨  根据用户配置使用 hyperkit 驱动程序
✅  正在使用镜像存储库 registry.cn-hangzhou.aliyuncs.com/google_containers
   Starting control plane node minikube in cluster minikube
   Creating hyperkit VM (CPUs=2, Memory=2048MB, Disk=20000MB) ...
   正在 Docker 19.03.12 中准备 Kubernetes v1.19.2…
   Verifying Kubernetes components...
   Enabled addons: default-storageclass, storage-provisioner
   Done! kubectl is now configured to use "minikube" by default

再试试,一切OK


iMac:~ neo$ minikube service list
|----------------------|---------------------------|---------------|---------------------------|
|      NAMESPACE       |           NAME            |  TARGET PORT  |            URL            |
|----------------------|---------------------------|---------------|---------------------------|
| default              | kubernetes                | No node port  |
| default              | mysql                     | No node port  |
| default              | redis                     | redis/6379    | http://192.168.64.8:30290 |
| default              | registry                  | registry/5000 | http://192.168.64.8:32050 |
| default              | welcome                   |          8080 | http://192.168.64.8:32663 |
| kube-system          | kube-dns                  | No node port  |
| kubernetes-dashboard | dashboard-metrics-scraper | No node port  |
| kubernetes-dashboard | kubernetes-dashboard      | No node port  |
|----------------------|---------------------------|---------------|---------------------------|

iMac:~ neo$ kubectl create deployment welcome --image=192.168.64.8:32050/netkiller/welcome:latest
deployment.apps/welcome created

iMac:~ neo$ kubectl expose deployment welcome --port=8080 --target-port=8080 --type=NodePort
service/welcome exposed

iMac:~ neo$ curl http://192.168.64.8:32663/
Hello world! 

iMac:~ neo$ curl http://192.168.64.8:32663/address
Address 172.17.0.7, Hostname welcome-784897f9c6-5vx5q

NodePort 端口是给外部使用的，容器内部还可以使用 Endpoints 地址

iMac:kubernetes neo$ kubectl describe service/registry
Name:                     registry
Namespace:                default
Labels:                   app=registry
Annotations:              <none>
Selector:                 app=registry
Type:                     NodePort
IP:                       10.10.0.188
Port:                     registry  5000/TCP
TargetPort:               5000/TCP
NodePort:                 registry  32050/TCP
Endpoints:                172.17.0.6:5000
Session Affinity:         None
External Traffic Policy:  Cluster
Events:                   <none>

由于 172.17.0.6:5000 没有事先加入到 --insecure-registry 所以会提示

Failed to pull image "172.17.0.6:5000/netkiller/welcome:latest": rpc error: code = Unknown desc = Error response from daemon: Get https://172.17.0.6:5000/v2/: http: server gave HTTP response to HTTPS client

最后一种方案是 minikube 自带的 registry 插件，但是我始终未成功

neo@MacBook-Pro-Neo ~ % minikube addons enable registry
  Verifying registry addon...

❌ Exiting due to MK_ENABLE: run callbacks: running callbacks: [verifying registry addon pods : timed out waiting for the condition: timed out waiting for the condition]

  If the above advice does not help, please let us know:
  https://github.com/kubernetes/minikube/issues/new/choose

已将问题反馈给官方

minikube addons enable registry 出错 · Issue #9378 · kubernetes/minikubegithub.com

netkiller-

关注

0
点赞
踩
1

收藏

觉得还不错? 一键收藏
打赏
0
评论
Kubernetes(minikube) 私有 registry 使用详解

Kubernetes(minikube) 私有 registry 使用详解Minikube 访问私有库有几种形式：registry 安装在宿主主机Docker下 registry 安装在minikube下。Minikube 启动可以选择多种驱动，有些驱动会启动一个虚拟机。因为127.0.0.1是虚拟机的本地地址，所以宿主主机的 127.0.0.1:5000 无法给 minikube 提供 registry 服务。docker run -d -p 5000:5000 --name reg.
复制链接

扫一扫