【云原生】Kubernetes----k8s免密使用harbor私有仓库

智能运维探索者

于 2024-07-29 14:52:49 发布

阅读量765

点赞数 28

文章标签：云原生 kubernetes 容器

本文链接：https://blog.csdn.net/hy199707/article/details/140767777

版权

引言

在Kubernetes（k8s）环境中，使用私有镜像仓库如Harbor来存储和管理容器镜像是一种常见做法。Harbor是由VMware公司开源的企业级Docker Registry管理项目，支持丰富的权限控制和完善的架构设计，尤其适合大规模Docker集群部署。然而，每次Pod拉取私有镜像时都需要进行身份验证，这可能会增加系统的复杂性和运维成本。本文将介绍如何在Kubernetes中配置免密使用Harbor私有仓库的方法。

环境准备

主机名	IP地址	部署服务	服务器类型
master01	192.168.83.30	k8s集群	控制节点、ETCD节点
node01	192.168.83.40	k8s集群	工作节点
node02	192.168.83.50	k8s集群	工作节点
harbor	192.168.83.60	harbor-offline-installer-v1.2.2	harbor私有仓库

一、搭建Harbor仓库

在harbor节点上进行操作

（一）关闭防护

[root@harbor ~]# systemctl stop firewalld.service
[root@harbor ~]# systemctl disable firewalld.service
[root@harbor ~]# setenforce 0

（二）安装docker

[root@harbor ~]#yum install -y yum-utils device-mapper-persistent-data lvm2
#安装依赖包
[root@harbor ~]#yum-config-manager --add-repo https://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo
#获取阿里云镜像源
[root@harbor ~]#yum install -y docker-ce-20.10.18 docker-ce-cli-20.10.18 containerd.io
#安装Docker-CE（社区版）20.10.18版本
[root@harbor ~]#systemctl enable --now docker.service
#设置开机自启并立即启动docker服务
[root@harbor ~]#cat > /etc/docker/daemon.json <<EOF
{
  "registry-mirrors": ["https://gix2yhc1.mirror.aliyuncs.com"]
}
EOF

（三）安装docker-compose

下载地址：Releases · docker/compose · GitHub

[root@harbor data]# wget https://github.com/docker/compose/releases/download/1.25.0/docker-compose-Linux-x86_64
[root@harbor data]# ls
docker-compose-Linux-x86_64
[root@harbor data]# mv docker-compose-Linux-x86_64 /usr/bin/docker-compose
[root@harbor data]# chmod +x /usr/bin/docker-compose 
[root@harbor data]# docker-compose --version
docker-compose version 1.25.0, build 0a186604

（四）安装harbor-offline

1.获取安装包

[root@harbor data]# wget http://harbor.orientsoft.cn/harbor-1.2.2/harbor-offline-installer-v1.2.2.tgz
#下载数据包，或者在官方网站下载完毕之后，上传到服务器当中
[root@harbor data]# ls
harbor-offline-installer-v1.2.2.tgz
[root@harbor data]# tar zxvf harbor-offline-installer-v1.2.2.tgz -C /usr/local/

2.修改配置文件

[root@harbor data]# cd /usr/local/harbor/
[root@harbor harbor]# ls
common                    docker-compose.notary.yml  harbor_1_1_0_template  harbor.v1.2.2.tar.gz  LICENSE  prepare
docker-compose.clair.yml  docker-compose.yml         harbor.cfg             install.sh            NOTICE   upgrade
[root@harbor harbor]# vim harbor.cfg
  5 hostname = 192.168.83.60               #修改仓库地址为本地地址
......
 59 harbor_admin_password = Harbor12345    #harbor登录密码，可自定义
......

3.启动服务

在配置好了 harbor.cfg 之后，执行 ./prepare 命令，为 harbor 启动的容器生成一些必要的文件（环境），再执行命令 ./install.sh 以 pull 镜像并启动容器

[root@harbor harbor]# ./prepare 
Generated and saved secret to file: /data/secretkey
Generated configuration file: ./common/config/nginx/nginx.conf
Generated configuration file: ./common/config/adminserver/env
Generated configuration file: ./common/config/ui/env
Generated configuration file: ./common/config/registry/config.yml
Generated configuration file: ./common/config/db/env
Generated configuration file: ./common/config/jobservice/env
Generated configuration file: ./common/config/jobservice/app.conf
Generated configuration file: ./common/config/ui/app.conf
Generated certificate, key file: ./common/config/ui/private_key.pem, cert file: ./common/config/registry/root.crt
The configuration files are ready, please use docker-compose to start the service.
[root@harbor harbor]# ./install.sh 

[Step 0]: checking installation environment ...

Note: docker version: 20.10.18

Note: docker-compose version: 1.25.0

[Step 1]: loading Harbor images ...
Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?
[root@harbor harbor]# systemctl start docker
[root@harbor harbor]# systemctl start docker
[root@harbor harbor]# ./install.sh 

[Step 0]: checking installation environment ...

Note: docker version: 20.10.18

Note: docker-compose version: 1.25.0

[Step 1]: loading Harbor images ...
dd60b611baaa: Loading layer [==================================================>]  133.2MB/133.2MB
abf0579c40fd: Loading layer [==================================================>]  1.536kB/1.536kB
ea1fc7bed9c5: Loading layer [==================================================>]  22.48MB/22.48MB
.......

4.登录仓库验证

浏览器访问http://harbo服务器ip/

用户名：admin

密码：Harbor12345

二、设置免密登录

（一）创建新项目

登录harbor仓库web界面

创建一个新项目。点击“+项目”按钮
填写项目名称为“new-project”，点击“确定”按钮，创建新项目

（二）添加仓库地址

在每个工作节点配置连接私有仓库，而后登录仓库

//node01节点
[root@node01 ~]#cat > /etc/docker/daemon.json <<EOF
{
  "registry-mirrors": ["https://gix2yhc1.mirror.aliyuncs.com"],
  "insecure-registries": ["192.168.83.60"]
}
EOF
[root@node01 ~]#systemctl daemon-reload
[root@node01 ~]#systemctl restart docker
[root@node01 ~]#docker login -u admin -p Harbor12345 http://192.168.83.60
WARNING! Using --password via the CLI is insecure. Use --password-stdin.
WARNING! Your password will be stored unencrypted in /root/.docker/config.json.
Configure a credential helper to remove this warning. See
https://docs.docker.com/engine/reference/commandline/login/#credentials-store

Login Succeeded



//node02节点
[root@node02 ~]#cat > /etc/docker/daemon.json <<EOF
{
  "registry-mirrors": ["https://gix2yhc1.mirror.aliyuncs.com"],
  "insecure-registries": ["192.168.83.60"]
}
EOF
[root@node02 ~]#systemctl daemon-reload
[root@node02 ~]#systemctl restart docker
[root@node02 ~]#docker login -u admin -p Harbor12345 http://192.168.83.60
WARNING! Using --password via the CLI is insecure. Use --password-stdin.
WARNING! Your password will be stored unencrypted in /root/.docker/config.json.
Configure a credential helper to remove this warning. See
https://docs.docker.com/engine/reference/commandline/login/#credentials-store

Login Succeeded

（三）添加镜像

将本地镜像添加到私有仓库

在其中一个节点下载nginx:1.18.0的镜像，并打上标签之后上传到私有仓库

[root@node02 ~]#docker pull nginx:1.18.0
#下载镜像
1.18.0: Pulling from library/nginx
f7ec5a41d630: Pull complete 
0b20d28b5eb3: Pull complete 
1576642c9776: Pull complete 
c12a848bad84: Pull complete 
03f221d9cf00: Pull complete 
Digest: sha256:e90ac5331fe095cea01b121a3627174b2e33e06e83720e9a934c7b8ccc9c55a0
Status: Downloaded newer image for nginx:1.18.0
docker.io/library/nginx:1.18.0
[root@node02 ~]#docker tag nginx:1.18.0 192.168.83.60/new-project/nginx:v1
#添加镜像标签
[root@node02 ~]#docker images |grep nginx
nginx                                latest     605c77e624dd   2 years ago     141MB
192.168.83.60/new-project/nginx      v1         c2c45d506085   3 years ago     133MB
nginx                                1.18.0     c2c45d506085   3 years ago     133MB
[root@node02 ~]#docker push 192.168.83.60/new-project/nginx:v1
#上传到harbo私有仓库
The push refers to repository [192.168.83.60/new-project/nginx]
4fa6704c8474: Pushed 
4fe7d87c8e14: Pushed 
6fcbf7acaafd: Pushed 
f3fdf88f1cb7: Pushed 
7e718b9c0c8c: Pushed 
v1: digest: sha256:9b0fc8e09ae1abb0144ce57018fc1e13d23abd108540f135dc83c0ed661081cf size: 1362

在web界面查看是否上传成功

（三）创建secret资源

1.查看登录凭据

//在任意工作节点上查看登录凭据
[root@node02 ~]#cat /root/.docker/config.json | base64 -w 0  
ewoJImF1dGhzIjogewoJCSIxOTIuMTY4LjgzLjYwIjogewoJCQkiYXV0aCI6ICJZV1J0YVc0NlNHRnlZbTl5TVRJek5EVT0iCgkJfQoJfQp9
#各节点登录凭据相同
#base64 -w 0：进行 base64 加密并禁止自动换行

2.创建登录凭据资源清单

在master节点创建harbor登录凭据资源清单用于 K8S 访问 Harbor 私服拉取镜像所需要的密钥权限凭证 secret 资源

[root@master01 data]#vim harbor-secret.yaml
[root@master01 data]#cat harbor-secret.yaml
apiVersion: v1
kind: Secret
metadata:
  name: harbor-secret
data:
  .dockerconfigjson: ewoJImF1dGhzIjogewoJCSIxOTIuMTY4LjgzLjYwIjogewoJCQkiYXV0aCI6ICJZV1J0YVc0NlNHRnlZbTl5TVRJek5EVT0iCgkJfQoJfQp9			#复制粘贴上述查看的登陆凭据
type: kubernetes.io/dockerconfigjson
[root@master01 data]#kubectl apply -f harbor-secret.yaml 
secret/harbor-secret created
[root@master01 data]#kubectl get secret harbor-secret
NAME            TYPE                             DATA   AGE
harbor-secret   kubernetes.io/dockerconfigjson   1      27s

3.删除镜像

删除之前在node节点下载的nginx镜像，已经自定义标签的镜像

[root@node02 ~]#docker rmi nginx:1.18.0 
Untagged: nginx:1.18.0
Untagged: nginx@sha256:e90ac5331fe095cea01b121a3627174b2e33e06e83720e9a934c7b8ccc9c55a0
[root@node02 ~]#docker rmi 192.168.83.60/new-project/nginx:v1 
Untagged: 192.168.83.60/new-project/nginx:v1
Untagged: 192.168.83.60/new-project/nginx@sha256:9b0fc8e09ae1abb0144ce57018fc1e13d23abd108540f135dc83c0ed661081cf
Deleted: sha256:c2c45d506085d300b72a6d4b10e3dce104228080a2cf095fc38333afe237e2be
Deleted: sha256:43d6c481a041dbcc1d8ea9c565b1b692bcb28da3414683c316703c669c012ebc
Deleted: sha256:defebc732c194dd0b5b39e20c4d014896ce120207f5dfdb41ed6696b0e8224d6
Deleted: sha256:4ea0f2550407442f808812429981c0b62d8dd6a531db8a412640293a1faf8f3c
Deleted: sha256:778ca58cf39b8fa0776ade88562750a035a24ec5afb7dc4ab2aa892b2c09769d
Deleted: sha256:7e718b9c0c8c2e6420fe9c4d1d551088e314fe923dce4b2caf75891d82fb227d

4.创建ngiinx

指定使用harbor仓库的镜像资源去创建pod

4.1 定义yaml文件

[root@master01 data]#vim nginx.yaml
[root@master01 data]#cat nginx.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx
spec:
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
      labels:
        app: nginx
    spec:
      imagePullSecrets:      #添加K8S访问Harbor私服拉取镜像所需要的secret资源选项
      - name: harbor-secret  #指定 secret 资源名称
      containers:
      - name: nginx
        image: 192.168.83.60/new-project/nginx:v1 #指定harbor中的镜像名
        ports:
        - containerPort: 80
---
apiVersion: v1
kind: Service
metadata:
  name: nginx
spec:
  type: NodePort
  ports:
  - port: 80
    targetPort: 80
    nodePort: 31111
  selector:
    app: nginx

4.2 创建资源

[root@master01 data]#kubectl apply -f nginx.yaml 
deployment.apps/nginx created
service/nginx created
[root@master01 data]#kubectl get pod,svc
NAME                         READY   STATUS    RESTARTS   AGE
pod/nginx-5b658db7f8-kldfk   1/1     Running   0          3s
pod/nginx-5b658db7f8-vd967   1/1     Running   0          3s

NAME                 TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)        AGE
service/kubernetes   ClusterIP   10.96.0.1       <none>        443/TCP        73d
service/nginx        NodePort    10.96.135.253   <none>        80:31111/TCP   3s

[root@master01 data]#curl 192.168.83.30:31111 -I
HTTP/1.1 200 OK
Server: nginx/1.18.0
Date: Mon, 29 Jul 2024 06:42:32 GMT
Content-Type: text/html
Content-Length: 612
Last-Modified: Tue, 21 Apr 2020 14:09:01 GMT
Connection: keep-alive
ETag: "5e9efe7d-264"
Accept-Ranges: bytes

4.3 查看镜像下载地址

[root@master01 data]#kubectl describe pod nginx-5b658db7f8-kldfk
Events:
  Type    Reason   Age   From     Message
  ----    ------   ----  ----     -------
  Normal  Pulling  22s   kubelet  Pulling image "192.168.83.60/new-project/nginx:v1"
  Normal  Pulled   13s   kubelet  Successfully pulled image "192.168.83.60/new-project/nginx:v1" in 9.569090599s
  Normal  Created  13s   kubelet  Created container nginx
  Normal  Started  12s   kubelet  Started container nginx
#可以发现镜像时从harbor下载的

刷新harbor页面，可以看到镜像的下载次数增加了

智能运维探索者

关注

28
点赞
踩
27

收藏

觉得还不错? 一键收藏
0
评论
【云原生】Kubernetes----k8s免密使用harbor私有仓库

在Kubernetes（k8s）环境中，使用私有镜像仓库如Harbor来存储和管理容器镜像是一种常见做法。Harbor是由VMware公司开源的企业级Docker Registry管理项目，支持丰富的权限控制和完善的架构设计，尤其适合大规模Docker集群部署。然而，每次Pod拉取私有镜像时都需要进行身份验证，这可能会增加系统的复杂性和运维成本。本文将介绍如何在Kubernetes中配置免密使用Harbor私有仓库的方法。
复制链接

扫一扫