Harbor仓库添加到k8s集群并提供服务

最新推荐文章于 2024-08-17 18:46:11 发布

博博的博客

最新推荐文章于 2024-08-17 18:46:11 发布

阅读量1.1k

点赞数

文章标签： k8s

本文链接：https://blog.csdn.net/weixin_48191138/article/details/110200692

版权

【harbor仓库部署】港口
Harbor是一个开放源代码注册中心，可通过策略和基于角色的访问控制来保护工件，确保扫描图像并使其免受漏洞侵害，并将图像签名为受信任的。Harbor是CNCF毕业的项目，可提供合规性，性能和互操作性，以帮助您跨Kubernetes和Docker等云原生计算平台持续，安全地管理工件。

本实验基于k8s多节点集群部署！上篇文章已做论述！

【环境】
主机名 IP地址部署的服务
nginx1 20.0.0.19 nginx、keepalived
nginx2 20.0.0.20 nginx、keepalived
master 20.0.0.15 apiserver、scheduler、controller-manager、etcd
master2 20.0.0.18 apiserver、scheduler、controller-manager
node1 20.0.0.16 kubelet、kube-proxy、docker、flannel、etcd
node2 20.0.0.17 kubelet、kube-proxy、docker、flannel、etcd
VIP 20.0.0.200
Harbor 20.0.0.21 docker、docker-compose、harbor

一、前期环境

1.1 修改主机名

hostnamectl set-hostname harbor
su

1.2 关闭防火墙、核心防护

sed -i 's/enforcing/disabled/' /etc/selinux/config 
setenforce 0
systemctl stop firewalld && systemctl disable firewalld

1.3 上传docker-compose和harbor软件包

'上传docker-compose和harbor软件包到/root目录下'
[root@harbor ~]# mv docker-compose /usr/bin/ && chmod +x  /usr/bin/docker-compose

二、安装docker

2.1 安装依赖包

rm -rf /var/run/yum.pid
yum install -y yum-utils device-mapper-persistent-data lvm2
'//yum-utils：提供yum-config-manager
//device-mapper：存储驱动程序需要device-mapper-persistent-data和lvm2
//device-mapper：是Linux2.6内核中支持逻辑卷管理的通用设备映射机制，为实现用于存储资源管理的块设备驱动提供了一个高度模块化的内核架构。'

2.2 设置阿里云镜像源

yum-config-manager --add-repo https://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo

2.3 安装docker-ce

yum -y install docker-ce
systemctl start docker
systemctl enable docker

2.4 镜像加速

'寻找镜像加速器方法：
登录阿里云官方网站-----》用自己的账户登录到控制台-----》导航栏搜索容器镜像服务，开通-----》选择镜像加速器------》下面的一串代码就是自己需要的。'
tee /etc/docker/daemon.json <<-'EOF'
{
  "registry-mirrors": ["https://2lb8t07e.mirror.aliyuncs.com"]
}
EOF
systemctl daemon-reload
systemctl restart docker

2.5 网络优化

echo "net.ipv4.ip_forward=1" >>/etc/sysctl.conf
sysctl -p
systemctl restart network
systemctl restart docker

三、部署harbor

3.1 安装harbor

[root@harbor ~]# tar xf harbor-offline-installer-v1.2.2.tgz -C /usr/local/
[root@harbor ~]# cd /usr/local/harbor/
[root@harbor harbor]# ls
common                     docker-compose.yml     harbor.v1.2.2.tar.gz  NOTICE
docker-compose.clair.yml   harbor_1_1_0_template  install.sh            prepare
docker-compose.notary.yml  harbor.cfg             LICENSE               upgrade

[root@harbor harbor]# vim harbor.cfg
hostname = 20.0.0.21
[root@harbor harbor]# sh install.sh	
'//若报错，则需要修改docker-compose权限'
[root@harbor harbor]# chmod +x /usr/bin/docker-compose
[root@harbor harbor]# sh install.sh

3.2 网页登陆

20.0.0.21
账号：admin 初始密码：Harbor12345

四、将harbor与k8s中的docker关联

4.1 两个node节点修改daemon.json（仅展示node1）

[root@node1 ~]# vim /etc/docker/daemon.json
{
  "registry-mirrors": ["https://2lb8t07e.mirror.aliyuncs.com"],
  "insecure-registries": ["20.0.0.21"]
}
'//使用docker pull nginx时，默认拉取的是docker共有仓库镜像
docker pull 20.0.0.21/public-harbor/nginx 拉取的是harbor仓库中public-harbor项目中的镜像'

[root@node1 ~]# systemctl restart docker

4.2 查看容器

[root@node1 ~]# docker ps -a

CONTAINER ID        IMAGE                                                                 COMMAND                  CREATED             STATUS                     PORTS               NAMES
1013bb053bac        nginx                                                                 "/docker-entrypoint.…"   2 minutes ago       Up 2 minutes                                   k8s_nginx_nginx-dbddb74b8-84xgw_default_21364fc8-2b3a-11eb-9e52-000c2980391a_1
c0b8e3a6d024        784cf2722f44                                                          "/dashboard --insecu…"   2 minutes ago       Up 2 minutes                                   k8s_kubernetes-dashboard_kubernetes-dashboard-7dffbccd68-hqvvp_kube-system_2f696e0c-2bcc-11eb-9e52-000c2980391a_1
20312f4e4233        registry.cn-hangzhou.aliyuncs.com/google-containers/pause-amd64:3.0   "/pause"                 2 minutes ago       Up 2 minutes                                   k8s_POD_kubernetes-dashboard-7dffbccd68-hqvvp_kube-system_2f696e0c-2bcc-11eb-9e52-000c2980391a_1
26f41402dd52        registry.cn-hangzhou.aliyuncs.com/google-containers/pause-amd64:3.0   "/pause"                 2 minutes ago       Up 2 minutes                                   k8s_POD_nginx-dbddb74b8-84xgw_default_21364fc8-2b3a-11eb-9e52-000c2980391a_1
c2edd0cb171e        siriuszg/kubernetes-dashboard-amd64                                   "/dashboard --insecu…"   About an hour ago   Exited (2) 2 minutes ago                       k8s_kubernetes-dashboard_kubernetes-dashboard-7dffbccd68-hqvvp_kube-system_2f696e0c-2bcc-11eb-9e52-000c2980391a_0
42a34795587b        registry.cn-hangzhou.aliyuncs.com/google-containers/pause-amd64:3.0   "/pause"                 About an hour ago   Exited (0) 2 minutes ago                       k8s_POD_kubernetes-dashboard-7dffbccd68-hqvvp_kube-system_2f696e0c-2bcc-11eb-9e52-000c2980391a_0
0ce1044b01e1        nginx                                                                 "/docker-entrypoint.…"   6 hours ago         Exited (0) 2 minutes ago                       k8s_nginx_nginx-dbddb74b8-84xgw_default_21364fc8-2b3a-11eb-9e52-000c2980391a_0
6b653333c95c        registry.cn-hangzhou.aliyuncs.com/google-containers/pause-amd64:3.0   "/pause"                 6 hours ago         Exited (0) 2 minutes ago                       k8s_POD_nginx-dbddb74b8-84xgw_default_21364fc8-2b3a-11eb-9e52-000c2980391a_0
198bcc12c6ac        centos:7                                                              "/bin/bash"              23 hours ago        Exited (0) 23 hours ago                        priceless_mayer

'//其中有四个业务容器因为重启服务正常退出，但是新出现4个up的容器，这是因为k8s为了保持pod的正常运转，会自动根据副本集创建新容器,所以重启docker不会影响业务，因为k8s会自动重启'

五、上传镜像到harbor

注意：在使用harbor下载镜像创建资源的时候，要保证node处于harbor登陆状态

5.1 两个节点都登录（仅展示node1操作）

[root@node1 ~]# docker login 20.0.0.21
Username: admin
Password: 	//密码Harbor12345
WARNING! Your password will be stored unencrypted in /root/.docker/config.json.
Configure a credential helper to remove this warning. See
https://docs.docker.com/engine/reference/commandline/login/#credentials-store

Login Succeeded

5.2 拉取一个公网仓库的tomcat以供测试

[root@node1 ~]# docker pull tomcat
[root@node1 ~]# docker images | grep tomcat
tomcat                                                            latest              e0bd8b34b4ea        2 days ago          649MB

5.3 上传镜像打标签

[root@node1 ~]# docker tag tomcat 20.0.0.21/object/tomcat-v1
[root@node1 ~]# docker push 20.0.0.21/object/tomcat-v1

5.4 到Harbor网站刷新查看，镜像上传成功

5.5 查看本地镜像

[root@node1 ~]# docker images | grep tomcat
20.0.0.21/object/tomcat-v1                                 latest              e0bd8b34b4ea        2 days ago          649MB
tomcat                                                            latest              e0bd8b34b4ea        2 days ago          649MB

5.6 把本地打标签的删掉，然后从harbor下载测试

[root@node1 ~]# docker rmi 20.0.0.21/object/tomcat-v1
[root@node1 ~]# docker images | grep tomcat

5.7 本地从Harbor仓库拉取镜像

[root@node1 ~]# docker pull 20.0.0.21/object/tomcat-v1

[root@node1 ~]# docker images | grep tomcat
20.0.0.21/object/tomcat-v1                                        latest              e0bd8b34b4ea        7 days ago          649MB
tomcat                                                            latest              e0bd8b34b4ea        7 days ago          649MB

六、通过K8s编辑yaml文件下载Harbor仓库的镜像

6.1 测试常规的kubectl run（master上）

docker pull tomcat:8.0.52

[root@master demo]#vim tomcat-deployment.yaml
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: my-tomcat
spec:
  replicas: 2
  template:
    metadata:
      labels:
        app: my-tomcat
    spec:
      containers:
      - name: my-tomcat
        image: docker.io/tomcat:8.0.52
        ports:
        - containerPort: 80
---
apiVersion: v1
kind: Service
metadata:
  name: my-tomcat
spec:
  type: NodePort
  ports:
  - port: 8080
    targetPort: 8080
    nodePort: 31111
  selector:
    app: my-tomcat


[root@master demo]# kubectl create -f tomcat-deployment.yaml

[root@master demo]#  kubectl get pods,deploy,svc
NAME                            READY   STATUS    RESTARTS   AGE
pod/my-tomcat-57667b9d9-8lnnh   1/1     Running   0          66s
pod/my-tomcat-57667b9d9-n4c8k   1/1     Running   0          66s
pod/nginx-dbddb74b8-nzz2p       1/1     Running   1          7h19m

NAME                              DESIRED   CURRENT   UP-TO-DATE   AVAILABLE   AGE
deployment.extensions/my-tomcat   2         2         2            2           66s
deployment.extensions/nginx       1         1         1            1           7h19m

NAME                 TYPE        CLUSTER-IP   EXTERNAL-IP   PORT(S)          AGE
service/kubernetes   ClusterIP   10.0.0.1     <none>        443/TCP          7h42m
service/my-tomcat    NodePort    10.0.0.247   <none>        8080:31111/TCP   66s


[root@master demo]# kubectl get pods
NAME                        READY   STATUS    RESTARTS   AGE
my-tomcat-57667b9d9-8lnnh   1/1     Running   0          116s
my-tomcat-57667b9d9-n4c8k   1/1     Running   0          116s
nginx-dbddb74b8-nzz2p       1/1     Running   1          7h20m

'//如果遇到处于Terminating状态的无法删除的资源如何处理'
'//这种情况下可以使用强制删除命令：'
kubectl delete pod [pod name] --force --grace-period=0 -n [namespace]
kubectl delete pod my-tomcat-57667b9d9-n4c8k --force --grace-period=0 -n default

6.2 node1上操作（之前登陆过harbor仓库的节点）

//镜像打标签
[root@node2 ~]# docker tag tomcat:8.0.52 20.0.0.21/object/tomcat-v1

6.3 查看登陆凭据

[root@node2 ~]# cat .docker/config.json |base64 -w 0
ewoJImF1dGhzIjogewoJCSIyMC4wLjAuMjEiOiB7CgkJCSJhdXRoIjogIllXUnRhVzQ2U0dGeVltOXlNVEl6TkRVPSIKCQl9Cgl9LAoJIkh0dHBIZWFkZXJzIjogewoJCSJVc2VyLUFnZW50IjogIkRvY2tlci1DbGllbnQvMTkuMDMuMTMgKGxpbnV4KSIKCX0KfQ==

[root@master demo]# vim registry-pull-secret.yaml
apiVersion: v1
kind: Secret
metadata:
  name: registry-pull-secret
data:
  .dockerconfigjson: ewoJImF1dGhzIjogewoJCSIyMC4wLjAuMjEiOiB7CgkJCSJhdXRoIjogIllXUnRhVzQ2U0dGeVltOXlNVEl6TkRVPSIKCQl9Cgl9LAoJIkh0dHBIZWFkZXJzIjogewoJCSJVc2VyLUFnZW50IjogIkRvY2tlci1DbGllbnQvMTkuMDMuMTMgKGxpbnV4KSIKCX0KfQ==
type: kubernetes.io/dockerconfigjson

6.4 创建secret资源

[root@master demo]# kubectl create -f registry-pull-secret.yaml
secret/registry-pull-secret created

6.5 查看secret资源

[root@master demo]# kubectl get secret
NAME                   TYPE                                  DATA   AGE
default-token-tggw6    kubernetes.io/service-account-token   3      7h48m
registry-pull-secret   kubernetes.io/dockerconfigjson        1      22s

6.6 创建资源从harbor中下载镜像

[root@master demo]# vim tomcat-deployment.yaml
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: my-tomcat
spec:
  replicas: 2
  template:
    metadata:
      labels:
        app: my-tomcat
    spec:
      imagePullSecrets:
      - name: registry-pull-secret
      containers:
      - name: my-tomcat
        image: 20.0.0.21/object/tomcat-v2
        ports:
        - containerPort: 80
---
apiVersion: v1
kind: Service
metadata:
  name: my-tomcat
spec:
  type: NodePort
  ports:
  - port: 8080
    targetPort: 8080
    nodePort: 31111
  selector:
    app: my-tomcat


[root@master demo]# kubectl create -f tomcat-deployment.yaml
'//私有仓库中的镜像被下载了2次'