【云原生Docker】12-Docker Harbor企业级镜像管理
文章目录
前言
上一章节我们介绍了Docker的官方的镜像仓库–registry,由于官方的仓库太过简单,因此并没有广泛应用在企业的实际生产环境中,这章节我们将着重介绍Harbor企业级镜像管理。
Harbor简介
简介
- Docker registry的一些缺陷:
- 缺少认证机制,任何人都可以随意拉去和上传镜像,安全性缺少;
- 缺乏镜像管理机制,镜像可以push不能删除,日积月累占用空间会越来越大;
- 缺乏相应的扩展机制。
Harbor由 VMware 公司中国研发中心云原生实验室原创,并于 2016 年 3 月开源。Harbor 在 Docker Distribution的基础上增加了企业用户必需的权限控制、镜像签名、安全漏洞扫描和远程复制等重要功能,还提供了图形管理界面及面向国内用户的中文支持,同时Harbor也提供了高级的安全特性,例如用户管理,访问控制,活动审计等等。
架构介绍
如上图所示,harbor模块:
- Proxy: Nginx前端代理,主要用于分发前端页面ui访问和镜像上传和下载流量;Harbor的registry、U1、token services等组件,都处在一个代理后边,该代理将来自浏览器、docker cdients的请求转发到后端服务上;
- Registry: 负责存情Docker像,以及处理Docker push/pul请求。因为Harbor强制要求对像的访问做权限控制,在每-次push/puli请求时Registry会强制要求客户瑞从token service那里获得一个有效的token。
- Job services: 主要用于镜像复制,本地镜像可以被同步到远程Harbor实例上;
- Log collector: 负责收集其他模块的日志到一个地方;
- Database(MySQL或Postgresql):为core services提供数据库服务,负责储存用户权限、审计日志、Docker image分组信息等数据;
- Clair:用于漏洞扫描的组件,可以对Docker镜像进行安全扫描,并提供漏洞报告和修复建议。
- Notary:用于镜像签名和验证的组件,可以保证镜像的完整性和来源可信性。
- Core services; Harbor的核心功能,主要包括如下3个服务:
- UI上作为Redistry Webhook,以图像用户界面的方式辅助用户管理接像。
- WebHook是在registry中置的种机制,当registry中镜像发生改变时,就可以通知到Hardor的webhook endpint、Harbor使用webhook来更新日志,初始化和同步job等;
- Token Service会根据用户在一个工程中的角色,为每一次的push/pull请求分配到对应的token。假如相应的请求没有包含token的话,registry会将请求重定向到token service;
组件间说明
需要说明的是,Harbor的每个组件都是以Dokcer容器的形式构建的,可以使用dokcer compose来进行部署,当然,如果你的环境中使用了kubernetes,hatbor也提供了kubernetes的配置文件。
Harbor组件:
- Ui :Harbor的核心服务,以图像用户界面的方式辅助用户管理接像;
- Log:运行着rsyslog容器,镜像日志收集,负责收集其他模块的日志到一个地方;
- DB:由mysql或Postgresql构成数据库容器,低版本使用的是mysql;
- proxy:使用nginx做反向代理;
- registry:官方的Docker registry;
- adminserver:harbor的配置数据管理器;
- jobservice:Harbor的任务管理服务;
- redis:用于存储session;
工作原理
docker login
- 首先登录请求会被proxy容器接收到,根据预先设置的匹配规则,该请求会被转发到后端的Registry容器;
- Registry接收到请求后,解析请求,因为配置了基于token的认证,所有会先查token,发现请求没有token后,返回错误代码401,以及token服务地址的URL;
- .Docker客户端接收到错误请求后,转而向token服务地址发送请求,并根据HTTP协议的BasicAuthentication 规范,将用户名密码组合并编码,放在请求头部(header)。
- 、同样,该请求会先发到Proxy容器,继而转发给ui/token的容器 该容器接受请求,将请求头解码,获取到用户名您码;
- .ui/token的容器获取到用户名密码后,通过查询数据库进行比对验证(如果是LDAP 的认证方式 就是与LDAP服务进行校验),比对成功后,返回成功的状码,并用密钥生成token,一并发送给Docker喜户端
docker push
- 同样的,首先与Registry进行通信,返回一个token服务的地址URL;
- Docker客户端会与token服务通信,指明要申请一个push image操作的token;
- token服务访问数据库验证当前用户是否有该操作的权限,如果有,会将image信息以及push操作进行骗码,用私签名,生成token返回给Docker客户端;
- Docker客户端再次与Registy通信,不过这次会将token放到请求header中,Registry收到请求后利用公钥解码并核对,核对成功,便可以开始push操作。
Harbor安装
会用到以下地址:
- Harbor版本下载地址:https://github.com/goharbor/harbor/releases
- Harbor安装手册:https://goharbor.io/docs/2.7.0/install-config/
- docker-compose下载:https://github.com/docker/compose/releases
版本介绍
- 目前github上Harbor有两个大的分支,一个是1.x版本,一个是2.x版本;
- 1.x最新的为v1.10.17;
- 2.x版本最新的为v2.8.0-rc1(rc表示"release candidate",即发布候选版本,通常用于测试和收集用户反馈。)
由于Harbor v2.x是Harbor v1.x的一个重大更新,因此v2.x版本包含了更多的新特性和改进。 - 功能:Harbor v2.x版本相对于v1.x版本增加了很多新的功能和改进,如OCI镜像支持、容器镜像签名、镜像复制的性能和可靠性等。此外,v2.x版本还支持Kubernetes原生部署模式,能够更好地与Kubernetes集成。 - 接口:Harbor v2.x版本的API接口与v1.x版本不同,需要进行适当的修改和迁移。 - 维护:Harbor v1.x版本已经不再更新和维护,而Harbor v2.x版本仍在积极开发和维护中,因此v2.x版本更具有长期支持和稳定性 - 同时部署方式也存在较大的差别。
安装
- 目前阶段按照官方的建议是直接使用2.x版本,2.x版本较1.x功能更加完善,更加安全,但是本章咱们还是将以1.x最新的v1.10.17版本为例,进行部署。
- Harbor支持http和https,https更加安全,
Http模式部署
Step1:配置yml文件,HTTP模式
- 最基础配置就是上述这些;
- data_volume:默认你使用本地文件系统,同时支持s3,nfs,swift,oss等共享存储
- 更多配置信息参考:https://goharbor.io/docs/2.7.0/install-config/configure-yml-file/
- 部分参数说明:
参数 | 说明 |
---|---|
hostname | Harbor的主机名,用于访问Harbor界面和Registry服务。默认值为reg.mydomain.com 。 |
http.port 和https.port | Harbor Web界面的监听端口,默认分别为80和443。如果需要修改端口号,可以在配置文件中设置新的端口号。 |
harbor_admin_password | Harbor管理员账户的密码。默认值为Harbor12345 。 |
database | Harbor所使用的数据库类型和连接参数。Harbor支持PostgreSQL和MySQL两种数据库,可以通过db.type 指定数据库类型,以及通过db.host 、db.port 、db.username 、db.password 和db.name 等参数指定数据库连接参数。 |
data_volume | 存储Harbor数据的卷路径。默认值为/data ,可以根据实际情况进行修改。可以为共享存储 |
clair | 是否启用Harbor的漏洞扫描服务,以及扫描服务所使用的配置参数。如果需要启用漏洞扫描服务,可以设置clair.enabled 为true ,并通过clair.server 、clair.output 、clair.threshold 等参数指定扫描服务的相关配置。 |
jobservice | Harbor任务调度服务的相关配置参数。如果需要启用任务调度服务,可以设置jobservice.enabled 为true ,并通过jobservice.max_job_workers 、jobservice.log_level 等参数指定任务调度服务的相关配置。 |
log | Harbor日志的相关配置参数。可以通过log.level 指定日志级别,以及通过log.rotate_size 、log.rotate_num 等参数指定日志轮转的相关配置。 |
Harbor所使用的SMTP服务器的相关配置参数。如果需要发送邮件,可以通过email.server 、email.server_port 、email.username 、email.password 等参数指定SMTP服务器的相关配置。 | |
ldap | 是否启用LDAP身份认证服务,以及LDAP服务所使用的配置参数。如果需要启用LDAP身份认证服务,可以设置ldap.enabled 为true ,并通过ldap.url 、ldap.base_dn 、ldap.filter 、ldap.uid 等参数指定LDAP服务的相关配置。 |
token_expiration | Harbor令牌的过期时间,单位为小时。默认值为24小时。 |
auth_mode | Harbor的认证方式,支持DB、LDAP或OIDC三种方式。可以通过auth_mode 参数指定认证方式,以及通过oidc 参数指定OIDC认证的相关配置。 |
Step2:安装docker-compose,要求1.18.0+
- 下载对应docker-compose版本:https://github.com/docker/compose/releases
[root@clinet bin]# wget https://github.com/docker/compose/releases/download/v2.17.1/docker-compose-linux-x86_64 -O /usr/local/bin/
[root@clinet bin]# chmod u+x docker-compose
[root@clinet bin]#
Step3:执行prepare文件,生成预配置文件以及docker-compose.yml
[root@clinet harbor]# ./prepare
prepare base dir is set to /usr/local/bin/harbor
/usr/src/app/utils/configs.py:100: YAMLLoadWarning: calling yaml.load() without Loader=... is deprecated, as the default Loader is unsafe. Please read https://msg.pyyaml.org/load for full details.
configs = yaml.load(f)
/usr/src/app/utils/configs.py:90: YAMLLoadWarning: calling yaml.load() without Loader=... is deprecated, as the default Loader is unsafe. Please read https://msg.pyyaml.org/load for full details.
versions = yaml.load(f)
Clearing the configuration file: /config/log/logrotate.conf
Clearing the configuration file: /config/log/rsyslog_docker.conf
Generated configuration file: /config/log/logrotate.conf
Generated configuration file: /config/log/rsyslog_docker.conf
Generated configuration file: /config/nginx/nginx.conf
Generated configuration file: /config/core/env
Generated configuration file: /config/core/app.conf
Generated configuration file: /config/registry/config.yml
Generated configuration file: /config/registryctl/env
Generated configuration file: /config/db/env
Generated configuration file: /config/jobservice/env
Generated configuration file: /config/jobservice/config.yml
Generated and saved secret to file: /secret/keys/secretkey
Generated certificate, key file: /secret/core/private_key.pem, cert file: /secret/registry/root.crt
Generated configuration file: /compose_location/docker-compose.yml
Clean up the input dir
[root@clinet harbor]#
Step4:执行install文件,安装Harbor
- 如果docker-compose没有安装或者版本不对,在安装的时候会直接提示。
[root@clinet harbor]# ./install.sh
[Step 0]: checking if docker is installed ...
Note: docker version: 23.0.3
[Step 1]: checking docker-compose is installed ...
Note: docker-compose version: 2.17.1
[Step 2]: loading Harbor images ...
2732dc5f9015: Loading layer [==================================================>] 9.553MB/9.553MB
e34f45312ba4: Loading layer [==================================================>] 11.49MB/11.49MB
c97135f1cb23: Loading layer [==================================================>] 11.49MB/11.49MB
Loaded image: goharbor/clair-adapter-photon:v1.10.17
8af5ee78ce53: Loading layer [==================================================>] 9.553MB/9.553MB
ffe47bed6a24: Loading layer [==================================================>] 59.94MB/59.94MB
a9a256439b2a: Loading layer [==================================================>] 3.072kB/3.072kB
4e1ed84790da: Loading layer [==================================================>] 3.584kB/3.584kB
fcca44955dc5: Loading layer [==================================================>] 60.77MB/60.77MB
5f70bf18a086: Loading layer [==================================================>] 1.024kB/1.024kB
Loaded image: goharbor/chartmuseum-photon:v1.10.17
857938cd97e9: Loading layer [==================================================>] 76.16MB/76.16MB
0092cc272697: Loading layer [==================================================>] 3.584kB/3.584kB
c55d79bd43dc: Loading layer [==================================================>] 3.072kB/3.072kB
d31468000ae8: Loading layer [==================================================>] 2.56kB/2.56kB
afd9f0713700: Loading layer [==================================================>] 3.072kB/3.072kB
f86db445974e: Loading layer [==================================================>] 3.584kB/3.584kB
079d47a859f5: Loading layer [==================================================>] 12.29kB/12.29kB
beb9cd44ffd5: Loading layer [==================================================>] 3.584kB/3.584kB
Loaded image: goharbor/harbor-log:v1.10.17
f71798125840: Loading layer [==================================================>] 127MB/127MB
5f70bf18a086: Loading layer [==================================================>] 1.024kB/1.024kB
3b4c9ddb3a90: Loading layer [==================================================>] 3.072kB/3.072kB
f3f3083c1980: Loading layer [==================================================>] 59.9kB/59.9kB
034ad9211562: Loading layer [==================================================>] 61.95kB/61.95kB
Loaded image: goharbor/redis-photon:v1.10.17
32728dfa19e9: Loading layer [==================================================>] 11.4MB/11.4MB
Loaded image: goharbor/nginx-photon:v1.10.17
7440f7386104: Loading layer [==================================================>] 9.548MB/9.548MB
89ce0f22a443: Loading layer [==================================================>] 5.683MB/5.683MB
2fb99f94cdf5: Loading layer [==================================================>] 13.33MB/13.33MB
b5f90f18512b: Loading layer [==================================================>] 26.35MB/26.35MB
5f7574a3965a: Loading layer [==================================================>] 22.02kB/22.02kB
b494e3f4da24: Loading layer [==================================================>] 13.33MB/13.33MB
Loaded image: goharbor/notary-signer-photon:v1.10.17
f0f9e2b791b7: Loading layer [==================================================>] 13.32MB/13.32MB
d6e37a84b84b: Loading layer [==================================================>] 39.06MB/39.06MB
5f70bf18a086: Loading layer [==================================================>] 1.024kB/1.024kB
Loaded image: goharbor/harbor-jobservice:v1.10.17
Loaded image: goharbor/prepare:v1.10.17
962dca0a458f: Loading layer [==================================================>] 13.32MB/13.32MB
4e1609a180bf: Loading layer [==================================================>] 33.88MB/33.88MB
393434c81f72: Loading layer [==================================================>] 5.632kB/5.632kB
ccee83c3f572: Loading layer [==================================================>] 40.45kB/40.45kB
5f70bf18a086: Loading layer [==================================================>] 1.024kB/1.024kB
9489b7a9e2e3: Loading layer [==================================================>] 2.56kB/2.56kB
Loaded image: goharbor/harbor-core:v1.10.17
3183473239e0: Loading layer [==================================================>] 9.553MB/9.553MB
d695dba545cf: Loading layer [==================================================>] 3.584kB/3.584kB
23bf03a61e8c: Loading layer [==================================================>] 17.32MB/17.32MB
703e31231944: Loading layer [==================================================>] 3.072kB/3.072kB
8c8b04df1a34: Loading layer [==================================================>] 7.41MB/7.41MB
8e1fb3ae2451: Loading layer [==================================================>] 25.55MB/25.55MB
Loaded image: goharbor/harbor-registryctl:v1.10.17
e7e1f676da83: Loading layer [==================================================>] 9.553MB/9.553MB
dacb0288fd24: Loading layer [==================================================>] 3.584kB/3.584kB
a749774e6f04: Loading layer [==================================================>] 3.072kB/3.072kB
312adf2d65b3: Loading layer [==================================================>] 17.32MB/17.32MB
790d4b88221b: Loading layer [==================================================>] 18.14MB/18.14MB
Loaded image: goharbor/registry-photon:v1.10.17
40a7e6304ef7: Loading layer [==================================================>] 11.4MB/11.4MB
2ecc73bf3408: Loading layer [==================================================>] 7.697MB/7.697MB
04eabfa24d0a: Loading layer [==================================================>] 223.2kB/223.2kB
d49f6d9a02e9: Loading layer [==================================================>] 195.1kB/195.1kB
7027176093e6: Loading layer [==================================================>] 15.36kB/15.36kB
2ce003f67643: Loading layer [==================================================>] 3.584kB/3.584kB
Loaded image: goharbor/harbor-portal:v1.10.17
8a3bf7c0e216: Loading layer [==================================================>] 65.32MB/65.32MB
643e807eadaf: Loading layer [==================================================>] 102.8MB/102.8MB
4812444b6617: Loading layer [==================================================>] 5.632kB/5.632kB
2245add4a393: Loading layer [==================================================>] 2.56kB/2.56kB
84cdb93637ca: Loading layer [==================================================>] 2.56kB/2.56kB
281478418c9f: Loading layer [==================================================>] 2.56kB/2.56kB
9dfacf46e588: Loading layer [==================================================>] 2.56kB/2.56kB
4d92353db9bb: Loading layer [==================================================>] 10.75kB/10.75kB
Loaded image: goharbor/harbor-db:v1.10.17
ba8f7a681d5d: Loading layer [==================================================>] 5.683MB/5.683MB
7ec30b9561a8: Loading layer [==================================================>] 14.86MB/14.86MB
e0507cd8b2c8: Loading layer [==================================================>] 26.35MB/26.35MB
7b10c0d8ff33: Loading layer [==================================================>] 22.02kB/22.02kB
207a9013a291: Loading layer [==================================================>] 14.86MB/14.86MB
Loaded image: goharbor/notary-server-photon:v1.10.17
531610669748: Loading layer [==================================================>] 120.3MB/120.3MB
4ea5eb14beb7: Loading layer [==================================================>] 11.97MB/11.97MB
393a9f42449e: Loading layer [==================================================>] 3.072kB/3.072kB
f163482ff464: Loading layer [==================================================>] 49.15kB/49.15kB
ac5d0eeaee30: Loading layer [==================================================>] 3.584kB/3.584kB
0f3788630ad3: Loading layer [==================================================>] 12.84MB/12.84MB
5f70bf18a086: Loading layer [==================================================>] 1.024kB/1.024kB
Loaded image: goharbor/clair-photon:v1.10.17
[Step 3]: preparing environment ...
[Step 4]: preparing harbor configs ...
prepare base dir is set to /usr/local/bin/harbor
Clearing the configuration file: /config/log/logrotate.conf
Clearing the configuration file: /config/log/rsyslog_docker.conf
Clearing the configuration file: /config/nginx/nginx.conf
Clearing the configuration file: /config/core/env
Clearing the configuration file: /config/core/app.conf
Clearing the configuration file: /config/registry/config.yml
Clearing the configuration file: /config/registryctl/env
Clearing the configuration file: /config/registryctl/config.yml
Clearing the configuration file: /config/db/env
Clearing the configuration file: /config/jobservice/env
Clearing the configuration file: /config/jobservice/config.yml
Generated configuration file: /config/log/logrotate.conf
Generated configuration file: /config/log/rsyslog_docker.conf
Generated configuration file: /config/nginx/nginx.conf
Generated configuration file: /config/core/env
Generated configuration file: /config/core/app.conf
Generated configuration file: /config/registry/config.yml
Generated configuration file: /config/registryctl/env
Generated configuration file: /config/db/env
Generated configuration file: /config/jobservice/env
Generated configuration file: /config/jobservice/config.yml
loaded secret from file: /secret/keys/secretkey
Generated configuration file: /compose_location/docker-compose.yml
/usr/src/app/utils/configs.py:100: YAMLLoadWarning: calling yaml.load() without Loader=... is deprecated, as the default Loader is unsafe. Please read https://msg.pyyaml.org/load for full details.
configs = yaml.load(f)
/usr/src/app/utils/configs.py:90: YAMLLoadWarning: calling yaml.load() without Loader=... is deprecated, as the default Loader is unsafe. Please read https://msg.pyyaml.org/load for full details.
versions = yaml.load(f)
Clean up the input dir
[Step 5]: starting Harbor ...
[+] Running 10/10
✔ Network harbor_harbor Created 0.4s
✔ Container harbor-log Started 0.7s
✔ Container harbor-portal Started 2.3s
✔ Container harbor-db Started 1.5s
✔ Container registryctl Started 1.7s
✔ Container registry Started 2.1s
✔ Container redis Started 2.2s
✔ Container harbor-core Started 2.7s
✔ Container harbor-jobservice Started 3.5s
✔ Container nginx Started 3.7s
✔ ----Harbor has been installed and started successfully.----
[root@clinet harbor]#
Step5:查看容器启动状况
Step5:登录
- 命令行界面登录测试
如果使用docker login登录的时候,提示上述错误则需要在/etc/docker/daemon.json中添加insecure-registry 参数。
[root@xhz harbor]# cat /etc/docker/daemon.json { "insecure-registries": ["http://xhz.example.com"] } [root@xhz harbor]#
- 重启docker后再次登录
Step6:上传镜像测试
- 镜像打标签
[root@xhz harbor]# docker tag goharbor/redis-photon:v1.10.17 xhz.example.com/library/redis:v1
[root@xhz harbor]# docker images |grep xhz.example.com/library/redis
xhz.example.com/library/redis v1 a4e5e1ea0833 6 weeks ago 160MB
[root@xhz harbor]#
- 镜像push
[root@xhz harbor]# docker push xhz.example.com/library/redis:v1
The push refers to repository [xhz.example.com/library/redis]
034ad9211562: Pushed
f3f3083c1980: Pushed
3b4c9ddb3a90: Pushed
5f70bf18a086: Pushed
f71798125840: Pushed
26d62a2ff7c9: Pushed
v1: digest: sha256:ac1ef0b681cccd6dc339b71893a4eaaa714cf7c4e68d7cc20f7c7a0206c38565 size: 1572
[root@xhz harbor]#
https模式部署
Step1:生成https的证书
若要配置HTTPS,必须创建SSL证书。您可以使用由受信任的第三方CA签名的证书,也可以使用自签名证书。本节介绍如何使用OpenSSL创建CA,以及如何使用CA对服务器证书和客户端证书进行签名
- 生成key文件
[root@clinet keys]# openssl genrsa -out ca.key 4096
Generating RSA private key, 4096 bit long modulus
..............................................................................++
.....................................................................++
e is 65537 (0x10001)
[root@clinet keys]#
- 生成ca证书
[root@clinet keys]# openssl req -x509 -new -nodes -sha512 -key ca.key -out ca.crt
You are about to be asked to enter information that will be incorporated
into your certificate request.
What you are about to enter is what is called a Distinguished Name or a DN.
There are quite a few fields but you can leave some blank
For some fields there will be a default value,
If you enter '.', the field will be left blank.
-----
Country Name (2 letter code) [XX]:hubei
string is too long, it needs to be less than 2 bytes long
Country Name (2 letter code) [XX]:hb
State or Province Name (full name) []:wh
Locality Name (eg, city) [Default City]:wh
Organization Name (eg, company) [Default Company Ltd]:
Organizational Unit Name (eg, section) []:
Common Name (eg, your name or your server's hostname) []:xhz.example.com
Email Address []:
[root@clinet keys]#
[root@clinet keys]# ll
-rw-r--r--. 1 root root 1992 Apr 13 14:41 ca.crt
-rw-r--r--. 1 root root 3243 Apr 13 14:39 ca.key
[root@clinet keys]#
Step2:配置yml文件,HTTPS模式
Step3:查看容器启动情况和端口监听情况
Step4:登录测试
[root@clinet harbor]# docker login https://xhz.example.com
Username: admin
Password:
WARNING! Your password will be stored unencrypted in /root/.docker/config.json.
Configure a credential helper to remove this warning. See
https://docs.docker.com/engine/reference/commandline/login/#credentials-store
Login Succeeded
[root@clinet harbor]#
- 登录的用户密码保存在/root/.docker/config.json文件中;
- 由于我们只创建了Harbor的证书,因此在login的时候还是需要添加insecure-registry 参数。
- 若不想添加参数,我们需要既给Harbor创建证书,也需要给Docker创建证书,具体步骤见官方文档:https://goharbor.io/docs/2.7.0/install-config/configure-https/
Step5:上传镜像命令
后面的章节我们将介绍:
- Harbor镜像仓库同步至阿里镜像仓库;
- Harbor对接共享存储,比如S3,oss等;
- Harbor的高可用部署以及新的2.x版本部署
bor]#
> - 登录的用户密码保存在/root/.docker/config.json文件中;
> - 由于我们只创建了Harbor的证书,因此在login的时候还是需要添加insecure-registry 参数。
>
> [外链图片转存中...(img-u0GmmnGe-1681394899764)]
>
> - 若不想添加参数,我们需要既给Harbor创建证书,也需要给Docker创建证书,具体步骤见官方文档:https://goharbor.io/docs/2.7.0/install-config/configure-https/
<b>Step5:上传镜像命令</b>
[外链图片转存中...(img-RqdRpDbn-1681394899764)]
---
> 后面的章节我们将介绍:
>
> - Harbor镜像仓库同步至阿里镜像仓库;
> - Harbor对接共享存储,比如S3,oss等;
> - Harbor的高可用部署以及新的2.x版本部署