Harbor镜像仓库的部署、简单使用
1 Docker Registry
-
网上有很多的Registry服务器都支持第三方用户注册,而后基于用户名去做自己的仓库,但是使用互联网上的Registry有一个缺陷,那就是我们去推送和下载镜像时都不会很快,而在生产环境中很可能并行启动的容器将达到几十、上百个,而且很有可能每个服务器本地是没有镜像的,此时如果通过互联网去下载镜像会有很多问题,比如下载速度会很慢、带宽会用很多等等,如果带宽不够的话,下载至启动这个过程可能要持续个几十分钟,这已然违背了使用容器会更加轻量、快速的初衷和目的。因此,很多时候我们很有可能需要去做自己的私有Registry。
-
Registry
用于保存docker镜像,包括镜像的层次结构和元数据。用户可以自建Registry,也可以使用官方的Docker Hub。
1.1 Docker Registry分类
-
Sponsor Registry:第三方的Registry,供客户和Docker社区使用
-
Mirror Registry:第三方的Registry,只让客户使用
-
Vendor Registry:由发布docker镜像的供应商提供的registry
-
Private Registry:通过设有防火墙和额外的安全层的私有实体提供的registry
事实上,如果运维的系统环境托管在云计算服务上,比如阿里云,那么用阿里云的Registry则是最好的选择。很多时候我们的生产环境不会在本地,而是托管在数据中心机房里,如果我们在数据中心机房里的某台主机上部署Registry,因为都在同一机房,所以属于同一局域网,此时数据传输走内网,效率会极大的提升。
所有的Registry
默认情况下都是基于https
工作的,这是Docker的基本要求,而我自建Registry时很可能是基于http工作的,但是Docker默认是拒绝使用http提供Registry服务的,除非明确的告诉它,我们就是要用http协议的Registry。
2 Docker Private Registry
- 为了帮助我们快速创建私有Registry,Docker专门提供了一个名为Docker Distribution的软件包,我们可以通过安装这个软件包快速构建私有仓库。
问:既然Docker是为了运行程序的,Docker Distribution能否运行在容器中?
-
容器时代,任何程序都应该运行在容器中,除了Kernel和init。而为了能够做Docker Private Registry,Docker Hub官方直接把Registry做成了镜像,我们可以直接将其pull到本地并启动为容器即可快速实现私有Registry。
-
Registry的主要作用是托管镜像,Registry运行在容器中,而容器自己的文件系统是随着容器的生命周期终止和删除而被删除的,所以当我们把Registry运行在容器中时,客户端上传了很多镜像,随着Registry容器的终止并删除,所有镜像都将化为乌有,因此这些镜像应该放在存储卷上,而且这个存储卷最好不要放在Docker主机本地,而应该放在一个网络共享存储上,比如NFS。
-
不过,镜像文件自己定义的存储卷,还是一个放在Docker本地、Docker管理的卷,我们可以手动的将其改成使用其它文件系统的存储卷。
这就是使用容器来运行Registry
的一种简单方式。自建Registry
的另一种方式,就是直接安装docker-distribution
软件。
2.1 使用docker-distribution自建Registry
注意:此方法适用于CentOS7上做
//在node02上自建Registry
[root@node02 ~]# yum -y install docker-distribution
[root@node02 ~]# vim /etc/docker-distribution/registry/config.yml
version: 0.1
log:
fields:
service: registry
storage:
cache:
layerinfo: inmemory
filesystem:
rootdirectory: /var/lib/registry # 修改此处为一个容量大的磁盘分区目录
http:
addr: :5000
[root@node02 ~]# systemctl start docker-distribution
[root@node02 ~]# ss -antl
State Recv-Q Send-Q Local Address:Port Peer Address:Port
LISTEN 0 100 127.0.0.1:25 *:*
LISTEN 0 128 *:22 *:*
LISTEN 0 100 [::1]:25 [::]:*
LISTEN 0 128 [::]:5000 [::]:*
LISTEN 0 128 [::]:22
[root@node01 ~]# vim /etc/docker/daemon.json
{
"registry-mirrors": ["https://in3617d8.mirror.aliyuncs.com","https://registry.docker-cn.com"],
"insecure-registries": ["register.example.com:5000"]
}
[root@node01 ~]# systemctl restart docker
[root@node01 ~]# docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
seancheng1002/b1 v0.2 42a777e26541 2 weeks ago 1.22MB
seancheng1002/b1 v0.1 bb54705dfd51 2 weeks ago 1.22MB
nginx latest 2073e0bcb60e 2 weeks ago 127MB
centos latest 470671670cac 5 weeks ago 237MB
busybox latest 6d5fcfe5ff17 8 weeks ago 1.22MB
[root@node01 ~]# docker tag nginx:latest register.example.com:5000/nginx:latest
[root@node01 ~]# docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
seancheng1002/b1 v0.2 42a777e26541 2 weeks ago 1.22MB
seancheng1002/b1 v0.1 bb54705dfd51 2 weeks ago 1.22MB
nginx latest 2073e0bcb60e 2 weeks ago 127MB
register.example.com:5000/nginx latest 2073e0bcb60e 2 weeks ago 127MB
centos latest 470671670cac 5 weeks ago 237MB
busybox latest 6d5fcfe5ff17 8 weeks ago 1.22MB
[root@node01 ~]# docker push register.example.com:5000/nginx
The push refers to repository [register.example.com:5000/nginx]
22439467ad99: Pushed
b4a29beac87c: Pushed
488dfecc21b1: Pushed
latest: digest: sha256:62f787b94e5faddb79f96c84ac0877aaf28fb325bfc3601b9c0934d4c107ba94 size: 948
2.2 使用官方镜像自建Registry
[root@node02 ~]# docker run -d -p 5000:5000 -v /opt/data/registry:/tmp/registry registry
[root@node02 ~]# ss -antl
State Recv-Q Send-Q Local Address:Port Peer Address:Port
LISTEN 0 100 127.0.0.1:25 *:*
LISTEN 0 128 *:22 *:*
LISTEN 0 100 [::1]:25 [::]:*
LISTEN 0 128 [::]:5000 [::]:*
LISTEN 0 128 [::]:22 [::]:*
3 Harbor
-
无论是使用Docker-distribution去自建仓库,还是通过官方镜像跑容器的方式去自建仓库,通过前面的演示我们可以发现其是非常的简陋的,还不如直接使用官方的Docker Hub去管理镜像来得方便,至少官方的Docker Hub能够通过web界面来管理镜像,还能在web界面执行搜索,还能基于Dockerfile利用Webhooks和Automated Builds实现自动构建镜像的功能,用户不需要在本地执行docker build,而是把所有build上下文的文件作为一个仓库推送到github上,让Docker Hub可以从github上去pull这些文件来完成自动构建。
-
但无论官方的Docker Hub有多强大,它毕竟是在国外,所以速度是最大的瓶颈,我们很多时候是不可能去考虑使用官方的仓库的,但是上面说的两种自建仓库方式又十分简陋,不便管理,所以后来就出现了一个被 CNCF 组织青睐的项目,其名为Harbor。
3.1 Harbor简介
- Harbor是由VMWare在Docker Registry的基础之上进行了二次封装,加进去了很多额外程序,而且提供了一个非常漂亮的web界面。
- Project Harbor是一个开源的受信任云本地注册表项目,用于存储、标记和扫描上下文。
- Harbor通过添加用户通常需要的功能,如安全性、身份标识和管理,扩展了开源Docker发行版。
- Harbor支持用户管理、访问控制、活动监控和实例之间的复制等高级特性。
3.2 Harbor的功能
Feathers:
- Multi-tenant content signing and validation 多租户内容签名和验证
- Security and vulnerability analysis 安全性和脆弱性分析
- Audit logging 审计日志记录
- Identity integration and role-based access control 身份集成和基于角色的访问控制
- Image replication between instances 实例间的镜像复制
- Extensible API and graphical UI 可扩展API和图形化界面
- Internationalization(currently English and Chinese) 国际化(目前为中英文)
4 Docker compose
Harbor在物理机上部署是非常难的,而为了简化Harbor的应用,Harbor官方直接把Harbor做成了在容器中运行的应用,而且这个容器在Harbor中依赖类似redis、mysql、pgsql等很多存储系统,所以它需要编排很多容器协同起来工作,因此VMWare Harbor在部署和使用时,需要借助于Docker的单机编排工具(Docker compose)来实现。
5 Harbor部署
环境说明:
主机名 | ip | 需要的应用 |
---|---|---|
registry.example.com | 192.168.200.145 | docker-ce 、docker-compose、Harbor |
node2 | 192.168.200.153 | docekr-ce |
//关闭防火墙
[root@registry ~]# systemctl disable --now firewalld
Removed /etc/systemd/system/multi-user.target.wants/firewalld.service.
Removed /etc/systemd/system/dbus-org.fedoraproject.FirewallD1.service.
[root@registry ~]# systemctl stop firewalld
[root@registry ~]# vi /etc/selinux/config
[root@registry ~]# setenforce 0
[root@registry ~]# getenforce 0
Permissive
安装Docker compose
[root@registry ~]# wget -O /etc/yum.repos.d/docker-ce.repo https://mirrors.tuna.tsinghua.edu.cn/docker-ce/linux/centos/docker-ce.repo
[root@registry ~]# sed -i 's@https://download.docker.com@https://mirrors.tuna.tsinghua.edu.cn/docker-ce@g' /etc/yum.repos.d/docker-ce.repo
[root@registry ~]# yum clean all
//安装docker
[root@registry ~]# yum -y install docker-ce
//安装docker compose
[root@registry ~]# curl -L "https://github.com/docker/compose/releases/download/1.29.2/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 664 100 664 0 0 1291 0 --:--:-- --:--:-- --:--:-- 1291
100 12.1M 100 12.1M 0 0 1696k 0 0:00:07 0:00:07 --:--:-- 1886k
[root@registry bin]# ls
docker-compose
//将执行权限应用于二进制文件
[root@registry ~]# chmod +x /usr/local/bin/docker-compose
[root@registry ~]# ls -l /usr/local/bin/docker-compose
-rwxr-xr-x 1 root root 12737304 12月 17 05:37 /usr/local/bin/docker-compose
[root@registry ~]# which docker-compose
/usr/local/bin/docker-compose
//查看版本号
[root@registry ~]# docker-compose version
docker-compose version 1.29.2, build 5becea4c
docker-py version: 5.0.0
CPython version: 3.7.10
OpenSSL version: OpenSSL 1.1.0l 10 Sep 2019
//要用md5sum与官网的md5sum文件进行对比,查看包是否完整
[root@registry ~]# ls
anaconda-ks.cfg harbor-offline-installer-v2.3.5.tgz
[root@registry ~]# md5sum harbor-offline-installer-v2.3.5.tgz
f1e01bbb4b62bf4a31a103d8c7c5a215 harbor-offline-installer-v2.3.5.tgz
[root@registry ~]#
//解压包到/usr/local目录下
[root@registry ~]# tar xf harbor-offline-installer-v2.3.5.tgz -C /usr/local/
//设置开机自启
[root@registry ~]# systemctl enable --now firewalld
Created symlink /etc/systemd/system/dbus-org.fedoraproject.FirewallD1.service → /usr/lib/systemd/system/firewalld.service.
Created symlink /etc/systemd/system/multi-user.target.wants/firewalld.service → /usr/lib/systemd/system/firewalld.service.
[root@registry ~]# systemctl status docker
● docker.service - Docker Application Container Engine
Loaded: loaded (/usr/lib/systemd/system/docker.service; enabled; vendor preset: disable>
Active: active (running) since Fri 2021-12-17 01:58:44 CST; 2h 29min ago
Docs: https://docs.docker.com
Main PID: 1141 (dockerd)
Tasks: 10
Memory: 123.2M
CGroup: /system.slice/docker.service
└─1141 /usr/bin/dockerd -H fd:// --containerd=/run/containerd/containerd.sock
12月 17 01:58:42 docker dockerd[1141]: time="2021-12-17T01:58:42.777717465+08:00" level=in>
12月 17 01:58:42 docker dockerd[1141]: time="2021-12-17T01:58:42.857851028+08:00" level=in>
12月 17 01:58:43 docker dockerd[1141]: time="2021-12-17T01:58:43.203588240+08:00" level=wa>
12月 17 01:58:43 docker dockerd[1141]: time="2021-12-17T01:58:43.203622439+08:00" level=wa>
12月 17 01:58:43 docker dockerd[1141]: time="2021-12-17T01:58:43.204471698+08:00" level=in>
//配置docker镜像加速器
[root@registry ~]# vi /etc/docker/daemon.json
[root@registry ~]# cat /etc/docker/daemon.json
{
"registry-mirrors": ["https://l6paj5i1.mirror.aliyuncs.com"]
}
[root@registry ~]# systemctl daemon-reload
[root@registry ~]# systemctl restart docker
[root@registry ~]#
//先修改要配置的主机名和域名映射
[root@registry ~]# cat /etc/hosts
127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4
::1 localhost localhost.localdomain localhost6 localhost6.localdomain6
192.168.200.145 registry.example.com
[root@registry ~]# ping registry.example.com
PING registry.example.com (192.168.200.145) 56(84) bytes of data.
64 bytes from registry.example.com (192.168.200.145): icmp_seq=1 ttl=64 time=0.043 ms
64 bytes from registry.example.com (192.168.200.145): icmp_seq=2 ttl=64 time=0.107 ms
^C
[root@registry ~]#
//修改harbor配置文件
[root@registry ~]# cd /usr/local/harbor/
[root@registry harbor]# ls
common.sh harbor.v2.3.5.tar.gz harbor.yml.tmpl install.sh LICENSE prepare
#备份文件
[root@registry harbor]# cp harbor.yml.tmpl harbor.yml
[root@registry harbor]# ls
common.sh harbor.v2.3.5.tar.gz harbor.yml harbor.yml.tmpl install.sh LICENSE prepare
[root@registry harbor]#
//harbor的web界面登陆密码和一些其他配置也在harbor.yml文件中,可根据你的需要进行修改
[root@registry harbor]# vim harbor.yml
[root@registry harbor]# cat harbor.yml
# Configuration file of Harbor
# The IP address or hostname to access admin UI and registry service.
# DO NOT use localhost or 127.0.0.1, because Harbor needs to be accessed by external clients.
hostname: registry.example.com #修改这里的内容为当前主机的主机名或IP地址
# http related config
http:
# port for http, default is 80. If https enabled, this port will redirect to https port
port: 80
# https related config #因为当前环境我们没有https证书所以要把https的相关内容注释掉
#https:
# https port for harbor, default is 443
# port: 443
# The path of cert and key files for nginx
#certificate: /your/certificate/path
# private_key: /your/private/key/path
.......
//安装harbor
[root@registry harbor]# ./install.sh
......
Creating network "harbor_harbor" with the default driver
Creating harbor-log ... done
Creating registry ... done
Creating harbor-portal ... done
Creating registryctl ... done
Creating redis ... done
Creating harbor-db ... done
Creating harbor-core ... done
Creating nginx ... done
Creating harbor-jobservice ... done
✔ ----Harbor has been installed and started successfully.----
[root@node1 harbor]#
[root@node1 harbor]# ls
common docker-compose.yml harbor.yml install.sh prepare
common.sh harbor.v2.3.5.tar.gz harbor.yml.tmpl LICENSE
[root@registry harbor]#
//查看镜像
[root@registry harbor]# docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
haproxy v1.2 219dc51c1897 4 days ago 54.5MB
haproxy v1.1 063d2fd06d69 4 days ago 111MB
haproxy v1.0 e6c982df7166 4 days ago 75.4MB
haproxy v0.4 2184300f3121 4 days ago 566MB
goharbor/harbor-exporter v2.3.5 1730c6f650e2 6 days ago 81.9MB
goharbor/chartmuseum-photon v2.3.5 47004f032938 6 days ago 179MB
goharbor/redis-photon v2.3.5 3d0cedc89a0d 6 days ago 156MB
goharbor/trivy-adapter-photon v2.3.5 5c0212e98070 6 days ago 133MB
goharbor/notary-server-photon v2.3.5 f20a76c65359 6 days ago 111MB
goharbor/notary-signer-photon v2.3.5 b9fa38eef4d7 6 days ago 108MB
goharbor/harbor-registryctl v2.3.5 7a52567a76ca 6 days ago 133MB
goharbor/registry-photon v2.3.5 cf22d3e386b8 6 days ago 82.6MB
goharbor/nginx-photon v2.3.5 5e3b6d9ce11a 6 days ago 45.7MB
goharbor/harbor-log v2.3.5 a03e4bc963d6 6 days ago 160MB
goharbor/harbor-jobservice v2.3.5 2ac32df5a2e0 6 days ago 211MB
goharbor/harbor-core v2.3.5 23baee01156f 6 days ago 193MB
goharbor/harbor-portal v2.3.5 bb545cdedf5a 6 days ago 58.9MB
goharbor/harbor-db v2.3.5 9826c57a5749 6 days ago 221MB
goharbor/prepare v2.3.5 a1ceaabe47b2 6 days ago 255MB
caiaoc/apache latest 3b1fe562c6f6 8 days ago 701MB
caiaoc/mysql v1.0 ba85e9d47f00 12 days ago 3.81GB
caiaoc/nginx v1.0 a985a6973aec 12 days ago 549MB
caiaoc/php v1.0 f5fe6827f02b 12 days ago 1.46GB
nginx latest f652ca386ed1 2 weeks ago 141MB
busybox latest d23834f29b38 2 weeks ago 1.24MB
alpine latest c059bfaa849c 3 weeks ago 5.59MB
httpd latest ad17c88403e2 3 weeks ago 143MB
centos latest 5d0da3dc9764 3 months ago 231MB
[root@node1 harbor]#
//查看允许的容器
[root@registry harbor]# docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
82baaff51fd8 goharbor/harbor-jobservice:v2.3.5 "/harbor/entrypoint.…" 2 minutes ago Up 2 minutes (healthy) harbor-jobservice
31be1fd0b631 goharbor/nginx-photon:v2.3.5 "nginx -g 'daemon of…" 2 minutes ago Up 2 minutes (healthy) 0.0.0.0:80->8080/tcp, :::80->8080/tcp nginx
6469912f37ab goharbor/harbor-core:v2.3.5 "/harbor/entrypoint.…" 2 minutes ago Up 2 minutes (healthy) harbor-core
19bf5ceb6e39 goharbor/harbor-db:v2.3.5 "/docker-entrypoint.…" 2 minutes ago Up 2 minutes (healthy) harbor-db
6c1be3a10cdd goharbor/harbor-portal:v2.3.5 "nginx -g 'daemon of…" 2 minutes ago Up 2 minutes (healthy) harbor-portal
5c6f04af25dd goharbor/redis-photon:v2.3.5 "redis-server /etc/r…" 2 minutes ago Up 2 minutes (healthy) redis
8703a6f5e42e goharbor/harbor-registryctl:v2.3.5 "/home/harbor/start.…" 2 minutes ago Up 2 minutes (healthy) registryctl
3ed692c8f1b2 goharbor/registry-photon:v2.3.5 "/home/harbor/entryp…" 2 minutes ago Up 2 minutes (healthy) registry
7bbc9d0aed50 goharbor/harbor-log:v2.3.5 "/bin/sh -c /usr/loc…" 2 minutes ago Up 2 minutes (healthy) 127.0.0.1:1514->10514/tcp harbor-log
[root@node1 harbor]#
//查看运行的harbor的端口号
[root@registry harbor]# ss -antl
State Recv-Q Send-Q Local Address:Port Peer Address:Port Process
LISTEN 0 128 0.0.0.0:22 0.0.0.0:*
LISTEN 0 128 127.0.0.1:1514 0.0.0.0:*
LISTEN 0 128 0.0.0.0:111 0.0.0.0:*
LISTEN 0 128 0.0.0.0:80 0.0.0.0:*
LISTEN 0 128 [::]:22 [::]:*
LISTEN 0 128 [::]:111 [::]:*
LISTEN 0 128 [::]:80 [::]:*
[root@node1 harbor]#
在浏览器上访问harbor web界面
注意
- 使用
docker-compose { start|stop }
来启动或停止Harbor,要注意的是要启动或停止Harbor的时候必须要在Harbor的工作目录中进行即/usr/local/harbor目录中。 - 因为docker-compose命令是基于docker-compose.yml来控制的,所以要在有这个文件的目录里执行命令。
使用Harbor的注意事项
- 在客户端上传镜像时一定要记得执行docker login进行用户认证,否则无法直接push
- 在客户端使用的时候如果不是用的https则必须要在客户端的/etc/docker/daemon.json配置文件中配置insecure-registries参数
- 数据存放路径应在配置文件中配置到一个容量比较充足的共享存储中
- Harbor是使用docker-compose命令来管理的,如果需要停止Harbor也应用docker-compose stop来停止,其他参数请–help
6 Harbor仓库的使用
//下载镜像源安装docker
[root@node2 ~]# wget -O /etc/yum.repos.d/docker-ce.repo https://mirrors.tuna.tsinghua.edu.cn/docker-ce/linux/centos/docker-ce.repo
[root@node2 ~]# sed -i 's@https://download.docker.com@https://mirrors.tuna.tsinghua.edu.cn/docker-ce@g' /etc/yum.repos.d/docker-ce.repo
[root@node2 ~]# yum clean all
21 files removed
[root@node2 ~]# yum -y install docker-ce
[root@node2 ~]# systemctl start docker
[root@node2 ~]# systemctl enable --now docker
Created symlink /etc/systemd/system/multi-user.target.wants/docker.service → /usr/lib/systemd/system/docker.service.
[root@node2 ~]#
[root@node2 ~]# vi /etc/docker/daemon.json
[root@node2 ~]# cat /etc/docker/daemon.json
{
"insecure-registries": ["registry.example.com"], #添加此行,填写registry的主机名或IP
"registry-mirrors": ["https://l6paj5i1.mirror.aliyuncs.com"]
}
[root@node2 ~]# systemctl daemon-reload
[root@node2 ~]# systemctl restart docker
[root@node2 ~]#
//在node2主机中配置harbor的域名解析
[root@node2 ~]# cat /etc/hosts
127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4
::1 localhost localhost.localdomain localhost6 localhost6.localdomain6
192.168.200.145 registry.example.com
[root@node2 ~]#
//登录harbor仓库
[root@node2 ~]# docker login registry.example.com
Username: admin
Password:
WARNING! Your password will be stored unencrypted in /root/.docker/config.json.
Configure a credential helper to remove this warning. See
https://docs.docker.com/engine/reference/commandline/login/#credentials-store
Login Succeeded
[root@node2 ~]#
在harbor上新建一个项目仓库
[root@node2 ~]# docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
busybox latest ffe9d497c324 8 days ago 1.24MB
registry.example.com/web/busybox latest ffe9d497c324 8 days ago 1.24MB
[root@node2 ~]# docker push registry.example.com/web/busybox
Using default tag: latest
The push refers to repository [registry.example.com/web/busybox]
64cac9eaf0da: Pushed
latest: digest: sha256:50e44504ea4f19f141118a8a8868e6c5bb9856efa33f2183f5ccea7ac62aacc9 size: 527
[root@node2 ~]#
在harbor仓库中查看
7 创建普通用户
![请添加图片描述](https://img-blog.csdnimg.cn/557598f0c6944dafbe8bdd5db4336f6d.png?x-oss-process=image/watermark,type_d3F5LXplbmhlaQ,shadow_50,text_Q1NETiBA5a-C5a-e55qE5qef5qaU5Li2,size_20,color_FFFFFF,t_70,g_se,x_16