docker(三）—镜像仓库、Registry工作原理、搭建私有仓库、搭建harbor仓库

1、docker仓库简介

2、Registry工作原理

客户端访问index，index返回镜像所需的地址以及index认证后的token，客户端拿着地址和token访问Registry，Registry会去访问index去验证token的合法性，index告诉registry合法，然后Registry才会将镜像传递给客户端，如下图：

客户端将push请求发给index，index会给客户端一个临时的token，然后客户端将镜像push到registry仓库，registry会访问index，验证token的合法性，验证成功registry会接收镜像

3、搭建私有仓库

私有仓库参考文档

[root@server1 docker]# docker pull registry  官方拉取仓库
 [root@server1 docker]#  docker run -d -p 5000:5000 --restart=always --name registry registry   运行， --restart=always表示每次在启动docker引擎的时侯自动开启
 [root@server1 docker]# docker ps  运行成功
CONTAINER ID        IMAGE               COMMAND                  CREATED             STATUS              PORTS                    NAMES
f47e78f9434e        registry            "/entrypoint.sh /etc…"   4 minutes ago       Up 4 minutes        0.0.0.0:5000->5000/tcp   registry
[root@server1 docker]# docker tag  yakexi007/game2048:latest localhost:5000/game2048   更改镜像名字，指定仓库地址：指向本机5000端口上传镜像
[root@server1 docker]# docker push localhost:5000/game2048  上传镜像
[root@server1 docker]# docker rmi yakexi007/game2048:latest   删除本地的game2048镜像
[root@server1 docker]# docker rmi localhost:5000/game2048:latest   删除本地的game2048镜像
[root@server1 docker]# docker pull localhost:5000/game2048 可以从本地镜像仓库拉取
latest: Pulling from game2048
534e72e7cedc: Pull complete 
f62e2f6dfeef: Pull complete 
fe7db6293242: Pull complete 
3f120f6a2bf8: Pull complete 
4ba4e6930ea5: Pull complete 

如何给私有仓库添加认证功能
开启一台虚拟机server2

[root@foundation50 ~]# cd /mnt/pub/docs/docker/
[root@foundation50 docker]# cp  -r docker-ce/ /var/www/html/   拷贝docker-ce/目录到http默认发布目录里
[root@server2 yum.repos.d]# vim docker.repo   配置软件仓库
[docker]
name=docker-ce
baseurl=http://172.25.254.50/docker-ce
gpgcheck=0
[root@server2 yum.repos.d]# yum install docker-ce -y  安装
[root@server2 yum.repos.d]# systemctl enable --now docker  启动docker
[root@server1 ~]# cd /etc/sysctl.d/  
[root@server1 sysctl.d]# scp docker.conf server2:/etc/sysctl.d/     
[root@server2 ~]# sysctl--system    使之生效
[root@server2 ~]# docker pull 172.25.50.1:5000/game2048  不支持远端非加密连接内网仓库
Using default tag: latest
Error response from daemon: Get https://172.25.50.1:5000/v2/: http: server gave HTTP response to HTTPS client   报错
[root@server2 ~]# cd /etc/docker/   
[root@server2 docker]# vim daemon.json  编辑，添加参数，支持远端非安全连接
{
  "insecure-registries" : ["172.25.50.1:5000"]
}
[root@server2 docker]# systemctl daemon-reload 
[root@server2 docker]# systemctl restart docker  重启daocker

[root@server2 docker]# docker pull 172.25.50.1:5000/game2048  可以拉取
Using default tag: latest
latest: Pulling from game2048
534e72e7cedc: Pull complete 
f62e2f6dfeef: Pull complete 
fe7db6293242: Pull complete 
3f120f6a2bf8: Pull complete 
4ba4e6930ea5: Pull complete 
Digest: sha256:8a34fb9cb168c420604b6e5d32ca6d412cb0d533a826b313b190535c03fe9390
Status: Downloaded newer image for 172.25.50.1:5000/game2048:latest
172.25.50.1:5000/game2048:latest

如何加密认证连接镜像仓库？？

[root@server1 sysctl.d]# docker rm -f  registry  删除
registry
[root@server1 sysctl.d]# docker volume prune  表示删除没有被使用的卷
WARNING! This will remove all local volumes not used by at least one container.
Are you sure you want to continue? [y/N] y
[root@server1 sysctl.d]# docker volume ls  查看卷，已经删除完成
DRIVER              VOLUME NAME
[root@server1 sysctl.d]# docker ps -a     查看容器，没有容器运行
CONTAINER ID        IMAGE               COMMAND             CREATED             STATUS              PORTS               NAMES

开启加密功能：

参靠文档
使用自签名证书

[root@server1 ~]# mkdir -p certs   建立目录
[root@server1 ~]# openssl req -newkey rsa:4096 -nodes -sha256 -keyout certs/westos.org.key -x509 -days 365 -out certs/westos.org.crt   生成key放到certs，再用key生成所需要的证书
Country Name (2 letter code) [XX]:cn  
State or Province Name (full name) []:shaanxi                  
Locality Name (eg, city) [Default City]:xi'an
Organization Name (eg, company) [Default Company Ltd]:westos
Organizational Unit Name (eg, section) []:linux
Common Name (eg, your name or your server's hostname) []:reg.westos.org   主机名
Email Address []:root@westos.org   
[root@server1 ~]# ls certs/   证书已经生成
westos.org.crt  westos.org.key
[root@server1 ~]# docker run -d \
>   --restart=always \
>   --name registry \
>   -v /opt/registry:/var/lib/registry \   表示宿主机/opt/registry目录，没有自动生成，挂载到容器存放镜像/var/lib/registry目录里，把仓库里的数据持久化到宿主机上
>   -v "$(pwd)"/certs:/certs \    -v表示手动挂载指定路经，把当前目录下的certs挂载到容器内的certs
>   -e REGISTRY_HTTP_ADDR=0.0.0.0:443 \     -e表示容器指令，表示启用443加密
>   -e REGISTRY_HTTP_TLS_CERTIFICATE=/certs/westos.org.crt \    指定生成的证书
>   -e REGISTRY_HTTP_TLS_KEY=/certs/westos.org.key \   指定生成的key
>   -p 443:443 \    做一个端口i映射，切记不要和宿主机冲突
>     registry  镜像名字
[root@server1 ~]# docker ps   查看是否运行
CONTAINER ID        IMAGE               COMMAND                  CREATED             STATUS                          PORTS               NAMES
f433469ca1d9        docker              "docker-entrypoint.s…"   6 seconds ago       Restarting (125) 1 second ago                       registry   已经运行
[root@server1 ~]# docker port registry 
443/tcp -> 0.0.0.0:443
[root@server1 ~]# vim /etc/hosts  远程连接的时侯要加解析
172.25.50.1   server1  reg.westos.org      reg.westos.org  为仓库名字

测试：从远程上传下载镜像
[root@server2 docker]# docker tag 172.25.50.1:5000/game2048:latest reg.westos.org/game2048  改镜像名字
[root@server2 docker]# docker push reg.westos.org/game2048  上传镜像,发现报错
The push refers to repository [reg.westos.org/game2048]
Get https://reg.westos.org/v2/: x509: certificate signed by unknown authority   server2上没有证书
[root@server2 docker]# mkdir certs.d   创建目录
[root@server2 docker]# cd certs.d/
[root@server2 certs.d]# pwd
/etc/docker/certs.d
[root@server2 certs.d]# mkdir reg.westos.org  建路目录，和软件仓库名保持一样
[root@server2 certs.d]# cd reg.westos.org/
[root@server2 reg.westos.org]# pwd
/etc/docker/certs.d/reg.westos.org
[root@server1 ~]# vim /etc/hosts
[root@server1 ~]# cd certs/
[root@server1 certs]# scp westos.org.crt server2:/etc/docker/certs.d/reg.westos.org/ca.crt   将server1上的证书westos.org.crt拷贝到server2上/etc/docker/certs.d/reg.westos.org/ 目录里
[root@server2 reg.westos.org]# pwd  
/etc/docker/certs.d/reg.westos.org      server2 上已经有证书了，注意证书一定要放到此目录里
[root@server2 reg.westos.org]# docker push reg.westos.org/game2048  此时再次上传，就可以上传了
The push refers to repository [reg.westos.org/game2048]
88fca8ae768a: Pushed 
6d7504772167: Pushed 
192e9fad2abc: Pushed 
36e9226e74f8: Pushed 
011b303988d2: Pushed 
latest: digest: sha256:8a34fb9cb168c420604b6e5d32ca6d412cb0d533a826b313b190535c03fe9390 size: 1364	

如何开启认证？

官方参考文档

[root@server1 certs]# cd --   
[root@server1 ~]# mkdir auth   创建认证目录
[root@server1 ~]# cd auth/
[root@server1 auth]# yum install -y httpd-tools   安装httpd工具
[root@server1 auth]# htpasswd  -B -c htpasswd wxh    -B强制使用最安全的加密算法  -c表示创建，在auth目录里创建htpasswd  ，wxh为添加的用户
[root@server1 auth]# cat htpasswd 
wxh:$2y$05$7QCbnhaOq0ashi6OxbwN5eii7RQaSB5yYcrtb6YkqozN3EEp0fv7S
[root@server1 auth]# htpasswd  -B  htpasswd admin     再次创建，第二次创建不要用-c，不然会把第一次的覆盖掉
New password: 
Re-type new password: 
Adding password for user admin
[root@server1 auth]# cat htpasswd            
wxh:$2y$05$7QCbnhaOq0ashi6OxbwN5eii7RQaSB5yYcrtb6YkqozN3EEp0fv7S
admin:$2y$05$Dn4RHzRpjdOQsriauva1gu56yPtXq3S1I5ZtyfPO7.XSZnkmzjaXm
[root@server1 ~]# docker rm -f registry   删除registry
registry
[root@server1 ~]# docker run -d --restart=always --name registry -v "$(pwd)"/certs:/certs -v /opt/registry:/var/lib/registry -v "$(pwd)"/auth:/auth -e "REGISTRY_AUTH=htpasswd" -e "REGISTRY_AUTH_HTPASSWD_REALM=Registry Realm" -e REGISTRY_AUTH_HTPASSWD_PATH=/auth/htpasswd -e REGISTRY_HTTP_ADDR=0.0.0.0:443 -e REGISTRY_HTTP_TLS_CERTIFICATE=/certs/westos.org.crt -e REGISTRY_HTTP_TLS_KEY=/certs/westos.org.key -p 443:443 registry 
fdfadff803e9969dc9d96499bcc9b80a539730a827ad85785e581c445e09f7a3
再次运行，添加htpasswd认证：
添加REGISTRY_AUTH=htpasswd认证方式 ，读取REGISTRY_AUTH_HTPASSWD_REALM=Registry Realm文件 
REGISTRY_AUTH_HTPASSWD_PATH=/auth/htpasswd  指定认证文件
[root@server1 ~]# docker ps   查看容器
CONTAINER ID        IMAGE               COMMAND                  CREATED             STATUS              PORTS                            NAMES
fdfadff803e9        registry            "/entrypoint.sh /etc…"   4 minutes ago       Up 4 minutes        0.0.0.0:443->443/tcp, 5000/tcp   registry
[root@server2 reg.westos.org]# docker pull reg.westos.org/game2048  此时拉取就需要认证
Using default tag: latest
Error response from daemon: Get https://reg.westos.org/v2/game2048/manifests/latest: no basic auth credentials   报错
[root@server2 reg.westos.org]# docker login reg.westos.org    登陆 
Username: wxh
Password: 
WARNING! Your password will be stored unencrypted in /root/.docker/config.json.
Configure a credential helper to remove this warning. See
https://docs.docker.com/engine/reference/commandline/login/#credentials-store

Login Succeeded

[root@server2 ~]# cd .docker/   认证文件存放位置
[root@server2 .docker]# ls
config.json
[root@server2 .docker]# cat config.json 
{
	"auths": {
		"reg.westos.org": {
			"auth": "d3hoOndlc3Rvcw=="    认证信息
		}
	},
	"HttpHeaders": {
		"User-Agent": "Docker-Client/19.03.15 (linux)"
	}
[root@server2 .docker]# docker pull  reg.westos.org/game2048  此时拉取就可以拉取成功
[root@server2 .docker]# docker logout  reg.westos.org  登出
Removing login credentials for reg.westos.org
[root@server2 .docker]# ls   
config.json
[root@server2 .docker]# cat config.json 
{
	"auths": {},    登出信息就没有了
	"HttpHeaders": {
		"User-Agent": "Docker-Client/19.03.15 (linux)"
	}

4、 harbor仓库

[root@server1 ~]# docker rm -f registry  删除registry
registry
[root@foundation50 docker]# scp harbor-offline-installer-v1.10.1.tgz server1:   将下载好的harbor软件包拷贝到server1上
[root@foundation50 docker]# cd compose/
[root@foundation50 compose]# scp docker-compose-Linux-x86_64-1.27.0  server1:   将下载好的docker-compose拷贝到server1上
[root@server1 ~]# tar zxf harbor-offline-installer-v1.10.1.tgz   解压缩
[root@server1 harbor]# mkdir /data  创建harbor数据目录
[root@server1 ~]# cp -r certs/ /data/  将证书拷贝到harbor数据目录里
[root@server1 harbor]# vim harbor.yml   编辑harbor配置文件

[root@server1 harbor]# [root@server1 harbor]# ./install.sh 启动 
./install.sh后面可以加功能：（--with-notary表示给镜像做签名  --with-clair  表示镜像扫描器 ，扫描漏洞缺陷  --with-chartmuseum k8s仓库，以及软件包管理)
[root@server1 ~]# mv docker-compose-Linux-x86_64-1.27.0 /usr/local/bin/docker-compose  将docker-compose软件包拷贝到bin目录里并改名为docker-compose
[root@server1 ~]# chmod +x /usr/local/bin/docker-compose  加上执行权限
[root@server1 ~]# cd harbor/  
[root@server1 harbor]# ./install.sh  再次执行
[root@server1 harbor]# docker-compose ps  启动harbor后必须在harbor]目录里执行此命令

[root@server1 harbor]# docker-compose start    启动一下，就会变成up

浏览器访问：172.25.50.1

[root@server1 ~]# docker login reg.westos.org   登陆
Username: admin
Password: 
Error response from daemon: Get https://reg.westos.org/v2/: x509: certificate signed by unknown authority  报错，缺少证书
[root@server1 ~]# cd /etc/docker/
[root@server1 docker]# mkdir certs.d
[root@server1 docker]# cd certs.d/
[root@server1 certs.d]# mkdir reg.westos.org
[root@server1 certs.d]# cd reg.westos.org/
[root@server1 reg.westos.org]# cp /data/certs/westos.org.crt ca.crt  将仓库证书放到指定目录里
[root@server1 ~]# docker login reg.westos.org  登陆
Username: admin
Password: 
WARNING! Your password will be stored unencrypted in /root/.docker/config.json.
Configure a credential helper to remove this warning. See
https://docs.docker.com/engine/reference/commandline/login/#credentials-store

Login Succeeded   成功
[root@server1 ~]# docker tag busybox:latest reg.westos.org/library/busybox:latest   修改标签，指定仓库位置
[root@server1 ~]# docker push reg.westos.org/library/busybox:latest   上传镜像成功
The push refers to repository [reg.westos.org/library/busybox]
d31505fd5050: Pushed 
latest: digest: sha256:b69959407d21e8a062e0416bf13405bb2b71ed7a84dde4158ebafacfa06f5578 size: 527

[root@server1 ~]# docker tag nginx:latest reg.westos.org/library/nginx:latest   改标签并指定仓库路经
[root@server1 ~]# docker push reg.westos.org/library/nginx:latest   上传镜像
The push refers to repository [reg.westos.org/library/nginx]
d874fd2bc83b: Pushed 
32ce5f6a5106: Pushed 
f1db227348d0: Pushed 
b8d6e692a25e: Pushed 
e379e8aedd4d: Pushed 
2edcec3590a4: Pushed 
latest: digest: sha256:ee89b00528ff4f02f2405e4ee221743ebc3f8e8dd0bfd5c4c20a2fa2aaa7ede3 size: 1570

新建项目westos，设置为不公开

[root@server1 harbor]# docker tag localhost:5000/game2048:latest reg.westos.org/westos/game2048  改标签并指定镜像路经
[root@server1 harbor]# docker push reg.westos.org/westos/game2048  上传镜像到westos项目上
[root@server2 ~]# docker pull reg.westos.org/library/nginx:latest   拉取成功，但是如何像官方一样不用写reg.westos.org/library/路经，直接docker pull  nginx:latest 
latest: Pulling from library/nginx
a2abf6c4d29d: Pull complete 
a9edb18cadd1: Pull complete 
589b7251471a: Pull complete 
186b1aaa4aa6: Pull complete 

上述问题解决方法：
[root@server2 ~]# cd /etc/docker/
[root@server2 docker]# vim daemon.json  编辑文件
{
  "registry-mirrors": ["https://reg.westos.org"]    更改默认仓库指向地址
}
[root@server2 docker]# systemctl daemon-reload 
[root@server2 docker]# systemctl restart docker
[root@server2 docker]# docker info  

测试：
[root@server2 docker]# docker pull busybox 直接拉取，默认镜像不用前面再跟仓库路经
Using default tag: latest
latest: Pulling from library/busybox
009932687766: Pull complete 
Digest: sha256:b69959407d21e8a062e0416bf13405bb2b71ed7a84dde4158ebafacfa06f5578
Status: Downloaded newer image for busybox:latest
docker.io/library/busybox:latest
注：拉取镜像默认去library项目里拉取，westos项目是私有的不能拉取
[root@server2 docker]# docker pull reg.westos.org/westos/game2048  拉取私有westos需要认证

创建一个普通用户


新添加成员

[root@server2 docker]# docker login reg.westos.org   登陆创建的普通用户
Username: wxh   
Password:  
WARNING! Your password will be stored unencrypted in /root/.docker/config.json.
Configure a credential helper to remove this warning. See
https://docs.docker.com/engine/reference/commandline/login/#credentials-store
Login Succeeded   登陆成功
[root@server2 docker]# docker pull reg.westos.org/westos/game2048:latest   此时拉取，就可以拉取westos里的镜像，私有拉取时仓库必须要指定仓库路经不能省略
latest: Pulling from westos/game2048
Digest: sha256:8a34fb9cb168c420604b6e5d32ca6d412cb0d533a826b313b190535c03fe9390
Status: Downloaded newer image for reg.westos.org/westos/game2048:latest

docker-compose 添加模块

[root@server1 harbor]# docker-compose stop  停止所有容器
Stopping nginx             ... done
Stopping harbor-jobservice ... done
Stopping harbor-core       ... done
Stopping registry          ... done
Stopping harbor-db         ... done
Stopping registryctl       ... done
Stopping harbor-portal     ... done
Stopping redis             ... done
Stopping harbor-log        ... done
[root@server1 harbor]# docker-compose rm   删除停掉的容器
Going to remove nginx, harbor-jobservice, harbor-core, registry, harbor-db, registryctl, harbor-portal, redis, harbor-log
Are you sure? [yN] y
Removing nginx             ... done
Removing harbor-jobservice ... done
Removing harbor-core       ... done
Removing registry          ... done
Removing harbor-db         ... done
Removing registryctl       ... done
Removing harbor-portal     ... done
Removing redis             ... done
Removing harbor-log        ... done
[root@server1 harbor]# ./install.sh --with-notary --with-clair --with-chartmuseum   添加模块，并启动

加上模块后可以添加扫描
手动扫描：

设置自动扫描

注意：自动扫描打开后，会导致子镜像越来越大

镜像签名

/etc/docker/certs.d/reg.westos.org/ca.crt  此证书已经部署了
做内容信任部署根证书
[root@server1 ~]# cd .docker/
[root@server1 .docker]# mkdir -p tls/reg.westos.org:4443/    4443为内容信任的服务端口
[root@server1 reg.westos.org:4443]# cp /etc/docker/certs.d/reg.westos.org/ca.crt .
[root@server1 reg.westos.org:4443]# pwd
/root/.docker/tls/reg.westos.org:4443
[root@server1 ~]# export DOCKER_CONTENT_TRUST=1  启用docker内容信任
[root@server1 ~]# export DOCKER_CONTENT_TRUST_SERVER=https://reg.westos.org:4443  指定谁提供的内容信任服务
[root@server1 ~]# docker push reg.westos.org/library/nginx:latest   拉取镜像
The push refers to repository [reg.westos.org/library/nginx]   
d874fd2bc83b: Pushed 
32ce5f6a5106: Pushed 
f1db227348d0: Pushed 
b8d6e692a25e: Pushed 
e379e8aedd4d: Pushed 
2edcec3590a4: Pushed 
latest: digest: sha256:ee89b00528ff4f02f2405e4ee221743ebc3f8e8dd0bfd5c4c20a2fa2aaa7ede3 size: 1570
Signing and pushing trust metadata
You are about to create a new root signing key passphrase. This passphrase
will be used to protect the most sensitive key in your signing system. Please
choose a long, complex passphrase and be careful to keep the password and the
key file itself secure and backed up. It is highly recommended that you use a
password manager to generate the passphrase and keep it safe. There will be no
way to recover this key. You can find the key in your config directory.
Enter passphrase for new root key with ID 44614d6:     设置根key的密码
Repeat passphrase for new root key with ID 44614d6:   确认根key的密码
Enter passphrase for new repository key with ID dc766c4:    设置仓库密码
Repeat passphrase for new repository key with ID dc766c4:   确认仓库密码
Finished initializing "reg.westos.org/library/nginx"
Successfully signed reg.westos.org/library/nginx:latest
注：如果上传的镜像名字变更了，根key密码就需要重新设置，如果只是版本名变更了只需要输入仓库密码即可

[root@server2 ~]# docker pull reg.westos.org/library/busybox:latest  此时拉取报错，不允许拉取没有签名的镜像
Error response from daemon: unknown: The image is not signed in Notary.
[root@server2 ~]# docker pull reg.westos.org/library/nginx:latest   nginx镜像做过签名，就可以拉取
latest: Pulling from library/nginx
Digest: sha256:ee89b00528ff4f02f2405e4ee221743ebc3f8e8dd0bfd5c4c20a2fa2aaa7ede3
Status: Downloaded newer image for reg.westos.org/library/nginx:latest
reg.westos.org/library/nginx:latest

把没有签名的busybox镜像删除

[root@server1 ~]# docker push reg.westos.org/library/busybox:latest     上传 busybox:latest镜像，并进行签名，注意镜像后面一定要加：latest
The push refers to repository [reg.westos.org/library/busybox]
d31505fd5050: Pushed 
latest: digest: sha256:b69959407d21e8a062e0416bf13405bb2b71ed7a84dde4158ebafacfa06f5578 size: 527
Signing and pushing trust metadata
Enter passphrase for root key with ID 44614d6:   设置根key的密码
Enter passphrase for new repository key with ID 8b33420:    确认根key的密码
Repeat passphrase for new repository key with ID 8b33420:  设置仓库密码
Finished initializing "reg.westos.org/library/busybox"   确认仓库密码
Successfully signed reg.westos.org/library/busybox:latest   确认仓库密码
[root@server2 ~]# docker pull reg.westos.org/library/busybox:latest  此时再次拉取busybox:latest 镜像就可以拉取了
latest: Pulling from library/busybox
Digest: sha256:b69959407d21e8a062e0416bf13405bb2b71ed7a84dde4158ebafacfa06f5578
Status: Downloaded newer image for reg.westos.org/library/busybox:latest
reg.westos.org/library/busybox:latest

删除签名：

[root@server1 ~]# docker trust revoke reg.westos.org/library/nginx:latest  删除签名
Enter passphrase for repository key with ID dc766c4: 
Successfully deleted signature for reg.westos.org/library/nginx:latest

恢复之前实验环境仓库不加模块

[root@server1 harbor]# docker-compose stop   停掉
[root@server1 harbor]# docker-compose rm  删掉所有镜像
[root@server1 harbor]# unset DOCKER_CONTENT_TRUST  去掉内容信任
[root@server1 harbor]# ./install.sh  --with-chartmuseum  只添加一个模块功能，k8s库

注：[root@server1 harbor]# vim harbor.yml   如果变更了此文件，就需要在./install之前执行
[root@server1 harbor]# ./prepare   此指令  可以