1.Docker Hub 公共镜像市场
地址:https://hub.docker.com 是官方提供的最大的公共镜像仓库
因为本地环境无法打开该网址,后续会补充进来,我们主要看看本地私有仓库
2.本地私有仓库
2.1 使用registry镜像创建私有仓库
docker -d -p 5000:5000 registry
[root@k8s-master /]# docker run -d -p 5000:5000 registry
Unable to find image 'registry:latest' locally
latest: Pulling from library/registry
7264a8db6415: Pull complete
c4d48a809fc2: Pull complete
88b450dec42e: Pull complete
121f958bea53: Pull complete
7417fa3c6d92: Pull complete
Digest: sha256:d5f2fb0940fe9371b6b026b9b66ad08d8ab7b0d56b6ee8d5c71cb9b45a374307
Status: Downloaded newer image for registry:latest
b230e1dfcca238de63240d790f9c86767c75b8cacdeef08e815dea0d946c0d9a
[root@k8s-master /]# docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
b230e1dfcca2 registry "/entrypoint.sh /etc…" 27 seconds ago Up 26 seconds 0.0.0.0:5000->5000/tcp, :::5000->5000/tcp thirsty_murdock
一般来说上传的镜像会在容器的/var/lib/registry下,我们可以用下面命令将其映射到本地路径来:
docker run -d -p 5000:5000 -v /opt/data/registry:/var/lib/registry registry
[root@k8s-master docker]# docker stop b230e1dfcca2
b230e1dfcca2
[root@k8s-master docker]# docker rm b230e1dfcca2
b230e1dfcca2
[root@k8s-master docker]# docker run -d -p 5000:5000 -v /opt/data/registry:/var/lib/registry registry
a73b864047d508453d34ea5c4781f719107db0d2f01e75822aa5380886a4e7aa
2.2 管理私有仓库
私有仓库地址:http://192.168.200.141:5000
上传镜像测试:docker push 192.168.200.141:5000/test
[root@k8s-master docker]# docker push 192.168.200.141:5000/test
Using default tag: latest
The push refers to repository [192.168.200.141:5000/test]
Get "https://192.168.200.141:5000/v2/": http: server gave HTTP response to HTTPS client
报错,因为docker是用的https协议去访问仓库,我们修改下daemon.json文件,重新上传即可
[root@k8s-master docker]# cat /etc/docker/daemon.json
{ "insecure-registries":["192.168.200.141:5000"] }
[root@k8s-master docker]# systemctl restart docker.service
[root@k8s-master docker]# docker run -d -p 5000:5000 -v /opt/date/registry:/var/lib/registry registry
6bc193c05d221a71c04988b2cba8d307c1ac9e2195e533b70cd5d7bec98bec37
[root@k8s-master docker]# docker push 192.168.200.141:5000/test
Using default tag: latest
The push refers to repository [192.168.200.141:5000/test]
dc0585a4b8b7: Pushed
latest: digest: sha256:d282a4713a33555177674cdc69b39246a739765ffd79b54679281d2da7f5e7be size: 529
tip:本地测试对镜像打标签时,直接用完整访问路径作为前缀
docker tag myservice:v1.1 localhost:5000/tang/myservice:v1.1 #把myservice:v1.1 打上带绝对路径的v1.1tag
3.私有仓库harbor的搭建
到这里算是步入了正题了,离线部署harbor仓库
3.1 安装docker-compose
[root@k8s-master docker]# wget -c https://github.com/docker/compose/releases/download/1.25.5/docker-compose-Linux-x86_64
--2023-09-25 10:44:59-- https://github.com/docker/compose/releases/download/1.25.5/docker-compose-Linux-x86_64
Resolving github.com (github.com)... 192.30.255.112
Connecting to github.com (github.com)|192.30.255.112|:443... connected.
HTTP request sent, awaiting response... 302 Found
Location: https://objects.githubusercontent.com/github-production-release-asset-2e65be/15045751/6e19c880-7b13-11ea-97d7-bec401ece2d4?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIWNJYAX4CSVEH53A%2F20230925%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20230925T024501Z&X-Amz-Expires=300&X-Amz-Signature=6e47fbafcc7073c9937ee14c568bdc41c72c7cf2a55d31f6d6dbe216ce5ae904&X-Amz-SignedHeaders=host&actor_id=0&key_id=0&repo_id=15045751&response-content-disposition=attachment%3B%20filename%3Ddocker-compose-Linux-x86_64&response-content-type=application%2Foctet-stream [following]
--2023-09-25 10:45:01-- https://objects.githubusercontent.com/github-production-release-asset-2e65be/15045751/6e19c880-7b13-11ea-97d7-bec401ece2d4?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIWNJYAX4CSVEH53A%2F20230925%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20230925T024501Z&X-Amz-Expires=300&X-Amz-Signature=6e47fbafcc7073c9937ee14c568bdc41c72c7cf2a55d31f6d6dbe216ce5ae904&X-Amz-SignedHeaders=host&actor_id=0&key_id=0&repo_id=15045751&response-content-disposition=attachment%3B%20filename%3Ddocker-compose-Linux-x86_64&response-content-type=application%2Foctet-stream
Resolving objects.githubusercontent.com (objects.githubusercontent.com)... 185.199.109.133, 185.199.111.133, 185.199.108.133
Connecting to objects.githubusercontent.com (objects.githubusercontent.com)|185.199.109.133|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 17586312 (17M) [application/octet-stream]
Saving to: ‘docker-compose-Linux-x86_64’
100%[=============================================================================================================================================================================================>] 17,586,312 500KB/s in 2m 46s
2023-09-25 10:47:50 (104 KB/s) - ‘docker-compose-Linux-x86_64’ saved [17586312/17586312]
[root@k8s-master docker]# ls
buildkit containers docker-compose-Linux-x86_64 engine-id image network overlay2 plugins runtimes swarm tmp volumes
[root@k8s-master docker]# mv docker-compose-Linux-x86_64 /usr/bin/docker-compose
[root@k8s-master docker]# chmod a+x /usr/bin/docker-compose
[root@k8s-master docker]# /usr/bin/docker-compose -v
docker-compose version 1.25.5, build 8a1c60f6
3.2 下载harbor离线安装包并解压
[root@k8s-master harbor]# wget -c https://github.com/goharbor/harbor/releases/download/v2.0.0/harbor-offline-installer-v2.0.0.tgz
[root@k8s-master harbor]# tar -xf harbor-offline-installer-v2.0.0.tgz
3.3 配置harbor
## 创建 https 证书
# 创建证书目录,并赋予权限
[root@k8s-master harbor]# mkdir /https/ca -p
[root@k8s-master harbor]# chmod -R 777 /https/ca/
[root@k8s-master harbor]# cd /https/ca/
# 生成私钥,需要设置密码:1234
[root@harbor ca]# openssl genrsa -des3 -out harbor.key 2048
Generating RSA private key, 2048 bit long modulus
.........+++
..................................+++
e is 65537 (0x10001)
Enter pass phrase for harbor.key:
Verifying - Enter pass phrase for harbor.key:
# 生成CA证书,需要输入密码1234
[root@harbor ca]# openssl req -sha512 -new \
> -subj "/C=CN/ST=JS/L=WX/O=zwx/OU=jhmy/CN=192.168.200.141" \
> -key harbor.key \
> -out harbor.csr
Enter pass phrase for harbor.key:1234
# 备份证书
[root@k8s-master ca]# cp harbor.key harbor.key.org
# 退掉私钥密码,以便docker访问(也可以参考官方进行双向认证)
[root@k8s-master ca]# openssl rsa -in harbor.key.org -out harbor.key
Enter pass phrase for harbor.key.org:
writing RSA key
# 使用证书进行签名
[root@k8s-master ca]# openssl x509 -req -days 100000 -in harbor.csr -signkey harbor.key -out harbor.crt
Signature ok
subject=/C=CN/ST=JS/L=WX/O=zwx/OU=jhmy/CN=192.168.200.141
Getting Private key
[root@harbor ~]# cd harbor/
[root@harbor harbor]# vim harbor.yml
hostname: 192.168.200.141
http:
port: 8080
https:
# https port for harbor, default is 443
port: 443
# The path of cert and key files for nginx
certificate: /https/ca/harbor.crt
private_key: /https/ca/harbor.key
harbor_admin_password: admin
3.4 安装harbor
#--with-clair参数是启用漏洞扫描功能
[root@harbor harbor]# ./install.sh --with-clair
[Step 0]: checking if docker is installed ...
Note: docker version: 19.03.9
[Step 1]: checking docker-compose is installed ...
Note: docker-compose version: 1.25.5
[Step 2]: loading Harbor images ...
Loaded image: goharbor/notary-signer-photon:v2.0.0
Loaded image: goharbor/clair-adapter-photon:v2.0.0
Loaded image: goharbor/chartmuseum-photon:v2.0.0
Loaded image: goharbor/harbor-log:v2.0.0
Loaded image: goharbor/harbor-registryctl:v2.0.0
Loaded image: goharbor/registry-photon:v2.0.0
Loaded image: goharbor/clair-photon:v2.0.0
Loaded image: goharbor/notary-server-photon:v2.0.0
Loaded image: goharbor/redis-photon:v2.0.0
Loaded image: goharbor/nginx-photon:v2.0.0
Loaded image: goharbor/harbor-core:v2.0.0
Loaded image: goharbor/harbor-db:v2.0.0
Loaded image: goharbor/harbor-jobservice:v2.0.0
Loaded image: goharbor/trivy-adapter-photon:v2.0.0
Loaded image: goharbor/prepare:v2.0.0
Loaded image: goharbor/harbor-portal:v2.0.0
[Step 3]: preparing environment ...
[Step 4]: preparing harbor configs ...
prepare base dir is set to /root/harbor
Clearing the configuration file: /config/log/logrotate.conf
Clearing the configuration file: /config/log/rsyslog_docker.conf
Clearing the configuration file: /config/nginx/nginx.conf
Clearing the configuration file: /config/core/env
Clearing the configuration file: /config/core/app.conf
Clearing the configuration file: /config/registry/passwd
Clearing the configuration file: /config/registry/config.yml
Clearing the configuration file: /config/registry/root.crt
Clearing the configuration file: /config/registryctl/env
Clearing the configuration file: /config/registryctl/config.yml
Clearing the configuration file: /config/db/env
Clearing the configuration file: /config/jobservice/env
Clearing the configuration file: /config/jobservice/config.yml
Clearing the configuration file: /config/clair/postgresql-init.d/README.md
Clearing the configuration file: /config/clair/postgres_env
Clearing the configuration file: /config/clair/config.yaml
Clearing the configuration file: /config/clair/clair_env
Clearing the configuration file: /config/clair-adapter/env
Generated configuration file: /config/log/logrotate.conf
Generated configuration file: /config/log/rsyslog_docker.conf
Generated configuration file: /config/nginx/nginx.conf
Generated configuration file: /config/core/env
Generated configuration file: /config/core/app.conf
Generated configuration file: /config/registry/config.yml
Generated configuration file: /config/registryctl/env
Generated configuration file: /config/registryctl/config.yml
Generated configuration file: /config/db/env
Generated configuration file: /config/jobservice/env
Generated configuration file: /config/jobservice/config.yml
loaded secret from file: /data/secret/keys/secretkey
Copying offline data file for clair DB
Generated configuration file: /config/clair/postgres_env
Generated configuration file: /config/clair/config.yaml
Generated configuration file: /config/clair/clair_env
Generated configuration file: /config/clair-adapter/env
Generated configuration file: /compose_location/docker-compose.yml
Clean up the input dir
Note: stopping existing Harbor instance ...
Stopping nginx ... done
Stopping harbor-jobservice ... done
Stopping clair-adapter ... done
Stopping harbor-core ... done
Stopping clair ... done
Stopping registryctl ... done
Stopping harbor-db ... done
Stopping redis ... done
Stopping registry ... done
Stopping harbor-portal ... done
Stopping harbor-log ... done
Removing nginx ... done
Removing harbor-jobservice ... done
Removing clair-adapter ... done
Removing harbor-core ... done
Removing clair ... done
Removing registryctl ... done
Removing harbor-db ... done
Removing redis ... done
Removing registry ... done
Removing harbor-portal ... done
Removing harbor-log ... done
Removing network harbor_harbor
Removing network harbor_harbor-clair
[Step 5]: starting Harbor ...
Creating network "harbor_harbor" with the default driver
Creating network "harbor_harbor-clair" with the default driver
Creating harbor-log ... done
Creating redis ... done
Creating harbor-portal ... done
Creating harbor-db ... done
Creating registryctl ... done
Creating registry ... done
Creating clair ... done
Creating harbor-core ... done
Creating clair-adapter ... done
Creating harbor-jobservice ... done
Creating nginx ... done
✔ ----Harbor has been installed and started successfully.----
````
```shell
[root@harbor ~]# docker ps | grep harbor
80e271e93f01 goharbor/nginx-photon:v2.0.0 "nginx -g 'daemon of…" 58 seconds ago Up 56 seconds (healthy) 0.0.0.0:80->8080/tcp nginx
381d9b6acd70 goharbor/harbor-jobservice:v2.0.0 "/harbor/entrypoint.…" 58 seconds ago Up 56 seconds (healthy) harbor-jobservice
a671be5e41de goharbor/clair-adapter-photon:v2.0.0 "/home/clair-adapter…" 59 seconds ago Up 57 seconds (healthy) 8080/tcp clair-adapter
7299713c6d14 goharbor/harbor-core:v2.0.0 "/harbor/entrypoint.…" 59 seconds ago Up 57 seconds (healthy) harbor-core
40a513e14e56 goharbor/clair-photon:v2.0.0 "./docker-entrypoint…" About a minute ago Up 54 seconds (healthy) 6060-6061/tcp clair
6332dfdc874d goharbor/harbor-db:v2.0.0 "/docker-entrypoint.…" About a minute ago Up 59 seconds (healthy) 5432/tcp harbor-db
f19de1b32a9b goharbor/redis-photon:v2.0.0 "redis-server /etc/r…" About a minute ago Up 59 seconds (healthy) 6379/tcp redis
7c216088e9bf goharbor/registry-photon:v2.0.0 "/home/harbor/entryp…" About a minute ago Up 59 seconds (healthy) 5000/tcp registry
420c77a7692a goharbor/harbor-registryctl:v2.0.0 "/home/harbor/start.…" About a minute ago Up 59 seconds (healthy) registryctl
00abe613b13c goharbor/harbor-portal:v2.0.0 "nginx -g 'daemon of…" About a minute ago Up 59 seconds (healthy) 8080/tcp harbor-portal
d7634d1b25e4 goharbor/harbor-log:v2.0.0 "/bin/sh -c /usr/loc…" About a minute ago Up About a minute (healthy) 127.0.0.1:1514->10514/tcp harbor-log
[root@harbor ~]# docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
goharbor/chartmuseum-photon v2.0.0 4db8d6aa63e9 2 weeks ago 127MB
goharbor/redis-photon v2.0.0 c89ea2e53cc0 2 weeks ago 72.2MB
goharbor/trivy-adapter-photon v2.0.0 6122c52b7e48 2 weeks ago 103MB
goharbor/clair-adapter-photon v2.0.0 dd2210cb7f53 2 weeks ago 62MB
goharbor/clair-photon v2.0.0 f7c7fcc52278 2 weeks ago 171MB
goharbor/notary-server-photon v2.0.0 983ac10ed8be 2 weeks ago 143MB
goharbor/notary-signer-photon v2.0.0 bee1b6d75e0d 2 weeks ago 140MB
goharbor/harbor-registryctl v2.0.0 c53c32d58d04 2 weeks ago 102MB
goharbor/registry-photon v2.0.0 afdc1b7ada36 2 weeks ago 84.5MB
goharbor/nginx-photon v2.0.0 17892f03e56c 2 weeks ago 43.6MB
goharbor/harbor-log v2.0.0 5f8ff08e795c 2 weeks ago 82MB
goharbor/harbor-jobservice v2.0.0 c68a2495bf55 2 weeks ago 116MB
goharbor/harbor-core v2.0.0 3aa3af64baf8 2 weeks ago 138MB
goharbor/harbor-portal v2.0.0 e0b1d3c894c4 2 weeks ago 52.4MB
goharbor/harbor-db v2.0.0 5c76f0296cec 2 weeks ago 154MB
goharbor/prepare v2.0.0 7266d49995ed 2 weeks ago 158MB
3.5 测试本地访问
tip:本地访问https://192.168.200.141:8080/harbo失败
查看了防火墙和selinux 都是关闭的,然后在宿主机或者k8s-node1节点机器telnet 192.168.200.141 8080失败,运行sysctl net.ipv4.ip_forward 查看值为0,表示未开启端口转发,修改1重启网络服务即可(systemctl restart network)
3.6 配置私有仓库
点击系统管理->用户管理->创建用户
3.6.1 用户管理
3.6.2 用户添加
3.6.3 用户信息
3.7 项目管理
点击项目->新建项目
3.7.1 新建项目
3.7.2 项目信息
3.8 拉取镜像,打标上传到我的私有仓库
3.8.1 修改仓库地址,重启docker,harbor服务
增加本地仓库地址"insecure-registries": ["http://192.168.200.141:8080"到/etc/docker/daemon.json文件
[root@k8s-master harbor]# cat /etc/docker/daemon.json
{
"registry-mirrors": ["https://7bc3o1s2.mirror.aliyuncs.com"],
"insecure-registries": ["http://192.168.200.141:8080"]
}
#重启docker服务
[root@k8s-master harbor]# systemctl restart docker.service
#重启harbor服务
[root@k8s-master harbor]# docker-compose up -d
3.8.2 拉取mysql作为测试镜像
[root@k8s-master harbor]# docker pull mysql
Using default tag: latest
latest: Pulling from library/mysql
72a69066d2fe: Pull complete
93619dbc5b36: Pull complete
99da31dd6142: Pull complete
626033c43d70: Pull complete
37d5d7efb64e: Pull complete
ac563158d721: Pull complete
d2ba16033dad: Pull complete
688ba7d5c01a: Pull complete
00e060b6d11d: Pull complete
1c04857f594f: Pull complete
4d7cfa90e6ea: Pull complete
e0431212d27d: Pull complete
Digest: sha256:e9027fe4d91c0153429607251656806cc784e914937271037f7738bd5b8e7709
Status: Downloaded newer image for mysql:latest
docker.io/library/mysql:latest
3.8.3 打tag标签
[root@k8s-master harbor]# docker tag mysql:latest 192.168.200.141:8080/test/mysql:test0.1
3.8.4 登录harbor私有仓库
[root@k8s-master harbor]# docker login http://192.168.200.141:8080
Username: test
Password:
WARNING! Your password will be stored unencrypted in /root/.docker/config.json.
Configure a credential helper to remove this warning. See
https://docs.docker.com/engine/reference/commandline/login/#credentials-store
Login Succeeded
3.8.5 上传tag标签镜像到harbor私有仓库
[root@k8s-master harbor]# docker push 192.168.200.141:8080/test/mysql:test0.1
The push refers to repository [192.168.200.141:8080/test/mysql]
d67a9f3f6569: Pushed
fc8a043a3c75: Pushed
118fee5d988a: Pushed
c654c2afcbba: Pushed
1d1f48e448f9: Pushed
aad27784b762: Pushed
0d17fee8db40: Pushed
d7a777f6c3a4: Pushed
a0c2a050fee2: Pushed
0798f2528e83: Pushed
fba7b131c5c3: Pushed
ad6b69b54919: Pushed
test0.1: digest: sha256:238cf050a7270dd6940602e70f1e5a11eeaf4e02035f445b7f613ff5e0641f7d size: 2828
登录仓库地址也能看到刚上传镜像:
4.注意事项
4.1 添加开机自启动服务
[root@k8s-master system]# cat /etc/systemd/system/harbor.service
[Unit]
Description=harbor
After=docker.service systemd-networkd.service systemd-resolved.service
Requires=docker.service
Documentation=http://github.com/vmware/harbor
[Service]
Type=simple
Restart=on-failure
RestartSec=5
ExecStart=/usr/bin/docker-compose -f /root/harbor/docker-compose.yml up
ExecStop=/usr/bin/docker-compose -f /root/harbor/docker-compose.yml down
#harbro installed dir:/root/harbor/docker-compose.yml
[Install]
WantedBy=multi-user.target
后续有遇到其他的tips会陆续在补充进来