安装harbor
harbor是容器镜像注册仓库
官方首页:
https://goharbor.io/
官方文档:
-
安装和配置文档(针对版本2.1)
https://goharbor.io/docs/2.1.0/install-config/ -
下载地址:
https://github.com/goharbor/harbor/releases -
将docker login的凭据保存在外部存储
https://docs.docker.com/engine/reference/commandline/login/#credentials-store
当前版本:2.1.3 (2021.2.1)
系统要求:
- docker 17.06.0-ce+
- docker-compose 1.18.0+
1.前置任务
准备如下包或软件
- SELinux :
permessive
- docker : 17.06.0-ce+
- docker-compose : 1.18.0+
- harbor离线安装包 : 文件名:
harbor-offline-installer-v2.1.3.tgz
.目前版本是2.1.3 (2021.2.1) - 自签名证书:
ca.crt
server.crt 和 server.key
注意:server.crt
中的DNS和IP必须包含本服务器的主机名和IPclient.crt
,client.key
docker daemon为私库的客户端,因此使用客户端证书
1.1 硬件要求:
https://goharbor.io/docs/2.1.0/install-config/installation-prereqs/
Resource Minimum Recommended
CPU 2 CPU 4 CPU
Mem 4 GB 8 GB
Disk 40 GB 160 GB
网络端口(可以在配置文件中修改):
Port Protocol Description
443 HTTPS
4443 HTTPS
80 HTTP
2.安装步骤:
安装分为2部分:
- 安装harbor私库
- 设置k8s的worker节点的docker,以便能访问harbor私库(k8s的master节点一般不会下载应用的镜像,因此无需设置master节点的docker)
2.1 安装harbor私库
1.下载docker-compose
,并拷贝到/usr/local/bin
目录
chmod +x docker-compose-Linux-x86_64
cp -p docker-compose-Linux-x86_64 /usr/local/bin/docker-compose
2.上载harbor离线安装包到~/tool
目录
3.编辑.~/.bashrc
,设置harbor的环境变量。并让设置生效
# 先备份
cp -p ~/.bashrc ~/.bashrc.20210202
tee -a >>~/.bashrc<<EOF
# harbor
HARBOR_ROOT=~/program
export HARBOR_ROOT
EOF
# 让设置生效
source ~/.bashrc
4.解压harbor的TAR包,得到harbor
子目录
# 创建目录
mkdir -p $HARBOR_ROOT
cd $HARBOR_ROOT
tar -zvxf ~/tool/harbor-offline-installer-v2.1.3.tgz
5.配置$HARBOR_HOME/harbor.yml
文件
cd $HARBOR_ROOT/harbor
# 从模板复制配置文件
cp -p harbor.yml.tmpl harbor.yml
必须配置的参数如下:
- hostname : 可以是IP地址
sed -i "s/^hostname:.*/hostname: 192.168.8.xxx(本机IP地址或域名)/" harbor.yml
- https下的
certificate
和private_key
假设服务端证书在~/certs
目录下
certificate: ~/certs/server.crt
private_key: ~/certs/server.key
- harbor_admin_password(可选) : 管理员密码.缺省用户名是
admin
,密码是Harbor12345
。此密码只用于第一次启动harbor,之后的密码在门户中设置 - data_volume(可选) : 存放harbor数据的目录.缺省是
/data
9.执行harbor的install.sh
脚本
cd $HARBOR_ROOT/harbor
# 启用上述所有选项
./install.sh
输出如下:
[Step 0]: checking if docker is installed ...
Note: docker version: 20.10.2
[Step 1]: checking docker-compose is installed ...
Note: docker-compose version: 1.28.2
[Step 2]: loading Harbor images ...
[Step 3]: preparing environment ...
...
[Step 4]: preparing harbor configs ...
prepare base dir is set to /root/program/harbor
Generated configuration file: /config/portal/nginx.conf
Generated configuration file: /config/log/logrotate.conf
Generated configuration file: /config/log/rsyslog_docker.conf
Generated configuration file: /config/nginx/nginx.conf
Generated configuration file: /config/core/env
Generated configuration file: /config/core/app.conf
Generated configuration file: /config/registry/config.yml
Generated configuration file: /config/registryctl/env
Generated configuration file: /config/registryctl/config.yml
Generated configuration file: /config/db/env
Generated configuration file: /config/jobservice/env
Generated configuration file: /config/jobservice/config.yml
Generated and saved secret to file: /data/secret/keys/secretkey
Successfully called func: create_root_cert
Successfully called func: create_root_cert
Successfully called func: create_cert
Copying certs for notary signer
Copying nginx configuration file for notary
Generated configuration file: /config/nginx/conf.d/notary.upstream.conf
Generated configuration file: /config/nginx/conf.d/notary.server.conf
Generated configuration file: /config/notary/server-config.postgres.json
Generated configuration file: /config/notary/server_env
Generated and saved secret to file: /data/secret/keys/defaultalias
Generated configuration file: /config/notary/signer_env
Generated configuration file: /config/notary/signer-config.postgres.json
Generated configuration file: /config/clair/postgres_env
Generated configuration file: /config/clair/config.yaml
Generated configuration file: /config/clair/clair_env
Generated configuration file: /config/clair-adapter/env
Generated configuration file: /config/chartserver/env
Generated configuration file: /compose_location/docker-compose.yml
Clean up the input dir
[Step 5]: starting Harbor ...
Building with native build. Learn about native build in Compose here: https://docs.docker.com/go/compose-native-build/
Creating network "harbor_harbor" with the default driver
Creating network "harbor_harbor-clair" with the default driver
Creating network "harbor_harbor-notary" with the default driver
Creating network "harbor_harbor-chartmuseum" with the default driver
Creating network "harbor_notary-sig" with the default driver
Creating harbor-log ... done
Creating registryctl ... done
Creating chartmuseum ... done
Creating harbor-portal ... done
Creating registry ... done
Creating redis ... done
Creating harbor-db ... done
Creating clair ... done
Creating notary-signer ... done
Creating harbor-core ... done
Creating clair-adapter ... done
Creating notary-server ... done
Creating nginx ... done
Creating harbor-jobservice ... done
✔ ----Harbor has been installed and started successfully.----
注意:如果执行install.sh
发生报错,则多半是harobr.yml
的格式错误导致
10.页面访问(假设本机IP为192.168.8.172
):
https://192.168.8.172
- 用户名:
admin
- 初始密码:
Harbor12345
11.设置harbor为系统服务
服务文件harbor.service
的内容如下:
[Unit]
Description=Start Harbor private registry
Documentation=https://goharbor.io/
After=docker.service
[Service]
Type=oneshot
RemainAfterExit=yes
WorkingDirectory=/root/program/harbor
ExecStart=/usr/local/bin/docker-compose up -d
ExecStop=/usr/local/bin/docker-compose down
[Install]
WantedBy=multi-user.target
1.修改harbor.service
中的WorkingDirectory
的值为$HARBOR_ROOT/harbor
的值,比如:/root/program/harbor
2.将harbor.service
复制到/etc/systemd/system
目录下
3.执行如下命令
systemctl daemon-reload
systemctl start harbor
systemctl status harbor
systemctl enable harbor
2.2 设置k8s的worker节点的docker,以便能访问harbor私库
假设harbor私库安装在192.168.8.172
1.为docker配置证书
在/etc/docker
目录下创建certs.d/192.168.8.172
子目录
cd /etc/docker
mkdir -p certs.d/192.168.8.172
2.将ca.crt,client.crt 和 client.key
拷贝到/etc/docker/certs.d/192.168.8.172
子目录下,且将文件更名为
- CA证书
ca.crt
- 服务端证书
client.crt
->client.cert
**注意:**证书的扩展名必须是.cert
- 服务端私钥
client.key
->client.key
3.重启docker
systemctl restart docker
3. 管理Harbor
在$HARBOR_HOME
目录下执行docker-compose
命令,以便手动启动或停止harbor
3.1停止harbor
cd $HARBOR_HOME
docker-compose down
3.2以后台方式启动harbor
docker-compose up -d
3.3重新配置harbor
# 删除容器且保持镜像和数据库
docker-compose down -v
vim harbor.yml
./prepare --with-notary --with-clair --with-chartmuseum
docker-compose up -d
缺省下,registry仓库数据存放在/data目录.日志在/var/log/harbor目录
4.将镜像推到harbor
1.对原镜像重新打标签.新标签必须是IP/library/xxx:yyy
格式,比如:192.168.8.172/library/pause:3.2
docker tag registry.aliyuncs.com/google_containers/pause:3.2 \
192.168.8.172/library/pause:3.2
# admin/Harbor12345
docker login 192.168.8.172
docker push 192.168.8.172/library/pause:3.2
在执行docker login 192.168.8.172
后,会生成~/.docker/config.json
,内容如下:
{
"auths": {
"192.168.8.172": {
"auth": "YWRtaW46SGFyYm9yMTIzNDU="
}
}
}
5.在k8s中拉取私库镜像
前置条件:
- 私库安装完毕
- 私库服务器已信任ca(需要重启服务器)
- k8s的worker节点已设置docker为私库的客户端
5.1 拉取原版镜像
docker pull alpine
5.2 重新打标签
docker tag alpine:latest 192.168.2.28/library/alpine:latest
5.3 登录私库
docker login 192.168.2.28 -u admin -p Harbor12345
5.4 推送镜像到私库
docker push 192.168.2.28/library/alpine:latest
5.5 创建k8s的secret对象
kubectl create secret docker-registry registry-secret
–docker-username=admin
–docker-password=Harbor12345
5.6 创建k8s的service account对象,并引用上面的secret
apiVersion: v1
kind: ServiceAccount
metadata:
name: mysa
imagePullSecrets:
- name: registry-secret
5.7 创建k8s的deployment对象,并设置podspec的serviceAccountName值为上述sa
apiVersion: apps/v1
kind: Deployment
metadata:
name: myjenkins
spec:
以下为pod模板
template:
spec:
restartPolicy: Always
# 用于运行和拉取镜像的SA
serviceAccountName: mysa
containers:
- name: jenkins
image: 192.168.2.28/library/alpine:latest
imagePullPolicy: IfNotPresent