容器技术
第三章 记录docker-compose使用和Harbor的部署过程
记录docker-compose使用和Harbor的部署过程
1. 使用CentOS 7.9系列的Linux操作系统
2. docker社区版：docker-ce
3. docker部署编排工具：docker-compose
4. docker私有镜像仓库管理工具：docker-harbor
5. docker轻量级可视化容器管理工具：Portainer
# 默认已初始化安装 CentOS 7.9 + docker-ce
CPU: 4核 * 2
Memory: 16G
Disk: 2块物理硬盘(sda,sdb) sda: 40GB(预装最小化Linux), sdb: 200GB
Swap: 12G
hostname: docker01.myside.com
ip: 10.0.0.210
gateway: 10.0.0.254
dns: 223.5.5.5 114.114.114.114
docker应用的映射存储目录: /opt/mydocker
设置docker服务端的目录结构
/opt/mydocker/
├── certs # 存放证书的目录, 按项目名称划分, ca证书放当前目录下
│ ├── ca.crt # CA根证书
│ ├── ca.key # CA私钥
│ └── harbor # Harbor项目的SSL证书目录
├── docker-root # docker-ce应用的目录
├── packages # 存储安装包、软件包
│ ├── harbor
│ └── harbor-offline-installer-v2.10.2.tgz
└── projects
└── harbor # Harbor项目的数据目录
Harbor
https方式部署：测试环境部署使用自签名SSL证书
测试环境部署
## harbor配置HTTPS方式部署, 测试环境使用自签名SSL证书, 正式环境使用企业颁发的SSL证书
# 配置CA证书(CN=mysite.com) 和 自签名SSL证书(CN=harbor.mysite.com SAN=DNS:harbor.mysite.com,DNS:*.harbor.mysite.com)
cd /opt/mydocker/certs
# 生成CA的私钥CA证书 ca.crt
openssl genrsa -out ca.key 4096
openssl req -x509 -new -nodes -sha512 -days 3650 -subj "/C=CN/ST=Beijing/L=Beijing/O=example/OU=Personal/CN=mysite.com" \
-key ca.key -out ca.crt
# 生成私钥和证书签名请求文件(CSR)
openssl genrsa -out harbor/harbor.mysite.com.key 4096
openssl req -sha512 -new -subj "/C=CN/ST=Beijing/L=Beijing/O=example/OU=Personal/CN=harbor.mysite.com" \
-key harbor/harbor.mysite.com.key -out harbor/harbor.mysite.com.csr
# 给harbor服务生成临时的x509 v3 扩展文件，使CA签发时附加SAN及更多扩展属性
cat <<EOF > v3.ext
authorityKeyIdentifier=keyid,issuer
basicConstraints=CA:FALSE
keyUsage = digitalSignature, nonRepudiation, keyEncipherment, dataEncipherment
extendedKeyUsage = serverAuth
subjectAltName = @alt_names
[alt_names]
DNS.1=harbor.mysite.com
DNS.2=*.harbor.mysite.com
EOF
# 使用CA证书给CSR文件签发CRT证书
openssl x509 -req -sha512 -days 3650 -extfile v3.ext -CA ca.crt -CAkey ca.key -CAcreateserial \
-in harbor/harbor.mysite.com.csr -out harbor/harbor.mysite.com.crt
https方式部署：正式环境部署使用企业颁发的SSL证书
正式环境部署
# 正式环境部署Harbor的SSL证书仅需要生成 私钥、证书签名请求(CSR)，使用企业CA给csr颁发crt证书即可
# 证书签名请求(CSR)一般要手工添加SAN扩展属性
openssl genrsa -out harbor/harbor.mysite.com.key 4096
openssl req -sha512 -new -subj "/C=CN/ST=Beijing/L=Beijing/O=Technology Co., Ltd./OU=Department/CN=harbor.mysite.com" \
-reqexts SAN -config <(cat /etc/pki/tls/openssl.cnf \
<(printf "[SAN]\nsubjectAltName=DNS:harbor.mysite.com,*.harbor.mysite.com")) \
-key harbor/harbor.mysite.com.key -out harbor/harbor.mysite.com.csr
给Docker守护进程添加Harbor的SSL证书认证
# Docker 守护进程会把 .crt当成CA证书 .cert当成客户端证书。所以这里要将harbor的ssl证书转换成后缀.cert的pem格式证书
# 文件内容是一样的可以用 cp ，不是pem格式则使用 openssl x509 -inform PEM 转换成pem格式
\cp harbor/harbor.mysite.com.crt harbor/harbor.mysite.com.cert
openssl x509 -inform PEM -in harbor/harbor.mysite.com.crt -out harbor/harbor.mysite.com.cert
# 复制证书文件到docker目录, 为后续的 docker login 提供证书认证。 login缺失下列文件将报错，见下文
mkdir -p /etc/docker/certs.d/harbor.mysite.com
cp ca.crt harbor/harbor.mysite.com.cert harbor/harbor.mysite.com.key /etc/docker/certs.d/harbor.mysite.com/
安装Harbor
安装 Harbor
## 官方tag包独立安装。 如下方法是获取到latest节点的版本，但仓库会存在有-rc的候选测试版本，建议从github仓库对照过最新的正式版本tag再构建二进制包的下载命令
HARBOR_TAG=`curl -s https://api.github.com/repos/goharbor/harbor/releases/latest | grep -i tag_name | awk -F '"' '{print $4}'`
wget -P /opt/mydocker/packages/ https://github.com/goharbor/harbor/releases/download/$HARBOR_TAG/harbor-offline-installer-$HARBOR_TAG.tgz
cd /opt/mydocker/packages/
tar zxvf harbor-offline-installer-$HARBOR_TAG.tgz; cd harbor
cp harbor.yml.tmpl harbor.yml
sed -i '/^$\|^\s*#/d' harbor.yml # 删除空行和注释行
# (可选)给core组件设置quota_update_provider，配额更新提供程序为redis
cat <<EOF >> harbor.yml
core:
quota_update_provider: redis
EOF
# harbor.yml
hostname: harbor.mysite.com # 主机名
http:
port: 80
https:
port: 443
certificate: /opt/mydocker/certs/harbor/harbor.mysite.com.crt # harbor的ssl证书，CN要跟hostname一致，否则会出现HSTS问题
private_key: /opt/mydocker/certs/harbor/harbor.mysite.com.key # harbor的ssl证书私钥
harbor_admin_password: Harbor12345
database:
password: root123
max_idle_conns: 100
max_open_conns: 900
conn_max_lifetime: 5m
conn_max_idle_time: 0
data_volume: /opt/mydocker/projects/harbor # 指定宿主机的数据目录
trivy:
ignore_unfixed: false
skip_update: false
skip_java_db_update: false
offline_scan: false
security_check: vuln
insecure: false
jobservice:
max_job_workers: 10
job_loggers:
- STD_OUTPUT
- FILE
logger_sweeper_duration: 1 #days
notification:
webhook_job_max_retry: 3
webhook_job_http_client_timeout: 3 #seconds
log:
level: info
local:
rotate_count: 50
rotate_size: 200M
location: /var/log/harbor
_version: 2.10.0
proxy:
http_proxy:
https_proxy:
no_proxy:
components:
- core
- jobservice
- trivy
upload_purging:
enabled: true
age: 168h
interval: 24h
dryrun: false
cache:
enabled: true # 开启cache缓存
expire_hours: 24
core:
quota_update_provider: redis
[root@docker01 harbor]# ./install.sh # 执行harbor目录下的install.sh安装脚本
[Step 5]: starting Harbor ...
WARN[0000] /opt/mydocker/packages/harbor/docker-compose.yml: `version` is obsolete
[+] Running 10/10
✔ Network harbor_harbor Created
✔ Container harbor-log Started
✔ Container harbor-db Started
✔ Container harbor-portal Started
✔ Container registry Started
✔ Container registryctl Started
✔ Container redis Started
✔ Container harbor-core Started
✔ Container harbor-jobservice Started
✔ Container nginx Started
✔ ----Harbor has been installed and started successfully.----
# 出现如上字样 说明harbor安装完成了
验证Harbor
# 查看harbor的监听端口和配置
# 在/opt/mydocker/packages/harbor/ 目录下执行 docker compose [OPTIONS]
# 或任意目录下执行 docker compose -f /opt/mydocker/packages/harbor/docker-compose.yml [OPTIONS]
[root@docker01 harbor]# docker compose ps
WARN[0000] /opt/mydocker/packages/harbor/docker-compose.yml: `version` is obsolete
NAME IMAGE COMMAND SERVICE CREATED STATUS PORTS
harbor-core goharbor/harbor-core:v2.10.2 "/harbor/entrypoint.…" core 6 minutes ago Up 6 minutes (healthy)
harbor-db goharbor/harbor-db:v2.10.2 "/docker-entrypoint.…" postgresql 6 minutes ago Up 6 minutes (healthy)
harbor-jobservice goharbor/harbor-jobservice:v2.10.2 "/harbor/entrypoint.…" jobservice 6 minutes ago Up 6 minutes (healthy)
harbor-log goharbor/harbor-log:v2.10.2 "/bin/sh -c /usr/loc…" log 6 minutes ago Up 6 minutes (healthy) 127.0.0.1:1514->10514/tcp
harbor-portal goharbor/harbor-portal:v2.10.2 "nginx -g 'daemon of…" portal 6 minutes ago Up 6 minutes (healthy)
nginx goharbor/nginx-photon:v2.10.2 "nginx -g 'daemon of…" proxy 6 minutes ago Up 6 minutes (healthy) 0.0.0.0:80->8080/tcp, 0.0.0.0:443->8443/tcp
redis goharbor/redis-photon:v2.10.2 "redis-server /etc/r…" redis 6 minutes ago Up 6 minutes (healthy)
registry goharbor/registry-photon:v2.10.2 "/home/harbor/entryp…" registry 6 minutes ago Up 6 minutes (healthy)
registryctl goharbor/harbor-registryctl:v2.10.2 "/home/harbor/start.…" registryctl 6 minutes ago Up 6 minutes (healthy)
[root@docker01 harbor]# iptables -t nat -nL DOCKER
[root@docker01 harbor]# netstat -tnlp | grep docker
构建自定义镜像
Dockerfile构建自定义镜像
# python+flask项目镜像, name:dingtalk-monitorbook tag:1.0
# 自定义镜像的ENV环境变量建议以 _ 开头，方便区分基础镜像与自定义镜像的ENV参数
[root@docker01 dingtalk-monitorbook]# vim Dockerfile
# 一阶段构建
FROM python:3.9-buster as builder-image
# 安装py项目的requirements依赖文件
COPY requirements.txt .
RUN pip3 install --no-cache-dir -r requirements.txt -i https://pypi.tuna.tsinghua.edu.cn/simple
# 二阶段构建
FROM python:3.9-slim-buster
# 作者信息
LABEL maintainer "ww1372247148@163.com"
# 复制python的二进制执行文件和包文件
COPY --from=builder-image /usr/local/bin /usr/local/bin
COPY --from=builder-image /usr/local/lib/python3.9/site-packages /usr/local/lib/python3.9/site-packages
# 环境变量: py应用的根目录 py应用的监听端口 设置容器不产生.pyc文件 设置关闭缓冲以方便容器日志记录
ENV _CODE_DIR=/app \
_APP_PORT=8000 \
PYTHONDONTWRITEBYTECODE=1 \
PYTHONUNBUFFERED=1
# 设置工作目录
WORKDIR $_CODE_DIR
# 复制源码文件
COPY . /app
# 推荐容器内使用非root用户运行进程, 配合.dockerignore 尽可能的删除非项目相关的文件
RUN adduser -u 5678 --disabled-password --gecos "" appuser; chown -R appuser /app; \
rm -r /app/requirements.txt
USER appuser
# 暴露TCP端口给容器做端口映射
EXPOSE $_APP_PORT
# 添加网站的健康检查
#HEALTHCHECK CMD curl --fail -s http://localhost:$_APP_PORT || exit 1
# 运行python命令
CMD ["python3","app.py"]
# 项目的main入口文件可能不是app.py，为保留项目的目录结构，使用 ln -s 挂载 app.py
[root@docker01 dingtalk-monitorbook]# ln -s [项目实际入口文件] ./app.py
# 新增.dockerignore文件, 根据项目目录结构添加忽略的文件
[root@docker01 dingtalk-monitorbook]# vim .dockerignore
*Dockerfile*
*docker-compose*.yml
readme.md
README.md
.git/
.gitignore
.dockerignore
assets/
public/
# Dockerfile构建镜像
docker build -t wwzhg77777/dingtalk-monitorbook:1.0 -f Dockerfile .
# 查看镜像构建历史
docker history wwzhg77777/dingtalk-monitorbook:1.0
docker run运行镜像，生成容器
# docker run 测试镜像是否可运行. 推荐设置 限制CPU核数、内存容量硬限制、容器不使用swap内存
docker run -itd --name dingtalk-monitorbook -p 8000:8000 --restart=unless-stopped --cpus=1 -m 256m --memory-swappiness=0 wwzhg77777/dingtalk-monitorbook:1.0
# 查看容器运行日志
docker logs dingtalk-monitorbook
使用docker-compose容器编排生成容器
# docker-compose.yml
version: "3"
services:
dingtalk-monitorbook:
container_name: dingtalk-monitorbook
ports:
- 8000:8000
restart: unless-stopped
deploy:
resources:
limits:
cpus: "1"
memory: 256m
networks:
- dingtalk-monitorbook
image: wwzhg77777/dingtalk-monitorbook:1.0
networks:
dingtalk-monitorbook:
name: dingtalk-monitorbook
docker-compose.yml构建镜像并生成容器
# docker-compose-build.yml
version: "3"
services:
dingtalk-monitorbook:
container_name: dingtalk-monitorbook
image: wwzhg77777/dingtalk-monitorbook:1.0
build:
context: . # 指定当前目录
dockerfile: Dockerfile # 指定Dockerfile的文件名称
ports:
- 8000:8000
restart: unless-stopped
deploy:
resources:
limits:
cpus: "1"
memory: 256m
networks:
- dingtalk-monitorbook
networks:
dingtalk-monitorbook:
name: dingtalk-monitorbook
# docker-compose 测试镜像是否可运行
docker compose up -d
# docker-compose 先build再生成容器
docker compose up -d --build
# 查看容器运行日志
docker compose logs
(可选)使用composerize生成docker-compose.yml
# (可选)使用 Composerize 将Container running之后的容器配置转换成 docker-compose.yml 再进行优化
# Composerize github: https://github.com/composerize/composerize
# Composerize的在线网站: https://www.composerize.com/
# Composerize的容器下载(仅支持web访问): https://hub.docker.com/r/gettionhub/composerize
# 自己构建一个composerize镜像, 设置alias别名 用于CLI执行composerize
cat <<EOF > Dockerfile
FROM node:14-alpine
RUN npm install composerize -g
ENTRYPOINT ["composerize"]
EOF
docker build -t local-composerize .
echo "alias composerize='docker run -it --rm local-composerize'" >> /etc/profile.d/_alias.sh
source /etc/profile.d/_alias.sh
[root@docker01 ~]# composerize docker run -itd --name dingtalk-monitorbook -p 8000:8000 --restart=unless-stopped --cpus=1 -m 256m wwzhg77777/dingtalk-monitorbook:1.0
name: <your project name>
services:
dingtalk-monitorbook:
stdin_open: true
tty: true
container_name: dingtalk-monitorbook
ports:
- 8000:8000
restart: unless-stopped
deploy:
resources:
limits:
cpus: 1
memory: 256m
image: wwzhg77777/dingtalk-monitorbook:1.0
上传镜像到harbor，修改harbor的镜像标签
# Docker守护进程已添加Harbor的SSL证书认证，执行 docker login 登录
[root@docker01 ~]# docker login -u admin -p Harbor12345 harbor.mysite.com
WARNING! Your password will be stored unencrypted in /root/.docker/config.json.
Configure a credential helper to remove this warning. See
https://docs.docker.com/engine/reference/commandline/login/#credentials-store
Login Succeeded
# 出现如上字样 说明docker登录harbor成功
# 给本地的镜像打tag，加上harbor服务器地址作为前缀，默认放library项目下
docker tag wwzhg77777/dingtalk-monitorbook:1.0 harbor.mysite.com/library/dingtalk-monitorbook:1.0
# push镜像到harbor
docker push harbor.mysite.com/library/dingtalk-monitorbook:1.0
Harbor注意事项
### Docker守护进程没有添加Harbor的SSL证书认证，测试登录Harbor
[root@docker02 ~]# docker login -u admin -p Harbor12345 harbor.mysite.com
Error response from daemon: Get "https://harbor.mysite.com/v2/": tls: failed to verify certificate: x509: certificate signed by unknown authority
# 添加CA证书到 /etc/docker/certs.d/harbor.mysite.com/ 目录即可解决
### push镜像时, 如果不加harbor服务器地址作为前缀, 将默认push到docker.io
[root@docker01 ~]# docker push wwzhg77777/dingtalk-monitorbook:1.0
The push refers to repository [docker.io/wwzhg77777/dingtalk-monitorbook]
### harbor所在服务器重启后，harbor服务无法正常自启动, 原因是docker compose up时才会按照depends_on的启动顺序
### 将harbor设置成systemd系统服务并设置开机启动
参考: [harbor开机不能自启问题](https://blog.csdn.net/weixin_43219672/article/details/124037012)
801
被折叠的 条评论
为什么被折叠?
到【灌水乐园】发言
