基于Kylin V10离线编译静态docker-ce

牛角上的男孩

已于 2022-05-17 09:35:10 修改

阅读量590

点赞数

分类专栏： docker 云计算文章标签： docker kylin golang

于 2022-05-16 11:06:17 首次发布

本文链接：https://blog.csdn.net/nanhai_happy/article/details/124793448

版权

云计算同时被 2 个专栏收录

38 篇文章 2 订阅

订阅专栏

docker

7 篇文章 0 订阅

订阅专栏

静态目标

在无互联网状态下，能完成编译（离线编译）
去dockerfile，处理网络请求问题，缩短编译耗时
编译static版本，避免系统库动态依赖问题
便于各种环境直接运行问题定位

编译环境

系统环境

如下所示：

[root@cp1 ~]# uname -a
Linux cp1 4.19.90-23.8.v2101.ky10.aarch64 #1 SMP Mon May 17 17:07:38 CST 2021 aarch64 aarch64 aarch64 GNU/Linux
[root@cp1 ~]# cat /etc/lsb-release
DISTRIB_ID=Kylin
DISTRIB_RELEASE=V10
DISTRIB_CODENAME=juniper
DISTRIB_DESCRIPTION="Kylin V10"
DISTRIB_KYLIN_RELEASE=V10
DISTRIB_VERSION_TYPE=enterprise
DISTRIB_VERSION_MODE=normal
[root@cp1 ~]# lscpu
架构：                           aarch64
CPU 运行模式：                   64-bit
字节序：                         Little Endian
CPU:                             64
在线 CPU 列表：                  0-63
每个核的线程数：                 1
每个座的核数：                   64
座：                             1
NUMA 节点：                      8
厂商 ID：                        Phytium
型号：                           2
型号名称：                       FT-2000+/64
步进：                           0x1
BogoMIPS：                       100.00
L1d 缓存：                       2 MiB
L1i 缓存：                       2 MiB
L2 缓存：                        256 MiB
NUMA 节点0 CPU：                 0-7
NUMA 节点1 CPU：                 8-15
NUMA 节点2 CPU：                 16-23
NUMA 节点3 CPU：                 24-31
NUMA 节点4 CPU：                 32-39
NUMA 节点5 CPU：                 40-47
NUMA 节点6 CPU：                 48-55
NUMA 节点7 CPU：                 56-63

docker相关源码下载

建立workspace目录，用于存储源码

cd workspace
git clone https://github.com/opencontainers/runc.git
git clone https://github.com/krallin/tini.git
git clone https://github.com/containerd/containerd.git
git clone https://github.com/docker/docker-ce.git
git clone https://github.com/docker/libnetwork.git

go下载

根据需要编译的docker-ce进行go版本的相关选择，可以参考
docker-ce/components/engine/Dockerfile.e2e
FROM golang这句从docker与golang对应dockerfile找到原始对应关系https://github.com/docker-library/golang

go二进制下载：https://go.dev/dl/
go源码下载：git clone https://github.com/golang/go.git

编译docker各组件

配置go的环境变量

最后选择基于go1.16.4版本进行编译，docker 18.09

mkdir -p /usr/local/go1.16.4
tar -C go1.16.4 -xvf go1.16.4.linux-arm64.tar.gz
go_version=/usr/local/go1.16.4/go
export PATH=${go_version}/bin/:$PATH
export GOROOT=${go_version}/
export GOPATH=/usr/local/docker
export DOCKER_GITCOMMIT=4c52b90/18.09

docker各组件分发

#docker dockerd docker-proxy
mkdir -p /usr/local/docker/src/github.com/docker
cp -r docker-ce /usr/local/docker/src/github.com/docker/
cp -r libnetwork /usr/local/docker/src/github.com/docker/
cd /usr/local/docker/src/github.com/docker
cp -r docker-ce/components/engine docker
cp -r docker-ce/components/cli cli
cd -

#docker-init
cp -r tini /usr/local/docker/src/github.com/

#docker-runc
mkdir -p /usr/local/docker/src/github.com/opencontainers
cp -r runc /usr/local/docker/src/github.com/opencontainers/


#docker-containerd
mkdir -p /usr/local/docker/src/github.com/containerd
cp -r containerd /usr/local/docker/src/github.com/containerd/

cd /usr/local/docker/src/github.com/docker/docker-ce
git checkout 18.09

docker各组件编译

安装基础软件包

基于本地源安装编译依赖包

yum group install -y 'Development Tools'
yum install -y bash ca-certificates cmake gcc git glibc-static libtool make
yum install -y l device-mapper-devel selinux-policy-devel systemd-devel

编译runc

根据docker-ce/components/engine/hack/dockerfile/install/runc.installer切换至对应commit id(RUNC_COMMIT=96ec2177ae841256168fcf76954f7177af9446eb)，通过ldd查看是否为静态二进制版本。

cd /usr/local/docker/src/github.com/opencontainers/runc
git checkout -q 96ec2177ae841256168fcf76954f7177af9446eb
make BUILDTAGS="seccomp apparmor selinux nokmem" static
ldd runc
cd -

编译containerd

据编译命令编译 docker-ce/components/engine/hack/dockerfile/install/containerd.installer CONTAINERD_COMMIT=9754871865f7fe2f4e74d43e2fc7ccd237edcbce # v1.2.2

cd /usr/local/docker/src/github.com/containerd/containerd/
git checkout -q 9754871865f7fe2f4e74d43e2fc7ccd237edcbce
make EXTRA_FLAGS="-buildmode pie" EXTRA_LDFLAGS='-extldflags "-fno-PIC -static"' BUILDTAGS="netgo osusergo static_build"
ldd bin/ctr
ldd bin/containerd*
cd -

编译docker-init

cd /usr/local/docker/src/github.com/tini
git checkout -q fec3683b971d9c3ef73f284f176672c44b44866
cmake .
make tini-static
ldd tini-static
cp tini-static docker-init
cd -

编译docker-proxy

cd /usr/local/docker/src/github.com/docker/libnetwork
git checkout -q 2cfbf9b1f98162a55829a21cc603c76072a75382
CGO_ENABLED=0 go build -o docker-proxy github.com/docker/libnetwork/cmd/proxy
ldd docker-proxy
cd -

编译docker

cd /usr/local/docker/src/github.com/docker/cli
export VERSION=18.09
export GITCOMMIT=4c52b90
make binary

ldd build/docker
build/docker -v
cd -

dockerd

cd /usr/local/docker/src/github.com/docker/cli
hack/make.sh binary

bundles/binary-daemon/dockerd -v
ldd bundles/binary-daemon/dockerd

静态二进制打包

mkdir /tmp/docker-ce-static-version
cd /usr/local/docker/src/github.com
cp docker/libnetwork/docker-proxy /tmp/docker-ce-static-version/
cp docker/cli/build/docker /tmp/docker-ce-static-version/
cp docker/cli/build/docker /tmp/docker-ce-static-version/
cp containerd/containerd/bin/ctr containerd/containerd/bin/containerd containerd/containerd/bin/containerd-shim /tmp/docker-ce-static-version/
cp tini/docker-init /tmp/docker-ce-static-version/
cp opencontainers/runc/runc /tmp/docker-ce-static-version/

静态二进制文件分发部署

可以docker-ce-static-version目录中编写安装替换脚本，替换原有系统的docker相关二进制文件，如下：

#!/bin/bash
systemctl stop docker && systemctl stop containerd
alias cp='cp'
cp containerd /usr/bin/
cp containerd-shim /usr/bin/
cp containerd-shim-runc-v2 /usr/bin/
cp ctr /usr/bin/
cp docker /usr/bin/
cp dockerd /usr/bin/
cp docker-init /usr/bin/
cp docker-proxy /usr/bin/
cp runc /usr/sbin/
alias cp='cp -i'
chown root:root /usr/bin/containerd /usr/bin/containerd-shim /usr/bin/containerd-shim-runc-v2 /usr/bin/ctr /usr/bin/docker /usr/bin/dockerd /usr/bin/docker-init /usr/bin/docker-proxy /usr/sbin/runc
systemctl restart containerd && systemctl daemon-reload && systemctl restart docker

问题记录：

cannot find package “crypto/ed25519” in any of
在编译docker cli时遇到如下错误：

[root@controller1 cli]# make binary


WARNING: you are not in a container.
Use "make -f docker.Makefile binary" or set
DISABLE_WARN_OUTSIDE_CONTAINER=1 to disable this warning.

Press Ctrl+C now to abort.

./scripts/build/binary
Building static docker-linux-amd64
+ go build -o build/docker-linux-amd64 -tags ' pkcs11' -ldflags ' -w -X "github.com/docker/cli/cli/version.GitCommit=4c52b90" -X "github.com/docker/cli/cli/version.BuildTime=2022-05-11T01:49:37Z" -X "github.com/docker/cli/cli/version.Version=18.09" -extldflags -static' -buildmode=pie github.com/docker/cli/cmd/docker
vendor/github.com/moby/buildkit/session/auth/authprovider/authprovider.go:5:2: cannot find package "crypto/ed25519" in any of:
        /usr/local/docker/src/github.com/docker/cli/vendor/crypto/ed25519 (vendor tree)
        /usr/local/go1.10.6/go/src/crypto/ed25519 (from $GOROOT)
        /usr/local/docker/src/crypto/ed25519 (from $GOPATH)
cli/compose/schema/schema.go:4:2: cannot find package "embed" in any of:
        /usr/local/docker/src/github.com/docker/cli/vendor/embed (vendor tree)
        /usr/local/go1.10.6/go/src/embed (from $GOROOT)
        /usr/local/docker/src/embed (from $GOPATH)
make: *** [Makefile:42：binary] 错误 1

解决方法：
由于最初采用的go是1.10.6，缺少crypto/ed25519所致，如下：

git clone https://github.com/golang/crypto.git
cp -r crypto/ed25519 /usr/local/go1.10.6/go/src/crypto

cannot find package “embed” in any of
在编译docker cli时遇到如下错误：

[root@controller1 cli]# make binary


WARNING: you are not in a container.
Use "make -f docker.Makefile binary" or set
DISABLE_WARN_OUTSIDE_CONTAINER=1 to disable this warning.

Press Ctrl+C now to abort.

./scripts/build/binary
Building static docker-linux-amd64
+ go build -o build/docker-linux-amd64 -tags ' pkcs11' -ldflags ' -w -X "github.com/docker/cli/cli/version.GitCommit=4c52b90" -X "github.com/docker/cli/cli/version.BuildTime=2022-05-11T01:55:25Z" -X "github.com/docker/cli/cli/version.Version=18.09" -extldflags -static' -buildmode=pie github.com/docker/cli/cmd/docker
import cycle not allowed
package github.com/docker/cli/cmd/docker
        imports github.com/docker/cli/cli/command/commands
        imports github.com/docker/cli/cli/command/builder
        imports github.com/docker/cli/cli/command/image
        imports github.com/moby/buildkit/session/auth/authprovider
        imports crypto/ed25519
        imports crypto/ed25519
cli/compose/schema/schema.go:4:2: cannot find package "embed" in any of:
        /usr/local/docker/src/github.com/docker/cli/vendor/embed (vendor tree)
        /usr/local/go1.10.6/go/src/embed (from $GOROOT)
        /usr/local/docker/src/embed (from $GOPATH)
make: *** [Makefile:42：binary] 错误 1

解决方法： /usr/local/docker/src/github.com/docker/cli/vendor/embed 。发现 embed 是 golang的内置函数，当前 golang 版本为 1.10.6，升级版本为1.16.4解决

go.mod file not found in current directory or any parent directory
docker编译报如下错误：

[root@controller1 docker]# hack/make.sh binary

Removing bundles/

---> Making bundle: binary (in bundles/binary)
Building: bundles/binary-daemon/dockerd-18.09
GOOS="" GOARCH="" GOARM=""
no required module provides package github.com/docker/docker/cmd/dockerd: go.mod file not found in current directory or any parent directory; see 'go help modules'

解决方法：go env -w GO111MODULE=off

参考文献：

官方linux静态版本：https://download.docker.com/linux/static/stable/
https://m.yisu.com/zixun/25424.html
https://www.cnblogs.com/xingzheanan/p/15700302.html

牛角上的男孩

关注

0
点赞
踩
0

收藏

觉得还不错? 一键收藏
0
评论
基于Kylin V10离线编译静态docker-ce

静态目标在无互联网状态下，能完成编译（离线编译）去dockerfile，处理网络请求问题，缩短编译耗时编译static版本，避免系统库动态依赖问题便于各种环境直接运行问题定位编译环境系统环境如下所示：[root@cp1 ~]# uname -aLinux cp1 4.19.90-23.8.v2101.ky10.aarch64 #1 SMP Mon May 17 17:07:38 CST 2021 aarch64 aarch64 aarch64 GNU/Linux[root@cp1
复制链接

扫一扫