环境
- 主机环境
cat /etc/os-release NAME="Ubuntu" VERSION="18.04.1 LTS (Bionic Beaver)" ID=ubuntu ID_LIKE=debian PRETTY_NAME="Ubuntu 18.04.1 LTS" VERSION_ID="18.04" HOME_URL="https://www.ubuntu.com/" SUPPORT_URL="https://help.ubuntu.com/" BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" VERSION_CODENAME=bionic UBUNTU_CODENAME=bionic - Docker环境
docker version Client: Version: 18.09.7 API version: 1.39 Go version: go1.10.1 Git commit: 2d0083d Built: Fri Aug 16 14:20:06 2019 OS/Arch: linux/amd64 Experimental: false Server: Engine: Version: 18.09.7 API version: 1.39 (minimum version 1.12) Go version: go1.10.1 Git commit: 2d0083d Built: Wed Aug 14 19:41:23 2019 OS/Arch: linux/amd64 Experimental: false - Golang
go version go version go1.13.12 linux/amd64 - k8s版本
分支: master 最后的commit id: 43fbe17dc682b703b0b663935a797244378f0aaf
编译
-
参考资料: https://github.com/kubernetes/kubernetes/blob/98814409bab818d942e3c341d76eb594f12675a5/build/ README.md
https://xuxinkun.github.io/2016/08/05/build-kubernetes/ -
执行命令:在kubernetes顶层目录执行
build/run.sh make,发生错误:Step 1/16 : FROM us.gcr.io/k8s-artifacts-prod/build-image/kube-cross:v1.14.6-1 Get https://us.gcr.io/v2/: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) -
安装本地docker registry(这里需要本地的registry,只需要本地docker有这个镜像就可以了)
- Deploy a registry server
- 先在香港节点的服务器上拉下镜像: us.gcr.io/k8s-artifacts-prod/build-image/kube-cross:v1.14.6-1, 然后
docker save -o kube-cross.tar us.gcr.io/k8s-artifacts-prod/build-image/kube-cross:v1.14.6-1 - 从香港节点的服务器用Filezilla拉取到本地windows
- 再用Filezilla传到虚拟机,然后
docker load -i kube-cross.tar - 把镜像推送到本地的registry
- docker tag us.gcr.io/k8s-artifacts-prod/build-image/kube-cross:v1.14.6-1 localhost:5000/my-kube-cross:v1.14.6-1
- docker push localhost:5000/my-kube-cross:v1.14.6-1
- Deploy a registry server
-
修改源文件
- vi ./build/build-image/Dockerfile
-FROM us.gcr.io/k8s-artifacts-prod/build-image/kube-cross:KUBE_BUILD_IMAGE_CROSS_TAG +FROM localhost:5000/my-kube-cross:KUBE_BUILD_IMAGE_CROSS_TAG
- vi ./build/build-image/Dockerfile
-
再次运行:
build/run.sh make,多撑了会root@ubuntu18:~/gomod/kubernetes# build/run.sh make +++ [0723 14:40:47] Verifying Prerequisites.... +++ [0723 14:41:04] Building Docker image kube-build:build-939996438d-5-v1.14.6-1 +++ [0723 14:42:04] Keeping container registry +++ [0723 14:42:04] Keeping container registry +++ [0723 14:42:04] Keeping container registry +++ [0723 14:42:04] Keeping image kube-build:build-939996438d-5-v1.14.6-1 +++ [0723 14:42:04] Creating data container kube-build-data-939996438d-5-v1.14.6-1 +++ [0723 14:42:27] Syncing sources to container +++ [0723 14:42:27] Stopping any currently running rsyncd container +++ [0723 14:42:27] Starting rsyncd container +++ [0723 14:42:28] Running rsync ERROR: password file must not be other-accessible rsync error: syntax or usage error (code 1) at authenticate.c(196) [sender=3.1.2] !!! [0723 14:42:28] Call tree: !!! [0723 14:42:28] 1: build/../build/common.sh:706 kube::build::rsync(...) !!! [0723 14:42:28] 2: build/../build/common.sh:444 kube::build::sync_to_container(...) !!! [0723 14:42:28] 3: build/run.sh:31 kube::build::build_image(...) !!! Error in build/../build/common.sh:686 Error in build/../build/common.sh:686. '((i<5-1))' exited with status 1 Call stack: 1: build/../build/common.sh:686 kube::build::rsync(...) 2: build/../build/common.sh:706 kube::build::sync_to_container(...) 3: build/../build/common.sh:444 kube::build::build_image(...) 4: build/run.sh:31 main(...) Exiting with status 1 -
上面的
ERROR: password file must not be other-accessible找了好半天的原因是因为,我是从windows宿主机通过mount把kubernetes的源文件挂载到linux的虚拟机导致的, 解决办法就是拷贝源代码文件到另一个位置不要让源文件被2个系统共享 -
然后又出现如下错误:
Makefile:1: *** missing separator. Stop. !!! [0724 14:29:54] Call tree: !!! [0724 14:29:54] 1: build/../build/common.sh:527 kube::build::run_build_command_ex(...) !!! [0724 14:29:54] 2: build/run.sh:39 kube::build::run_build_command(...) !!! Error in build/../build/common.sh:604 Error in build/../build/common.sh:604. '((i<4-1))' exited with status 2 Call stack: 1: build/../build/common.sh:604 kube::build::run_build_command_ex(...) 2: build/../build/common.sh:527 kube::build::run_build_command(...) 3: build/run.sh:39 main(...) Exiting with status 1 -
还是因为被windows系统打开过导致文件格式出现问题,重新下载kubernetes源码并直接copy到linux系统上,不再被windows系统染指
-
执行
build/run.sh make, 报磁盘空间不足,清理磁盘空间 -
再次执行
build/run.sh make,报错:/usr/local/go/pkg/tool/linux_amd64/link: signal: killed !!! [0724 16:22:59] Call tree: !!! [0724 16:22:59] 1: /go/src/k8s.io/kubernetes/hack/lib/golang.sh:709 kube::golang::build_some_binaries(...) !!! [0724 16:22:59] 2: /go/src/k8s.io/kubernetes/hack/lib/golang.sh:864 kube::golang::build_binaries_for_platform(...) !!! [0724 16:22:59] 3: hack/make-rules/build.sh:27 kube::golang::build_binaries(...) !!! Error in /go/src/k8s.io/kubernetes/hack/lib/golang.sh:673 Error in /go/src/k8s.io/kubernetes/hack/lib/golang.sh:673. 'go install "${build_args[@]}" "$@"' exited with status 1 Call stack: 1: /go/src/k8s.io/kubernetes/hack/lib/golang.sh:673 kube::golang::build_some_binaries(...) 2: /go/src/k8s.io/kubernetes/hack/lib/golang.sh:709 kube::golang::build_binaries_for_platform(...) 3: /go/src/k8s.io/kubernetes/hack/lib/golang.sh:864 kube::golang::build_binaries(...) 4: hack/make-rules/build.sh:27 main(...) Exiting with status 1 !!! [0724 16:22:59] Call tree: !!! [0724 16:22:59] 1: hack/make-rules/build.sh:27 kube::golang::build_binaries(...) !!! Error in /go/src/k8s.io/kubernetes/hack/lib/golang.sh:865 Error in /go/src/k8s.io/kubernetes/hack/lib/golang.sh:865. '( kube::golang::set_platform_envs "${platform}"; kube::golang::build_binaries_for_platform "${platform}" )' exited with status 1 Call stack: 1: /go/src/k8s.io/kubernetes/hack/lib/golang.sh:865 kube::golang::build_binaries(...) 2: hack/make-rules/build.sh:27 main(...) Exiting with status 1 !!! [0724 16:22:59] Call tree: !!! [0724 16:22:59] 1: hack/make-rules/build.sh:27 kube::golang::build_binaries(...) !!! Error in /go/src/k8s.io/kubernetes/hack/lib/golang.sh:868 Error in /go/src/k8s.io/kubernetes/hack/lib/golang.sh:868. '( kube::golang::setup_env; V=2 kube::log::info "Go version: $(go version)"; local host_platform; host_platform=$(kube::golang::host_platform); local goflags goldflags goasmflags gogcflags gotags; goldflags="${GOLDFLAGS=-s -w -buildid=} $(kube::version::ldflags)"; goasmflags="-trimpath=${KUBE_ROOT}"; gogcflags="${GOGCFLAGS:-} -trimpath=${KUBE_ROOT}"; gotags="selinux,$(echo "${GOFLAGS:-}" | sed -e 's|.*-tags=\([^-]*\).*|\1|')"; local -a targets=(); local arg; for arg in "$@"; do if [[ "${arg}" == -* ]]; then goflags+=("${arg}"); else targets+=("${arg}"); fi; done; if [[ ${#targets[@]} -eq 0 ]]; then targets=("${KUBE_ALL_TARGETS[@]}"); fi; local -a platforms; IFS=" " read -ra platforms <<< "${KUBE_BUILD_PLATFORMS:-}"; if [[ ${#platforms[@]} -eq 0 ]]; then platforms=("${host_platform}"); fi; local -a binaries; while IFS="" read -r binary; do binaries+=("$binary"); done < <(kube::golang::binaries_from_targets "${targets[@]}"); local parallel=false; if [[ ${#platforms[@]} -gt 1 ]]; then local gigs; gigs=$(kube::golang::get_physmem); if [[ ${gigs} -ge ${KUBE_PARALLEL_BUILD_MEMORY} ]]; then kube::log::status "Multiple platforms requested and available ${gigs}G >= threshold ${KUBE_PARALLEL_BUILD_MEMORY}G, building platforms in parallel"; parallel=true; else kube::log::status "Multiple platforms requested, but available ${gigs}G < threshold ${KUBE_PARALLEL_BUILD_MEMORY}G, building platforms in serial"; parallel=false; fi; fi; if [[ "${parallel}" == "true" ]]; then kube::log::status "Building go targets for {${platforms[*]}} in parallel (output will appear in a burst when complete):" "${targets[@]}"; local platform; for platform in "${platforms[@]}"; do ( kube::golang::set_platform_envs "${platform}"; kube::log::status "${platform}: build started"; kube::golang::build_binaries_for_platform "${platform}"; kube::log::status "${platform}: build finished" ) &> "/tmp//${platform//\//_}.build" & done; local fails=0; for job in $(jobs -p); do wait "${job}" || (( fails+=1 )); done; for platform in "${platforms[@]}"; do cat "/tmp//${platform//\//_}.build"; done; exit ${fails}; else for platform in "${platforms[@]}"; do kube::log::status "Building go targets for ${platform}:" "${targets[@]}"; ( kube::golang::set_platform_envs "${platform}"; kube::golang::build_binaries_for_platform "${platform}" ); done; fi )' exited with status 1 Call stack: 1: /go/src/k8s.io/kubernetes/hack/lib/golang.sh:868 kube::golang::build_binaries(...) 2: hack/make-rules/build.sh:27 main(...) Exiting with status 1 make: *** [Makefile:93: all] Error 1 !!! [0724 16:23:01] Call tree: !!! [0724 16:23:01] 1: build/../build/common.sh:527 kube::build::run_build_command_ex(...) !!! [0724 16:23:01] 2: build/run.sh:39 kube::build::run_build_command(...) !!! Error in build/../build/common.sh:603 Error in build/../build/common.sh:603. '((i<4-1))' exited with status 2 Call stack: 1: build/../build/common.sh:603 kube::build::run_build_command_ex(...) 2: build/../build/common.sh:527 kube::build::run_build_command(...) 3: build/run.sh:39 main(...) Exiting with status 1 -
发现系统负载特别大, 调高了虚拟机的配置,升到 8核8G
-
再次执行
build/run.sh make,好像成功了,生成的文件在_output目录下Go version: go version go1.14.6 linux/amd64 +++ [0724 16:55:59] Building go targets for linux/amd64: cmd/kube-proxy cmd/kube-apiserver cmd/kube-controller-manager cmd/kubelet cmd/kubeadm cmd/kube-scheduler vendor/k8s.io/kube-aggregator vendor/k8s.io/apiextensions-apiserver cluster/gce/gci/mounter cmd/kubectl cmd/gendocs cmd/genkubedocs cmd/genman cmd/genyaml cmd/genswaggertypedocs cmd/linkcheck vendor/github.com/onsi/ginkgo/ginkgo test/e2e/e2e.test cluster/images/conformance/go-runner cmd/kubemark vendor/github.com/onsi/ginkgo/ginkgo test/e2e_node/e2e_node.test Env for linux/amd64: GOOS=linux GOARCH=amd64 GOROOT=/usr/local/go CGO_ENABLED= CC= Coverage is disabled. Coverage is disabled. +++ [0724 17:03:03] Placing binaries +++ [0724 17:03:22] Syncing out of container +++ [0724 17:03:22] Stopping any currently running rsyncd container +++ [0724 17:03:22] Starting rsyncd container +++ [0724 17:03:24] Running rsync +++ [0724 17:03:33] Stopping any currently running rsyncd container -
_output目录结构root@ubuntu18:~/kubernetes/_output# tree . ├── dockerized │ └── bin │ ├── go-bindata │ └── linux │ └── amd64 │ ├── apiextensions-apiserver │ ├── conversion-gen │ ├── deepcopy-gen │ ├── defaulter-gen │ ├── e2e_node.test │ ├── e2e.test │ ├── gendocs │ ├── genkubedocs │ ├── genman │ ├── genswaggertypedocs │ ├── genyaml │ ├── ginkgo │ ├── go2make │ ├── go-bindata │ ├── go-runner │ ├── kubeadm │ ├── kube-aggregator │ ├── kube-apiserver │ ├── kube-controller-manager │ ├── kubectl │ ├── kubelet │ ├── kubemark │ ├── kube-proxy │ ├── kube-scheduler │ ├── linkcheck │ ├── mounter │ ├── openapi-gen │ └── prerelease-lifecycle-gen └── images └── kube-build:build-f1967cbefe-5-v1.14.6-1 ├── Dockerfile ├── localtime ├── rsyncd.password └── rsyncd.sh- 只编译某个组件:
bash build/run.sh hack/build-go.sh cmd/kubelet, 建议使用:bash build/run.sh make WHAT='cmd/kubelet' - 总结:
- 编译的虚拟机配置要高: 至少8核8G
- 不要通过mount的方式在windows宿主机和编译的linux虚拟机之间共享kubernets源码,会导致rsync出问题
- 不要用windows打开任何文件,可能导致Makefile出现tab键的问题
- 编译时要用到的docker镜像通过别的方式下载下来,放到本地docker就可以了,用不着本地启动registry
- 只编译某个组件:
948
被折叠的 条评论
为什么被折叠?
到【灌水乐园】发言
