原文地址:openeuler离线部署K8S(v1.25.4)
配置清单
主机名 | IP地址 | 组件 |
---|---|---|
node1 | 192.168.0.244 | etcd,apiserver,controller-manager,scheduler |
apiserver.cluster.local | 192.168.0.244 | vip |
说明:
因为只拿到了一台openeuler:22.03的arm架构主机,所以就先用一台测试。
流程是一样的,不管是在新增master节点,还是新增worker节点都是一样的。
操作系统及软件版本信息
操作系统
openEuler 22.03 (LTS-SP2)
软件版本
kernel:5.10.0
kubelet:1.25.4
kubeadm:1.25.4
kubectl:1.25.4
cri-tools:1.26.0
socat:1.7.3.2
containerd:1.6.10
nerdctl:1.5.0
etcd:3.5.6
cni-plugins:1.1.1
crictl:1.25.0
用到的脚本
01-rhel_init.sh 用于初始化服务器的操作,并配置检查部署K8S的基础条件是否满足。
02-containerd-install.sh 用于安装containerd容器运行时。
03-kubeadm-mater1-init.sh 用于安装kubeadm等服务,并初始化master1节点,创建出token,用于其他节点注册。
04-kubeadm-mater-install.sh 用于其他节点安装kubeadm等服务,并向master1进行注册。
copy-certs.sh 用于CA证书等在master1节点向其他master节点分发。
准备工作
修改主机名
上传离线包
修改03-kubeadm-mater1-init.sh脚本中的hosts_init方法,把规划集群的IP和主机名加入进去
修改kubeadm-config.yaml集群初始化文件中的IP地址,把规划的主机IP进行修改
依次执行shell脚本文件,注意:在初始化系统后一定要至少重启一次服务器。
开始部署
当前系统
[root@node1 ~]# uname -a
Linux node1 5.10.0-153.27.0.103.oe2203sp2.aarch64 #1 SMP Wed Sep 13 19:04:10 CST 2023 aarch64 aarch64 aarch64 GNU/Linux
离线包目录文件结构如下
[root@node1 ~]# tree .
.
├── 01-rhel_init.sh
├── 02-containerd-install.sh
├── 03-kubeadm-mater1-init.sh
├── bin
│ ├── etcdctl
│ ├── nerdctl
│ └── runc
├── conf
│ ├── containerd.service
│ ├── docker.service
│ ├── k8s.conf
│ └── sysctl.conf
├── images_v1.25.4.tar
├── k8s_init.log
├── kubeadm-config.yaml
├── packages
│ ├── cni-plugins-linux-arm64-v1.1.1.tgz
│ ├── containerd-1.6.10-linux-arm64.tar.gz
│ ├── cri-containerd-1.6.10-linux-arm64.tar.gz
│ ├── cri-containerd-cni-1.6.10-linux-arm64.tar.gz
│ ├── crictl-v1.25.0-linux-arm64.tar.gz
│ ├── docker-20.10.21.tgz
│ ├── etcd-v3.5.6-linux-arm64.tar.gz
│ └── nerdctl-1.5.0-linux-arm64.tar.gz
├── py_join.py
├── rely
│ ├── centos7
│ └── openeuler
│ ├── binutils-2.37-23.oe2203sp2.aarch64.rpm
│ ├── bpftool-5.10.0-153.27.0.103.oe2203sp2.aarch64.rpm
│ ├── cpio-2.13-10.oe2203sp2.aarch64.rpm
│ ├── curl-7.79.1-23.oe2203sp2.aarch64.rpm
│ ├── dnf-4.14.0-15.oe2203sp2.noarch.rpm
│ ├── dnf-data-4.14.0-15.oe2203sp2.noarch.rpm
│ ├── file-5.41-3.oe2203sp2.aarch64.rpm
│ ├── file-libs-5.41-3.oe2203sp2.aarch64.rpm
│ ├── gawk-5.1.1-5.oe2203sp2.aarch64.rpm
│ ├── gdb-11.1-6.oe2203sp2.aarch64.rpm
│ ├── gdb-headless-11.1-6.oe2203sp2.aarch64.rpm
│ ├── ghostscript-9.55.0-5.oe2203sp2.aarch64.rpm
│ ├── gnutls-3.7.2-9.oe2203sp2.aarch64.rpm
│ ├── gnutls-utils-3.7.2-9.oe2203sp2.aarch64.rpm
│ ├── grub2-common-2.06-35.oe2203sp2.noarch.rpm
│ ├── grub2-efi-aa64-2.06-35.oe2203sp2.aarch64.rpm
│ ├── grub2-tools-2.06-35.oe2203sp2.aarch64.rpm
│ ├── grub2-tools-extra-2.06-35.oe2203sp2.aarch64.rpm
│ ├── grub2-tools-minimal-2.06-35.oe2203sp2.aarch64.rpm
│ ├── htop-3.1.2-1.oe2203sp2.aarch64.rpm
│ ├── iftop-1.0pre4-1.oe2203sp2.aarch64.rpm
│ ├── iotop-0.6-24.oe2203sp2.noarch.rpm
│ ├── json-c-0.16-4.oe2203sp2.aarch64.rpm
│ ├── kernel-5.10.0-153.27.0.103.oe2203sp2.aarch64.rpm
│ ├── kernel-devel-5.10.0-153.27.0.103.oe2203sp2.aarch64.rpm
│ ├── kernel-headers-5.10.0-153.27.0.103.oe2203sp2.aarch64.rpm
│ ├── kernel-tools-5.10.0-153.27.0.103.oe2203sp2.aarch64.rpm
│ ├── krb5-devel-1.19.2-9.oe2203sp2.aarch64.rpm
│ ├── krb5-libs-1.19.2-9.oe2203sp2.aarch64.rpm
│ ├── libcurl-7.79.1-23.oe2203sp2.aarch64.rpm
│ ├── libnghttp2-1.46.0-4.oe2203sp2.aarch64.rpm
│ ├── librsvg2-2.50.5-4.oe2203sp2.aarch64.rpm
│ ├── libsmbclient-4.17.5-7.oe2203sp2.aarch64.rpm
│ ├── libtiff-4.3.0-32.oe2203sp2.aarch64.rpm
│ ├── libtiff-devel-4.3.0-32.oe2203sp2.aarch64.rpm
│ ├── libwbclient-4.17.5-7.oe2203sp2.aarch64.rpm
│ ├── lm_sensors-3.6.0-6.oe2203sp2.aarch64.rpm
│ ├── lrzsz-0.12.20-46.oe2203sp2.aarch64.rpm
│ ├── ncurses-6.3-12.oe2203sp2.aarch64.rpm
│ ├── ncurses-base-6.3-12.oe2203sp2.noarch.rpm
│ ├── ncurses-libs-6.3-12.oe2203sp2.aarch64.rpm
│ ├── nmap-7.92-6.oe2203sp2.aarch64.rpm
│ ├── openssh-8.8p1-21.oe2203sp2.aarch64.rpm
│ ├── openssh-clients-8.8p1-21.oe2203sp2.aarch64.rpm
│ ├── openssh-server-8.8p1-21.oe2203sp2.aarch64.rpm
│ ├── openssl-1.1.1m-22.oe2203sp2.aarch64.rpm
│ ├── openssl-devel-1.1.1m-22.oe2203sp2.aarch64.rpm
│ ├── openssl-libs-1.1.1m-22.oe2203sp2.aarch64.rpm
│ ├── pcp-5.3.7-3.oe2203sp2.aarch64.rpm
│ ├── pcp-conf-5.3.7-3.oe2203sp2.aarch64.rpm
│ ├── pcp-help-5.3.7-3.oe2203sp2.noarch.rpm
│ ├── pcp-pmda-dm-5.3.7-3.oe2203sp2.aarch64.rpm
│ ├── pcp-pmda-nfsclient-5.3.7-3.oe2203sp2.aarch64.rpm
│ ├── pcp-pmda-openmetrics-5.3.7-3.oe2203sp2.aarch64.rpm
│ ├── pcp-selinux-5.3.7-3.oe2203sp2.aarch64.rpm
│ ├── pcp-system-tools-5.3.7-3.oe2203sp2.aarch64.rpm
│ ├── pcp-zeroconf-5.3.7-3.oe2203sp2.aarch64.rpm
│ ├── pcre2-10.39-9.oe2203sp2.aarch64.rpm
│ ├── pcre2-devel-10.39-9.oe2203sp2.aarch64.rpm
│ ├── perl-5.34.0-9.oe2203sp2.aarch64.rpm
│ ├── perl-devel-5.34.0-9.oe2203sp2.aarch64.rpm
│ ├── perl-libs-5.34.0-9.oe2203sp2.aarch64.rpm
│ ├── perl-PCP-PMDA-5.3.7-3.oe2203sp2.aarch64.rpm
│ ├── procps-ng-4.0.2-10.oe2203sp2.aarch64.rpm
│ ├── python3-3.9.9-25.oe2203sp2.aarch64.rpm
│ ├── python3-dnf-4.14.0-15.oe2203sp2.noarch.rpm
│ ├── python3-pcp-5.3.7-3.oe2203sp2.aarch64.rpm
│ ├── python3-perf-5.10.0-153.27.0.103.oe2203sp2.aarch64.rpm
│ ├── samba-client-libs-4.17.5-7.oe2203sp2.aarch64.rpm
│ ├── samba-common-4.17.5-7.oe2203sp2.aarch64.rpm
│ ├── screen-4.9.0-2.oe2203sp2.aarch64.rpm
│ ├── sqlite-3.37.2-6.oe2203sp2.aarch64.rpm
│ ├── sysstat-12.5.4-9.oe2203sp2.aarch64.rpm
│ ├── telnet-0.17-78.oe2203sp2.aarch64.rpm
│ ├── vim-common-9.0-17.oe2203sp2.aarch64.rpm
│ ├── vim-enhanced-9.0-17.oe2203sp2.aarch64.rpm
│ ├── vim-filesystem-9.0-17.oe2203sp2.noarch.rpm
│ ├── vim-minimal-9.0-17.oe2203sp2.aarch64.rpm
│ └── yum-4.14.0-15.oe2203sp2.noarch.rpm
└── repo
├── centos7
└── openeuler
├── conntrack-tools-1.4.6-6.oe2203sp2.aarch64.rpm
├── containernetworking-plugins-1.1.1-2.oe2203sp2.aarch64.rpm
├── cri-tools-1.26.0-0.aarch64.rpm
├── ebtables-2.0.11-11.oe2203sp2.aarch64.rpm
├── kubeadm-1.25.4-0.aarch64.rpm
├── kubectl-1.25.4-0.aarch64.rpm
├── kubelet-1.25.4-0.aarch64.rpm
├── libnetfilter_cthelper-1.0.0-16.oe2203sp2.aarch64.rpm
├── libnetfilter_cttimeout-1.0.0-15.oe2203sp2.aarch64.rpm
├── libnetfilter_queue-1.0.5-2.oe2203sp2.aarch64.rpm
└── socat-1.7.3.2-8.oe2203sp2.aarch64.rpm
9 directories, 112 files
包含除主要K8S依赖程序、离线镜像外,还包含日常运维工具命令包。
执行初始化脚本 --01-rhel_init.sh
[root@node1 ~]# chmod +x 01-rhel_init.sh
[root@node1 ~]# sh 01-rhel_init.sh all
执行用户检测: [ok]
操作系统检测: [ok]
外网权限检查: [ok]
CPU配置检查: [ok]
内存配置检查: [ok]
关闭防火墙 : [ok]
关闭交换分区: [ok]
历史命令格式 [ok]
node1 2023-09-19 15:17:43: 将安装的运维命令: 【gcc、bash-completion、vim、screen、lrzsz、tree、psmisc、zip、unzip、bzip2、gdisk、telnet、net-tools、sysstat、iftop、lsof、iotop、htop、dstat】
请选择软件包安装方式:
1. 联网在线下载并安装
2. 离线下载安装
3. 跳过安装操作
请输入你的选择(1/2/3):2
node1 2023-09-19 15:17:49: 离线更新软件包: 【gcc、bash-completion、vim、screen、lrzsz、tree、psmisc、zip、unzip、bzip2、gdisk、telnet、net-tools、sysstat、iftop、lsof、iotop、htop、dstat】
node1 2023-09-19 15:17:49: openeuler
软件包安装: [ok]
# ntpdate未安装,开始进行安装....
ntpdate安装成功: [failed]
时间同步检测: [ok]
添加内核参数: [ok]
启用ipvs模块: [ok]
------------------------------------------------------------------------------
【node1 2023-09-19 15:18:30】 服务器部署K8S的基础环境初始化操作已经完成,请在确认无误后重启服务器,以便配置文件生效。
------------------------------------------------------------------------------
node1 2023-09-19 15:18:30: openeuler内核不做单独更新,如有特殊需求,请手动更新
[root@node1 ~]# reboot
运维工具包提供离线和跳过方式,有更多选择。
安装containerd容器运行时 --02-containerd-install.sh
[root@node1 ~]# chmod +x 02-containerd-install.sh
[root@node1 ~]# sh 02-containerd-install.sh
Created symlink /etc/systemd/system/multi-user.target.wants/containerd.service → /etc/systemd/system/containerd.service.
Containerd containerd-1.6.10 已安装并配置为systemd服务!
使用如下命令进行测试是否安装成功:nerdctl run -d -p 8080:80 --name nginx nginx:alpine
[root@node1 ~]# nerdctl ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
初始化kubernetes --03-kubeadm-mater1-init.sh
[root@node1 ~]# sh 03-kubeadm-mater1-init.sh all
hosts写入: [ok]
ipvs检测: [ok]
内核检测: [ok]
containerd检测: [ok]
系统检测: [ok]
当前系统为openEuler,版本为22.03
kubeadm检查: [ok]
【node1 2023-09-19 15:21:49】 开始导入离线镜像
【node1 2023-09-19 15:21:54】 kubeadm开始初始化master节点
K8s初始化: [ok]
【node1 2023-09-19 15:22:06】 master和worker节点的加入连接如下
03-kubeadm-mater1-init.sh:行179: python:未找到命令
[root@node1 ~]# python
python3 python-argcomplete-check-easy-install-script
python3.9 python-argcomplete-tcsh
[root@node1 ~]# python3 py_join.py
Control plane information:
kubeadm join apiserver.cluster.local:6443 --token abcdef.0123456789abcdef \
--discovery-token-ca-cert-hash sha256:f02750c443bb61e8e429f0007ed1738fe974c163463fe38d236295c672085669 \
--control-plane
Worker node information:
kubeadm join apiserver.cluster.local:6443 --token abcdef.0123456789abcdef \
--discovery-token-ca-cert-hash sha256:f02750c443bb61e8e429f0007ed1738fe974c163463fe38d236295c672085669 \
[root@node1 ~]# kubectl get nodes
NAME STATUS ROLES AGE VERSION
node1 NotReady control-plane 85s v1.25.4
到这里K8S节点1已经初始化完成了,其他节点依次加入集群即可。
忘记openeuler22.03中已经没有python2了。
安装网络插件flannel
[root@node1 ~]# kubectl apply -f kube-flannel.yml
namespace/kube-flannel created
clusterrole.rbac.authorization.k8s.io/flannel created
clusterrolebinding.rbac.authorization.k8s.io/flannel created
serviceaccount/flannel created
configmap/kube-flannel-cfg created
daemonset.apps/kube-flannel-ds created
[root@node1 ~]# kubectl get nodes
NAME STATUS ROLES AGE VERSION
node1 NotReady control-plane 33m v1.25.4
[root@node1 ~]# kubectl get nodes
NAME STATUS ROLES AGE VERSION
node1 Ready control-plane 34m v1.25.4
到这里openeuler arm64架构的测试就算通过了,后续再测试集群版了。