kubeadm极速部署Kubernetes 1.26版本集群
1.1 环境准备
1.1.1 主机操作系统说明
序号 | 操作系统及版本 | 备注 |
---|---|---|
1 | Ubuntu 20.04.5 LTS |
1.1.2 修改Ubuntu镜像
用你熟悉的编辑器打开 /etc/apt/sources.list
# 备份
cp /etc/apt/sources.list /etc/apt/sources.list.bak
替换默认的 http://archive.ubuntu.com/
为 http://mirrors.aliyun.com/
ubuntu 20.04 LTS (focal) 配置如下
deb https://mirrors.aliyun.com/ubuntu/ focal main restricted universe multiverse
deb-src https://mirrors.aliyun.com/ubuntu/ focal main restricted universe multiverse
deb https://mirrors.aliyun.com/ubuntu/ focal-security main restricted universe multiverse
deb-src https://mirrors.aliyun.com/ubuntu/ focal-security main restricted universe multiverse
deb https://mirrors.aliyun.com/ubuntu/ focal-updates main restricted universe multiverse
deb-src https://mirrors.aliyun.com/ubuntu/ focal-updates main restricted universe multiverse
# deb https://mirrors.aliyun.com/ubuntu/ focal-proposed main restricted universe multiverse
# deb-src https://mirrors.aliyun.com/ubuntu/ focal-proposed main restricted universe multiverse
deb https://mirrors.aliyun.com/ubuntu/ focal-backports main restricted universe multiverse
deb-src https://mirrors.aliyun.com/ubuntu/ focal-backports main restricted universe multivers
修改后更新
sudo apt update && sudo apt upgrade
1.1.3 主机配置
1.1.3.1 主机名配置
由于本次使用3台主机完成kubernetes集群部署,其中1台为master节点,名称为master;其中2台为worker节点,名称分别为:node1及node2
master节点
# hostnamectl set-hostname master
node1节点
# hostnamectl set-hostname node1
node2节点
# hostnamectl set-hostname node2
1.1.3.2 主机IP地址配置
主机名成解析 编辑三台服务器的/etc/hosts
文件,添加下面内容
根据自己情况配置,这里我有一台master节点对应 192.168.90.100,node1节点对应 192.168.90.106,node2节点对应 192.168.90.107
所有集群主机均需要进行配置。
192.168.90.100 master
192.168.90.106 node1
192.168.90.107 node2
角色 | IP地址 | 组件 |
---|---|---|
master | 192.168.90.100 | containerd,kubectl,kubeadm,kubelet |
node1 | 192.168.90.106 | containerd,kubectl,kubeadm,kubelet |
node2 | 192.168.90.107 | containerd,kubectl,kubeadm,kubelet |
1.1.3.4 防火墙配置
所有主机均需要操作。
# 关闭现有防火墙firewalld
[root@master ~]# systemctl stop firewalld
[root@master ~]# systemctl disable firewalld
# 关闭iptables服务
[root@master ~]# systemctl stop iptables
[root@master ~]# systemctl disable iptables
# 关闭iptables服务
[root@master ~]# sudo ufw disable
[root@master ~]# sudo systemctl stop ufw
[root@master ~]# sudo systemctl disable ufw
1.1.3.6 时间同步配置
所有主机均需要操作
# chronyd方式
[root@master ~]# systemctl start chronyd
[root@master ~]# systemctl enable chronyd
[root@master ~]# date
ntp 方式
sudo apt update
sudo apt install -y ntp配置 NTP 服务器
sudo vim /etc/ntp.conf
#添加ntp服务器 中国开源免费NTP服务器
server cn.pool.ntp.org
#重新启动 NTP 服务:
sudo systemctl restart ntp
#验证时间同步状态:
#查看 NTP 服务器状态
ntpq -p
#查看系统时间
date
1.1.3.7 升级操作系统内核
修改linux的内核采纳数,添加网桥过滤和地址转发功能
所有主机均需要操作。
-
创建
/etc/sysctl.d/kubernetes.conf
文件,添加如下配置:net.bridge.bridge-nf-call-ip6tables = 1 net.bridge.bridge-nf-call-iptables = 1 net.ipv4.ip_forward = 1
-
重新加载配置
[root@master ~]# sysctl -p 1
-
加载网桥过滤模块
[root@master ~]# modprobe br_netfilter 1
-
查看网桥过滤模块是否加载成功
[root@master ~]# lsmod | grep br_netfilter br_netfilter 22256 0 bridge 151336 1 br_netfilter
-
如果查询结果为0(未生效),需重启服务器(sudo reboot)生效
cat /proc/sys/net/ipv4/ip_forward 1
1.1.3.9 安装ipset及ipvsadm
所有主机均需要操作。
# 1.安装ipset和ipvsadm
[root@master ~]# apt install ipvsadm ipset sysstat conntrack -y
# 2.添加需要加载的模块写入脚本文件
[root@master ~]# cat >> /etc/modules-load.d/ipvs.conf <<EOF
ip_vs
ip_vs_rr
ip_vs_wrr
ip_vs_sh
nf_conntrack
ip_tables
ip_set
xt_set
ipt_set
ipt_rpfilter
ipt_REJECT
ipip
EOF
# 3.重启服务
[root@master ~]# systemctl restart systemd-modules-load.service
# 4.查看对应的模块是否加载成功
[root@master ~]# lsmod | grep -e ip_vs -e nf_conntrack
ip_vs_sh 16384 0
ip_vs_wrr 16384 0
ip_vs_rr 16384 0
ip_vs 155648 6 ip_vs_rr,ip_vs_sh,ip_vs_wrr
nf_conntrack 139264 1 ip_vs
nf_defrag_ipv6 24576 2 nf_conntrack,ip_vs
nf_defrag_ipv4 16384 1 nf_conntrack
libcrc32c 16384 4 nf_conntrack,btrfs,raid456,ip_vs
1.1.3.10 关闭SWAP分区
修改完成后需要重启操作系统,如不重启,可临时关闭,命令为swapoff -a
swapoff -a # 临时禁用
sed -ri 's/.*swap.*/#&/' /etc/fstab # 永久禁用
1.2 安装containerd
如果过去安装过 docker 或者 containerd,需要先卸载
$ for pkg in docker.io docker-doc docker-compose podman-docker containerd runc; do sudo apt-get remove $pkg; done
安装依赖
-
ca-certificates:证书管理工具(Ubuntu 22 默认已安装)
-
gnupg2:GNU Privacy Guard ( GnuPG 或 GPG ),是一种加密软件,有多个版本(Ubuntu 22 默认已安装 gnupg)
-
apt-transport-https:使 apt 支持 https
-
software-properties-common:提供了 apt 的一些简便工具,例如:add-apt-repository
$ sudo apt-get update
$ sudo apt -y install apt-transport-https ca-certificates software-properties-common gnupg2 curl
配置仓库
此处使用了阿里云的镜像源
信任 Docker 的 GPG 公钥
$ sudo install -m 0755 -d /etc/apt/keyrings
$ curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /etc/apt/keyrings/docker.gpg
$ sudo chmod a+r /etc/apt/keyrings/docker.gpg
添加软件仓库(清华源)
$ echo \
"deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.gpg] https://mirrors.tuna.tsinghua.edu.cn/docker-ce/linux/ubuntu \
$(lsb_release -cs) stable" | sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
更新缓存
$ apt update
安装
安装 Docker 引擎、containerd 和 Docker Compose
旧版 k8s:
$ sudo apt-get install docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin
配置docker 阿里云镜像
加速器地址在阿里云控制台申请(每个人的不一样!your.mirror.aliyuncs.com 改为你自己的)
sudo mkdir -p /etc/docker sudo tee /etc/docker/daemon.json <<-'EOF' { "registry-mirrors": ["https://your.mirror.aliyuncs.com"] } EOF sudo systemctl daemon-reload sudo systemctl restart docker #查看是否配置成功 docker info
新版k8s:只需要安装containerd即可(默认安装最新版)
$ sudo apt-get install containerd.io
安装指定containerd版本
# 查找 containerd 版本 $ apt-cache madison containerd.io containerd.io | 1.6.28-1 | https://mirrors.aliyun.com/docker-ce/linux/ubuntu focal/stable amd64 Packages containerd.io | 1.6.27-1 | https://mirrors.aliyun.com/docker-ce/linux/ubuntu focal/stable amd64 Packages containerd.io | 1.6.26-1 | https://mirrors.aliyun.com/docker-ce/linux/ubuntu focal/stable amd64 Packages containerd.io | 1.6.25-1 | https://mirrors.aliyun.com/docker-ce/linux/ubuntu focal/stable amd64 Packages containerd.io | 1.6.24-1 | https://mirrors.aliyun.com/docker-ce/linux/ubuntu focal/stable amd64 Packages containerd.io | 1.6.22-1 | https://mirrors.aliyun.com/docker-ce/linux/ubuntu focal/stable amd64 Packages containerd.io | 1.6.21-1 | https://mirrors.aliyun.com/docker-ce/linux/ubuntu focal/stable amd64 Packages containerd.io | 1.6.20-1 | https://mirrors.aliyun.com/docker-ce/linux/ubuntu focal/stable amd64 Packages ...... # 安装指定版本的 containerd: sudo apt-get -y install containerd.io=[VERSION] sudo apt install -y containerd.io=1.6.28-1
通过运行镜像来验证Docker Engine安装是否成功 hello-world
sudo docker run hello-world
开启开机自启
systemctl start containerd
配置config.toml
$ mkdir -p /etc/containerd/
$ containerd config default > /etc/containerd/config.toml
# 配置 systemd cgroup驱动
$ sed -i 's/SystemdCgroup \= false/SystemdCgroup \= true/g' /etc/containerd/config.toml
$ systemctl restart containerd
- 配置containerd的阿里云镜像
(每个人的不一样!your.mirror.aliyuncs.com 改为你自己的)
$ sudo vim /etc/containerd/config.toml #修改以下位置的值,修改完需重启containerd(systemctl restart containerd) [plugins."io.containerd.grpc.v1.cri".registry.mirrors] [plugins."io.containerd.grpc.v1.cri".registry.mirrors."docker.io"] endpoint = ["https://your.mirror.aliyuncs.com"] [plugins."io.containerd.grpc.v1.cri".registry.mirrors."registry.aliyuncs.com/google_containers"] endpoint = ["https://registry.aliyuncs.com/k8sxio"]
1.3 kubernetes 1.26.X 集群部署
1.3.1 集群软件及版本说明
kubeadm | kubelet | kubectl | |
---|---|---|---|
版本 | 1.26.X | 1.26.X | 1.26.X |
安装位置 | 集群所有主机 | 集群所有主机 | 集群所有主机 |
作用 | 初始化集群、管理集群等 | 用于接收api-server指令,对pod生命周期进行管理 | 集群应用命令行管理工具 |
1.3.2 kubernetes 镜像源
1.3.2.2 阿里云 参考https://developer.aliyun.com/mirror/kubernetes?spm=a2c6h.13651102.0.0.2e611b11BvTeZL
# 如果没有/etc/apt/keyrings目录,需要手动创建
$ apt-get update && apt-get install -y apt-transport-https
# 下载 Google Cloud 公开签名秘钥(阿里云镜像)
$ curl -fsSL https://mirrors.aliyun.com/kubernetes-new/core/stable/v1.26/deb/Release.key |
gpg --dearmor -o /etc/apt/keyrings/kubernetes-apt-keyring.gpg
# 添加 Kubernetes apt 仓库(阿里云镜像)
$ echo "deb [signed-by=/etc/apt/keyrings/kubernetes-apt-keyring.gpg] https://mirrors.aliyun.com/kubernetes-new/core/stable/v1.26/deb/ /" |
tee /etc/apt/sources.list.d/kubernetes.list
$ apt-get update
# 查看镜像版本
$ apt-cache madison kubectl kubelet kubeadm
1.3.3 集群软件安装
所有节点均可安装
# 安装默认最新版本
$ sudo apt-get install -y kubelet kubeadm kubectl
# 安装指定版本
$ apt-get install -y kubectl=1.26.2-1.1 kubelet=1.26.2-1.1 kubeadm=1.26.2-1.1
1.3.4 集群初始化
-
重载沙箱(pause)镜像
vim /etc/containerd/config.toml #修改disabled_plugins这行为[] disabled_plugins = [] # 替换sandbox_image的值为registry.aliyuncs.com/google_containers/pause:3.9 [plugins."io.containerd.grpc.v1.cri"] sandbox_image = "registry.aliyuncs.com/google_containers/pause:3.9" # 重启containerd systemctl restart containerd
-
使用配置文件初始化
# 生成配置文件 kubeadm config print init-defaults > /home/xctech/k8s/init-default.yaml vim /home/xctech/k8s/init-default.yaml # 修改 imageRepository: k8s.gcr.io,改为 registry.aliyuncs.com/google_containers imageRepository: registry.aliyuncs.com/google_containers # 修改kubernetesVersion,改为kubernetesVersion: v1.13.1 kubernetesVersion: v1.26.2 #advertiseAddress表示控制面的稳定ip和端口,填当前节点的ip advertiseAddress: 192.168.110.244 # podSubnet 为 Pod 所使用的子网。 默认配置没有该选项,在Network选项下给其增加该配置 podSubnet: 10.244.0.0/16
参考配置如下
apiVersion: kubeadm.k8s.io/v1beta3 bootstrapTokens: - groups: - system:bootstrappers:kubeadm:default-node-token token: abcdef.0123456789abcdef ttl: 24h0m0s usages: - signing - authentication kind: InitConfiguration localAPIEndpoint: advertiseAddress: 192.168.110.244 bindPort: 6443 nodeRegistration: criSocket: unix:///var/run/containerd/containerd.sock imagePullPolicy: IfNotPresent name: node taints: null --- apiServer: timeoutForControlPlane: 4m0s apiVersion: kubeadm.k8s.io/v1beta3 certificatesDir: /etc/kubernetes/pki clusterName: kubernetes controllerManager: {} dns: {} etcd: local: dataDir: /var/lib/etcd imageRepository: registry.aliyuncs.com/google_containers kind: ClusterConfiguration kubernetesVersion: 1.26.2 networking: dnsDomain: cluster.local serviceSubnet: 10.96.0.0/12 podSubnet: 10.244.0.0/16 scheduler: {} --- apiVersion: kubelet.config.k8s.io/v1beta1 kind: KubeletConfiguration cgroupDriver: systemd
初始化master
root@master:/etc# kubeadm init --config=init-default.yaml
初始化完成后 会输出这样一段话:
our Kubernetes control-plane has initialized successfully!
To start using your cluster, you need to run the following as a regular user:
mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config
Alternatively, if you are the root user, you can run:
export KUBECONFIG=/etc/kubernetes/admin.conf
You should now deploy a pod network to the cluster.
Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at:
https://kubernetes.io/docs/concepts/cluster-administration/addons/
Then you can join any number of worker nodes by running the following on each as root:
kubeadm join 192.168.110.244:6443 --token abcdef.0123456789abcdef \
--discovery-token-ca-cert-hash sha256:a9d295199d283cd3a718cb11867c648c7eaf2e04921710d8bfed84be4372cb6f
最后两行记得拷贝记录一下,后面初始化node的时候会用到
如果是root用户,要运行
export KUBECONFIG=/etc/kubernetes/admin.conf
然后运行
mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config
1.3.5 集群网络准备
安装 flannel(master操作)
kubectl apply -f https://raw.githubusercontent.com/flannel-io/flannel/master/Documentation/kube-flannel.yml
1.3.9 初始化worker
node的初始化和master的初始化步骤基本一致,除了
初始化master
的步骤(但要包括那错误
例子中pause的配置)不需要做,其他的参考上面步骤重新做一遍即可,不同的是init命令换成了join命令
[root@k8s-node ~]# kubeadm join 192.168.110.244:6443 --token abcdef.0123456789abcdef \
--discovery-token-ca-cert-hash sha256:a9d295199d283cd3a718cb11867c648c7eaf2e04921710d8bfed84be4372cb6f
就是让你们拷贝的那段,如果忘记了,可以使用以下命令重新获取
kubeadm token create --print-join-command
查看nodes的状态
kubectl get nodes
如果报以下错误
couldn't get current server API group list: Get "http://localhost:8080/api?timeout=32s": dial tcp 127.0.0.1:8080: connect: connection refused
查看/etc/kubernetes/kubelet.conf文件是否存在
不存在就先执行
kubeadm reset 后重新 kubeadm join
然后添加到环境变量中
echo "export KUBECONFIG=/etc/kubernetes/kubelet.conf" >> /etc/profile
source /etc/profile
再执行就正常了
kubectl get nodes
看到node和master的状态都是Ready就没问题了!
二、 验证集群可用性
#查看所有的节点
[root@k8s-master01 ~]# kubectl get nodes
NAME STATUS ROLES AGE VERSION
k8s-master01 Ready control-plane 12h v1.26.2
k8s-worker01 Ready <none> 12h v1.26.2
k8s-worker02 Ready <none> 12h v1.26.2
#查看集群健康情况
[root@k8s-master01 ~]# kubectl get cs
Warning: v1 ComponentStatus is deprecated in v1.19+
NAME STATUS MESSAGE ERROR
controller-manager Healthy ok
scheduler Healthy ok
etcd-0 Healthy {"health":"true","reason":""}
#查看kubernetes集群pod运行情况
[root@k8s-master01 ~]# kubectl get pods -n kube-system
NAME READY STATUS RESTARTS AGE
coredns-6d4b75cb6d-js5pl 1/1 Running 0 12h
coredns-6d4b75cb6d-zm8pc 1/1 Running 0 12h
etcd-k8s-master01 1/1 Running 0 12h
kube-apiserver-k8s-master01 1/1 Running 0 12h
kube-controller-manager-k8s-master01 1/1 Running 0 12h
kube-proxy-7nhr7 1/1 Running 0 12h
kube-proxy-fv4kr 1/1 Running 0 12h
kube-proxy-vv5vg 1/1 Running 0 12h
kube-scheduler-k8s-master01 1/1 Running 0 12h