一、部署思路
1)所有节点进行初始化,安装 容器运行时、kubeadm、kubelet、kubectl;
2)执行 kubeadm config print init-defaults 命令生成集群初始化配置文件并进行修改;
3)执行 kubeadm init 命令根据初始化配置文件初始化生成K8S master控制管理节点;
4)安装 cni 网络插件(flannel/calico);
5)在其它节点执行 kubeadm join 命令将将node节点或者其它master节点加入到K8S集群中。
二、基本架构
Server | IP |
---|---|
Master01 | 192.168.10.200 |
Node01 | 192.168.10.201 |
Node02 | 192.168.10.202 |
2.1 系统初始化操作
2.1.1 关闭防火墙、selinux和swap分区
关闭防火墙和selinux
#关闭防火墙
systemctl stop ufw
安装基本的软件
apt install chrony ipvsadm tree ipset -y
关闭交换分区
交换分区必须要关闭。
K8s的各个组件和容器都需要足够的内存来运行,而Swap的使用可能导致性能下降,甚至是应用程序的奔溃。
关闭Swap可以确保集群的可预测性和稳定性,避免不必要的磁盘交换。
swapoff -a
sed -i '/swap.img/s/^/#/' /etc/fstab
#永久关闭swap分区,&符号在sed命令中代表上次匹配的结果
修改主机名
hostnamectl set-hostname master01
hostnamectl set-hostname node01
hostnamectl set-hostname node02
修改本地hosts文件
vim /etc/hosts
192.168.10.200 master01
192.168.10.201 node01
192.168.10.202 node02
安装containerd
apt update
apt install containerd[=版本号] -y
/lib/systemd/system/containerd.service (二进制安装参考此配置)
mkdir /etc/containerd/ ## 存放containerd的配置文件
#配置containerd
containerd config default > /etc/containerd/config.toml
修改pause镜像地址:
sandbox_image = "registry.aliyuncs.com/google_containers/pause:3.9" #阿里镜像地址
sandbox_image = "harbor.hiuiu.com/kubernetes/google_containers/pause:3.9" #本地harbor镜像地址
修改镜像加速配置
168 [plugins."io.containerd.grpc.v1.cri".registry.mirrors]
169 [plugins."io.containerd.grpc.v1.cri".registry.mirrors."docker.io"]
170 endpoint = ["https://hzsbbqty.mirror.aliyuncs.com"]
修改为true
137 SystemdCgroup = true
安装nerdctl工具
下载路径
wget https://github.com/containerd/nerdctl/releases/download/v1.3.0/nerdctl-1.3.0-linux-amd64.tar.gz
安装
tar xvf nerdctl-1.7.6-linux-amd64.tar.gz -C /usr/local/bin/
nerdctl version
mkdir /etc/nerdctl
cat > /etc/nerdctl/nerdctl.toml <<EOF
namespace = "k8s.io"
debug = false
debug_full = false
insecure_registry = true
EOF
CNI工具
为容器提供网桥,如果不安装CNI,容器只有host网络模式。
安装CNI:
wget https://github.com/containernetworking/plugins/releases/download/v1.2.0/cni-plugins-linux-amd64-v1.2.0.tgz
tar xvf cni-plugins-linux-amd64-v1.5.1.tgz -C /opt/cni/bin/
加载模块
modprobe br_netfilter && lsmod |grep br_netfilter
modprobe ip_conntrack && lsmod | grep conntrack
cat >/etc/modules-load.d/modules.conf<<EOF
ip_vs
ip_vs_lc
ip_vs_lblc
ip_vs_lblcr
ip_vs_rr
ip_vs_wrr
ip_vs_sh
ip_vs_dh
ip_vs_fo
ip_vs_nq
ip_vs_sed
ip_vs_ftp
ip_tables
ip_set
ipt_set
ipt_rpfilter
ipt_REJECT
ipip
xt_set
br_netfilter
nf_conntrack
overlay
EOF
systemctl restart systemd-modules-load.service
lsmod | grep -e ip_vs -e nf_conntrack
配置时间服务器(时间同步)
sed -i 's/pool ntp.ubuntu.com/pool 时间服务器地址/' /etc/chrony/chrony.conf
systemctl restart chronyd
修改内核参数
net.ipv4.ip_forward=1
vm.max_map_count=262144
kernel.pid_max=4194303
fs.file-max=1000000
net.ipv4.tcp_max_tw_buckets=6000
net.netfilter.nf_conntrack_max=2097152
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1
vm.swappiness=0
vim /etc/sysctl.conf
sysctl -p
三、正式安装K8S-kubeadm方式
3.1安装 kubeadm 、kubelet、 kubectl
apt update
apt install apt-transport-https ca-certificates curl gpg
mkdir -p -m 755 /etc/apt/keyrings
curl -fsSL https://pkgs.k8s.io/core:/stable:/v1.30/deb/Release.key | sudo gpg --dearmor -o /etc/apt/keyrings/kubernetes-apt-keyring.gpg
##阿里源
echo "deb [signed-by=/etc/apt/keyrings/kubernetes-apt-keyring.gpg] https://mirrors.aliyun.com/kubernetes-new/core/stable/v1.30/deb/ /" | tee /etc/apt/sources.list.d/kubernetes.list
## 安装命令
apt-get update && apt-cache madison kubeadm
apt-get install -y kubelet=1.30.3-1.1 kubeadm=1.30.3-1.1 kubectl=1.30.3-1.1
3.2 拉取镜像
nerdctl pull registry.cn-hangzhou.aliyuncs.com/google_containers/kube-apiserver:v1.30.3
nerdctl pull registry.cn-hangzhou.aliyuncs.com/google_containers/kube-controller-manager:v1.30.3
nerdctl pull registry.cn-hangzhou.aliyuncs.com/google_containers/kube-scheduler:v1.30.3
nerdctl pull registry.cn-hangzhou.aliyuncs.com/google_containers/kube-proxy:v1.30.3
nerdctl pull registry.cn-hangzhou.aliyuncs.com/google_containers/pause:3.9
nerdctl pull registry.cn-hangzhou.aliyuncs.com/google_containers/etcd:3.5.12-0
nerdctl pull registry.cn-hangzhou.aliyuncs.com/google_containers/coredns:v1.11.1
3.3 创建MASTER
kubeadm init --apiserver-advertise-address=192.168.10.200 --apiserver-bind-port=6443 --kubernetes-version=v1.30.3 --pod-network-cidr=10.200.0.0/16 --service-cidr=10.96.0.0/16 --service-dns-domain=cluster.local --image-repository registry.aliyuncs.com/google_containers --ignore-preflight-errors=swap
按照提示执行:
mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf$HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config
export KUBECONFIG=/etc/kubernetes/admin.conf
3.4 节点加入集群
kubeadm join 192.168.10.200:6443 --token s1nty1.w4lnkiwy9tl284qz \
--discovery-token-ca-cert-hash sha256:de83ae983240c5d8fd476b328798121ad02655f17153d154979653a35c92ef3d
3.5 查看集群节点
kubectl get nodes
四、网络组件calico
三台机子都安装calico
tar -zxvf calico-release-v3.28.0.tgz
cd release-v3.28.0/
release-v3.28.0/manifests/calico.yaml
# 找calico配置文件
修改配置文件:calico.yaml
(1) pod网络地址:
calico.yaml
#/CALICO_IPV4POOL_CIDR
- name: CALICO_IPV4POOL_CIDR
value: "10.200.0.0/16"
将name和value前的#号删除,将ip改为初始化时设置的pod网段
镜像导入
cd /release-v3.28.0/images
release-v3.28.0/images# ll
total 876528
drwxrwxr-x 2 1001 1001 213 May 11 08:20 ./
drwxrwxr-x 5 1001 1001 48 May 11 08:20 ../
-rw------- 1 1001 1001 208990208 May 11 08:20 calico-cni.tar
-rw------- 1 1001 1001 41927168 May 11 08:19 calico-dikastes.tar
-rw------- 1 1001 1001 128270336 May 11 08:19 calico-flannel-migration-controller.tar
-rw------- 1 1001 1001 79177216 May 11 08:19 calico-kube-controllers.tar
-rw------- 1 1001 1001 354560512 May 11 08:20 calico-node.tar
-rw------- 1 1001 1001 13446144 May 11 08:19 calico-pod2daemon.tar
-rw------- 1 1001 1001 71183360 May 11 08:20 calico-typha.tar
导入K8S环境里面(三台节点都要载入镜像)
nerdctl load -i calico-cni.tar
nerdctl load -i calico-kube-controllers.tar
nerdctl load -i calico-node.tar
master上执行安装
kubectl apply -f calico.yaml
#验证
kubectl run nginx21 --image=harbor.hiuiu.com/basic_image/centos7_filebeat_nginx:2408.u sleep 10000
kubectl run nginx22 --image=harbor.hiuiu.com/basic_image/centos7_filebeat_nginx:2408.u sleep 10000
#查看pod状态
kubectl get pod -o wide
#分别进入nginx21,nginx22
kubectl exec nginx21 -it bash
kubectl exec nginx22 -it bash
使用nginx11 ping nginx22来验证
在nginx22上使用ip a查看ip
tip 如果遇到报错可使用kubectl describe pod pod名 -n kube-system来进行排错