rockeylinux 搭建k8s 1.28.10

1.关闭防火墙
systemctl stop firewalld
systemctl disable firewalld

2.关闭selinux
# 临时禁用selinux
# 将 SELinux 设置为 permissive 模式（相当于将其禁用）
setenforce 0
sed -i 's/^SELINUX=enforcing$/SELINUX=permissive/' /etc/selinux/config

3.网路配置
cat <<EOF | sudo tee /etc/sysctl.d/k8s.conf
net.bridge.bridge-nf-call-iptables  = 1
net.bridge.bridge-nf-call-ip6tables = 1
net.ipv4.ip_forward                 = 1
EOF
 执行 sysctl --system 使生效
 

 4.关闭swap
 # 禁用交换分区
swapoff -a
# 永久禁用，打开/etc/fstab注释掉swap那一行
sed -i 's/.*swap.*/#&/' /etc/fstab

5.配置host文件
master节点
hostnamectl set-hostname master
node节点
hostnamectl set-hostname node01
hostnamectl set-hostname node02

执行 hostname 查看是否成功

6.安装iptables
yum install iptables

7.安装 kubeadm kubectl kubelet

由于官网未开放同步方式, 可能会有索引gpg检查失败的情况, 这时请用 yum install -y --nogpgcheck kubelet kubeadm kubectl 安装

yum install -y --nogpgcheck kubeadm
设置 kubelet 开机自启
systemctl enable kubelet

8.安装docker
编译安装 docker-19.03.15.tgz
tar -xf docker-19.03.15.tgz
cp docker/* /usr/bin

vim /etc/systemd/system/docker.service
复制下面内容

[Unit]
Description=Docker Application Container Engine
Documentation=https://docs.docker.com
# BindsTo=containerd.service
# After=network-online.target firewalld.service containerd.service
After=network-online.target firewalld.service
Wants=network-online.target
# Requires=docker.socket

[Service]
Type=notify
# the default is not to use systemd for cgroups because the delegate issues still
# exists and systemd currently does not support the cgroup feature set required
# for containers run by docker
# ExecStart=/usr/bin/dockerd -H fd:// --containerd=/run/containerd/containerd.sock
ExecStart=/usr/bin/dockerd
ExecReload=/bin/kill -s HUP $MAINPID
TimeoutSec=0
RestartSec=2
Restart=always

# Note that StartLimit* options were moved from "Service" to "Unit" in systemd 229.
# Both the old, and new location are accepted by systemd 229 and up, so using the old location
# to make them work for either version of systemd.
StartLimitBurst=3

# Note that StartLimitInterval was renamed to StartLimitIntervalSec in systemd 230.
# Both the old, and new name are accepted by systemd 230 and up, so using the old name to make
# this option work for either version of systemd.
StartLimitInterval=60s

# Having non-zero Limit*s causes performance problems due to accounting overhead
# in the kernel. We recommend using cgroups to do container-local accounting.
LimitNOFILE=infinity
LimitNPROC=infinity
LimitCORE=infinity

# Comment TasksMax if your systemd version does not support it.
# Only systemd 226 and above support this option.
# TasksMax=infinity

# set delegate yes so that systemd does not reset the cgroups of docker containers
Delegate=yes

# kill only the docker process, not all processes in the cgroup
KillMode=process

[Install]
WantedBy=multi-user.target

创建docker文件夹
mkdir /etc/docker
编辑daemon.json文件,主从节点都需要执行此步骤
vi /etc/docker/daemon.json
{
  "exec-opts": ["native.cgroupdriver=systemd"]
}
chmod +x /etc/systemd/system/docker.service #添加文件权限并启动docker
systemctl daemon-reload  #重载unit配置文件
systemctl start docker  #启动Docker
systemctl enable docker  #设置开机自启
systemctl status docker  #查看docker状态

9.安装 cri-dockerd

下载 cri-dockerd-0.3.14.amd64.tgz
tar -xf cri-dockerd-0.3.14.amd64.tgz
cp cri-dockerd/cri-dockerd /usr/local/bin/

cat > /etc/systemd/system/cri-dockerd.service<<-EOF
[Unit]
Description=CRI Interface for Docker Application Container Engine
Documentation=https://docs.mirantis.com
After=network-online.target firewalld.service docker.service
Wants=network-online.target
 
[Service]
Type=notify
ExecStart=/usr/local/bin/cri-dockerd --pod-infra-container-image=registry.cn-hangzhou.aliyuncs.com/google_containers/pause:3.9 --network-plugin=cni --cni-conf-dir=/etc/cni/net.d --cni-bin-dir=/opt/cni/bin --container-runtime-endpoint=unix:///var/run/cri-dockerd.sock --cri-dockerd-root-directory=/var/lib/dockershim --docker-endpoint=unix:///var/run/docker.sock --
cri-dockerd-root-directory=/var/lib/docker
ExecReload=/bin/kill -s HUP $MAINPID
TimeoutSec=0
RestartSec=2
Restart=always
StartLimitBurst=3
StartLimitInterval=60s
LimitNOFILE=infinity
LimitNPROC=infinity
LimitCORE=infinity
TasksMax=infinity
Delegate=yes
KillMode=process
[Install]
WantedBy=multi-user.target
EOF

 cat > /etc/systemd/system/cri-dockerd.socket <<-EOF
[Unit]
Description=CRI Docker Socket for the API
PartOf=cri-docker.service
[Socket]
ListenStream=/var/run/cri-dockerd.sock
SocketMode=0660
SocketUser=root
SocketGroup=docker
[Install]
WantedBy=sockets.target
EOF

启动服务
设置服务开机自启动
systemctl daemon-reload
systemctl enable --now cri-dockerd.service

10.准备k8s其它相关镜像

查看需要的镜像，执行命令

kubeadm config images list

registry.k8s.io/kube-apiserver:v1.28.10
registry.k8s.io/kube-controller-manager:v1.28.10
registry.k8s.io/kube-scheduler:v1.28.10
registry.k8s.io/kube-proxy:v1.28.10
registry.k8s.io/pause:3.9
registry.k8s.io/etcd:3.5.12-0
registry.k8s.io/coredns/coredns:v1.10.1

修改成国内的镜像源，拉取镜像

docker pull registry.cn-hangzhou.aliyuncs.com/google_containers/kube-apiserver:v1.28.10
docker pull registry.cn-hangzhou.aliyuncs.com/google_containers/kube-controller-manager:v1.28.10
docker pull registry.cn-hangzhou.aliyuncs.com/google_containers/kube-scheduler:v1.28.10
docker pull registry.cn-hangzhou.aliyuncs.com/google_containers/kube-proxy:v1.28.10
docker pull registry.cn-hangzhou.aliyuncs.com/google_containers/pause:3.9
docker pull registry.cn-hangzhou.aliyuncs.com/google_containers/etcd:3.5.12-0
docker pull registry.cn-hangzhou.aliyuncs.com/google_containers/coredns:v1.10.1

修改镜像的tag

docker tag registry.cn-hangzhou.aliyuncs.com/google_containers/kube-apiserver:v1.28.10 kubernetes-register.openlab.cn/google_containers/kube-apiserver:v1.28.10
docker tag registry.cn-hangzhou.aliyuncs.com/google_containers/kube-controller-manager:v1.28.10 kubernetes-register.openlab.cn/google_containers/kube-controller-manager:v1.28.10
docker tag registry.cn-hangzhou.aliyuncs.com/google_containers/kube-scheduler:v1.28.10 kubernetes-register.openlab.cn/google_containers/kube-scheduler:v1.28.10
docker tag registry.cn-hangzhou.aliyuncs.com/google_containers/kube-proxy:v1.28.10 kubernetes-register.openlab.cn/google_containers/kube-proxy:v1.28.10
docker tag registry.cn-hangzhou.aliyuncs.com/google_containers/pause:3.9 kubernetes-register.openlab.cn/google_containers/pause:3.9
docker tag registry.cn-hangzhou.aliyuncs.com/google_containers/etcd:3.5.12-0 kubernetes-register.openlab.cn/google_containers/etcd:3.5.12-0
docker tag registry.cn-hangzhou.aliyuncs.com/google_containers/coredns:v1.10.1 kubernetes-register.openlab.cn/google_containers/coredns:v1.10.1

注意：上面拉取镜像和给镜像打tag 最好也取node节点上执行一遍，因为k8s自己调度之后，不一定给你部署到哪个节点上，到时候就出出现 镜像拉取失败的问题，当然不执行也没啥问题，等出现镜像拉取失败的问题，到对应node节点上在执行也行，多折腾几次就行了。 

11.初始化集群

kubeadm init --kubernetes-version=1.28.10 \
--apiserver-advertise-address=192.168.2.74 \ --修改成自己的ip
--image-repository kubernetes-register.openlab.cn/google_containers \
--service-cidr=10.96.0.0/12 \
--pod-network-cidr=10.244.0.0/16 \
--ignore-preflight-errors=Swap \
--cri-socket=unix:///var/run/cri-dockerd.sock 

出现以上的日志，代表执行成功了，在根据日志提示，执行对应的命令

mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config 

12. 到其他node节点上执行日志提示的，kubeadm join xxxx命令

13.安装网络组件flannel

wget https://github.com/flannel-io/flannel/releases/latest/download/kube-flannel.yml

sed -i 's|quay.io/coreos/flannel|registry.cn-hangzhou.aliyuncs.com/acs/flannel:v0.14.0|g' kube-flannel.yml

kubectl apply -f  kube-flannel.yml

至此，k8s部署成功

14.注意如果中间出现pod状态不对的，执行kubectl describe pod xx查看对应的信息 

出现截图上的错误

Error response from daemon: cgroups: cgroup mountpoint does not exist: unknown

参考下面的链接解决Error response from daemon: cgroups: cgroup mountpoint does not exist: unknown-CSDN博客