kubeadm部署k8s高可用集群

简介

K8S集群通常由两部分构成：master结点和node结点
master结点的主要工作是负责集群的控制，对pod进行调度，对令牌的管理等
node结点的主要工作是负责干活，启动容器和管理容器
master结点和node结点一般采用分布式部署

前期准备

准备三台Centos7虚拟机，修改IP地址和hostname，关闭防火墙和selinux，同步三个结点的时间，修改IP地址与hostname映射关系

hostname	ip
192.168.29.130	master1
192.168.29.131	master2
192.168.29.132	master3
192.168.29.133	node1
192.168.29.134	node2
192.168.29.100	VirtualIP

master1,master2,master3作为集群主结点，node1和node2作为从结点
关闭swap

临时关闭
# swapoff -a
解除挂载
vi /etc/fstab
#UUID=5586e861-1922-462d-86a9-3236085f6653 swap                    swap    defaults        0 0

结点之间进行SSH免密登陆

# ssh-keygen #创建密钥
# ssh-copy-id root@master1
# ssh-copy-id root@master2
# ssh-copy-id root@master3
# ssh-copy-id root@node1
# ssh-copy-id root@node2
#验证
# ssh root@master1
# ssh root@master2
# ssh root@master3
# ssh root@node1
# ssh root@node2

所有结点安装kubernetes以及docker

配置kubernetes的yum源

[kubernetes]
name=Kubernetes repo
baseurl=https://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64/
enabled=1
gpgcheck=0

获取docker包

tar -zxvf docker-20.10.9.tgz
cp -p -f docker/* /usr/bin

配置docker服务

vi /usr/lib/systemd/system/docker.service
[Unit]
Description=Docker Application Container Engine
Documentation=http://docs.docker.com
After=network.target docker.socket
[Service]
Type=notify
EnvironmentFile=-/run/flannel/docker
WorkingDirectory=/usr/local/bin
ExecStart=/usr/bin/dockerd -H tcp://0.0.0.0:4243 -H unix:///var/run/docker.sock --selinux-enabled=false --log-opt max-size=1g
ExecReload=/bin/kill -s HUP
# Having non-zero Limit*s causes performance problems due to accounting overhead
# in the kernel. We recommend using cgroups to do container-local accounting. 
LimitNOFILE=infinity
LimitNPROC=infinity
LimitCORE=infinity
# Uncomment TasksMax if your systemd version supports it. 
# Only systemd 226 and above support this version. 
#TasksMax=infinity
TimeoutStartSec=0
# set delegate yes so that systemd does not reset the cgroups of docker containers 
Delegate=yes
# kill only the docker process, not all processes in the cgroup 
KillMode=process
Restart=on-failure
[Install]
WantedBy=multi-user.target

yum makecache
yum install kubelet-1.21.7 kubeadm-1.21.7 kubectl-1.21.7 -y

安装完成后启动服务

systemctl daemon-reload
systemctl enable --now kubelet
systemctl enable --now docker

修改配置文件

vi /etc/docker/daemon.json
{
  "exec-opts": ["native.cgroupdriver=systemd"]
}
mkdir /var/lib/kubelet -pv
vi /var/lib/kubelet/config.yaml
apiVersion: kubelet.config.k8s.io/v1beta1
kind: KubeletConfiguration
cgroupDriver: systemd

重启服务

systemctl restart kubelet docker

转发配置

# echo 1 > /proc/sys/net/bridge/bridge-nf-call-iptables

部署keepalived

可参考:https://blog.csdn.net/xixixilalalahaha/article/details/106781086

部署集群

master1结点进行集群初始化

[root@master1 ~]# kubeadm init --apiserver-advertise-address=192.168.29.100  --control-plane-endpoint "192.168.29.100" --apiserver-bind-port 6443 --image-repository  registry.aliyuncs.com/google_containers --kubernetes-version v1.21.7  --service-cidr=10.96.0.0/12 --pod-network-cidr=10.244.0.0/16  --upload-certs

执行初始化完成后根据提示进行以下操作，同时需要记录下提示画面中的token等信息，在后续结点加入集群中需要使用

[root@master1 ~]# mkdir -p $HOME/.kube
[root@master1 ~]# cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
[root@master1 ~]# chown $(id -u):$(id -g) $HOME/.kube/config

把master2和master3结点加入到集群中

[root@master2 ~]#kubeadm join 192.168.29.100:6443 --token jt0aws.p9ni06om5c6vqc41 --discovery-token-ca-cert-hash sha256:dde3b1db5dec438df389776202476af779d69eb0b38c8a2a77c9afa8301a94b3 --control-plane  --certificate-key f8902e114ef118304e561c3ecd4d0b543adc226b7a07f675f56564185ffe0c07
[root@master3 ~]#kubeadm join 192.168.29.100:6443 --token jt0aws.p9ni06om5c6vqc41 --discovery-token-ca-cert-hash sha256:dde3b1db5dec438df389776202476af779d69eb0b38c8a2a77c9afa8301a94b3  --control-plane  --certificate-key f8902e114ef118304e561c3ecd4d0b543adc226b7a07f675f56564185ffe0c07

[root@master2 ~]# mkdir -p $HOME/.kube
[root@master2 ~]# cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
[root@master2 ~]# chown $(id -u):$(id -g) $HOME/.kube/config

[root@master3 ~]# mkdir -p $HOME/.kube
[root@master3 ~]# cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
[root@master3 ~]# chown $(id -u):$(id -g) $HOME/.kube/config

把node1和node2结点加入到集群中
若忘记记录token信息或token超过时限(24小时)，可在node1结点执行以下命令重新生成token

[root@master1 ~]# kubeadm token create #创建token
[root@master1 ~]# kubeadm token list #查看token
[root@master1 ~]# openssl x509 -pubkey -in /etc/kubernetes/pki/ca.crt | openssl rsa -pubin -outform der 2>/dev/null | openssl dgst -sha256 -hex | sed 's/^.* //' #生成验证信息

根据初始化完毕后的提示页面中或重新创建的token信息，在node2和node3结点执行以下命令把结点加入集群中

[root@node1 ~]#kubeadm join 192.168.29.100:6443 --token jt0aws.p9ni06om5c6vqc41 --discovery-token-ca-cert-hash sha256:dde3b1db5dec438df389776202476af779d69eb0b38c8a2a77c9afa8301a94b3
[root@node2 ~]#kubeadm join 192.168.29.100:6443 --token jt0aws.p9ni06om5c6vqc41 --discovery-token-ca-cert-hash sha256:dde3b1db5dec438df389776202476af779d69eb0b38c8a2a77c9afa8301a94b3

查看集群情况

[root@master1 ~]# kubectl get nodes