kubernetes集群扩缩容

10.98.4.1 master
10.98.4.2 node01
10.98.4.3 node02
扩容节点 10.98.4.4 master02

初始化扩容节点

需要预先有docker环境
#关闭防火墙
systemctl stop firewalld
systemctl disable firewalld
#关闭selinux
sed -i ‘s/enforcing/disabled/’ /etc/selinux/config # 永久
setenforce 0 # 临时
#关闭swap
swapoff -a # 临时
sed -ri ‘s/.swap./#&/’ /etc/fstab # 永久
#根据规划设置主机名
#hostnamectl set-hostname
#在master添加hosts
cat > /etc/hosts << EOF
127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4
::1 localhost localhost.localdomain localhost6 localhost6.localdomain6
10.98.4.1 k8s-master
10.98.4.2 k8s-node1
10.98.4.3 k8s-node2
10.98.4.4 k8s-mater02
EOF

#将桥接的IPv4流量传递到iptables的链
cat > /etc/sysctl.d/k8s.conf << EOF
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1
EOF

sysctl --system # 生效
#时间同步
yum install ntpdate -y
ntpdate time.windows.com

kubeadm扩容步骤

#配置安装源
cat > /etc/yum.repos.d/kubernetes.repo << EOF
[kubernetes]
name=Kubernetes
baseurl=https://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64
enabled=1
gpgcheck=0
repo_gpgcheck=0
gpgkey=https://mirrors.aliyun.com/kubernetes/yum/doc/yum-key.gpg https://mirrors.aliyun.com/kubernetes/yum/doc/rpm-package-key.gpg
EOF
1，安装集群同版本组件
yum install -y kubelet-1.23.0 kubeadm-1.23.0 kubectl-1.23.0
systemctl enable kubelet

2，生成加入集群token
kubeadm token create --print-join-command
kubeadm join 10.98.4.1:6443 --token e4vrfd.tzgxfgxzy14dluj4 --discovery-token-ca-cert-hash sha256:ace5ee01eace4fba6f5ed134788049fdf4e7741417bd1046d9735585616e51f8

3，生成证书供新master节点使用，1.16版本前参数为--experimental-upload-certs，1.16及1.16版本以后为--upload-certs 
kubeadm init phase upload-certs --upload-certs 
ab072631de5093870c266a0455fd92a8264ed3a781a7a3d915853280542a3342

4，如果扩容的是node节点直接使用token加入集群
kubeadm join 10.98.4.1:6443 --token e4vrfd.tzgxfgxzy14dluj4 --discovery-token-ca-cert-hash 

5，如果是master节点使用token和证书拼接命令，同样，1.16版本前参数为--experimental-control-plane --certificate-key ，1.16及1.16版本以后为
--control-plane --certificate-key
kubeadm join 10.98.4.1:6443 --token e4vrfd.tzgxfgxzy14dluj4 --discovery-token-ca-cert-hash sha256:ace5ee01eace4fba6f5ed134788049fdf4e7741417bd1046d9735585616e51f8 --control-plane --certificate-key ab072631de5093870c266a0455fd92a8264ed3a781a7a3d915853280542a3342

6，扩容的master节点第一次加入集群需要原master修改配置文件
kubectl -n kube-system get cm kubeadm-config -o yaml 
kubectl -n kube-system edit cm kubeadm-config
kind: ClusterConfiguration
    kubernetesVersion: v1.18.0
    controlPlaneEndpoint: 10.98.4.1:6443 #添加此行

二进制安装kubernetes集群扩容

1，拷贝所有证书文件到master02
scp -r /opt/kubernetes root@10.98.4.4:/opt
scp -r /opt/etcd/ssl root@10.98.4.4:/opt/etcd
scp /usr/lib/systemd/system/kube* root@192.168.4.4:/usr/lib/systemd/system
scp /usr/bin/kubectl  root@10.98.4.4:/usr/bin
scp -r ~/.kube root@10.98.4.4:~

2，删除kubelet证书和kubeconfig文件：
rm -f /opt/kubernetes/cfg/kubelet.kubeconfig 
rm -f /opt/kubernetes/ssl/kubelet*

3，修改apiserver、kubelet和kube-proxy配置文件为master02的IP
vim /opt/kubernetes/cfg/kube-apiserver.conf 
...
--bind-address=10.98.4.4 \
--advertise-address=10.98.4.4 \
...

vim /opt/kubernetes/cfg/kube-controller-manager.kubeconfig
server: https://10.98.4.4:6443

vim /opt/kubernetes/cfg/kube-scheduler.kubeconfig
server: https://10.98.4.4:6443

vim /opt/kubernetes/cfg/kubelet.conf
--hostname-override=k8s-master2

vim /opt/kubernetes/cfg/kube-proxy-config.yml
hostnameOverride: k8s-master2

vim ~/.kube/config
...
server: https://10.98.4.4:6443

4，配置服务启动
systemctl daemon-reload
systemctl start kube-apiserver kube-controller-manager kube-scheduler kubelet kube-proxy
systemctl enable kube-apiserver kube-controller-manager kube-scheduler kubelet kube-proxy
systemctl status kube-apiserver kube-controller-manager kube-scheduler kubelet kube-proxy

5，在master节点上配置master02 证书申请
kubectl get csr
kubectl certificate approve node-csr-rRJ0VtVz7AJlQt56mwwgc6iv0nUkrrcnSf3ttPmaRTU

6，查看集群node
kubectl get node
NAME        STATUS   ROLES    AGE   VERSION
k8s-master1    Ready    <none>   34h   v1.20.4
k8s-master2    Ready    <none>   2m   v1.20.4
k8s-node1     Ready    <none>   33h   v1.20.4
k8s-node2     Ready    <none>   33h   v1.20.4

node节点下线

1，设置不可调度
kubectl cordon k8s-master2
2，驱逐节点pod
kubectl drain k8s-node3 --delete-local-data  --ignore-daemonsets
3，检查节点pod
kubectl describe nodes k8s-master2
4，摘除node
kubectl delete nodes k8s-master2