K8S IP迁移流程

一、安装etcdctl

参阅本部分的etcd原理及其使用。

二、etcd备份

ETCDCTL_API=3 etcdctl --endpoints={master节点IP:2379} --cert="/etc/kubernetes/pki/etcd/server.crt" --cacert="/etc/kubernetes/pki/etcd/ca.crt" --key="/etc/kubernetes/pki/etcd/server.key" snapshot save /data/etcd_backup_dir/etcd-snapshot-`date +%Y%m%d-%H`.db 

PS：备份目录不存在需要手动创建

三、修改kubeadm-config.yaml（在每个master上操作）

apiVersion: kubeadm.k8s.io/v1beta2
kind: InitConfiguration
localAPIEndpoint:
  advertiseAddress: 192.168.120.31 //需要修改为修改后的主机IP
  bindPort: 6443
nodeRegistration:
  criSocket: /run/containerd/containerd.sock
---
apiVersion: kubeadm.k8s.io/v1beta2
kind: ClusterConfiguration
kubernetesVersion: v1.20.0
controlPlaneEndpoint: apiserver.cluster.local:6443
imageRepository: k8s.gcr.io
networking:
  # dnsDomain: cluster.local
  podSubnet: 100.64.0.0/10
  serviceSubnet: 10.96.0.0/12
apiServer:
  certSANs:
  - 127.0.0.1
  - apiserver.cluster.local
  - 192.168.120.31
  - 192.168.120.32
  - 192.168.120.33
  - 10.103.97.2 //lvscare维护的虚IP
  - 192.9.213.41
  - 192.9.213.42
  - 192.9.213.43 //增加所有迁移后的master节点IP
  extraArgs:
    feature-gates: TTLAfterFinished=true
  extraVolumes:
  - name: localtime
    hostPath: /etc/localtime
    mountPath: /etc/localtime
    readOnly: true
    pathType: File
controllerManager:
  extraArgs:
    feature-gates: TTLAfterFinished=true
    experimental-cluster-signing-duration: 876000h
  extraVolumes:
  - hostPath: /etc/localtime
    mountPath: /etc/localtime
    name: localtime
    readOnly: true
    pathType: File
scheduler:
  extraArgs:
    feature-gates: TTLAfterFinished=true
  extraVolumes:
  - hostPath: /etc/localtime
    mountPath: /etc/localtime
    name: localtime
    readOnly: true
    pathType: File
---
apiVersion: kubeproxy.config.k8s.io/v1alpha1
kind: KubeProxyConfiguration
mode: ipvs
ipvs:
  excludeCIDRs:
  - 10.103.97.2/32 //lvscare使用的IP需要排除

四、备份所有证书（在每个master上执行）

cp -r /etc/kubernetes/pki/* /etc/kubernetes/bak-pki/ 

五、在IP迁移前重新签发apiserver的证书（在每个master上分别执行）

1，删除apiserver证书

rm /etc/kubernetes/pki/apiserver.* 

2，重新签发新的apiserver证书

kubeadm init phase certs apiserver --config /root/kubeadm-config.yaml 

3，刷新admin.conf

kubeadm certs renew admin.conf 

4，重启kube-apiserver

kubectl delete pod {pod-name} -n kube-system 

六、IP迁移后重新签发etcd的server和peer证书（在每个master分别执行）

1，删除server和peer证书

rm /etc/kubernetes/pki/etcd/peer* rm /etc/kubernetes/pki/etcd/server* 

2，重新签发server和peer证书

kubeadm init phase certs etcd-server --config /root/kubeadm-config.yaml kubeadm init phase certs etcd-peer --config /root/kubeadm-config.yaml 

PS：etcd证书更换不需要重启，是动态生效的
查看CRT证书详细信息命令：

openssl x509 -text -noout -in client.crt 

七、etcd集群信息迁移

1，将步骤二中备份的数据复制到每个master节点的/tmp目录下 /tmp/snapshot.db
2，修改每个master节点上静态POD（api-server,etcd）的地址信息
静态POD由kubelet管理，yaml文件存放于/etc/kubernetes/manifests/ 目录下 修改此处的api-server和etcd的地址数据
3，在每个master节点上删除etcd数据目录

rm -rf /var/lib/etcd 

4，在每个master节点上执行操作

ETCDCTL_API=3 etcdctl --cacert=/etc/kubernetes/pki/etcd/ca.crt --cert=/etc/kubernetes/pki/etcd/healthcheck-client.crt --key=/etc/kubernetes/pki/etcd/healthcheck-client.key snapshot restore /tmp/snapshot.db --name=master02 --initial-cluster=master02=https://192.9.213.42:2380,master01=https://192.9.213.41:2380 --initial-advertise-peer-urls=https://192.9.213.42:2380 --data-dir=/var/lib/etcd 

PS：注意修改–name=master02为当前节点名，并更换后面的IP地址
5，修改每个worker节点上的lvscare的静态POD 主节点地址