k8s修改pod IP地址池-Calico CIDR

说明

1 k8s通过二进制方式部署
2 k8s官网更加推荐集群重新部署
3 下述方法会使k8s集群整体停止服务一段时间

方式

1. 将所有的deployment缩减到0，或者删除部署
2. 删除所有Node节点

#查询所有节点
kubectl get node
#删除包括所有master在内的节点
kubectl delete node xxx

3. 修改master上服务controller-manager

vim /usr/lib/systemd/system/kube-controller-manager.service
#修改--
cluster-cidr=172.32.0.0/16 
#重启服务
systemctl daemon-reload
systemctl restart kube-controller-manager.service

4. 修改kube-proxy

vim /etc/kubernetes/kube-proxy.conf
#修改
clusterCIDR: 172.32.0.0/16

将kube-proxy配置文件发送到所有节点，包括master和node

for NODE in master01 node03 node06; do
   scp /etc/kubernetes/kube-proxy.conf $NODE:/etc/kubernetes/kube-proxy.conf
done

#登录到其他节点执行服务重启
systemctl daemon-reload
systemctl restart kube-proxy

5. 修改calico
   系统的Calico通过calico-etcd.yaml部署，yaml 从Calico官网下载

wget https://docs.projectcalico.org/archive/v3.14/manifests/calico-etcd.yaml --no-check-certificate

修改Calico文件并生效

cd /home/k8s_backup/k8spckg/calico
vim calico-etcd.yaml
#修改
 # The default IPv4 pool to create on startup if none exists. Pod IPs will be
 # chosen from this range. Changing this value after installation will have
 # no effect. This should fall within `--cluster-cidr`.
 # - name: CALICO_IPV4POOL_CIDR
 #   value: "192.168.0.0/16"
- name: CALICO_IPV4POOL_CIDR
              value: 172.32.0.0/16

#配置生效              
kubectl apply -f calico-etcd.yaml

根据yaml文件中描述，如果Calico已经安装过，修改的cidr并不会生效，此时我们需要通过calicoctl工具进行操作，修改calico ipam地址池。

#下载calicoctl工具
curl -o /usr/local/bin/calicoctl -O -L https://github.com/projectcalico/calico/releases/download/v3.15.1/calicoctl-linux-amd64
chmod +x /usr/local/bin/calicoctl

#测试
 calicoctl ipam show --allow-version-mismatch
+----------+---------------+-----------+------------+--------------+
| GROUPING |     CIDR      | IPS TOTAL | IPS IN USE |   IPS FREE   |
+----------+---------------+-----------+------------+--------------+
| IP Pool  | 172.17.0.0/16 |     65536 | 5 (0%)     | 65531 (100%) |
+----------+---------------+-----------+------------+--------------+

calicoctl 关联到k8s

mkdir /etc/calico

#如果通过kubeadminitiate 安装，配置calicoctl连接关联kube-apiserver
cat >/etc/calico/calicoctl.cfg<<'EOF'
apiVersion: projectcalico.org/v3
kind: CalicoAPIConfig
metadata:
spec:
  datastoreType: "kubernetes"
  kubeconfig: "/root/.kube/config"
EOF

#如果通过二进制安装，etcd 额外部署
cat >/etc/calico/calicoctl.cfg<<'EOF'
apiVersion: projectcalico.org/v3
kind: CalicoAPIConfig
metadata:
spec:
  datastoreType: "etcdv3"
  etcdEndpoints: 'https://192.168.0.10:2379'
  etcdKeyFile: "/etc/kubernetes/pki/etcd/etcd-key.pem"
  etcdertFile: "/etc/kubernetes/pki/etcd/etcd.pem"
  etcdCACertFile: "/etc/kubernetes/pki/etcd/etcd-ca.pem"
EOF

#测试：
calicoctl get nodes --allow-version-mismatch

查看旧的地址池

calicoctl get ippool

导出旧地址池文件

caclico get ippool default-ipv4-ippool -o yaml > old.yaml --allow-version-mismatch

创建新的地址池文件

vim newippool.yaml

apiVersion: projectcalico.org/v3
kind: IPPool
metadata:
 name: new-pool
spec:
 cidr: 172.32.0.0/16
 ipipMode: Always
 natOutgoing: true

应用新的地址池

calicoctl apply -f new.yaml --allow-version-mismatch

停止旧的地址池，启用新地址池

#修改旧地址池yaml文件
vim old.yaml

spec:
 disabled: true
 
#应用旧地址池yaml文件
calicoctl apply -f new.yaml --allow-version-mismatch
calicoctl get ippool -o wide --allow-version-mismatch


#重启所有pod，重新创建所有现有工作负载
#通过运行以下命令检查新工作负载现在是否在新IP池中具有地址： 
calicoctl get wep --all-namespaces  --allow-version-mismatch

#删除旧地址池
calicoctl delete pool default-ipv4-ippool

5. 重启所有节点的kubelet,把节点都加回来

参考资料
https://blog.csdn.net/suiyingday/article/details/114965791
https://www.cnblogs.com/ygbh/p/17279505.html
https://docs.tigera.io/calico/latest/operations/calicoctl/install