提示:以下配置都要确认calico的类型是“calico-ipam”
cat /etc/cni/net.d/10-calico.conflist "ipam": { "type": "calico-ipam" }, |
一、IPPool配置
全局配置不区分某一个namespace
apiVersion: projectcalico.org/v3 kind: IPPool metadata: name: zone-west-ippool2 spec: cidr: 172.122.2.0/24 #填写创建集群时规划的cidr地址段 ipipMode: Never #不使用IPIP模式相反为Always时代表使用IPIP模式 natOutgoing: true #nat转发 nodeSelector: all() allowedUses: - Workload - Tunnel blockSize: 26 vxlanMode: CrossSubnet #vxlanMode可以为Always或者是CrossSubnet当为CrossSubnet时calico可以智能的判断calico节点是否在一个二层网络如果在使用BGP,如果需要依赖三层网络则使用VXlan模式 |
二、根据拓扑分配不同段的IP地址
如果您在不同的区域、专区或机架中有工作负载,您可能希望它们从同一个 IP 池中获取 IP 地址。此策略可用于减少网络中所需的路由数量,或满足外部防火墙设备或策略强加的要求。Calico 使用带有节点标签和节点选择器的 IP 池资源可以轻松完成此操作。
在下面的示例中,我们创建了两个 IP 池,它只为标签为zone=west和zone=west2的节点分配 IP 地址
kind: IPPool metadata: name: zone-west-ippool1 spec: cidr: 172.122.1.0/24 ipipMode: Always natOutgoing: true nodeSelector: zone == "west" --- apiVersion: projectcalico.org/v3 kind: IPPool metadata: name: zone-west-ippool2 spec: cidr: 172.122.2.0/24 ipipMode: Always natOutgoing: true nodeSelector: zone == "west2" 然后,我们用 zone=west 标记一个节点。例如: kubectl label nodes node1 zone=west kubectl label nodes node2 zone=west2 kubectl label node master1 zone=west kubectl label node master2 zone=west2 kubectl label node master3 zone=west2 [root@master1 install]# calicoctl get IPPOOL -o wide NAME CIDR NAT IPIPMODE VXLANMODE DISABLED DISABLEBGPEXPORT SELECTOR zone-west-ippool1 172.122.1.0/24 true Always Never false false zone == "west" zone-west-ippool2 172.122.2.0/24 true Always Never false false zone == "west2" 可以看到我们为节点配置好不同的标签后,会根据标签的值来匹配到对应的IPPOOL,然后为POD分配对应网段的IP [root@master1 install]# kubectl get pod -o wide NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES nginx-569bf69fb7-5669s 1/1 Running 0 12m 172.122.1.129 node1 nginx-569bf69fb7-8fjbm 1/1 Running 0 12m 172.122.1.64 master1 nginx-569bf69fb7-9jl5c 1/1 Running 0 12m 172.122.2.66 master3 nginx-569bf69fb7-dlmm9 1/1 Running 0 12m 172.122.2.2 node2 nginx-569bf69fb7-hlcl8 1/1 Running 0 12m 172.122.1.65 master1 nginx-569bf69fb7-hzzd8 1/1 Running 0 12m 172.122.2.128 master2 nginx-569bf69fb7-jxg2c 1/1 Running 0 12m 172.122.1.130 node1 nginx-569bf69fb7-k22x8 1/1 Running 0 12m 172.122.2.129 master2 nginx-569bf69fb7-vrppw 1/1 Running 0 12m 172.122.2.64 master3 nginx-569bf69fb7-zkmrn 1/1 Running 0 12m 172.122.2.65 master3 |
注意:如果节点没有这些标签的话,pod调度到这台机器上就会创建失败,如下图
三、Pod IP 从Pool迁移到另一个Pool
IP 池和集群 CIDR
Calico 支持在集群中使用多个不相交的 IP 池 CIDR。但是,Kubernetes 期望所有 pod 的地址都在同一个集群 CIDR 中。这意味着虽然在集群 CIDR 之外创建 IP 池在技术上是可行的,但我们不建议这样做。在 Kubernetes 集群 CIDR 之外分配地址的 Pod 将失去网络连接。
- IPv4:20-32(含)
- IPv6:116-128(含)
-
添加新 IP 池
我们添加了一个新的IPPool,其 CIDR 范围为10.0.0.0/16kind: IPPool metadata: name: new-pool spec: cidr: 10.0.0.0/16 ipipMode: Always natOutgoing: true [root@master1 install]# kubectl apply -f new-ippool.yaml [root@master1 install]# calicoctl get ippool -o wide NAME CIDR NAT IPIPMODE VXLANMODE DISABLED DISABLEBGPEXPORT SELECTOR new-pool 10.0.0.0/16 true Always Never false false all() zone-west-ippool1 172.122.1.0/24 true Never CrossSubnet false false zone == "west" zone-west-ippool2 172.122.2.0/24 true Never CrossSubnet false false zone == "west2"
-
禁用旧的IP池
[root@master1 install]# kubectl edit IPPOOL zone-west-ippool1 在spec下修改或增加以下配置,之后wq保存退出 disabled: true 查看zone-west-ippool1 DISABLED已经变为了true [root@master1 install]# calicoctl get ippool -o wide NAME CIDR NAT IPIPMODE VXLANMODE DISABLED DISABLEBGPEXPORT SELECTOR new-pool 10.0.0.0/16 true Always Never false false all() zone-west-ippool1 172.122.1.0/24 true Never CrossSubnet true false zone == "west" zone-west-ippool2 172.122.2.0/24 true Never CrossSubnet false false zone == "west2"
-
删除之前使用172.122.1.0/24 网段的pod
[root@master1 install]# kubectl get pod -o wide | grep 172.122.1 | awk '{print $1}' | xargs kubectl delete pod pod "nginx-569bf69fb7-5669s" deleted pod "nginx-569bf69fb7-8fjbm" deleted pod "nginx-569bf69fb7-hlcl8" deleted pod "nginx-569bf69fb7-jxg2c" deleted 查看更新的pod及IP网段,可以看到已经从172.122.1.0 变为了10.0网段 [root@master1 install]# kubectl get pod -o wide NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES nginx-569bf69fb7-2js4g 1/1 Running 0 17s 10.0.137.130 master1 nginx-569bf69fb7-5z2w2 1/1 Running 0 16s 10.0.180.1 master2 nginx-569bf69fb7-9jl5c 1/1 Running 0 112m 172.122.2.66 master3 nginx-569bf69fb7-dlmm9 1/1 Running 0 112m 172.122.2.2 node2 nginx-569bf69fb7-jbs2v 1/1 Running 0 16s 10.0.166.129 node1 nginx-569bf69fb7-jvwtl 1/1 Running 0 17m 172.122.2.131 master2 nginx-569bf69fb7-k46hr 1/1 Running 0 17m 172.122.2.132 master2 nginx-569bf69fb7-qp9ht 1/1 Running 0 16s 10.0.137.131 master1 nginx-569bf69fb7-vrppw 1/1 Running 0 112m 172.122.2.64 master3 nginx-569bf69fb7-zkmrn 1/1 Running 0 112m 172.122.2.65 master3
四、更改IP池块blockSize
默认情况下,Calico 使用 64 个地址的 IPAM 块大小 - /26 用于 IPv4,/122 用于 IPv6。但是,可以根据 IP 池地址系列更改块大小。
- IPv4:20-32(含)
- IPv6:116-128(含)
在安装清单中,每个协议只能有一个默认 IP 池。在此示例中,有一个用于 IPv4 (/26) 的 IP 池和一个用于 IPv6 (/122) 的 IP 池.
apiVersion: operator.tigera.io/v1 kind: Installation metadata: name: default pec: # Configures Calico networking. calicoNetwork: # Note: The ipPools section cannot be modified post-install. ipPools: - blockSize: 26 cidr: 10.48.0.0/21 encapsulation: IPIP natOutgoing: Enabled nodeSelector: all() - blockSize: 122 cidr: 2001::00/64 encapsulation: None natOutgoing: Enabled nodeSelector: all() |
扩大或缩小 IP 池块大小
默认情况下,IP 池的 Calico IPAM 块大小为 /26。要从默认大小 /26 扩展,请降低blockSize
(例如,/24)。要从默认 /26 缩大blockSize
,请提高数字(例如,/28)
注意:扩容BlockSize时需要将集群内所有原网段以分配的pod杀死,使用临时的IPPOOL的IP。生产环境一定要三思而后行。
1. 创建临时 IP 池
我们添加了一个 CIDR 范围为 10.0.0.0/16 的新 IPPool。
创建一个temporary-pool.yaml
apiVersion: projectcalico.org/v3 kind: IPPool metadata: name: temporary-pool spec: cidr: 10.0.0.0/16 ipipMode: Always natOutgoing: true [root@master1 ~]# calicoctl get ippool -o wide NAME CIDR NAT IPIPMODE DISABLED default-ipv4-ippool 172.112.0.0/16 true Always false temporary-pool 10.0.0.0/16 true Always false 禁用现有 IP 池 [root@master1 ~]# calicoctl patch ippool default-ipv4-ippool -p '{"spec": {"disabled": true}}' [root@master1 ~]# calicoctl get ippool -o wide NAME CIDR NAT IPIPMODE DISABLED default-ipv4-ippool 192.168.0.0/16 true Always true temporary-pool 10.0.0.0/16 true Always false 注意:扩容BlockSize时需要将集群内所有原网段以分配的pod杀死,使用临时的IPPOOL的IP。生产环境一定要三思而后行 只需一个命令即可重新启动所有 pod。 [root@master1 ~]# $kubectl delete pod -A --all 三思而行 现在您已验证 pod 正在从新范围获取 IP,您可以安全地删除现有池。 [root@master1 ~]# calicoctl delete ippool default-ipv4-ippool |
2. 创建具有所需块大小的新 IP 池
在这一步中,我们使用 (/28) 的新块大小更新 IPPool。
apiVersion: projectcalico.org/v3
kind: IPPool
metadata:
name: default-ipv4-ippool
spec:
blockSize: 28
cidr: 172.122.0.0/16
ipipMode: Always
natOutgoing: true
calicoctl apply -f pool.yaml
禁用临时 IP 池
[root@master1 ~]# calicoctl patch ippool temporary-pool -p '{"spec": {"disabled": true}}'
[root@master1 ~]# $kubectl delete pod -A --all 三思而行
通过运行以下命令验证您的 pod 和块大小是否正确:
[root@master1 ~]# kubectl get pods --all-namespaces -o wide
[root@master1 ~]# calicoctl ipam show --show-blocks
+----------+--------------------+-----------+------------+--------------+
| GROUPING | CIDR | IPS TOTAL | IPS IN USE | IPS FREE |
+----------+--------------------+-----------+------------+--------------+
| IP Pool | 172.122.0.0/16 | 65536 | 33 (0%) | 65503 (100%) |
| Block | 172.122.101.80/28 | 16 | 16 (100%) | 0 (0%) |
| Block | 172.122.101.96/28 | 16 | 4 (25%) | 12 (75%) |
| Block | 172.122.180.112/28 | 16 | 3 (19%) | 13 (81%) |
| Block | 172.122.74.16/28 | 16 | 2 (12%) | 14 (88%) |
| Block | 172.122.75.96/28 | 16 | 5 (31%) | 11 (69%) |
| Block | 172.122.90.32/28 | 16 | 3 (19%) | 13 (81%) |
+----------+--------------------+-----------+------------+--------------+
删除临时 IP 池
[root@master1 ~]# calicoctl delete pool temporary-pool
[root@master1 install]# calicoctl get IPPOOL -o wide
NAME CIDR NAT IPIPMODE VXLANMODE DISABLED DISABLEBGPEXPORT SELECTOR
default-ipv4-ippool 172.122.0.0/16 true Always Never false false all() |