kubeadm 搭建ipv4/ipv6双栈集群

前置准备

1.三台虚机

node1:192.168.58.197

node2:192.168.58.198

node3:192.168.58.199

vip：192.168.58.16 （绑定在node1上）

2.设置内核参数

net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1
net.ipv4.ip_forward = 1
net.ipv4.conf.all.rp_filter = 0
net.ipv6.conf.all.forwarding = 1
net.ipv4.conf.all.forwarding = 1

3.准备kubeadm,kubelet,kubelet v1.23.3

4、安装好keepalived，让vip绑定在master上

5.给集群配置ipv6地址

部署安装

1.配置ipv4/ipv6 双栈参数

kube-apiserver:
--service-cluster-ip-range=<IPv4 CIDR>,<IPv6 CIDR>
kube-controller-manager:
--cluster-cidr=<IPv4 CIDR>,<IPv6 CIDR>
--service-cluster-ip-range=<IPv4 CIDR>,<IPv6 CIDR>
--node-cidr-mask-size-ipv4|--node-cidr-mask-size-ipv6 defaults to /24 for IPv4 and /64 for IPv6
kube-proxy:
--cluster-cidr=<IPv4 CIDR>,<IPv6 CIDR>
kubelet:
when there is no --cloud-provider the administrator can pass a comma-separated pair of IP addresses via --node-ip to manually configure dual-stack .status.addresses for that Node. If a Pod runs on that node in HostNetwork mode, the Pod reports these IP addresses in its .status.podIPs field. All podIPs in a node match the IP family preference defined by the .status.addresses field for that Node.

2.在master 节点上通过kubeadm-init-master.yaml 文件来进行配置.

---
apiVersion: kubeadm.k8s.io/v1beta3
kind: InitConfiguration # 初始Master节点的私有配置
bootstrapTokens:        # 可以指定bootstrapToken，默认24小过期自动删除
- token: "9a08jv.c0izixklcxtmnze7"
  description: "kubeadm bootstrap token"
  ttl: "24h"
certificateKey: "e6a2eb8581237ab72a4f494f30285ec12a9694d750b9785706a83bfcbbbd2204"      # 可以指定certificateKey,默认两小时过期自动删除
localAPIEndpoint:
  advertiseAddress: "192.168.58.197"    # 控制平台通信使用ipv4
nodeRegistration:
  name: node1
  kubeletExtraArgs:
    node-ip: 192.168.58.197,fd92::102    # 控制平台通信使用ipv4，把ipv4地址放前面
 
---
apiVersion: kubeadm.k8s.io/v1beta3
kind: ClusterConfiguration  # 所有Master节点的公共配置
imageRepository: registry.aliyuncs.com/google_containers
kubernetesVersion: v1.23.3
controlPlaneEndpoint: 192.168.58.16:6443    # 控制平台我们使用ipv4
networking:
  podSubnet: 172.26.0.0/16,172:26::/64    # ipv4放在前面，那么kubectl get node时显示的是ipv4地址
  serviceSubnet: 10.96.0.0/16,10:96::/112    # ipv4放在前面，那么kubectl get service时显示的是ipv4地址
etcd:
  local:
    extralArgs:
      listen-metrics-urls: http://[::]:2381    # 同时监听ipv4与ipv6
apiServer:
  certSANs: ["192.168.58.16", "fd92::200"]
  extraArgs:
    service-cluster-ip-range: 10.96.0.0/16,10:96::/112
    bind-address: "::"
    secure-port: "6443"
    insecure-bind-address: "::"
    insecure-port: "0"
scheduler:
  extraArgs:
    bind-address: "::"
controllerManager:
  extraArgs:
    bind-address: "::"
 
---
apiVersion: kubelet.config.k8s.io/v1beta1
kind: KubeletConfiguration
evictionHard:
  imagefs.available: 5%
  memory.available: 5%
  nodefs.available: 5%
  nodefs.inodesFree: 5%
healthzBindAddress: "::"
healthzPort: 10248
readOnlyPort: 10255
 
---
apiVersion: kubeproxy.config.k8s.io/v1alpha1
kind: KubeProxyConfiguration
bindAddress: "::"
clusterCIDR: "172.26.0.0/16,172:26::/64"    # Pod的地址范围
mode: "iptables"

...

kubectl taint nodes --all node-role.kubernetes.io/master-

calico 网络配置

1. 集群前置环境准备好后，编辑calico.yml 文件，编辑CNI 配置（calico-config 的configmap)如下

   "ipam": {
       "type": "calico-ipam",
       "assign_ipv4": "true",
       "assign_ipv6": "true"
   },

2. 将以下变量添加到calico-code 容器的env数组中：

   key	value
   IP6	autodetect
   FELIX_IPV6SUPPORT	true
   CALICO_IPV6POOL_CIDR	（与kubeadm-init 文件中一致ipv6格式）
   IP6_AUTODETECTION_METHOD	interface=ens33(实际情况填写 ）

   3.kubectl apply -f calico.yml