kubernetes1.20版本启用ipvs模式

 

在1.19版本之前,kubeadm部署方式启用ipvs模式时,初始化配置文件需要添加以下内容:

---
apiVersion: kubeproxy.config.k8s.io/v1alpha1
kind: KubeProxyConfiguration
featureGates:
  SupportIPVSProxyMode: true
mode: ipvs

 

本次在1.20.2版本中,使用kubeadm进行集群初始化时,虽然可以正常部署,但是查看pod情况的时候可以看到kube-proxy无法运行成功,报错部分内容如下:

]# kubectl get pod -A
NAMESPACE NAME READY STATUS RESTARTS AGE

....#省略其他输出内容
kube-system kube-proxy-7vrbv 0/1 CrashLoopBackOff 9 43m
kube-system kube-proxy-ghs7h 0/1 CrashLoopBackOff 9 43m
kube-system kube-proxy-l9twb 0/1 CrashLoopBackOff 1 7s
kube-system kube-proxy-mzfrf 0/1 CrashLoopBackOff 9 42m
kube-system kube-proxy-nxpls 0/1 CrashLoopBackOff 9 3h4m
kube-system kube-proxy-pmmtq 0/1 CrashLoopBackOff 8 42m

#查看日志信息
]# kubectl  logs kube-proxy-l9twb -n kube-system
F0114 12:58:34.042769       1 server.go:488] failed complete: unrecognized feature gate: SupportIPVSProxyMode
goroutine 1 [running]:
k8s.io/kubernetes/vendor/k8s.io/klog/v2.stacks(0xc00000e001, 0xc0004b6000, 0x6e, 0xc0)
    /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/vendor/k8s.io/klog/v2/klog.go:1026 +0xb9
k8s.io/kubernetes/vendor/k8s.io/klog/v2.(*loggingT).output(0x29b65c0, 0xc000000003, 0x0, 0x0, 0xc0003d8230, 0x28edbc9, 0x9, 0x1e8, 0x0)
    /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/vendor/k8s.io/klog/v2/klog.go:975 +0x19b
k8s.io/kubernetes/vendor/k8s.io/klog/v2.(*loggingT).printf(0x29b65c0, 0xc000000003, 0x0, 0x0, 0x0, 0x0, 0x1b3a573, 0x13, 0xc000431310, 0x1, ...)
    /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/vendor/k8s.io/klog/v2/klog.go:750 +0x191
k8s.io/kubernetes/vendor/k8s.io/klog/v2.Fatalf(...)

通过报错可以看到kube-proxy无法识别SupportIPVSProxyMode这个字段,于是访问官方查看最新版本ipvs开启的正确配置,通过https://github.com/kubernetes/kubernetes/blob/master/pkg/proxy/ipvs/README.md可以看到官方说明:

Cluster Created by Kubeadm

If you are using kubeadm with a configuration file, you have to add mode: ipvs below the kubeProxy field as part of the kubeadm configuration.

...
kubeProxy:
  config:
    mode: ipvs
...

由于集群已经初始化成功了,所以现在改kubeadm初始化配置文件没有意义,因为我们需要直接修改kube-proxy的启动配置

通过查看kube-pxory的资源清单可以知道, kube-proxy的配置文件是通过configmap方式挂载到容器中的,因此我们只需要对应修改configmap中的配置内容,就可以将无效字段删除

]# kubectl -n kube-system get pod kube-proxy-24tkb -o yaml
apiVersion: v1
kind: Pod
metadata:
.....　　#其他内容省略
  containers:
  - command:
    - /usr/local/bin/kube-proxy
    - --config=/var/lib/kube-proxy/config.conf
    - --hostname-override=$(NODE_NAME)
.....　　#其他内容省略
    volumeMounts:
    - mountPath: /var/lib/kube-proxy
      name: kube-proxy
.....　　#其他内容省略
  volumes:
  - configMap:
      defaultMode: 420
      name: kube-proxy
    name: kube-proxy

 

]# kubectl get cm -n kube-system
NAME                                 DATA   AGE
coredns                              1      5h18m
extension-apiserver-authentication   6      5h18m
kube-proxy                           2      5h18m
kube-root-ca.crt                     1      5h18m
kubeadm-config                       2      5h18m
kubelet-config-1.20                  1      5h18m

]# kubectl edit cm kube-proxy -n kube-system　　　　
#在编辑模式中找到以下字段,删除后保存退出
featureGates: 
  SupportIPVSProxyMode: true

 

然后将删除所有kube-proxy进行重启,查看pod运行情况

]# kubectl get pod -n kube-system
NAME                                          READY   STATUS    RESTARTS   AGE
...　　#其他内容省略
kube-proxy-24tkb                              1/1     Running   0          122m
kube-proxy-9cl7j                              1/1     Running   0          123m
kube-proxy-cxbg5                              1/1     Running   0          123m
kube-proxy-cxgqk                              1/1     Running   0          123m
kube-proxy-hkq54                              1/1     Running   0          123m
kube-proxy-ttdqb                              1/1     Running   0          122m

在服务器上安装ipvsadm,查看ipvs模式是否启用成功

]# yum install ipvsadm -y
]# ipvsadm -Ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
  -> RemoteAddress:Port           Forward Weight ActiveConn InActConn
TCP  10.96.0.1:443 rr
  -> 和谐ip:6443          Masq    1      0          0         
  -> 和谐ip:6443          Masq    1      0          0         
  -> 和谐ip:6443          Masq    1      0          0         
TCP  10.96.0.10:53 rr
TCP  10.96.0.10:9153 rr
UDP  10.96.0.10:53 rr