K8S-SRIOV 及 multus CNI 安装

环境

k8s 版本： v1.22.3
SRIOV 网卡：E810

组件功能

• Multus Plugin： 多网卡支持。
• SRIOV-Network-Device Plugin：通过规则实现SR-IOV的VF自动选取。
• SRIOV CNI：具体干活的，把VF挂载到POD里面，设置MAC，VLAN等等。

SRIOV CNI 安装

git clone https://github.com/intel/sriov-cni.git
cd sriov-cni
make
cp build/sriov /opt/cni/bin
cd /opt/cni/bin
chmod 777 sriov

每个需要创建 vf 的网卡都需要安装

SRIOV network device plugin

可以直接 pull 镜像

docker pull nfvpe/sriov-device-plugin

可以 make image

git clone https://github.com/intel/sriov-network-device-plugin.git
cd sriov-network-device-plugin
make image

同样每个节点都需要

安装

ConfigMap
vendors:devices 可以通过 lspci -nn 进行查询

apiVersion: v1
kind: ConfigMap
metadata:
  name: sriovdp-config
  namespace: kube-system
data:
  config.json: |
    {
        "resourceList": [{
                "resourceName": "intel_sriov_netdevice",
                "resourcePrefix": "intel.com",
                "selectors": {
                    "vendors": ["8086"],
                    "devices": ["1889"],
                    "drivers": ["iavf"]
                }
            }
        ]
    }

sriov-crd.yaml

apiVersion: "k8s.cni.cncf.io/v1"
kind: NetworkAttachmentDefinition
metadata:
  name: sriov-net1
  annotations:
    k8s.v1.cni.cncf.io/resourceName: intel.com/intel_sriov_netdevice
spec:
  config: '{
  "type": "sriov",
  "cniVersion": "0.3.1",
  "name": "sriov-network",
  "ipam": {
    "type": "host-local",
    "subnet": "10.56.217.0/24",
    "routes": [{
      "dst": "0.0.0.0/0"
    }],
    "gateway": "10.56.217.1"
  }
}'

kubectl create -f sriov-network-device-plugin/deployments/k8s-v1.16/sriovdp-daemonset.yaml

pod 使用

pod-iperf-vf.yaml

apiVersion: v1
kind: Pod
metadata:
  name: iperf-vf
  namespace: default
  annotations:
    k8s.v1.cni.cncf.io/networks: sriov-net1
spec:
  containers:
  - name: iperf-vf
    image: sirot/netperf-latest
    command:
    - sleep
    - "infinity"
    resources:
      requests:
        intel.com/intel_sriov_netdevice: '1'
      limits:
        intel.com/intel_sriov_netdevice: '1'
  nodeSelector:
    kubernetes.io/hostname: master

参考

参考至同事泓宇 https://www.jianshu.com/p/ddd788d2458f