安装kata container with cri-o

ImmerSommer

已于 2022-11-24 01:18:01 修改

阅读量518

点赞数

文章标签： ubuntu linux 运维

于 2022-11-07 00:58:54 首次发布

本文链接：https://blog.csdn.net/immerarbeiten/article/details/127723732

版权

Install CRI-O Container Runtime on Ubuntu 20.04

参考教程： https://computingforgeeks.com/install-cri-o-container-runtime-on-ubuntu-linux/

Step1：更新系统

sudo apt update && sudo apt upgrade

Step2：安装CRI-O相关

cri-o版本应与Kubernetes版本相对应。此处使用的Kubernetes版本为1.24，因此CRI-O版本也使用1.24。

OS=xUbuntu_20.04
CRIO_VERSION=1.24
echo "deb https://download.opensuse.org/repositories/devel:/kubic:/libcontainers:/stable/$OS/ /"|sudo tee /etc/apt/sources.list.d/devel:kubic:libcontainers:stable.list
echo "deb http://download.opensuse.org/repositories/devel:/kubic:/libcontainers:/stable:/cri-o:/$CRIO_VERSION/$OS/ /"|sudo tee /etc/apt/sources.list.d/devel:kubic:libcontainers:stable:cri-o:$CRIO_VERSION.list

设置GPG key （忽略此步后续会产生报错）

curl -L https://download.opensuse.org/repositories/devel:kubic:libcontainers:stable:cri-o:$CRIO_VERSION/$OS/Release.key | sudo apt-key add -
curl -L https://download.opensuse.org/repositories/devel:/kubic:/libcontainers:/stable/$OS/Release.key | sudo apt-key add -

Step3：在Ubuntu上安装cri-o

sudo apt update
sudo apt install cri-o cri-o-runc

检查cri-o版本

$ apt show cri-o
Package: cri-o
Version: 1.24.3~0
Priority: optional
Section: devel
Maintainer: Peter Hunt <haircommander@fedoraproject.org>
Installed-Size: 96.1 MB
Depends: libgpgme11, libseccomp2, conmon, containers-common (>= 0.1.27) | golang-github-containers-common, tzdata
Suggests: cri-o-runc | runc (>= 1.0.0), containernetworking-plugins
Replaces: cri-o-1.19, cri-o-1.20, cri-o-1.21
Homepage: https://github.com/cri-o/cri-o
Download-Size: 20.6 MB
APT-Manual-Installed: yes
APT-Sources: http://download.opensuse.org/repositories/devel:/kubic:/libcontainers:/stable:/cri-o:/1.24/xUbuntu_20.04  Packages
Description: OCI-based implementation of Kubernetes Container Runtime Interface.

启动cri-o

sudo systemctl enable crio.service
sudo systemctl start crio.service

检查运行状态

$ systemctl status crio
● crio.service - Container Runtime Interface for OCI (CRI-O)
     Loaded: loaded (/lib/systemd/system/crio.service; enabled; vendor preset: enabled)
     Active: active (running) since Sun 2022-11-06 14:08:53 CET; 3h 20min ago
       Docs: https://github.com/cri-o/cri-o
   Main PID: 2702634 (crio)
      Tasks: 30
     Memory: 17.6M
     CGroup: /system.slice/crio.service
             └─2702634 /usr/bin/crio

安装kata container相关组件

下载测试文档

git clone https://github.com/kata-containers/tests.git

检查是否有残余kata组件存在，如若存在，则卸载干净

~/tests/cmd/kata-manager$ ./kata-manager.sh  remove-packages

接着进行安装

~/tests/cmd/kata-manager$ ./kata-manager.sh install-packages

可能会出现错误

Err:4 http://download.opensuse.org/repositories/home:/katacontainers:/releases:/x86_64:/master/xUbuntu_20.04  InRelease
  The following signatures were invalid: EXPKEYSIG D0B37B826063F3ED home:katacontainers OBS Project <home:katacontainers@build.opensuse.org>
E: The repository 'http://download.opensuse.org/repositories/home:/katacontainers:/releases:/x86_64:/master/xUbuntu_20.04  InRelease' is not signed.

采用以下方法解决 Apt-Key expired · Issue #545 · kata-containers/kata-containers · GitHub

~/tests/cmd/kata-manager$ sudo apt-get -o Acquire::AllowInsecureRepositories=true update
~/tests/cmd/kata-manager$ sudo apt-get --allow-unauthenticated -y install kata-runtime kata-proxy kata-shim kata-ksm-throttler

成功安装

Setting up kata-proxy (1.13.0~alpha0-50) ...
Setting up kata-containers-image (1.13.0~alpha0-49) ...
Setting up kata-shim (1.13.0~alpha0-48) ...
Setting up kata-linux-container (5.4.60.91-52) ...
Setting up kata-ksm-throttler (1.13.0~alpha0-52) ...
Setting up kata-runtime (1.13.0~alpha0-57) ...

cri-o配置文件

参考： documentation/run-kata-with-k8s.md at master · kata-containers/documentation · GitHub

更改cri-o配置文件（默认路径 /etc/crio/crio.conf）

manage_ns_lifecycle = true

[crio.runtime.runtimes.kata-runtime]
  runtime_path = "/usr/bin/kata-runtime"
  runtime_type = "oci"

该文件进行任何更改后，都要进行重启

sudo systemctl restart crio

kubernetes安装

配置/etc/systemd/system/kubelet.service.d/0-crio.conf

[Service]
Environment="KUBELET_EXTRA_ARGS=--container-runtime=remote --runtime-request-timeout=15m --container-runtime-endpoint=unix:///var/run/crio/crio.sock"

创建一个集群

关闭交换

sudo swapoff -a
sudo sed -i '/ swap / s/^/#/' /etc/fstab

初始化集群

$ sudo systemctl daemon-reload
$ sudo systemctl restart kubelet
$ sudo kubeadm init --cri-socket /var/run/crio/crio.sock --pod-network-cidr=10.244.0.0/16 --ignore-preflight-errors=ALL

添加网络

kubectl apply -f https://raw.githubusercontent.com/flannel-io/flannel/master/Documentation/kube-flannel.yml

让pod在主节点上运行

$ kubectl get node
NAME          STATUS   ROLES           AGE    VERSION
epyc-maggie   Ready    control-plane   2d8h   v1.25.3
$ kubectl taint node epyc-maggie node-role.kubernetes.io/control-plane:NoSchedule-
node/epyc-maggie untainted

创建kata runtime

apiVersion: node.k8s.io/v1

kind: RuntimeClass

metadata:

name: kata-origin

handler: kata-runtime

$ kubectl get runtimeclass
NAME          HANDLER        AGE
kata-origin   kata-runtime   2d8h
kata-sev      kata-sev       2d4h

创建pod

apiVersion: v1

kind: Pod

metadata:

name: test-pod-origin

labels:

app: origin

spec:

runtimeClassName: kata-origin

containers:

- name: origin

image: nginx

ports:

- containerPort: 22

成功运行

$ kubectl get pod
NAME              READY   STATUS              RESTARTS   AGE
test-pod-origin   1/1     Running             1          2d4h

运行包含SEV的kata容器

方式一：

采用新的路径创建runtime

[crio.runtime.runtimes.kata-sev]

runtime_path = "/home/zxxx/kata-runtime-2.x-SEV/src/runtime/kata-runtime"

runtime_type = "oci"

$ kubectl get runtimeclass
NAME          HANDLER        AGE
kata-sev      kata-sev       3d1h

用新的kata runtime 运行pod，会产生错误

Node-Selectors:              <none>
Tolerations:                 node.kubernetes.io/not-ready:NoExecute op=Exists for 300s
                             node.kubernetes.io/unreachable:NoExecute op=Exists for 300s
Events:
  Type     Reason                  Age               From               Message
  ----     ------                  ----              ----               -------
  Normal   Scheduled               17s               default-scheduler  Successfully assigned default/test-pod-sev to epyc-maggie
  Warning  FailedCreatePodSandBox  5s (x2 over 17s)  kubelet            Failed to create pod sandbox: rpc error: code = Unknown desc = container create failed: Invalid command "create"

方法二：

原始路径代码直接覆盖

错误相同

猜想：kata-runtime版本不同导致 https://github.com/kata-containers/kata-containers/issues/1133

原版使用1.0.0版本，新版使用2.0.0版本，新版缺少语句

查看容器内部

kubectl exec -i -t <pod-name> -- /bin/bash

尝试另一种kata容器定义

[crio.runtime.runtimes.kata-runtime]

runtime_path = "/usr/bin/containerd-shim-kata-v2"

runtime_type = "vm"

runtime_root = "/run/vc"

privileged_without_host_devices = true

pod 可正常运行

将SEV相关覆盖源代码，出现错误

Events:
  Type     Reason                  Age   From               Message
  ----     ------                  ----  ----               -------
  Normal   Scheduled               12s   default-scheduler  Successfully assigned default/test-pod-shim-origin-sev to epyc-maggie
  Warning  FailedCreatePodSandBox  12s   kubelet            Failed to create pod sandbox: rpc error: code = Unknown desc = CreateContainer failed: failed to launch qemu: exit status 1, error messages from qemu log: qemu-vanilla-system-x86_64: -device vhost-vsock-pci,disable-modern=false,vhostfd=3,id=vsock-2921524591,guest-cid=2921524591,romfile=,iommu_platform=true,iommu_platform=on: VIRTIO_F_IOMMU_PLATFORM was supported by neither legacy nor transitional device
: unknown

Bug #1915509 “QEMU 1:4.2-3ubuntu6.12 : Unable to start SEV enabl...” : Bugs : qemu package : Ubuntu

猜测：包含SEV的kata容器不和kubectl兼容