containerd的客户端工具ctr命令

lldhsds

于 2024-06-22 22:46:24 发布

阅读量946

点赞数 15

分类专栏： kubernetes 文章标签： containerd

本文链接：https://blog.csdn.net/codelearning/article/details/139889511

版权

kubernetes 专栏收录该内容

41 篇文章 1 订阅

订阅专栏

containerd的客户端工具ctr命令

类似docker为docker-shim容器运行时的客户端工具，ctr 是 containerd 的客户端工具，安装containerd作为容器运行时后，会自动安装ctr。

root@master1:~# ctr -v
ctr github.com/containerd/containerd 1.7.12

1. 帮助文档

ctr帮助文档

root@master1:~# ctr --help

containerd CLI

USAGE:
   ctr [global options] command [command options] [arguments...]

VERSION:
   1.7.12

DESCRIPTION:

ctr is an unsupported debug and administrative client for interacting
with the containerd daemon. Because it is unsupported, the commands,
options, and operations are not guaranteed to be backward compatible or
stable from release to release of the containerd project.

COMMANDS:
   plugins, plugin            Provides information about containerd plugins
   version                    Print the client and server versions
   containers, c, container   Manage containers
   content                    Manage content
   events, event              Display containerd events
   images, image, i           Manage images
   leases                     Manage leases
   namespaces, namespace, ns  Manage namespaces
   pprof                      Provide golang pprof outputs for containerd
   run                        Run a container
   snapshots, snapshot        Manage snapshots
   tasks, t, task             Manage tasks
   install                    Install a new package
   oci                        OCI tools
   sandboxes, sandbox, sb, s  Manage sandboxes
   info                       Print the server info
   deprecations
   shim                       Interact with a shim directly
   help, h                    Shows a list of commands or help for one command

GLOBAL OPTIONS:
   --debug                      Enable debug output in logs
   --address value, -a value    Address for containerd's GRPC server (default: "/run/containerd/containerd.sock") [$CONTAINERD_ADDRESS]
   --timeout value              Total timeout for ctr commands (default: 0s)
   --connect-timeout value      Timeout for connecting to containerd (default: 0s)
   --namespace value, -n value  Namespace to use with commands (default: "default") [$CONTAINERD_NAMESPACE]
   --help, -h                   show help
   --version, -v                print the version

注意：

ctr与docker较大的一个区别再用ctr有命名空间的概念，下面的命令执行时候如果没加-n参数，表示default命名空间。当containerd结合k8s使用时，相关镜像一般存放到k8s.io，相关操作需要加-n k8s.io，且将参数放到最前面。

子命令帮助文档，通过ctr+子命令查看帮助信息：

root@master1:~# ctr plugins
NAME:
   ctr plugins - Provides information about containerd plugins

USAGE:
   ctr plugins command [command options] [arguments...]

COMMANDS:
   list, ls  Lists containerd plugins

OPTIONS:
   --help, -h  show help

2. 镜像操作

帮助信息：

root@master1:~# ctr images
NAME:
   ctr images - Manage images

USAGE:
   ctr images command [command options] [arguments...]

COMMANDS:
   check                    Check existing images to ensure all content is available locally
   export                   Export images
   import                   Import images
   list, ls                 List images known to containerd
   mount                    Mount an image to a target path
   unmount                  Unmount the image from the target
   pull                     Pull an image from a remote
   push                     Push an image to a remote
   prune                    Remove unused images
   delete, del, remove, rm  Remove one or more images by reference
   tag                      Tag an image
   label                    Set and clear labels for an image
   convert                  Convert an image
   usage                    Display usage of snapshots for a given image ref

OPTIONS:
   --help, -h  show help

拉取镜像:

# 帮助信息
[root@k8s ~]# ctr images pull --help
NAME:
   ctr images pull - pull an image from a remote

USAGE:
   ctr images pull [command options] [flags] <ref>

DESCRIPTION:
   Fetch and prepare an image for use in containerd.

After pulling an image, it should be ready to use the same reference in a run
command. As part of this process, we do the following:

1. Fetch all resources into containerd.
2. Prepare the snapshot filesystem with the pulled resources.
3. Register metadata for the image.


OPTIONS:
   --skip-verify, -k                 skip SSL certificate validation
   --plain-http                      allow connections using plain HTTP
   --user value, -u value            user[:password] Registry user and password
   --refresh value                   refresh token for authorization server
   --hosts-dir value                 Custom hosts configuration directory
   --tlscacert value                 path to TLS root CA
   --tlscert value                   path to TLS client certificate
   --tlskey value                    path to TLS client key
   --http-dump                       dump all HTTP request/responses when interacting with container registry
   --http-trace                      enable HTTP tracing for registry interactions
   --snapshotter value               snapshotter name. Empty value stands for the default value. [$CONTAINERD_SNAPSHOTTER]
   --label value                     labels to attach to the image
   --platform value                  Pull content from a specific platform
   --all-platforms                   pull content and metadata from all platforms
   --all-metadata                    Pull metadata for all platforms
   --print-chainid                   Print the resulting image's chain ID
   --max-concurrent-downloads value  Set the max concurrent downloads for each pull (default: 0)


# 拉取nginx镜像，注意与docker拉取镜像不同，镜像名不能简写。
[root@k8s ~]# ctr i pull docker.io/library/nginx:latest
docker.io/library/nginx:latest:                                                   resolved       |++++++++++++++++++++++++++++++++++++++|
index-sha256:9c367186df9a6b18c6735357b8eb7f407347e84aea09beb184961cb83543d46e:    done           |++++++++++++++++++++++++++++++++++++++|
manifest-sha256:4e02e85a6f060a8406978fa53aafd2d828d0cedf5259275d191bab9afc33249e: done           |++++++++++++++++++++++++++++++++++++++|
layer-sha256:a06b6fd631e8e2091ce18db1a1b063f14f06a63d0513cafc51500ce7cb1ae2f4:    done           |++++++++++++++++++++++++++++++++++++++|
config-sha256:e0c9858e10ed8be697dc2809db78c57357ffc82de88c69a3dee5d148354679ef:   done           |++++++++++++++++++++++++++++++++++++++|
layer-sha256:2cc3ae149d28a36d28d4eefbae70aaa14a0c9eab588c3790f7979f310b893c44:    done           |++++++++++++++++++++++++++++++++++++++|
layer-sha256:1018f2b8dba8ffec0afc981d5bae673e38ecac6ca29a861d12cccbd820d53f8d:    done           |++++++++++++++++++++++++++++++++++++++|
layer-sha256:b831e78d8e20641c11c527baff6e2c7bae6a43cc1ab4316a5532885d6461b1e7:    done           |++++++++++++++++++++++++++++++++++++++|
layer-sha256:3ab22521e91957c19caeb12eadcad2823fdca853477acb38cd5e9a1ebe57e96e:    done           |++++++++++++++++++++++++++++++++++++++|
layer-sha256:5112bf42775bbb6a896ccd3ad63cbf864976fb1c047a4c56f316cf78d3edd756:    done           |++++++++++++++++++++++++++++++++++++++|
layer-sha256:cbdaf9e4ee2d8507bf2e162c560cfb0b37567db3870235b8940aeb157d628327:    done           |++++++++++++++++++++++++++++++++++++++|
elapsed: 17.1s                                                                    total:  67.7 M (4.0 MiB/s)                                 
unpacking linux/amd64 sha256:9c367186df9a6b18c6735357b8eb7f407347e84aea09beb184961cb83543d46e...
done: 1.164220475s

备注：

测试发现ctr拉取镜像如果要配置代理的话，走的是linux全局代理配置方式，而不是docker或者containerd单独的代理配置方式。参考方式：
export proxy=http://192.168.0.210:808
export all_proxy=$proxy
export http_proxy=$proxy
export https_proxy=$proxy
export no_proxy="localhost, 127.0.0.1, ::1"

打印镜像列表:

# ctr images ls 等同于 ctr -n default images ls;同时images可以简写为image或i
root@master1:~# ctr images ls
REF     TYPE          DIGEST                    SIZE      PLATFORMS             LABELS
docker.io/library/redis:6.0.5                                        application/vnd.docker.distribution.manifest.list.v2+json sha256:800f2587bf3376cb01e6307afe599ddce9439deafbd4fb8562829da96085c9c5 36.4 MiB  linux/386,linux/amd64,linux/arm/v5,linux/arm/v7,linux/arm64/v8,linux/mips64le,linux/ppc64le,linux/s390x -  -
# -q，只打印镜像名称
root@master1:~# ctr images ls -q
docker.io/library/redis:6.0.5

检测本地镜像:

# 检查本地镜像的完整性
root@master1:~# ctr images check
...
root@master1:~# ctr -n k8s.io images check
...

镜像重新打标签:

root@master1:~# ctr i tag docker.io/library/redis:6.0.5 docker.io/lldhsds/redis:6.0.5
docker.io/lldhsds/redis:6.0.5

root@master1:~# ctr i ls -q
docker.io/library/redis:6.0.5
docker.io/lldhsds/redis:6.0.5

删除镜像:

root@master1:~# ctr i rm docker.io/lldhsds/redis:6.0.5
docker.io/lldhsds/redis:6.0.5

将镜像导出为压缩包:

# 指定命名空间，指定--platform参数
root@worker1:~# ctr -n k8s.io image export --platform=linux/amd64 guestbook.tar docker.io/library/redis:6.0.5
root@worker1:~# ls
guestbook.tar  guestbook.tar.gz

从压缩包导入镜像:

# 导入到default命名空间下
root@master1:~# ctr image import --platform=linux/amd64 guestbook.tar
unpacking docker.io/library/redis:6.0.5 (sha256:800f2587bf3376cb01e6307afe599ddce9439deafbd4fb8562829da96085c9c5)...done
# 导入到k8s.io命名空间下
root@master1:~# ctr -n k8s.io image import --platform=linux/amd64 guestbook.tar
unpacking docker.io/library/redis:6.0.5 (sha256:800f2587bf3376cb01e6307afe599ddce9439deafbd4fb8562829da96085c9c5)...done

备注：

使用ctr导出、导入镜像时一般都需要加上--platform=linux/amd64参数(x86-64架构)，否则会出现类似问题ctr: content digest sha256:a630f3b108d2562d7a6747a0d3b5181b8f5a9aa489b8915fccd04de94cdc96bf: not found
如果是从docker侧打包的镜像，使用ctr打包，测试可以不加--platform=linux/amd64参数

将镜像挂载到主机目录:

root@worker1:~# ctr i mount docker.io/library/redis:6.0.5 /mnt
ctr: image "docker.io/library/redis:6.0.5": not found
root@worker1:~# ctr -n k8s.io i mount docker.io/library/redis:6.0.5 /mnt
sha256:852691351e76013456bccc5ca476ea5998cd4ef829ff88f36aa6152d26752a9f
/mnt
root@worker1:~# ls /mnt -alh
total 88K
drwxr-xr-x  1 root root            4.0K Jun 21 21:27 .
drwxr-xr-x 23 root root            4.0K Jun  1 01:04 ..
drwxr-xr-x  2 root root            4.0K Jun  7  2020 bin
drwxr-xr-x  2 root root            4.0K May  3  2020 boot
drwxr-xr-x  2  999 systemd-journal 4.0K Jun 10  2020 data
drwxr-xr-x  2 root root            4.0K Jun  7  2020 dev
drwxr-xr-x  1 root root            4.0K Jun 10  2020 etc
drwxr-xr-x  2 root root            4.0K May  3  2020 home
drwxr-xr-x  1 root root            4.0K Jun 10  2020 lib
drwxr-xr-x  2 root root            4.0K Jun  7  2020 lib64
drwxr-xr-x  2 root root            4.0K Jun  7  2020 media
drwxr-xr-x  2 root root            4.0K Jun  7  2020 mnt
drwxr-xr-x  2 root root            4.0K Jun  7  2020 opt
drwxr-xr-x  2 root root            4.0K May  3  2020 proc
drwx------  1 root root            4.0K Jun 10  2020 root
drwxr-xr-x  3 root root            4.0K Jun  7  2020 run
drwxr-xr-x  2 root root            4.0K Jun  7  2020 sbin
drwxr-xr-x  2 root root            4.0K Jun  7  2020 srv
drwxr-xr-x  2 root root            4.0K May  3  2020 sys
drwxrwxrwt  1 root root            4.0K Jun 10  2020 tmp
drwxr-xr-x  1 root root            4.0K Jun  7  2020 usr
drwxr-xr-x  1 root root            4.0K Jun  7  2020 var

将镜像从主机目录上卸载:

root@worker1:~# ctr i unmount /mnt
/mnt
root@worker1:~# ls /mnt

3. 容器操作

帮助信息：

root@worker1:~# ctr container
NAME:
   ctr containers - Manage containers

USAGE:
   ctr containers command [command options] [arguments...]

COMMANDS:
   create                   Create container
   delete, del, remove, rm  Delete one or more existing containers
   info                     Get info about a container
   list, ls                 List containers
   label                    Set and clear labels for a container
   checkpoint               Checkpoint a container
   restore                  Restore a container from checkpoint

OPTIONS:
   --help, -h  show help

创建容器：

# 创建一个nginx容器
root@master1:~# ctr c create docker.io/library/nginx:1.27.0 nginx
# 打印容器列表
root@master1:~# ctr c ls
CONTAINER    IMAGE                             RUNTIME
nginx        docker.io/library/nginx:1.27.0    io.containerd.runc.v2

查看容器详细信息：

root@master1:~# ctr c info nginx
{
    "ID": "nginx",
    "Labels": {
        "io.containerd.image.config.stop-signal": "SIGQUIT",
        "maintainer": "NGINX Docker Maintainers \u003cdocker-maint@nginx.com\u003e"
    },
    "Image": "docker.io/library/nginx:1.27.0",
    "Runtime": {
        "Name": "io.containerd.runc.v2",
        "Options": {
            "type_url": "containerd.runc.v1.Options"
        }
    },
    "SnapshotKey": "nginx",
    "Snapshotter": "overlayfs",
    "CreatedAt": "2024-06-22T07:08:10.924977879Z",
    "UpdatedAt": "2024-06-22T07:08:10.924977879Z",
    "Extensions": {},
...

删除容器：

root@master1:~# ctr c ls
CONTAINER    IMAGE                             RUNTIME
nginx        docker.io/library/nginx:1.27.0    io.containerd.runc.v2
root@master1:~# ctr c rm nginx
root@master1:~# ctr c ls
CONTAINER    IMAGE    RUNTIME

4. 任务

通过 container create 命令创建的容器，并没有处于运行状态，仅仅只是一个创建容器的声明。一个 container 对象包含了运行一个容器所需的资源及相关配置数据，表示 namespaces、rootfs 和容器的配置都已经初始化成功了，只是用户进程还没有启动。

启动并进入容器：

root@master1:~# ctr c create docker.io/library/nginx:1.27.0 nginx
root@master1:~# ctr c ls
CONTAINER    IMAGE                             RUNTIME
nginx        docker.io/library/nginx:1.27.0    io.containerd.runc.v2
# 启动一个容器
root@master1:~# ctr task start -d nginx
/docker-entrypoint.sh: /docker-entrypoint.d/ is not empty, will attempt to perform configuration
/docker-entrypoint.sh: Looking for shell scripts in /docker-entrypoint.d/
/docker-entrypoint.sh: Launching /docker-entrypoint.d/10-listen-on-ipv6-by-default.sh
# 查看task列表，即运行的容器列表
root@master1:~# ctr task ls
TASK     PID      STATUS
nginx    63588    RUNNING
# 进入容器里面，需要指定--exec-id参数，值可以随意。
root@master1:~# ctr task exec --exec-id 0 -t nginx sh
# ls
bin   dev                  docker-entrypoint.sh  home  lib64  mnt  proc  run   srv  tmp  var
boot  docker-entrypoint.d  etc                   lib   media  opt  root  sbin  sys  usr
#

暂停/恢复容器：

root@master1:~# ctr task ls
TASK     PID      STATUS
nginx    63588    RUNNING
# 暂停容器
root@master1:~# ctr task pause nginx
root@master1:~# ctr task ls
TASK     PID      STATUS
nginx    63588    PAUSED
# 恢复容器
root@master1:~# ctr t resume  nginx
root@master1:~# ctr task ls
TASK     PID      STATUS
nginx    63588    RUNNING

杀死容器，同停止容器：

root@master1:~# ctr t kill nginx
root@master1:~# ctr task ls
TASK     PID      STATUS
nginx    63588    STOPPED

删除task：

root@master1:~# ctr t ls
TASK     PID      STATUS
nginx    63588    STOPPED
root@master1:~# ctr t rm nginx

# 运行状态下删除task：
root@master1:~# ctr task ls
TASK     PID      STATUS
nginx    67604    RUNNING
root@master1:~# ctr t rm nginx
ERRO[0000] unable to delete nginx     error="task must be stopped before deletion: running: failed precondition"
ctr: task must be stopped before deletion: running: failed precondition

说明：

删除task之前必须停止或kill容器。

获取容器的 cgroup 相关信息：

root@master1:~# ctr t metrics nginx
ID       TIMESTAMP
nginx    seconds:1719041526  nanos:710831637

METRIC                VALUE
pids.current          5
pids.limit            18446744073709551615
cpu.usage_usec        24082
cpu.user_usec         6337
cpu.system_usec       17745
cpu.nr_periods        0
cpu.nr_throttled      0
cpu.throttled_usec    0
memory.usage          4849664
memory.usage_limit    18446744073709551615
memory.swap_usage     0
memory.swap_limit     18446744073709551615

查看容器中所有进程在宿主机中的 PID：

root@master1:~# ctr t ps nginx
PID      INFO
67604    -
67644    -
67645    -
67646    -
67647    -
# 与ps命令查到的进程ID匹配。
root@master1:~# ps -elf | grep nginx | grep -v grep
0 S root       67583       1  0  80   0 - 309523 futex_ 15:29 ?       00:00:00 /usr/bin/containerd-shim-runc-v2 -namespace default -id nginx -address /run/containerd/containerd.sock
4 S root       67604   67583  0  80   0 -  2851 sigsus 15:29 ?        00:00:00 nginx: master process nginx -g daemon off;
5 S message+   67644   67604  0  80   0 -  2975 ep_pol 15:29 ?        00:00:00 nginx: worker process
5 S message+   67645   67604  0  80   0 -  2975 ep_pol 15:29 ?        00:00:00 nginx: worker process
5 S message+   67646   67604  0  80   0 -  2975 ep_pol 15:29 ?        00:00:00 nginx: worker process
5 S message+   67647   67604  0  80   0 -  2975 ep_pol 15:29 ?        00:00:00 nginx: worker process

5. 命名空间

Containerd中是用命名空间进行资源区分的。

命名空间生命周期管理：

# 查看ns列表
root@master1:~# ctr ns ls
NAME    LABELS
default
k8s.io
# 创建命名空间
root@master1:~# ctr ns create test
root@master1:~# ctr ns ls
NAME    LABELS
default
k8s.io
test
# 删除命名空间
root@master1:~# ctr ns rm test
test

指定命名空间选项：

root@master1:~# ctr i ls -q
docker.io/library/nginx:1.27.0
docker.io/library/redis:6.0.5
us-docker.pkg.dev/google-samples/containers/gke/gb-frontend:v5
us-docker.pkg.dev/google-samples/containers/gke/gb-redis-follower:v2
root@master1:~# ctr -n k8s.io i ls -q
docker.io/flannel/flannel-cni-plugin:v1.4.1-flannel1
docker.io/flannel/flannel-cni-plugin@sha256:e88c0d84fa89679eb6cb6a28bc257d652ced8d1b2e44d54a592f0a2cd85dba53
docker.io/flannel/flannel:v0.25.4
docker.io/flannel/flannel@sha256:17415d91743e53fc4b852676a30a08915f131a2b6848d891ba5786eacd447076
docker.io/library/redis:6.0.5
quay.io/tigera/operator:v1.34.0
quay.io/tigera/operator@sha256:479ddc7ff9ab095058b96f6710bbf070abada86332e267d6e5dcc1df36ba2cc5
registry.aliyuncs.com/google_containers/coredns:v1.10.1

备注：

在使用ctr指定命名空间进行操作时，将-n xxx选项放到命令的前面。

6. ctr与docker

ctr 是 containerd 自带的工具，有 namespaces 的概念，对于上层编排系统的支持，ctr 客户端主要区分了 3 个命名空间分别是k8s.io、moby和default。

较新版本的Docker 底层默认调用的也是containerd。 Docker 使用 containerd 下面的命名空间默认 moby，而不是 default，所以假如我们有用 docker 启动容器，那么我们也可以通过 ctr -n moby 来定位下面的容器：

[root@k8s ~]# docker run -d --name nginx nginx:latest
b6f9e425cc80ef576a1878ac5f94727cdee6397b845d2f0101263cd0ab00996c

[root@k8s ~]# ctr -n moby c ls | grep b6f9e425cc
b6f9e425cc80ef576a1878ac5f94727cdee6397b845d2f0101263cd0ab00996c    -        io.containerd.runc.v2

遗留问题：

按照上面的说法类推，测试发现docker的镜像与ctr的镜像在moby命名空间下是对不上的，目前没有查到相关解释，作为遗留问题记录。测试版本：docker client:26.1.4，docker server:20.10.6，containerd.io 1.6.33。

7. ctr与k8s、ctrictl

Kubernetes 下使用的 containerd 默认命名空间是 k8s.io。所以在导入镜像时需要指定命令空间为 k8s.io，否则使用 crictl images 无法查询到。

#docker导出镜像
docker save -o kube-controllers.tar docker.io/calico/kube-controllers:v3.26.1

# ctr导出镜像(也支持多个镜像一起打包)
ctr -n k8s.io images export k8s-v1.28.2.tar registry.k8s.io/kube-apiserver:v1.28.2 registry.k8s.io/kube-controller-manager:v1.28.2 registry.k8s.io/kube-proxy:v1.28.2 registry.k8s.io/kube-scheduler:v1.28.2 registry.k8s.io/coredns/coredns:v1.10.1 registry.k8s.io/etcd:3.5.9-0
 
#ctr导入镜像
ctr -n k8s.io image import kube-controllers.tar

#查看镜像是否导入
crictl images

我们用crictl操作的均在k8s.io命名空间，使用ctr 看镜像列表就需要加上-n 参数。crictl 是只有一个k8s.io命名空间，但是没有-n 参数。

ctr images pull 拉取的镜像默认放在default，而 crictl pull 和 kubelet 默认拉取的镜像都在 k8s.io 命名空间下。所以通过ctr导入镜像的时候特别注意一点，最好指定命名空间。

# 注意-n不能放在命令最后面，下面几行查看的镜像是一样的
ctr -n=k8s.io image ls
ctr -n k8s.io image ls

# crictl 没有-n参数，操作都在k8s.io命名空间下。
crictl images

# crictl image list/ls 等同于 ctr -n=k8s.io image list/ls

使用ctr命令指定命名空间导入镜像
ctr -n=k8s.io image import dashboard.tar

#查看镜像，可以看到可以查询到了
crictl images

lldhsds

关注

15
点赞
踩
19

收藏

觉得还不错? 一键收藏
打赏
0
评论
containerd的客户端工具ctr命令

类似docker为docker-shim容器运行时的客户端工具，ctr 是 containerd 的客户端工具，安装containerd作为容器运行时后，会自动安装ctr。
复制链接

扫一扫