NVIDIA NGC镜像使用笔记

最新推荐文章于 2024-08-15 15:02:58 发布

大健的日常

最新推荐文章于 2024-08-15 15:02:58 发布

阅读量6.1k

点赞数

分类专栏： docker 文章标签： NVIDIA NGC

本文链接：https://blog.csdn.net/womenrendeme/article/details/106873170

版权

docker 专栏收录该内容

5 篇文章 0 订阅

订阅专栏

NGC镜像列表官方web地址：

https://ngc.nvidia.com/catalog/all

NGC命令官方使用说明地址：

https://docs.nvidia.com/ngc/ngc-catalog-cli-user-guide/index.html

查看NGC镜像列表

# ngc registry image list
+--------------------------------------+--------------------------------------+---------------------------------+------------+--------------+------------+
| Name                                 | Repository                           | Latest Tag                      | Image Size | Updated Date | Permission |
+--------------------------------------+--------------------------------------+---------------------------------+------------+--------------+------------+
| CUDA                                 | nvidia/cuda                          | latest                          | 1.76 GB    | Jun 19, 2020 | unlocked   |
| Caffe2                               | nvidia/caffe2                        | 18.08-py3                       | 1.3 GB     | Oct 18, 2019 | unlocked   |
| NVCaffe                              | nvidia/caffe                         | 20.03-py3                       | 2.08 GB    | Mar 26, 2020 | unlocked   |
| Microsoft Cognitive Toolkit          | nvidia/cntk                          | 18.08-py3                       | 2.4 GB     | Oct 18, 2019 | unlocked   |
| MXNet                                | nvidia/mxnet                         | 20.03-py3                       | 2.56 GB    | Mar 26, 2020 | unlocked   |
| PyTorch                              | nvidia/pytorch                       | 20.03-py3                       | 3.38 GB    | Mar 26, 2020 | unlocked   |
| TensorFlow                           | nvidia/tensorflow                    | 20.03-tf2-py3                   | 2.78 GB    | Jun 19, 2020 | unlocked   |
| Theano                               | nvidia/theano                        | 18.08                           | 1.49 GB    | Oct 18, 2019 | unlocked   |
| Torch                                | nvidia/torch                         | 18.08-py2                       | 1.24 GB    | Oct 18, 2019 | unlocked   |
...

输入#号后命令后，返回的是NGC镜像列表。列表解释：

Title	Describe
Name	镜像名称
Repository	镜像路径/地址
Lasted Tag	默认镜像的镜像版本
Image Size	默认镜像的大小
Updated Date	默认镜像的更新时间
Permission	未知

查看某个镜像中所有镜像版本/tag

ngc registry image info [镜像路径/地址]

以镜像列表的caffe2为例，需要注意的是命令用到的是caffe2的镜像地址repository，非镜像名称name。

# ngc registry image info nvidia/caffe     
--------------------------------------------------
  Image Repository Information
    Name: NVCaffe
    Short Description: NVIDIA Caffe, also known as NVCaffe, is an NVIDIA-maintained fork of Berkeley Vision and Learning Center (BVLC) Caffe tuned for NVIDIA GPUs, particularly in multi-GPU configurations.
    Built By: NVIDIA
    Publisher: NVIDIA
    Logo: https://static.nvidiagrid.net/ngc/containers/nvcaffe.png
    Labels: Deep Learning, Training
    Public: Yes
    Last Updated: Mar 26, 2020
    Latest Image Size: 2.08 GB
    Latest Tag: 20.03-py3
    Tags: 
        20.03-py3
        20.02-py3
        20.01-py3
        19.12-py3
        19.11-py3
        19.10-py2
        19.09-py2
        19.08-py2
        19.07-py2
        19.06-py2
  ...

查看某一镜像tag的详细信息

我们是在使用NGC时不免会想得到一个镜像的详细信息，如镜像中使用到了什么版本的cuda，cudnn等，这时我们可以通过json格式输出镜像的详细信息，以此来选择合适的镜像。

ngc registry image --format_type json info [镜像路径]:[镜像tag]

以镜像caffe2中tag 19.11-py3为例，需要注意的是命令用到的是caffe2的镜像地址repository，非镜像仓库名称name。

# ngc registry image --format_type json info nvidia/caffe:19.11-py3    
{
    "architecture": "amd64",
    "fsLayers": [
        {
            "blobSum": "sha256:b591970b7b882ee9d190d3f8ea11a075782eb47aaee74df8553f84413281ee1a"
        },
...
...
    ],
    "history": [
        {
            "v1Compatibility": "{\"architecture\":\"amd64\",\"config\":{\"Hostname\":\"\",\"Domainname\":\"\",\"User\":\"\",\"AttachStdin\":false,\"AttachStdout\":false,\"AttachStderr\":false,\"Tty\":false,\"OpenStdin\":false,\"StdinOnce\":false,\"Env\":[\"PATH=/usr/local/mpi/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin\",\"CUDA_VERSION=10.2.89\",\"CUDA_DRIVER_VERSION=440.30.00\",\"CUDA_CACHE_DISABLE=1\",\"_CUDA_COMPAT_PATH=/usr/local/cuda/compat\",\"ENV=/etc/shinit_v2\",\"BASH_ENV=/etc/bash.bashrc\",\"NVIDIA_REQUIRE_CUDA=cuda\\u003e=9.0\",\"NCCL_VERSION=2.5.6\",\"CUBLAS_VERSION=10.2.2.89\",\"CUDNN_VERSION=7.6.5.32\",\"TRT_VERSION=6.0.1.8\",\"NSIGHT_SYSTEMS_VERSION=2019.5.2\",\"DALI_VERSION=0.15.0\",\"DALI_BUILD=947079\",\"LD_LIBRARY_PATH=/usr/local/cuda/compat/lib:/usr/local/nvidia/lib:/usr/local/nvidia/lib64\",\"NVIDIA_VISIBLE_DEVICES=all\",\"NVIDIA_DRIVER_CAPABILITIES=compute,utility,video\",\"MOFED_VERSION=4.4-1.0.0\",\"IBV_DRIVERS=/usr/lib/libibverbs/libmlx5\",\"OPENMPI_VERSION=3.1.4\",\"OMPI_MCA_btl_vader_single_copy_mechanism=none\",\"LIBRARY_PATH=/usr/local/cuda/lib64/stubs:\",\"CAFFE_VERSION=0.17.3\",\"NVIDIA_CAFFE_VERSION=19.11\",\"PYTHONPATH=:/usr/local/python\",\"NVIDIA_BUILD_ID=8776850\"],\"Cmd\":null,\"ArgsEscaped\":true,\"Image\":\"sha256:0a69bd6b813ec5dc5dd2b73b1db842d4e1050ff86dd3b478387c33ea4df844c2\",\"Volumes\":null,\"WorkingDir\":\"/workspace\",\"Entrypoint\":[\"/usr/local/bin/nvidia_entrypoint.sh\"],\"OnBuild\":null,\"Labels\":{\"com.nvidia.build.id\":\"8776850\",\"com.nvidia.build.ref\":\"6b1ea48d7c7e81888171e242f1e205e985ae73bb\",\"com.nvidia.caffe.version\":\"0.17.3\",\"com.nvidia.cublas.version\":\"10.2.2.89\",\"com.nvidia.cuda.version\":\"9.0\",\"com.nvidia.cudnn.version\":\"7.6.5.32\",\"com.nvidia.nccl.version\":\"2.5.6\",\"com.nvidia.nsightsystems.version\":\"2019.5.2\",\"com.nvidia.tensorrt.version\":\"6.0.1.8\",\"com.nvidia.volumes.needed\":\"nvidia_driver\"}},\"container\":\"4825a54e4a1fd7a0d15dee754baef65db7e6f8fba489efd75d40bc80dc74c116\",\"container_config\":{\"Hostname\":\"\",\"Domainname\":\"\",\"User\":\"\",\"AttachStdin\":false,\"AttachStdout\":false,\"AttachStderr\":false,\"Tty\":false,\"OpenStdin\":false,\"StdinOnce\":false,\"Env\":[\"PATH=/usr/local/mpi/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin\",\"CUDA_VERSION=10.2.89\",\"CUDA_DRIVER_VERSION=440.30.00\",\"CUDA_CACHE_DISABLE=1\",\"_CUDA_COMPAT_PATH=/usr/local/cuda/compat\",\"ENV=/etc/shinit_v2\",\"BASH_ENV=/etc/bash.bashrc\",\"NVIDIA_REQUIRE_CUDA=cuda\\u003e=9.0\",\"NCCL_VERSION=2.5.6\",\"CUBLAS_VERSION=10.2.2.89\",\"CUDNN_VERSION=7.6.5.32\",\"TRT_VERSION=6.0.1.8\",\"NSIGHT_SYSTEMS_VERSION=2019.5.2\",\"DALI_VERSION=0.15.0\",\"DALI_BUILD=947079\",\"LD_LIBRARY_PATH=/usr/local/cuda/compat/lib:/usr/local/nvidia/lib:/usr/local/nvidia/lib64\",\"NVIDIA_VISIBLE_DEVICES=all\",\"NVIDIA_DRIVER_CAPABILITIES=compute,utility,video\",\"MOFED_VERSION=4.4-1.0.0\",\"IBV_DRIVERS=/usr/lib/libibverbs/libmlx5\",\"OPENMPI_VERSION=3.1.4\",\"OMPI_MCA_btl_vader_single_copy_mechanism=none\",\"LIBRARY_PATH=/usr/local/cuda/lib64/stubs:\",\"CAFFE_VERSION=0.17.3\",\"NVIDIA_CAFFE_VERSION=19.11\",\"PYTHONPATH=:/usr/local/python\",\"NVIDIA_BUILD_ID=8776850\"],\"Cmd\":[\"|2\",\"NVIDIA_BUILD_REF=6b1ea48d7c7e81888171e242f1e205e985ae73bb\",\"PYVER=3.6\",\"/bin/sh\",\"-c\",\"rm -rf /opt/mellanox/DEBS/4.7-1.0.0/DEBS\"],\"ArgsEscaped\":true,\"Image\":\"sha256:0a69bd6b813ec5dc5dd2b73b1db842d4e1050ff86dd3b478387c33ea4df844c2\",\"Volumes\":null,\"WorkingDir\":\"/workspace\",\"Entrypoint\":null,\"OnBuild\":null,\"Labels\":{\"com.nvidia.build.id\":\"8776850\",\"com.nvidia.build.ref\":\"6b1ea48d7c7e81888171e242f1e205e985ae73bb\",\"com.nvidia.caffe.version\":\"0.17.3\",\"com.nvidia.cublas.version\":\"10.2.2.89\",\"com.nvidia.cuda.version\":\"9.0\",\"com.nvidia.cudnn.version\":\"7.6.5.32\",\"com.nvidia.nccl.version\":\"2.5.6\",\"com.nvidia.nsightsystems.version\":\"2019.5.2\",\"com.nvidia.tensorrt.version\":\"6.0.1.8\",\"com.nvidia.volumes.needed\":\"nvidia_driver\"}},\"created\":\"2019-11-13T01:25:47.958284932Z\",\"docker_version\":\"18.06.0-ce\",\"id\":\"5c4e301aec4260b05edb24eda582a5c59b498e9a2f796c33bce7ca9a4f56fffe\",\"os\":\"linux\",\"parent\":\"fda4f4fd76f910dc8f69f6079556cef18cd4bf470daf55a98cf2440f8fbe71b9\"}"
        },
...
...

每个镜像的详细信息不同，上例中我们可以从history中查看到镜像中所使用的环境信息。

下载NGC镜像

在镜像中找到合适的镜像tag后，通过此命令将镜像下载到本地

ngc registry image pull [镜像路径]:[镜像tag]

以镜像caffe2中tag 19.11-py3为例，需要注意的是命令用到的是caffe2的镜像地址repository，非镜像仓库名称name。

# ngc registry image pull nvidia/caffe:19.11-py3
...
5c5837a082b9: Waiting
6f25f220bc2b: Waiting
b591970b7b88: Waiting
7ddbc47eeb70: Downloading [====================>                              ]  10.86MB/26.69MB
c1bbdc448b72: Download complete
8c3b70e39044: Download complete
45d437916d57: Download complete
08675124ee65: Downloading [==============================================>    ]  8.354MB/9.054MB
258aa4bcaca7: Download complete
f2e8775ec207: Downloading [>                                                  ]  10.25MB/715.2MB
...

2021年8月17日10:51:59更新：

nvidia官方已经开放了ngc镜像下载，可以直接通过https://ngc.nvidia.com/catalog下载需要的镜像。