Docker安装
1.安装docker
https://docs.docker.com/engine/install/ubuntu/
sudo apt-get update
sudo apt-get install \
apt-transport-https \
ca-certificates \
curl \
gnupg \
lsb-release
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /usr/share/keyrings/docker-archive-keyring.gpg
echo \
"deb [arch=arm64 signed-by=/usr/share/keyrings/docker-archive-keyring.gpg] https://download.docker.com/linux/ubuntu \
$(lsb_release -cs) stable" | sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
sudo apt-get update
sudo apt-get install docker-ce docker-ce-cli containerd.io
2.安装nvidia-container-runtime
curl -s -L https://nvidia.github.io/nvidia-docker/gpgkey | sudo apt-key add -
distribution=$(. /etc/os-release;echo $ID$VERSION_ID)
echo $distribution
curl -s -L https://nvidia.github.io/nvidia-docker/$distribution/nvidia-docker.list | sudo tee /etc/apt/sources.list.d/nvidia-docker.list
wget https://nvidia.github.io/nvidia-container-runtime/ubuntu14.04/amd64/./nvidia-container-runtime-hook_1.4.0-1_amd64.deb
sudo apt-get -y update
sudo apt-get install -y nvidia-container-toolkit
sudo apt-get install -y nvidia-container-runtime
sudo systemctl restart docker
Docker化部署
一、运行时检查及配置
1、检查 docker 配置是否为可用的运行时:
docker info | grep Runtime
此处的结果不一致也没有关系,因为这里我已经配置好了,接着往下走
2、使用一个简单的运行时运行GPU容器,理想结果输出会是 Result=FAIL
docker run -it jitteam/devicequery ./deviceQuery
3、修改docker配置文件, 设置nvidia为默认运行时
vi /etc/docker/daemon.json
内容如下:
{
"default-runtime":"nvidia",
"runtimes": {
"nvidia": {
"path": "nvidia-container-runtime",
"runtimeArgs": []
}
}
}
4、重启docker
systemctl restart docker
5、再次检查运行时:
docker info | grep Runtime
6、接下来我们指定一下运行时再次测试:
docker run -it --runtime nvidia jitteam/devicequery ./deviceQuery
二、创建自己的容器并配置环境
1、创建文件夹docker-test
mkdir docker-test
cd docker-test
2、创建Dockerfile文件
vi Dockerfile
内容如下:
FROM arm64v8/ubuntu:16.04
#ENV LD_LIBRARY_PATH=/usr/lib/aarch64-linux-gnu
ENV LD_LIBRARY=/usr/lib/aarch64-linux-gnu/tegra
#ENV CUDA_HOME=/usr/local/cuda-10.0:/usr/local/cuda-10.0/include:/usr/local/cuda-10.0/lib
RUN mkdir /cudaSamples
COPY deviceQuery /cudaSamples/
ENV LANG C.UTF-8
#ENV LIB_PATH=/lib/aarch64-linux-gnu/
RUN apt-get update -y && apt-get upgrade -y
RUN apt-get install -y tzdata && ln -sf /usr/share/zoneinfo/Asia/Shanghai /etc/localtime
RUN apt-get -y install vim
#安装sqlite3
RUN apt-get install -y libsqlite3-dev
RUN apt-get -y install gcc
#解决ssl No module named _ssl
RUN apt-get install libssl-dev -y
RUN apt-get install make
RUN apt-get -y install zlib*
#安装wget
RUN apt-get -y install wget
#解决opencv 报错
#RUN apt-get install -y libgl1-mesa-glx
#RUN apt-get install -y libglib2.0-dev
#RUN wget https://www.python.org/ftp/python/3.6.5/Python-3.6.5.tgz
#解压tgz
#RUN tar -xvzf Python-3.6.5.tgz
#
#RUN cd Python-3.6.5
#RUN ./configure --with-ssl
#RUN make -4 && make install
#RUN ln -sf /usr/local/bin/python3 /usr/bin/python
#RUN python3 -m pip install --upgrade pip
3、 创建镜像
docker build -t nano_env_img .
4、运行容器
在运行之前,我们需要知道以下几个路径:
/dev/nvhost-ctrl
/dev/nvhost-ctrl-gpu
/dev/nvhost-prof-gpu
/dev/nvmap
/dev/nvhost-gpu
/dev/nvhost-as-gpu
这些路径都是宿主机上的一些路径,这是容器运行需要访问的路径,我们可以通过 --device 指令添加。
还有一条比较重要,容器也需要访问驱动程序,路径如下:
/usr/lib/aarch64-linux-gnu/tegra
可以通过 -v 指令添加。
如果程序还需要其他的访问路径,例如环境变量,cuda库的位置等,都可以通过以上指令添加。
接下来正式启动容器:
docker run --device=/dev/nvhost-ctrl --device=/dev/nvhost-ctrl-gpu --device=/dev/nvhost-prof-gpu --device=/dev/nvmap --device=/dev/nvhost-gpu --device=/dev/nvhost-as-gpu -v /usr/lib/aarch64-linux-gnu/tegra:/usr/lib/aarch64-linux-gnu/tegra -it --name nano_test nano_env_img
然后你就可以在容器中配置你自己的环境,配置完以后可以将该容器保存为镜像,然后移植到其它jetson 设备上。
参考资料
K3S and Nvidia Jetson Nano https://www.virtualthoughts.co.uk/2020/03/24/k3s-and-nvidia-jetson-nano/
使用文章中提到的jitteam/devicequery作为基像,程序可以运行了。
安装arm版cuda10.2:https://developer.nvidia.com/cuda-toolkit/arm
NVIDIA Container Runtime on Jetson:https://github.com/NVIDIA/nvidia-docker/wiki/NVIDIA-Container-Runtime-on-Jetson