本文用流水账的形式记录如何使用nvdla的virtual platform编译生成NVDLA硬件可识别代码并且在QEMU中模拟运行。NVDLA硬件为full模式,并没有对硬件做任何的配置。
使用Docker运行VP
-
下载nvdla/vp中提到的docker,所有的编译工具都已经预编译并且放在/usr/local/nvdla/中
docker pull nvdla/vp
-
准备caffe模型
可以从以下链接下载编译好了的caffe模型,并从以下链接获取测试样例。此处需要注意,caffe模型是使用黑底白字训练的,而我们得到的测试样例是白底黑字,所以需要使用以下代码来做转换[1]:import sys import numpy from PIL import Image img = Image.open(sys.argv[1]).convert('L') im = numpy.array(img) im = 255 - im result = Image.fromarray(im) result.save(sys.argv[1].split('.')[0]+'_inv.'+sys.argv[1].split('.')[1])
-
编译.nvdla模型
在docker容器下运行以下命令,如果没有报错,会生成.nvdla文件。此处需要注意的是,设置好nvdla_compiler的路径,否则会报找不到nvdla_compiler的错误。./nvdla_compiler --caffemodel lenet_iter_10000.caffemodel --prototxt lenet.prototxt
-
在QEMU中运行runtime
4.1 使用以下命令进入QEMUcd /usr/local/nvdla # start the virtual simulator aarch64_toplevel -c aarch64_nvdla.lua # Login the kernel with account 'root' and password 'nvdla'
4.2 挂载当前文件夹并加载所需驱动
mount -t 9p -o trans=virtio r /mnt # mount pwd cd /mnt insmod drm.ko # install drm driver insmod opendla_1.ko # install nvdla driver
4.3 运行runtime,时间有点久,需要耐心等待
./nvdla_runtime --loadable fast-math.nvdla --image 8_inv.pgm --rawdump
4.4 查看运行结果,可以看到模型能够正确的识别出8(黑底白字)
cat output.dimg
使用源代码编译
NVDLA完全开源了sw的compiler和runtime部分,分别在/sw/umd/app和/sw/umd/core下面。从虚拟机进入/sw/umd目录底下。下载buildroot编译工具buildroot-2017.11-rc1[3],并解压缩到/sw文件夹。注意!使用绝对路径!
wget --no-check-certificate https://git.busybox.net/buildroot/snapshot/buildroot-2017.11-rc1.tar.bz2
tar -xf buildroot-2017.11-rc1.tar.bz2
# install necessary laibrary
apt-get install libncurses-dev
apt-get install cpio
# configuration
make qemu_aarch64_virt_defconfig
make menuconfig
* Target Options -> Target Architecture -> AArch64 (little endian)
* Target Options -> Target Architecture Variant -> cortex-A57
* Toolchain -> Custom kernel headers series -> 4.13.x
* Toolchain -> Toolchain type -> External toolchain
* Toolchain -> Toolchain -> Linaro AArch64 2017.08
* Toolchain -> Toolchain origin -> Toolchain to be downloaded and installed
* Kernel -> () Kernel version -> 4.13.3
* Kernel -> Kernel configuration -> Use the architecture default configuration
* System configuration -> Enable root login with password -> Y
* System configuration -> Root password -> nvdla
* Target Packages -> Show packages that are also provided by busybox -> Y
* Target Packages -> Networking applications -> openssh -> Y
# compile
make -j4
编译nvdla_compiler
注意!使用绝对路径!
export TOP=/vp/sw/umd
make compiler
make runtime TOOLCHAIN_PREFIX= (path_to_)buildroot-2017.11-rc1/output/host/opt/ext-toolchain/bin/aarch64-linux-gnu-
编译nvdla_runtime
编译KMD,注意!使用绝对路径!
cd sw/kmd
make KDIR=(path_to)buildroot-2017.11-rc1/output/build/linux-4.13.3/ \
ARCH=arm64 \
CROSS_COMPILE=(path_to_)buildroot-2017.11-rc1/output/host/bin/aarch64-linux-gnu-
将buildroot生成的Linux内核位置在aarch64_nvdla.lua中设置好,然后即可开始编译.nvdla模型并且在runtime中运行。但是!使用自己编译的compiler和runtime编译出来的模型,目前并没有在QEMU中被正确执行!
部分文件的位置:
sw/umd/out/core/src/compiler/libnvdla_compiler/libnvdla_compiler.so #umd生成的应用compiler和runtime程序以及动态库
sw/umd/out/core/src/runtime/libnvdla_runtime/libnvdla_runtime.so
sw/umd/out/apps/compiler/nvdla_compiler/nvdla_compiler
sw/umd/out/apps/runtime/nvdla_runtime/nvdla_runtime
sw/kmd/port/linux/opendla.ko # kmd生成的nvdla驱动
buildroot/output/build/linux-4.13.3/drivers/gpu/drm/drm.ko # vp环境对应的显卡gpu驱动
buildroot-2017.11-rc1/output/images # buildroot生成的Linux image
runtime pre-compiled | runtime from source | |
---|---|---|
compiler pre-compiled | Y | Y |
compiler from source | N | N |
参考资料:
[1] Run LeNet on NVDLA
[2] NVDLA Open Source Virtual Platform
[3] Building Linux Kernel for NVDLA Virtual Simulator
[4] nvdla整个build的flow
[5] NVDLA RunTime编译运行