V831上部署resnet18分类网络

V831上部署resnet18分类网络

前期准备

在V831上使用resnet18分类网络，我们需要在linux环境下进行。windows系统可以使用虚拟机，或者是使用WSL，具体的安装教程请自行百度，这里就不过多的进行描述

安装pytorch环境

我们需要在系统中安装pytorch，通过在pytorch官网上可以知道安装pytorch需要执行

pip3 install torch==1.9.0+cpu torchvision==0.10.0+cpu torchaudio==0.9.0 -f https://download.pytorch.org/whl/torch_stable.html

或者是通过conda环境进行安装

conda install pytorch torchvision torchaudio cpuonly -c pytorch

我们还需要安装一个torchsummary库来进行神经网络的可视化

pip3 install torchsummary

编译ncnn转换工具

通过 git clone https://github.com/Tencent/ncnn.git 将ncnn的仓库拉取到本地，进行编译

安装编译环境的依赖

sudo apt update
sudo apt install build-essential git cmake libprotobuf-dev protobuf-compiler libvulkan-dev vulkan-utils libopencv-dev

编译ncnn需要使用到 Vulkan 后端
要使用 Vulkan 后端，请安装 Vulkan 头文件、一个 vulkan 驱动程序加载器、GLSL 到 SPIR-V 编译器和 vulkaninfo 工具。或者从https://vulkan.lunarg.com/sdk/home下载并安装完整的 Vulkan SDK（大约 200MB；它包含所有头文件、文档和预构建的加载程序，以及一些额外的工具和所有源代码）

wget https://sdk.lunarg.com/sdk/download/1.2.182.0/linux/vulkansdk-linux-x86_64-1.2.182.0.tar.gz
tar xvf vulkansdk-linux-x86_64-1.2.182.0.tar.gz
export VULKAN_SDK=$(pwd)/1.2.182.0/x86_64

拉取ncnn的子仓库

cd ncnn
git submodule update --init

开始编译ncnn

mkdir -p build
cd build
cmake -DCMAKE_BUILD_TYPE=Release -DNCNN_VULKAN=ON -DNCNN_SYSTEM_GLSLANG=ON -DNCNN_BUILD_EXAMPLES=ON ..
make -j$(nproc)

编译结束之后会在build/tools/onnx/下的到onnx2ncnn可执行文件，这个是就用ncnn的转换工具

获取模型并进行推理

以下代码建议在jupyter中运行

通过pytorch hub来获取resnet18的预训练模型，这里并不细说训练的过程和模型定义

label下载
使用以下代码进行模型的下载和推理

import os
import torch
from torchsummary import summary
torch.hub._validate_not_a_forked_repo=lambda a,b,c: True
## model
model = torch.hub.load('pytorch/vision:v0.6.0', 'resnet18', pretrained=True)
model.eval()
input_shape = (3, 224, 224)
summary(model, input_shape, device="cpu")
## test image
filename = "out/dog.jpg"
if not os.path.exists(filename):
    if not os.path.exists("out"):
        os.makedirs("out")
    import urllib
    url, filename = ("https://github.com/pytorch/hub/raw/master/images/dog.jpg", filename)
    try: urllib.URLopener().retrieve(url, filename)
    except: urllib.request.urlretrieve(url, filename)
print("test image:", filename)
## preparing input data
from PIL import Image
import numpy as np
from torchvision import transforms
input_image = Image.open(filename)
# input_image.show()
preprocess = transforms.Compose([
    transforms.Resize(max(input_shape[1:3])),
    transforms.CenterCrop(input_shape[1:3]),
    transforms.ToTensor(),
    transforms.Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225]),
])
input_tensor = preprocess(input_image)
print("input data max value: {}, min value: {}".format(torch.max(input_tensor), torch.min(input_tensor)))
input_batch = input_tensor.unsqueeze(0) # create a mini-batch as expected by the model
## forward model
# move the input and model to GPU for speed if available
if torch.cuda.is_available():
    input_batch = input_batch.to('cuda')
    model.to('cuda')
with torch.no_grad():
    output = model(input_batch)
## result    
# Tensor of shape 1000, with confidence scores over Imagenet's 1000 classes
# print(output[0])
# The output has unnormalized scores. To get probabilities, you can run a softmax on it.
max_1000 = torch.nn.functional.softmax(output[0], dim=0)
max_idx = int(torch.argmax(max_1000))
with open("imagenet_classes.txt") as f:
    labels = f.read().split("\n")
print("result: idx:{}, name:{}".format(max_idx, labels[max_idx]))

wsl下是无法使用GPU需要将cuda：0修改成cpu

运行后得到结果:

Using cache found in /home/neucrack/.cache/torch/hub/pytorch_vision_v0.6.0
----------------------------------------------------------------
        Layer (type)               Output Shape         Param #
======