Jenson nano部署yolov5+Tensor Rt加速

已于 2023-05-15 16:06:29 修改
阅读量1.2k 收藏 11
点赞数 6
分类专栏： jenson nano  yolov5 tensorrt 文章标签： YOLO 人工智能
于 2023-05-04 17:41:51 首次发布
版权声明：本文为博主原创文章，遵循 CC 4.0 BY-SA 版权协议，转载请附上原文出处链接和本声明。
本文链接：https://blog.csdn.net/adgbh/article/details/130431428
版权

前言

在工作中接触到了jenson nano 部署，在网上找了很多资料也入了很多坑，经过不断的努力终于整出来啦，决定写下这篇文章

一、准备工具

1、读卡器

2、SD卡

3、螺丝刀

二、 烧录

1、下载烧录镜像，可以去官网下载，根据自己所需的版本，我下载的是4.4.1地址如下：

 JetPack SDK 4.4.1 archive | NVIDIA Developerr

​8bcc36c796774c2baa350b537c7c55d7.png​

 2、下载好后进行解压

8f492876d1914ea7a6eb405de04c8049.png

 3、下载SD卡格式化工具，网址如下

Get Started With Jetson Nano Developer Kit | NVIDIA Developer

4decab1a9f5f48ddb4ee450c7f018825.png

 同时下载烧录工具

a2e9e2ca41dd413a8378dfd357125a0b.png

 4、开始格式化SD卡

310f900d39c647dfa53a827596bb5e1c.png

 5、 开始烧录镜像 （大概二十分钟左右）

0972802afc084c519428102d569d73bd.png

6、烧录完成后，插卡开机

f3d4f24287d54cf48d3136e0c5fb2ac4.png

7、开机后，简单设置一下（密码，时区，语言，输入法等）就会这样的页面

d8073ae9f79d4e248f1a02070154a513.jpeg

三、搭建环境

1、配置cuda

打开终端（ctrl+alt+t）

sudo gedit ~/.bashrc

打开文档在最后加入以下代码，ctrl+S保存后关闭文档

export CUDA_HOME=/usr/local/cuda-10.2
export LD_LIBRARY_PATH=/usr/local/cuda-10.2/lib64:$LD_LIBRARY_PATH
export PATH=/usr/local/cuda-10.2/bin:$PATH

验证是否配置成功

nvcc -V

2、配置conda

因为它得架构与windows和liunx不同，所以这里我们要安装archiconda

在终端输入下载命令

wget https://github.com/Archiconda/build-tools/releases/download/0.2.3/Archiconda3-0.2.3-Linux-aarch64.sh

下载好后执行以下命令

bash Archiconda3-0.2.3-Linux-aarch64.sh

然后就是傻瓜式安装（过程与anaconda一样）

配置环境变量

sudo gedit ~/.bashrc

在文档最后加入以下命令，保存退出

export PATH=~/archiconda3/bin:$PATH

查看版本号

conda -V

3、创建虚拟环境(我这里创建的是以nano命名的，你可以改成自己喜欢的名字)

conda create -n nano python=3.6

这里可以进入一下刚创建的虚拟环境（可以省略）

conda activate nano

退出虚拟环境

conda deactivate

4、换源

先备份source.list文件

sudo cp /etc/apt/sources.list /etc/apt/sources.list.bak

进入source.list文件

sudo gedit /etc/apt/sources.list

按 i 开始输入，然后ctrl+a删除内容，之后将以下内容复制进去，Esc 结束输入，然后保存退出

deb http://mirrors.tuna.tsinghua.edu.cn/ubuntu-ports/ bionic main multiverse restricted universe
deb http://mirrors.tuna.tsinghua.edu.cn/ubuntu-ports/ bionic-security main multiverse restricted universe
deb http://mirrors.tuna.tsinghua.edu.cn/ubuntu-ports/ bionic-updates main multiverse restricted universe
deb http://mirrors.tuna.tsinghua.edu.cn/ubuntu-ports/ bionic-backports main multiverse restricted universe
deb-src http://mirrors.tuna.tsinghua.edu.cn/ubuntu-ports/ bionic main multiverse restricted universe
deb-src http://mirrors.tuna.tsinghua.edu.cn/ubuntu-ports/ bionic-security main multiverse restricted universe
deb-src http://mirrors.tuna.tsinghua.edu.cn/ubuntu-ports/ bionic-updates main multiverse restricted universe
deb-src http://mirrors.tuna.tsinghua.edu.cn/ubuntu-ports/ bionic-backports main multiverse restricted universe

更新软件列表，保存到本地

sudo apt-get update

更新软件

sudo apt-get upgrade

有可能会出现如下错误

4aad5e3397d546c0b04061836768579e.jpeg

在dpkg文件下打开终端用如下命令可以解决

sudo mkdir info

5、安装pip

升级所有已安装的软件包，并解决依赖关系

sudo apt-get dist-upgrade

之后就是安装pip

sudo apt-get install python3-pip libopenblas-base libopenmpi-dev

更新pip到最新版本

pip3 install --upgrade pip

6、下载torch和torchvision

去英伟达官网下载所需版本，我这里是1.8.0，网址如下：

​​​​​​PyTorch for Jetson - version 1.10 now available - Jetson Nano - NVIDIA Developer Forums04a0c340906c4e3da358e387ce4af33b.png 

如果官网进不去的话可以在网盘里直接下载

链接: https://pan.baidu.com/s/1DpKXfMRcLV4TSJpvH7iCmA?pwd=2yxf

提取码: 2yxf 

下载好后，这里推荐一个工具（MobaXterm）用于连接板子进行互传，当然你也可以直接下载到U盘，用U盘实现互传

具体操作可以看这个博主的博客：MobaXterm（终端工具）下载&安装&使用教程_蜗牛也不慢......的博客-CSDN博客 

连接成功后把网盘里的东西解压后，传过去，建议传到home目录下

7、安装torch

在文件目录下打开终端，然后进入你创建的虚拟环境，输入以下命令

sudo apt-get install python3-pip libopenblas-base libopenmpi-dev 
pip install torch-1.8.0-cp36-cp36m-linux_aarch64.whl

这时需要安装numpy，不然测试torch时不显示安装成功

sudo apt install python3-numpy

测试torch，是否安装成功

import torch
print(torch.__version__)

这是会报错，继续执行以下命令

export OPENBLAS_CORETYPE=ARMV8

8、安装torchvision，逐个执行以下命令（如果执行第三个命令报错，继续执行第四第五行命令，如果不报错就直接cd ..）

cd torchvision
 
export BUILD_VERSION=0.9.0
 
sudo python setup.py install 

python setup.py build

python setup.py install
 
cd ..

如果执行完第五个命令有这个错，可以不用管，接着往下走

2a84e57d100347178a6e71cc5f6eeafa.png

测试torchvision

python

import torchvision

print(torchvision.__version__)

在测试时如果报错PIL，就安装pillow，命令如下（在第二步如果报权限错误，就在开头加sudo，或在结尾加--user）

sudo apt-get install libjpeg8 libjpeg62-dev libfreetype6 libfreetype6-dev

python3 -m pip install -i https://mirrors.aliyun.com/pypi/simple pillow

到次环境就搭建好啦，下面开始部署yolov5

四、试跑Yolov5

1、去官网上下载需要的版本，我这里下载的是5.0版本，下载的时候要把对应的权重也要下载下来网址如下：

ultralytics/yolov5 at v5.0 (github.com)

这里可以选择版本

ee72613a365240c29babd8f280121ea6.png

权重网址如下：

Releases · ultralytics/yolov5 (github.com)

e53f9fdb9eaf43e286b3b6be0a82f98a.png

2、通过我们之前介绍的工具将v5和模型拉到home目录下，然后打开终端执行以下命令

pip install -r requirements.txt -i https://pypi.tuna.tsinghua.edu.cn/simple

这时应该会报错，可以查看你得numpy版本，将版本降到1.19.4就OK了，命令如下

pip install numpy==1.19.4 -i https://pypi.tuna.tsinghua.edu.cn/simple

解决完报错后，会进行安装相关依赖包，当安装opencv时会出现Building wheel for opencv-python (pyroject.toml)...  这种情况等就可以啦

3、安装好所有依赖包后，执行以下命令

python3 detect.py --weights yolov5s.pt

e0032511a88c467da90dd69fc268a4be.png

 如果遇到这个错误，在common.py中再添加一个类就可以

import warnings

class SPPF(nn.Module):
    # Spatial Pyramid Pooling - Fast (SPPF) layer for YOLOv5 by Glenn Jocher
    def __init__(self, c1, c2, k=5):  # equivalent to SPP(k=(5, 9, 13))
        super().__init__()
        c_ = c1 // 2  # hidden channels
        self.cv1 = Conv(c1, c_, 1, 1)
        self.cv2 = Conv(c_ * 4, c2, 1, 1)
        self.m = nn.MaxPool2d(kernel_size=k, stride=1, padding=k // 2)

    def forward(self, x):
        x = self.cv1(x)
        with warnings.catch_warnings():
            warnings.simplefilter('ignore')  # suppress torch 1.9.0 max_pool2d() warning
            y1 = self.m(x)
            y2 = self.m(y1)
            return self.cv2(torch.cat([x, y1, y2, self.m(y2)], 1))

下面我们开始部署yolov5

五、tensorrt部署yolov5

1、tensorrt官网下载v5，网址如下，确定下载是5.0版本

https://gitcode.net/mirrors/wang-xinyu/tensorrtx?utm_source=csdn_github_accelerator

b0d18cd91d8c4d41960a4ad45985763d.png

2、通过之前的工具传到home目录下

传完之后在文件夹中找到gen_wts.py,复制到我们刚才试跑的v5文件夹中，在文件中打开终端，运行如下命令生成.wts文件

python3 gen_wts.py --w yolov5s.pt

把生成的.wts文件复制到tensorrtx-yolov5-v5.0\yolov5中

3、然后找到yololayer.h文件，打开之后可以修改类别数量及图片大小（根据自己的需求而定）

da6b8eb268b04a14856c7568cfe2e264.png

 4、在当前目录下创建文件build，命令如下

mkdir build #创建build文件夹

cd build #进入build

cmake .. #构建项目

#将我们上面生成的.wts文件复制到build文件夹中

make

5、在当前终端运行如下命令，目地：将yolov5s.wts文件转为.engine(引擎文件)

sudo ./yolov5 -s yolov5s.wts yolov5s.engine s

6、在tensorrtx-yolov5-v5.0\yolov5下新建sample文件夹，在里面放一张需要测试的图片进行测试，命令如下

sudo ./yolov5 -d yolov5s.engine ../sample

但是看不出效果，这个我们交付时可用用的都是摄像头，下面我们开始调用摄像头

7、调用摄像头

修改yolov5.cpp文件,将里面内容替换成如下代码

#include <iostream>
#include <chrono>
#include "cuda_utils.h"
#include "logging.h"
#include "common.hpp"
#include "utils.h"
#include "calibrator.h"
 
#define USE_FP16  // set USE_INT8 or USE_FP16 or USE_FP32
#define DEVICE 0  // GPU id
#define NMS_THRESH 0.4
#define CONF_THRESH 0.5
#define BATCH_SIZE 1
 
// stuff we know about the network and the input/output blobs
static const int INPUT_H = Yolo::INPUT_H;
static const int INPUT_W = Yolo::INPUT_W;
static const int CLASS_NUM = Yolo::CLASS_NUM;
static const int OUTPUT_SIZE = Yolo::MAX_OUTPUT_BBOX_COUNT * sizeof(Yolo::Detection) / sizeof(float) + 1;  // we assume the yololayer outputs no more than MAX_OUTPUT_BBOX_COUNT boxes that conf >= 0.1
const char* INPUT_BLOB_NAME = "data";
const char* OUTPUT_BLOB_NAME = "prob";
static Logger gLogger;
 
//修改为自己的类别
char *my_classes[]={ "person", "bicycle", "car", "motorcycle", "airplane", "bus", "train", "truck", "boat", "traffic light",
         "fire hydrant", "stop sign", "parking meter", "bench", "bird", "cat", "dog", "horse", "sheep", "cow",
         "elephant", "bear", "zebra", "giraffe", "backpack", "umbrella", "handbag", "tie", "suitcase", "frisbee",
         "skis", "snowboard", "sports ball", "kite", "baseball bat", "baseball glove", "skateboard","surfboard",
         "tennis racket", "bottle", "wine glass", "cup", "fork", "knife", "spoon", "bowl", "banana", "apple",
         "sandwich", "orange", "broccoli", "carrot", "hot dog", "pizza", "donut", "cake", "chair", "couch",
         "potted plant", "bed", "dining table", "toilet", "tv", "laptop", "mouse", "remote", "keyboard", "cell phone",
         "microwave", "oven", "toaster", "sink", "refrigerator", "book", "clock", "vase", "scissors", "teddy bear",
         "hair drier", "toothbrush" };
 
static int get_width(int x, float gw, int divisor = 8) {
    //return math.ceil(x / divisor) * divisor
    if (int(x * gw) % divisor == 0) {
        return int(x * gw);
    }
    return (int(x * gw / divisor) + 1) * divisor;
}
 
static int get_depth(int x, float gd) {
    if (x == 1) {
        return 1;
    }
    else {
        return round(x * gd) > 1 ? round(x * gd) : 1;
    }
}
 //#创建engine和network
ICudaEngine* build_engine(unsigned int maxBatchSize, IBuilder* builder, IBuilderConfig* config, DataType dt, float& gd, float& gw, std::string& wts_name) {
    INetworkDefinition* network = builder->createNetworkV2(0U);
 
    // Create input tensor of shape {3, INPUT_H, INPUT_W} with name INPUT_BLOB_NAME
    ITensor* data = network->addInput(INPUT_BLOB_NAME, dt, Dims3{ 3, INPUT_H, INPUT_W });
    assert(data);
 
    std::map<std::string, Weights> weightMap = loadWeights(wts_name);
 
    /* ------ yolov5 backbone------ */
    auto focus0 = focus(network, weightMap, *data, 3, get_width(64, gw), 3, "model.0");
    auto conv1 = convBlock(network, weightMap, *focus0->getOutput(0), get_width(128, gw), 3, 2, 1, "model.1");
    auto bottleneck_CSP2 = C3(network, weightMap, *conv1->getOutput(0), get_width(128, gw), get_width(128, gw), get_depth(3, gd), true, 1, 0.5, "model.2");
    auto conv3 = convBlock(network, weightMap, *bottleneck_CSP2->getOutput(0), get_width(256, gw), 3, 2, 1, "model.3");
    auto bottleneck_csp4 = C3(network, weightMap, *conv3->getOutput(0), get_width(256, gw), get_width(256, gw), get_depth(9, gd), true, 1, 0.5, "model.4");
    auto conv5 = convBlock(network, weightMap, *bottleneck_csp4->getOutput(0), get_width(512, gw), 3, 2, 1, "model.5");
    auto bottleneck_csp6 = C3(network, weightMap, *conv5->getOutput(0), get_width(512, gw), get_width(512, gw), get_depth(9, gd), true, 1, 0.5, "model.6");
    auto conv7 = convBlock(network, weightMap, *bottleneck_csp6->getOutput(0), get_width(1024, gw), 3, 2, 1, "model.7");
    auto spp8 = SPP(network, weightMap, *conv7->getOutput(0), get_width(1024, gw), get_width(1024, gw), 5, 9, 13, "model.8");
 
    /* ------ yolov5 head ------ */
    auto bottleneck_csp9 = C3(network, weightMap, *spp8->getOutput(0), get_width(1024, gw), get_width(1024, gw), get_depth(3, gd), false, 1, 0.5, "model.9");
    auto conv10 = convBlock(network, weightMap, *bottleneck_csp9->getOutput(0), get_width(512, gw), 1, 1, 1, "model.10");
 
    auto upsample11 = network->addResize(*conv10->getOutput(0));
    assert(upsample11);
    upsample11->setResizeMode(ResizeMode::kNEAREST);
    upsample11->setOutputDimensions(bottleneck_csp6->getOutput(0)->getDimensions());
 
    ITensor* inputTensors12[] = { upsample11->getOutput(0), bottleneck_csp6->getOutput(0) };
    auto cat12 = network->addConcatenation(inputTensors12, 2);
    auto bottleneck_csp13 = C3(network, weightMap, *cat12->getOutput(0), get_width(1024, gw), get_width(512, gw), get_depth(3, gd), false, 1, 0.5, "model.13");
    auto conv14 = convBlock(network, weightMap, *bottleneck_csp13->getOutput(0), get_width(256, gw), 1, 1, 1, "model.14");
 
    auto upsample15 = network->addResize(*conv14->getOutput(0));
    assert(upsample15);
    upsample15->setResizeMode(ResizeMode::kNEAREST);
    upsample15->setOutputDimensions(bottleneck_csp4->getOutput(0)->getDimensions());
 
    ITensor* inputTensors16[] = { upsample15->getOutput(0), bottleneck_csp4->getOutput(0) };
    auto cat16 = network->addConcatenation(inputTensors16, 2);
 
    auto bottleneck_csp17 = C3(network, weightMap, *cat16->getOutput(0), get_width(512, gw), get_width(256, gw), get_depth(3, gd), false, 1, 0.5, "model.17");
 
    // yolo layer 0
    IConvolutionLayer* det0 = network->addConvolutionNd(*bottleneck_csp17->getOutput(0), 3 * (Yolo::CLASS_NUM + 5), DimsHW{ 1, 1 }, weightMap["model.24.m.0.weight"], weightMap["model.24.m.0.bias"]);
    auto conv18 = convBlock(network, weightMap, *bottleneck_csp17->getOutput(0), get_width(256, gw), 3, 2, 1, "model.18");
    ITensor* inputTensors19[] = { conv18->getOutput(0), conv14->getOutput(0) };
    auto cat19 = network->addConcatenation(inputTensors19, 2);
    auto bottleneck_csp20 = C3(network, weightMap, *cat19->getOutput(0), get_width(512, gw), get_width(512, gw), get_depth(3, gd), false, 1, 0.5, "model.20");
    //yolo layer 1
    IConvolutionLayer* det1 = network->addConvolutionNd(*bottleneck_csp20->getOutput(0), 3 * (Yolo::CLASS_NUM + 5), DimsHW{ 1, 1 }, weightMap["model.24.m.1.weight"], weightMap["model.24.m.1.bias"]);
    auto conv21 = convBlock(network, weightMap, *bottleneck_csp20->getOutput(0), get_width(512, gw), 3, 2, 1, "model.21");
    ITensor* inputTensors22[] = { conv21->getOutput(0), conv10->getOutput(0) };
    auto cat22 = network->addConcatenation(inputTensors22, 2);
    auto bottleneck_csp23 = C3(network, weightMap, *cat22->getOutput(0), get_width(1024, gw), get_width(1024, gw), get_depth(3, gd), false, 1, 0.5, "model.23");
    IConvolutionLayer* det2 = network->addConvolutionNd(*bottleneck_csp23->getOutput(0), 3 * (Yolo::CLASS_NUM + 5), DimsHW{ 1, 1 }, weightMap["model.24.m.2.weight"], weightMap["model.24.m.2.bias"]);
 
    auto yolo = addYoLoLayer(network, weightMap, "model.24", std::vector<IConvolutionLayer*>{det0, det1, det2});
    yolo->getOutput(0)->setName(OUTPUT_BLOB_NAME);
    network->markOutput(*yolo->getOutput(0));
 
    // Build engine
    builder->setMaxBatchSize(maxBatchSize);
    config->setMaxWorkspaceSize(16 * (1 << 20));  // 16MB
#if defined(USE_FP16)
    config->setFlag(BuilderFlag::kFP16);
#elif defined(USE_INT8)
    std::cout << "Your platform support int8: " << (builder->platformHasFastInt8() ? "true" : "false") << std::endl;
    assert(builder->platformHasFastInt8());
    config->setFlag(BuilderFlag::kINT8);
    Int8EntropyCalibrator2* calibrator = new Int8EntropyCalibrator2(1, INPUT_W, INPUT_H, "./coco_calib/", "int8calib.table", INPUT_BLOB_NAME);
    config->setInt8Calibrator(calibrator);
#endif
 
    std::cout << "Building engine, please wait for a while..." << std::endl;
    ICudaEngine* engine = builder->buildEngineWithConfig(*network, *config);
    std::cout << "Build engine successfully!" << std::endl;
 
    // Don't need the network any more
    network->destroy();
 
    // Release host memory
    for (auto& mem : weightMap)
    {
        free((void*)(mem.second.values));
    }
 
    return engine;
}
 
ICudaEngine* build_engine_p6(unsigned int maxBatchSize, IBuilder* builder, IBuilderConfig* config, DataType dt, float& gd, float& gw, std::string& wts_name) {
    INetworkDefinition* network = builder->createNetworkV2(0U);
 
    // Create input tensor of shape {3, INPUT_H, INPUT_W} with name INPUT_BLOB_NAME
    ITensor* data = network->addInput(INPUT_BLOB_NAME, dt, Dims3{ 3, INPUT_H, INPUT_W });
    assert(data);
 
    std::map<std::string, Weights> weightMap = loadWeights(wts_name);
 
    /* ------ yolov5 backbone------ */
    auto focus0 = focus(network, weightMap, *data, 3, get_width(64, gw), 3, "model.0");
    auto conv1 = convBlock(network, weightMap, *focus0->getOutput(0), get_width(128, gw), 3, 2, 1, "model.1");
    auto c3_2 = C3(network, weightMap, *conv1->getOutput(0), get_width(128, gw), get_width(128, gw), get_depth(3, gd), true, 1, 0.5, "model.2");
    auto conv3 = convBlock(network, weightMap, *c3_2->getOutput(0), get_width(256, gw), 3, 2, 1, "model.3");
    auto c3_4 = C3(network, weightMap, *conv3->getOutput(0), get_width(256, gw), get_width(256, gw), get_depth(9, gd), true, 1, 0.5, "model.4");
    auto conv5 = convBlock(network, weightMap, *c3_4->getOutput(0), get_width(512, gw), 3, 2, 1, "model.5");
    auto c3_6 = C3(network, weightMap, *conv5->getOutput(0), get_width(512, gw), get_width(512, gw), get_depth(9, gd), true, 1, 0.5, "model.6");
    auto conv7 = convBlock(network, weightMap, *c3_6->getOutput(0), get_width(768, gw), 3, 2, 1, "model.7");
    auto c3_8 = C3(network, weightMap, *conv7->getOutput(0), get_width(768, gw), get_width(768, gw), get_depth(3, gd), true, 1, 0.5, "model.8");
    auto conv9 = convBlock(network, weightMap, *c3_8->getOutput(0), get_width(1024, gw), 3, 2, 1, "model.9");
    auto spp10 = SPP(network, weightMap, *conv9->getOutput(0), get_width(1024, gw), get_width(1024, gw), 3, 5, 7, "model.10");
    auto c3_11 = C3(network, weightMap, *spp10->getOutput(0), get_width(1024, gw), get_width(1024, gw), get_depth(3, gd), false, 1, 0.5, "model.11");
 
    /* ------ yolov5 head ------ */
    auto conv12 = convBlock(network, weightMap, *c3_11->getOutput(0), get_width(768, gw), 1, 1, 1, "model.12");
    auto upsample13 = network->addResize(*conv12->getOutput(0));
    assert(upsample13);
    upsample13->setResizeMode(ResizeMode::kNEAREST);
    upsample13->setOutputDimensions(c3_8->getOutput(0)->getDimensions());
    ITensor* inputTensors14[] = { upsample13->getOutput(0), c3_8->getOutput(0) };
    auto cat14 = network->addConcatenation(inputTensors14, 2);
    auto c3_15 = C3(network, weightMap, *cat14->getOutput(0), get_width(1536, gw), get_width(768, gw), get_depth(3, gd), false, 1, 0.5, "model.15");
 
    auto conv16 = convBlock(network, weightMap, *c3_15->getOutput(0), get_width(512, gw), 1, 1, 1, "model.16");
    auto upsample17 = network->addResize(*conv16->getOutput(0));
    assert(upsample17);
    upsample17->setResizeMode(ResizeMode::kNEAREST);
    upsample17->setOutputDimensions(c3_6->getOutput(0)->getDimensions());
    ITensor* inputTensors18[] = { upsample17->getOutput(0), c3_6->getOutput(0) };
    auto cat18 = network->addConcatenation(inputTensors18, 2);
    auto c3_19 = C3(network, weightMap, *cat18->getOutput(0), get_width(1024, gw), get_width(512, gw), get_depth(3, gd), false, 1, 0.5, "model.19");
 
    auto conv20 = convBlock(network, weightMap, *c3_19->getOutput(0), get_width(256, gw), 1, 1, 1, "model.20");
    auto upsample21 = network->addResize(*conv20->getOutput(0));
    assert(upsample21);
    upsample21->setResizeMode(ResizeMode::kNEAREST);
    upsample21->setOutputDimensions(c3_4->getOutput(0)->getDimensions());
    ITensor* inputTensors21[] = { upsample21->getOutput(0), c3_4->getOutput(0) };
    auto cat22 = network->addConcatenation(inputTensors21, 2);
    auto c3_23 = C3(network, weightMap, *cat22->getOutput(0), get_width(512, gw), get_width(256, gw), get_depth(3, gd), false, 1, 0.5, "model.23");
 
    auto conv24 = convBlock(network, weightMap, *c3_23->getOutput(0), get_width(256, gw), 3, 2, 1, "model.24");
    ITensor* inputTensors25[] = { conv24->getOutput(0), conv20->getOutput(0) };
    auto cat25 = network->addConcatenation(inputTensors25, 2);
    auto c3_26 = C3(network, weightMap, *cat25->getOutput(0), get_width(1024, gw), get_width(512, gw), get_depth(3, gd), false, 1, 0.5, "model.26");
 
    auto conv27 = convBlock(network, weightMap, *c3_26->getOutput(0), get_width(512, gw), 3, 2, 1, "model.27");
    ITensor* inputTensors28[] = { conv27->getOutput(0), conv16->getOutput(0) };
    auto cat28 = network->addConcatenation(inputTensors28, 2);
    auto c3_29 = C3(network, weightMap, *cat28->getOutput(0), get_width(1536, gw), get_width(768, gw), get_depth(3, gd), false, 1, 0.5, "model.29");
 
    auto conv30 = convBlock(network, weightMap, *c3_29->getOutput(0), get_width(768, gw), 3, 2, 1, "model.30");
    ITensor* inputTensors31[] = { conv30->getOutput(0), conv12->getOutput(0) };
    auto cat31 = network->addConcatenation(inputTensors31, 2);
    auto c3_32 = C3(network, weightMap, *cat31->getOutput(0), get_width(2048, gw), get_width(1024, gw), get_depth(3, gd), false, 1, 0.5, "model.32");
 
    /* ------ detect ------ */
    IConvolutionLayer* det0 = network->addConvolutionNd(*c3_23->getOutput(0), 3 * (Yolo::CLASS_NUM + 5), DimsHW{ 1, 1 }, weightMap["model.33.m.0.weight"], weightMap["model.33.m.0.bias"]);
    IConvolutionLayer* det1 = network->addConvolutionNd(*c3_26->getOutput(0), 3 * (Yolo::CLASS_NUM + 5), DimsHW{ 1, 1 }, weightMap["model.33.m.1.weight"], weightMap["model.33.m.1.bias"]);
    IConvolutionLayer* det2 = network->addConvolutionNd(*c3_29->getOutput(0), 3 * (Yolo::CLASS_NUM + 5), DimsHW{ 1, 1 }, weightMap["model.33.m.2.weight"], weightMap["model.33.m.2.bias"]);
    IConvolutionLayer* det3 = network->addConvolutionNd(*c3_32->getOutput(0), 3 * (Yolo::CLASS_NUM + 5), DimsHW{ 1, 1 }, weightMap["model.33.m.3.weight"], weightMap["model.33.m.3.bias"]);
 
    auto yolo = addYoLoLayer(network, weightMap, "model.33", std::vector<IConvolutionLayer*>{det0, det1, det2, det3});
    yolo->getOutput(0)->setName(OUTPUT_BLOB_NAME);
    network->markOutput(*yolo->getOutput(0));
 
    // Build engine
    builder->setMaxBatchSize(maxBatchSize);
    config->setMaxWorkspaceSize(16 * (1 << 20));  // 16MB
#if defined(USE_FP16)
    config->setFlag(BuilderFlag::kFP16);
#elif defined(USE_INT8)
    std::cout << "Your platform support int8: " << (builder->platformHasFastInt8() ? "true" : "false") << std::endl;
    assert(builder->platformHasFastInt8());
    config->setFlag(BuilderFlag::kINT8);
    Int8EntropyCalibrator2* calibrator = new Int8EntropyCalibrator2(1, INPUT_W, INPUT_H, "./coco_calib/", "int8calib.table", INPUT_BLOB_NAME);
    config->setInt8Calibrator(calibrator);
#endif
 
    std::cout << "Building engine, please wait for a while..." << std::endl;
    ICudaEngine* engine = builder->buildEngineWithConfig(*network, *config);
    std::cout << "Build engine successfully!" << std::endl;
 
    // Don't need the network any more
    network->destroy();
 
    // Release host memory
    for (auto& mem : weightMap)
    {
        free((void*)(mem.second.values));
    }
 
    return engine;
}
 
void APIToModel(unsigned int maxBatchSize, IHostMemory** modelStream, float& gd, float& gw, std::string& wts_name) {
    // Create builder
    IBuilder* builder = createInferBuilder(gLogger);
    IBuilderConfig* config = builder->createBuilderConfig();
 
    // Create model to populate the network, then set the outputs and create an engine
    ICudaEngine* engine = build_engine(maxBatchSize, builder, config, DataType::kFLOAT, gd, gw, wts_name);
    assert(engine != nullptr);
 
    // Serialize the engine
    (*modelStream) = engine->serialize();
 
    // Close everything down
    engine->destroy();
    builder->destroy();
    config->destroy();
}
 
void doInference(IExecutionContext& context, cudaStream_t& stream, void** buffers, float* input, float* output, int batchSize) {
    // DMA input batch data to device, infer on the batch asynchronously, and DMA output back to host
    CUDA_CHECK(cudaMemcpyAsync(buffers[0], input, batchSize * 3 * INPUT_H * INPUT_W * sizeof(float), cudaMemcpyHostToDevice, stream));
    context.enqueue(batchSize, buffers, stream, nullptr);
    CUDA_CHECK(cudaMemcpyAsync(output, buffers[1], batchSize * OUTPUT_SIZE * sizeof(float), cudaMemcpyDeviceToHost, stream));
    cudaStreamSynchronize(stream);
}
 
bool parse_args(int argc, char** argv, std::string& engine) {
    if (argc < 3) return false;
    if (std::string(argv[1]) == "-v" && argc == 3) {
        engine = std::string(argv[2]);
    }
    else {
        return false;
    }
    return true;
}
 
int main(int argc, char** argv) {
    cudaSetDevice(DEVICE);
 
    //std::string wts_name = "";
    std::string engine_name = "";
    //float gd = 0.0f, gw = 0.0f;
    //std::string img_dir;
 
    if (!parse_args(argc, argv, engine_name)) {
        std::cerr << "arguments not right!" << std::endl;
        std::cerr << "./yolov5 -v [.engine] // run inference with camera" << std::endl;
        return -1;
    }
 
    std::ifstream file(engine_name, std::ios::binary);
    if (!file.good()) {
        std::cerr << " read " << engine_name << " error! " << std::endl;
        return -1;
    }
    char* trtModelStream{ nullptr };
    size_t size = 0;
    file.seekg(0, file.end);
    size = file.tellg();
    file.seekg(0, file.beg);
    trtModelStream = new char[size];
    assert(trtModelStream);
    file.read(trtModelStream, size);
    file.close();
 
 
    // prepare input data ---------------------------
    static float data[BATCH_SIZE * 3 * INPUT_H * INPUT_W];
    //for (int i = 0; i < 3 * INPUT_H * INPUT_W; i++)
    //    data[i] = 1.0;
    static float prob[BATCH_SIZE * OUTPUT_SIZE];
    IRuntime* runtime = createInferRuntime(gLogger);
    assert(runtime != nullptr);
    ICudaEngine* engine = runtime->deserializeCudaEngine(trtModelStream, size);
    assert(engine != nullptr);
    IExecutionContext* context = engine->createExecutionContext();
    assert(context != nullptr);
    delete[] trtModelStream;
    assert(engine->getNbBindings() == 2);
    void* buffers[2];
    // In order to bind the buffers, we need to know the names of the input and output tensors.
    // Note that indices are guaranteed to be less than IEngine::getNbBindings()
    const int inputIndex = engine->getBindingIndex(INPUT_BLOB_NAME);
    const int outputIndex = engine->getBindingIndex(OUTPUT_BLOB_NAME);
    assert(inputIndex == 0);
    assert(outputIndex == 1);
    // Create GPU buffers on device
    CUDA_CHECK(cudaMalloc(&buffers[inputIndex], BATCH_SIZE * 3 * INPUT_H * INPUT_W * sizeof(float)));
    CUDA_CHECK(cudaMalloc(&buffers[outputIndex], BATCH_SIZE * OUTPUT_SIZE * sizeof(float)));
    // Create stream
    cudaStream_t stream;
    CUDA_CHECK(cudaStreamCreate(&stream));
 
     //#读取本地视频
    //cv::VideoCapture capture("/home/nano/Videos/video.mp4");
     //#调用本地usb摄像头,我的默认参数为1,如果1报错,可修改为0.
    cv::VideoCapture capture(1);
    if (!capture.isOpened()) {
        std::cout << "Error opening video stream or file" << std::endl;
        return -1;
    }
 
    int key;
    int fcount = 0;
    while (1)
    {
        cv::Mat frame;
        capture >> frame;
        if (frame.empty())
        {
            std::cout << "Fail to read image from camera!" << std::endl;
            break;
        }
        fcount++;
        //if (fcount < BATCH_SIZE && f + 1 != (int)file_names.size()) continue;
        for (int b = 0; b < fcount; b++) {
            //cv::Mat img = cv::imread(img_dir + "/" + file_names[f - fcount + 1 + b]);
            cv::Mat img = frame;
            if (img.empty()) continue;
            cv::Mat pr_img = preprocess_img(img, INPUT_W, INPUT_H); // letterbox BGR to RGB
            int i = 0;
            for (int row = 0; row < INPUT_H; ++row) {
                uchar* uc_pixel = pr_img.data + row * pr_img.step;
                for (int col = 0; col < INPUT_W; ++col) {
                    data[b * 3 * INPUT_H * INPUT_W + i] = (float)uc_pixel[2] / 255.0;
                    data[b * 3 * INPUT_H * INPUT_W + i + INPUT_H * INPUT_W] = (float)uc_pixel[1] / 255.0;
                    data[b * 3 * INPUT_H * INPUT_W + i + 2 * INPUT_H * INPUT_W] = (float)uc_pixel[0] / 255.0;
                    uc_pixel += 3;
                    ++i;
                }
            }
        }
 
        // Run inference
        auto start = std::chrono::system_clock::now();//#获取模型推理开始时间
        doInference(*context, stream, buffers, data, prob, BATCH_SIZE);
        auto end = std::chrono::system_clock::now();//#结束时间
        //std::cout << std::chrono::duration_cast<std::chrono::milliseconds>(end - start).count() << "ms" << std::endl;
        int fps = 1000.0 / std::chrono::duration_cast<std::chrono::milliseconds>(end - start).count();
        std::vector<std::vector<Yolo::Detection>> batch_res(fcount);
        for (int b = 0; b < fcount; b++) {
            auto& res = batch_res[b];
            nms(res, &prob[b * OUTPUT_SIZE], CONF_THRESH, NMS_THRESH);
        }
        for (int b = 0; b < fcount; b++) {
            auto& res = batch_res[b];
            //std::cout << res.size() << std::endl;
            //cv::Mat img = cv::imread(img_dir + "/" + file_names[f - fcount + 1 + b]);
            for (size_t j = 0; j < res.size(); j++) {
                cv::Rect r = get_rect(frame, res[j].bbox);
                cv::rectangle(frame, r, cv::Scalar(0x27, 0xC1, 0x36), 2);
                std::string label = my_classes[(int)res[j].class_id];
                cv::putText(frame, label, cv::Point(r.x, r.y - 1), cv::FONT_HERSHEY_PLAIN, 1.2, cv::Scalar(0xFF, 0xFF, 0xFF), 2);
                std::string jetson_fps = "FPS: " + std::to_string(fps);
                cv::putText(frame, jetson_fps, cv::Point(11, 80), cv::FONT_HERSHEY_PLAIN, 3, cv::Scalar(0, 0, 255), 2, cv::LINE_AA);
            }
            //cv::imwrite("_" + file_names[f - fcount + 1 + b], img);
        }
        cv::imshow("yolov5", frame);
        key = cv::waitKey(1);
        if (key == 'q') {
            break;
        }
        fcount = 0;
    }
 
    capture.release();
    // Release stream and buffers
    cudaStreamDestroy(stream);
    CUDA_CHECK(cudaFree(buffers[inputIndex]));
    CUDA_CHECK(cudaFree(buffers[outputIndex]));
    // Destroy the engine
    context->destroy();
    engine->destroy();
    runtime->destroy();
 
    return 0;
}

改完之后执行如下命令，调用摄像头成功

cd build
make
sudo ./yolov5 -v yolov5s.engine

如图

4ed275328dc949b69df04a51ea95bb28.png

到此就大功告成啦

​

adgbh
关注
专栏目录
yolov5-tensorrtx进行tensorrt加速：
04-23
YoloV5 pt 编译转 engine 模型加速后在jenson agx xavier边缘设备中部署。文章链接： ...
Jetson Nano TensorRT加速Yolov5
博客
06-27 826
【代码】Jetson Nano TensorRT加速Yolov5。
Jetson Nano B01部署YOLOv5项目全流程（五）TensorRT加速配置
人工智能
08-22 1930
五、TensorRT加速配置，主要包含Pycuda安装以及TensorRT加速。这里我们用自己的项目为例，使用YOLOv5自带的export.py文件进行engine文件转换，实现给自己的模型加速。
jetson nano: python调用yolov5-v6.0的tensorrt加速的c++代码，推理速度稳定在12fps，显存占用在0.8G以内
qq_30841655的博客
06-01 1951
python调用tensorrt加速的yolov5-v6.0的c++代码，使得推理速度12fps，显存控制在0.8G以内
Jetson Nano部署YOLOv5与Tensorrtx加速——（自己走一遍全过程记录）
热门推荐
Mr_LanGX的博客
11-30 2万+
Jetson Nano部署YOLOv5与Tensorrtx加速
在NVIDIA Jetson Nano上部署YOLOv5算法，并使用TensorRT和DeepStream进行加速
最新发布
qq_18677445的博客
05-22 1074
部署YOLOv5算法在NVIDIA Jetson Nano上并使用TensorRT和DeepStream进行加速涉及几个关键步骤。
Jenson
10-24
【Jenson：经典字体的魅力与应用】 在计算机和设计领域，字体扮演着至关重要的角色，为文本赋予了视觉表达和风格。"Jenson"是一种历史悠久的字体，它源自15世纪，由著名铸字工匠尼古拉斯·乔森（Nicolas Jenson）...
Jenson TX2配置tf-openpose.doc
06-24
本文讲述了在 jenson TX2开发板上，在配置 tf-openpose 时自己所遇到的一些问题和解决过程
Humane_Jenson_Normal
10-24
"Humane_Jenson_Normal"这个名称可能指的是一个特定的字体家族，它可能是由设计师或开发团队创造的，旨在提供一种易于阅读且具有美学吸引力的文本样式。在这个场景下，我们将深入探讨字体的相关知识，包括其分类、...
Xming 6.9 + 7.5
06-10
Xming是一款专为Windows设计的开源X Window System服务器软件，它允许用户在Windows操作系统上运行Linux或Unix应用程序，特别是那些依赖于X Window环境的应用。Xming的核心是Xorg服务器的一个移植版本，它使得...
Jetson Nano部署YOLOv5与Tensorrtx加速
君莫笑的博客
12-01 4758
Jetson Nano部署YOLOv5与Tensorrtx加速
Nvidia Jetson Nano Developer KIT配置全过程（五）：Pycuda及TensorRT安装配置
weixin_64298980的博客
01-06 2531
本系列主要讲解JETSON NANO DEVELOPER KIT开发板配置全过程，包括系统烧录安装、torch及cuda安装配置、pycharm及pycuda等安装配置。
（二）archiconda3和jetpack的安装
weixin_42823014的博客
01-11 2144
注意nvidia jetson tx2的架构是arrch64的，系统是ubuntu10.04的，但在安装需要的包和库时，要注意安装的是arrch64的 （1)进行anaconda的安装 linux_x86_64的可以安装成功，但无法使用，因此要安装archiconda3,安装包可以直接在这里下载：https://github.com/Archiconda/build-tools/releases，下载之后在对应的文件存储地址进行安装， 执行： sudo ./Archiconda3-0.2.3-Linu
在英伟达 jeason nano配置yolov5
laoli_的博客
12-07 1785
伟达 jeason nano配置yolov5
jeson nano 部署yolov5
qq_63946402的博客
02-09 208
jeson nano 部署yolov5
本人毕设记错--jeston nano TX1雷达小车实现yolov5 图像计数
qq_53883961的博客
04-21 183
充电口找错了，导致以为系统坏了，重装了一边系统，重新换了个小车，好几天的功夫都白弄了。
YOLOv5目标检测实战：Jetson Nano部署
04-06
PyTorch版的YOLOv5是高性能的实时目标检测方法。Jetson Nano是英伟达含有GPU的人工智能硬件。本课程讲述如何部署YOLOv5在Jetson Nano开发板上。部署完成后可进行图像、视频文件和摄像头视频的实时目标检测。部署时使用AI视频处理加速引擎TensorRT和DeepStream。 课程内容包括：原理篇（DeepStream介绍、TensorRT介绍、 tensorrtx介绍）、实践篇（Nano硬件搭建、烧录系统镜像、安装远程登录工具、安装和测试DeepStream、安装yolov5、生成yolov5s.wts文件、生成yolov5s.engine、使用DeepStream部署yolov5、Nano部署测试）
jenson nano对YoloV5进行部署以及TensorRT加速
very_big_house的博客
05-12 1625
部署是软件开发生命周期中非常重要的环节，它直接影响到系统的安全性、可用性和性能。因此，在进行部署前需要仔细检查和准备，确保软件能够在目标环境中顺利运行，并进行后期的监控和维护。那么今天这篇文章就来记录一下我是怎么进行对YoloV5的部署的。
jetson nano 部署人员检测
08-29
要在Jetson Nano上部署人员检测，您可以采用以下步骤： 1. 获取Jetson Nano开发套件，并按照NVIDIA开发者网站上的说明进行设置和配置。 2. 下载人脸佩戴口罩状态检测的训练好的模型，该模型已经适用于Jetson Nano。您可以使用PaddlePaddle-GPU和SSD-Mobilenet模型进行人员检测。 3. 确保您已经安装了适用于Jetson Nano的PyTorch版本。这将帮助您在Jetson Nano上进行深度学习任务。 4. 将下载的模型和必要的配置文件移动到Jetson Nano上。 5. 在Jetson Nano上运行您的人员检测代码。您可能需要编写一些代码来加载模型、进行推理和检测人员。 请注意，Jetson Nano是一款强大的嵌入式计算平台，可以用于运行各种深度学习任务。您可以根据具体需求选择适合的模型和算法进行人员检测。<span class="em">1</span><span class="em">2</span><span class="em">3</span> #### 引用[.reference_title] - *1* *3* [Jetson nano部署Yolov5目标检测 + Tensor RT加速（超级详细版）](https://blog.csdn.net/qq_67381784/article/details/130638235)[target="_blank" data-report-click={"spm":"1018.2226.3001.9630","extra":{"utm_source":"vip_chatgpt_common_search_pc_result","utm_medium":"distribute.pc_search_result.none-task-cask-2~all~insert_cask~default-1-null.142^v93^chatsearchT3_1"}}] [.reference_item style="max-width: 50%"] - *2* [人脸佩戴口罩状态检测的训练好的模型（jetson nano端）](https://download.csdn.net/download/qq_41675931/40325118)[target="_blank" data-report-click={"spm":"1018.2226.3001.9630","extra":{"utm_source":"vip_chatgpt_common_search_pc_result","utm_medium":"distribute.pc_search_result.none-task-cask-2~all~insert_cask~default-1-null.142^v93^chatsearchT3_1"}}] [.reference_item style="max-width: 50%"] [ .reference_list ]
评论 3
添加红包

请填写红包祝福语或标题

个

红包个数最小为10个

元

红包金额最低5元

当前余额3.43元 前往充值 >
需支付：10.00元
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付元
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明：

1.余额是钱包充值的虚拟货币，按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载，可以购买VIP、付费专栏及课程。

余额充值