Win11系统C++将ONNX转换为engineer文件,TensorRT加速推理

已于 2022-10-30 22:28:51 修改
阅读量3.2k 收藏 30
点赞数 6
分类专栏： C++ CUDA pytorch 文章标签： c++ onnx
于 2022-10-30 22:25:06 首次发布
版权声明：本文为博主原创文章，遵循 CC 4.0 BY-SA 版权协议，转载请附上原文出处链接和本声明。
本文链接：https://blog.csdn.net/Vertira/article/details/127602349
版权
pytorch 同时被 3 个专栏收录
86 篇文章 20 订阅
订阅专栏
C++
14 篇文章 0 订阅
订阅专栏
CUDA
14 篇文章 3 订阅
订阅专栏

准确工作，安装配置好CUDA,cudnn，vs2019，TensorRT

可以参考我博客（下面博客有CUDA11.2,CUDNN11.2，vs2019，TensorRT配置方法）

(70条消息) WIN11+CUAD11.2+vs2019+tensorTR8.6+Yolov3/4/5模型加速_Vertira的博客-CSDN博客

下面我们在上面的基础上，下载opencv4（备用），然后创建onnx2TensorRT 项目

1.vs2019创建控制台程序，

  如果你是初次安装，没有c++套件（如果你安装了C++套件 ，忽略此步，直接进行第2步），

打开 Visual Studio Installer , 点击 " 修改 " 按钮 

进入 可以安装套件的界面

安装 " 使用 C++ 的桌面开发 " 组件 ; 有很多组件，这里咱们只安装C++

 选中后 , 右下角会显示 " 修改 " 按钮 , 点击该按钮 , 即可开始

 

然后等待安装完成即可 ;（时间有点长）

然后：

2、创建并运行 Windows 控制台程序 

 文件-->新建---》项目

选择创建 " 控制台应用 " ,

下一步，

在弹出的界面中 定义好工程名称onnx2TensorRT  和路径，确定创建  即可 

 然后进入工程文件夹，打开工程，工程自动创建带有输出hello world的cpp源文件，把文件清空

然后粘贴下面的代码：（由于没有配置路径，会出现很多红线，这个不用担心，下面就开始配置CUDA,TensorRT,这里没有用的图片显示，我暂时没有配置opencv）

#include <iostream>
#include <fstream>
#include "NvInfer.h"
#include "NvOnnxParser.h"

// 实例化记录器界面。捕获所有警告消息，但忽略信息性消息
class Logger : public nvinfer1::ILogger
{
    void log(Severity severity, const char* msg) noexcept override
    {
        // suppress info-level messages
        if (severity <= Severity::kWARNING)
            std::cout << msg << std::endl;
    }
} logger;


void ONNX2TensorRT(const char* ONNX_file, std::string save_ngine)
{
    // 1.创建构建器的实例
    nvinfer1::IBuilder* builder = nvinfer1::createInferBuilder(logger);

    // 2.创建网络定义
    uint32_t flag = 1U << static_cast<uint32_t>(nvinfer1::NetworkDefinitionCreationFlag::kEXPLICIT_BATCH);
    nvinfer1::INetworkDefinition* network = builder->createNetworkV2(flag);

    // 3.创建一个 ONNX 解析器来填充网络
    nvonnxparser::IParser* parser = nvonnxparser::createParser(*network, logger);

    // 4.读取模型文件并处理任何错误
    parser->parseFromFile(ONNX_file, static_cast<int32_t>(nvinfer1::ILogger::Severity::kWARNING));
    for (int32_t i = 0; i < parser->getNbErrors(); ++i)
    {
        std::cout << parser->getError(i)->desc() << std::endl;
    }

    // 5.创建一个构建配置，指定 TensorRT 应该如何优化模型
    nvinfer1::IBuilderConfig* config = builder->createBuilderConfig();

    // 6.设置属性来控制 TensorRT 如何优化网络
    // 设置内存池的空间
    config->setMemoryPoolLimit(nvinfer1::MemoryPoolType::kWORKSPACE, 16 * (1 << 20));
    // 设置低精度   注释掉为FP32
    if (builder->platformHasFastFp16())
    {
        config->setFlag(nvinfer1::BuilderFlag::kFP16);
    }

    // 7.指定配置后，构建引擎
    nvinfer1::IHostMemory* serializedModel = builder->buildSerializedNetwork(*network, *config);

    // 8.保存TensorRT模型
    std::ofstream p(save_ngine, std::ios::binary);
    p.write(reinterpret_cast<const char*>(serializedModel->data()), serializedModel->size());

    // 9.序列化引擎包含权重的必要副本，因此不再需要解析器、网络定义、构建器配置和构建器，可以安全地删除
    delete parser;
    delete network;
    delete config;
    delete builder;

    // 10.将引擎保存到磁盘，并且可以删除它被序列化到的缓冲区
    delete serializedModel;
}


void exportONNX(const char* ONNX_file, std::string save_ngine)
{
    std::ifstream file(ONNX_file, std::ios::binary);
    if (!file.good())
    {
        std::cout << "Load ONNX file failed! No file found from:" << ONNX_file << std::endl;
        return ;
    }

    std::cout << "Load ONNX file from: " << ONNX_file << std::endl;
    std::cout << "Starting export ..." << std::endl;

    ONNX2TensorRT(ONNX_file, save_ngine);

    std::cout << "Export success, saved as: " << save_ngine << std::endl;

}


int main(int argc, char** argv)
{
    // 输入信息
    const char* ONNX_file  = "../weights/test.onnx";
    std::string save_ngine = "../weights/test.engine";

    exportONNX(ONNX_file, save_ngine);

    return 0;
}

 

 然后，开始配置

属性界面如下： 

 包含目录：是CUDA的include文件夹所在路径和TensorRT的 include文件夹

$(CUDA_PATH)\include                            
D:\software\TensorRT-8.4.1.5_CUDA11.6_Cudnn8.4.1\include
D:\software\TensorRT-8.4.1.5_CUDA11.6_Cudnn8.4.1\samples\common


库目录：指的是 CUDA的lib 和TensorRT的lib

$(CUDA_PATH)\lib
$(CUDA_PATH)\lib\x64
D:\software\TensorRT-8.4.1.5_CUDA11.6_Cudnn8.4.1\lib

注意：$(CUDA_PATH)  也可以用绝对路径代替

3. 设置属性 — 链接器 

 输入的内容：

nvinfer.lib
nvinfer_plugin.lib
nvonnxparser.lib
nvparsers.lib
cudnn.lib
cublas.lib
cudart.lib

然后确定，确定  即可

5. 创建模型文件夹

新建weights文件夹，将你的onnx文件放在里面

      

 注意：weights路径的位置一定要和程序  最下面的路径位置一样

 然后，在debug模型在运行，如果你需要release版本，那就在release模式下运行。

 运行结果：

Load ONNX file from: ../weights/model.onnx
Starting export ...
onnx2trt_utils.cpp:369: Your ONNX model has been generated with INT64 weights, while TensorRT does not natively support INT64. Attempting to cast down to INT32.
TensorRT was linked against cuDNN 8.4.1 but loaded cuDNN 8.2.1
Weights [name=Conv_3 + Relu_4.weight] had the following issues when converted to FP16:
 - Subnormal FP16 values detected.
 - Values less than smallest positive FP16 Subnormal value detected. Converting to FP16 minimum subnormalized value.
If this is not the desired behavior, please modify the weights or retrain with regularization to reduce the magnitude of the weights.
Weights [name=Conv_3 + Relu_4.weight] had the following issues when converted to FP16:
 - Subnormal FP16 values detected.
 - Values less than smallest positive FP16 Subnormal value detected. Converting to FP16 minimum subnormalized value.
If this is not the desired behavior, please modify the weights or retrain with regularization to reduce the magnitude of the weights.
Weights [name=Conv_3 + Relu_4.weight] had the following issues when converted to FP16:
 - Subnormal FP16 values detected.
 - Values less than smallest positive FP16 Subnormal value detected. Converting to FP16 minimum subnormalized value.
If this is not the desired behavior, please modify the weights or retrain with regularization to reduce the magnitude of the weights.
Weights [name=Conv_3 + Relu_4.weight] had the following issues when converted to FP16:
 - Subnormal FP16 values detected.
 - Values less than smallest positive FP16 Subnormal value detected. Converting to FP16 minimum subnormalized value.
If this is not the desired behavior, please modify the weights or retrain with regularization to reduce the magnitude of the weights.
Weights [name=Conv_3 + Relu_4.weight] had the following issues when converted to FP16:
 - Subnormal FP16 values detected.
 - Values less than smallest positive FP16 Subnormal value detected. Converting to FP16 minimum subnormalized value.
If this is not the desired behavior, please modify the weights or retrain with regularization to reduce the magnitude of the weights.
Weights [name=Conv_3 + Relu_4.weight] had the following issues when converted to FP16:
 - Subnormal FP16 values detected.
 - Values less than smallest positive FP16 Subnormal value detected. Converting to FP16 minimum subnormalized value.
If this is not the desired behavior, please modify the weights or retrain with regularization to reduce the magnitude of the weights.
Weights [name=Conv_3 + Relu_4.weight] had the following issues when converted to FP16:
 - Subnormal FP16 values detected.
 - Values less than smallest positive FP16 Subnormal value detected. Converting to FP16 minimum subnormalized value.
If this is not the desired behavior, please modify the weights or retrain with regularization to reduce the magnitude of the weights.
Weights [name=Conv_3 + Relu_4.weight] had the following issues when converted to FP16:
 - Subnormal FP16 values detected.
 - Values less than smallest positive FP16 Subnormal value detected. Converting to FP16 minimum subnormalized value.
If this is not the desired behavior, please modify the weights or retrain with regularization to reduce the magnitude of the weights.
Weights [name=Gemm_8 + Relu_9.weight] had the following issues when converted to FP16:
 - Subnormal FP16 values detected.
 - Values less than smallest positive FP16 Subnormal value detected. Converting to FP16 minimum subnormalized value.
If this is not the desired behavior, please modify the weights or retrain with regularization to reduce the magnitude of the weights.
Weights [name=Gemm_8 + Relu_9.bias] had the following issues when converted to FP16:
 - Subnormal FP16 values detected.
If this is not the desired behavior, please modify the weights or retrain with regularization to reduce the magnitude of the weights.
Weights [name=Gemm_8 + Relu_9.weight] had the following issues when converted to FP16:
 - Subnormal FP16 values detected.
 - Values less than smallest positive FP16 Subnormal value detected. Converting to FP16 minimum subnormalized value.
If this is not the desired behavior, please modify the weights or retrain with regularization to reduce the magnitude of the weights.
Weights [name=Gemm_8 + Relu_9.bias] had the following issues when converted to FP16:
 - Subnormal FP16 values detected.
If this is not the desired behavior, please modify the weights or retrain with regularization to reduce the magnitude of the weights.
Weights [name=Gemm_8 + Relu_9.weight] had the following issues when converted to FP16:
 - Subnormal FP16 values detected.
 - Values less than smallest positive FP16 Subnormal value detected. Converting to FP16 minimum subnormalized value.
If this is not the desired behavior, please modify the weights or retrain with regularization to reduce the magnitude of the weights.
Weights [name=Gemm_8 + Relu_9.bias] had the following issues when converted to FP16:
 - Subnormal FP16 values detected.
If this is not the desired behavior, please modify the weights or retrain with regularization to reduce the magnitude of the weights.
Weights [name=Gemm_8 + Relu_9.weight] had the following issues when converted to FP16:
 - Subnormal FP16 values detected.
 - Values less than smallest positive FP16 Subnormal value detected. Converting to FP16 minimum subnormalized value.
If this is not the desired behavior, please modify the weights or retrain with regularization to reduce the magnitude of the weights.
Weights [name=Gemm_8 + Relu_9.bias] had the following issues when converted to FP16:
 - Subnormal FP16 values detected.
If this is not the desired behavior, please modify the weights or retrain with regularization to reduce the magnitude of the weights.
Weights [name=Gemm_8 + Relu_9.weight] had the following issues when converted to FP16:
 - Subnormal FP16 values detected.
 - Values less than smallest positive FP16 Subnormal value detected. Converting to FP16 minimum subnormalized value.
If this is not the desired behavior, please modify the weights or retrain with regularization to reduce the magnitude of the weights.
Weights [name=Gemm_8 + Relu_9.bias] had the following issues when converted to FP16:
 - Subnormal FP16 values detected.
If this is not the desired behavior, please modify the weights or retrain with regularization to reduce the magnitude of the weights.
Weights [name=Gemm_8 + Relu_9.weight] had the following issues when converted to FP16:
 - Subnormal FP16 values detected.
 - Values less than smallest positive FP16 Subnormal value detected. Converting to FP16 minimum subnormalized value.
If this is not the desired behavior, please modify the weights or retrain with regularization to reduce the magnitude of the weights.
Weights [name=Gemm_8 + Relu_9.bias] had the following issues when converted to FP16:
 - Subnormal FP16 values detected.
If this is not the desired behavior, please modify the weights or retrain with regularization to reduce the magnitude of the weights.
Weights [name=Gemm_8 + Relu_9.weight] had the following issues when converted to FP16:
 - Subnormal FP16 values detected.
 - Values less than smallest positive FP16 Subnormal value detected. Converting to FP16 minimum subnormalized value.
If this is not the desired behavior, please modify the weights or retrain with regularization to reduce the magnitude of the weights.
Weights [name=Gemm_8 + Relu_9.bias] had the following issues when converted to FP16:
 - Subnormal FP16 values detected.
If this is not the desired behavior, please modify the weights or retrain with regularization to reduce the magnitude of the weights.
Weights [name=Gemm_8 + Relu_9.weight] had the following issues when converted to FP16:
 - Subnormal FP16 values detected.
 - Values less than smallest positive FP16 Subnormal value detected. Converting to FP16 minimum subnormalized value.
If this is not the desired behavior, please modify the weights or retrain with regularization to reduce the magnitude of the weights.
Weights [name=Gemm_8 + Relu_9.bias] had the following issues when converted to FP16:
 - Subnormal FP16 values detected.
If this is not the desired behavior, please modify the weights or retrain with regularization to reduce the magnitude of the weights.
Weights [name=Gemm_8 + Relu_9.weight] had the following issues when converted to FP16:
 - Subnormal FP16 values detected.
 - Values less than smallest positive FP16 Subnormal value detected. Converting to FP16 minimum subnormalized value.
If this is not the desired behavior, please modify the weights or retrain with regularization to reduce the magnitude of the weights.
Weights [name=Gemm_8 + Relu_9.bias] had the following issues when converted to FP16:
 - Subnormal FP16 values detected.
If this is not the desired behavior, please modify the weights or retrain with regularization to reduce the magnitude of the weights.
Weights [name=Gemm_8 + Relu_9.weight] had the following issues when converted to FP16:
 - Subnormal FP16 values detected.
 - Values less than smallest positive FP16 Subnormal value detected. Converting to FP16 minimum subnormalized value.
If this is not the desired behavior, please modify the weights or retrain with regularization to reduce the magnitude of the weights.
Weights [name=Gemm_8 + Relu_9.bias] had the following issues when converted to FP16:
 - Subnormal FP16 values detected.
If this is not the desired behavior, please modify the weights or retrain with regularization to reduce the magnitude of the weights.
Weights [name=Gemm_8 + Relu_9.weight] had the following issues when converted to FP16:
 - Subnormal FP16 values detected.
 - Values less than smallest positive FP16 Subnormal value detected. Converting to FP16 minimum subnormalized value.
If this is not the desired behavior, please modify the weights or retrain with regularization to reduce the magnitude of the weights.
Weights [name=Gemm_8 + Relu_9.bias] had the following issues when converted to FP16:
 - Subnormal FP16 values detected.
If this is not the desired behavior, please modify the weights or retrain with regularization to reduce the magnitude of the weights.
Weights [name=Gemm_8 + Relu_9.weight] had the following issues when converted to FP16:
 - Subnormal FP16 values detected.
 - Values less than smallest positive FP16 Subnormal value detected. Converting to FP16 minimum subnormalized value.
If this is not the desired behavior, please modify the weights or retrain with regularization to reduce the magnitude of the weights.
Weights [name=Gemm_8 + Relu_9.bias] had the following issues when converted to FP16:
 - Subnormal FP16 values detected.
If this is not the desired behavior, please modify the weights or retrain with regularization to reduce the magnitude of the weights.
Weights [name=Gemm_8 + Relu_9.weight] had the following issues when converted to FP16:
 - Subnormal FP16 values detected.
 - Values less than smallest positive FP16 Subnormal value detected. Converting to FP16 minimum subnormalized value.
If this is not the desired behavior, please modify the weights or retrain with regularization to reduce the magnitude of the weights.
Weights [name=Gemm_8 + Relu_9.bias] had the following issues when converted to FP16:
 - Subnormal FP16 values detected.
If this is not the desired behavior, please modify the weights or retrain with regularization to reduce the magnitude of the weights.
Weights [name=Gemm_8 + Relu_9.weight] had the following issues when converted to FP16:
 - Subnormal FP16 values detected.
 - Values less than smallest positive FP16 Subnormal value detected. Converting to FP16 minimum subnormalized value.
If this is not the desired behavior, please modify the weights or retrain with regularization to reduce the magnitude of the weights.
Weights [name=Gemm_8 + Relu_9.bias] had the following issues when converted to FP16:
 - Subnormal FP16 values detected.
If this is not the desired behavior, please modify the weights or retrain with regularization to reduce the magnitude of the weights.
Weights [name=Gemm_10.weight] had the following issues when converted to FP16:
 - Subnormal FP16 values detected.
If this is not the desired behavior, please modify the weights or retrain with regularization to reduce the magnitude of the weights.
Weights [name=Gemm_10.weight] had the following issues when converted to FP16:
 - Subnormal FP16 values detected.
If this is not the desired behavior, please modify the weights or retrain with regularization to reduce the magnitude of the weights.
Weights [name=Gemm_10.weight] had the following issues when converted to FP16:
 - Subnormal FP16 values detected.
If this is not the desired behavior, please modify the weights or retrain with regularization to reduce the magnitude of the weights.
Weights [name=Gemm_10.weight] had the following issues when converted to FP16:
 - Subnormal FP16 values detected.
If this is not the desired behavior, please modify the weights or retrain with regularization to reduce the magnitude of the weights.
Weights [name=Gemm_10.weight] had the following issues when converted to FP16:
 - Subnormal FP16 values detected.
If this is not the desired behavior, please modify the weights or retrain with regularization to reduce the magnitude of the weights.
Weights [name=Gemm_10.weight] had the following issues when converted to FP16:
 - Subnormal FP16 values detected.
If this is not the desired behavior, please modify the weights or retrain with regularization to reduce the magnitude of the weights.
Weights [name=Gemm_10.weight] had the following issues when converted to FP16:
 - Subnormal FP16 values detected.
If this is not the desired behavior, please modify the weights or retrain with regularization to reduce the magnitude of the weights.
Weights [name=Gemm_10.weight] had the following issues when converted to FP16:
 - Subnormal FP16 values detected.
If this is not the desired behavior, please modify the weights or retrain with regularization to reduce the magnitude of the weights.
Weights [name=Gemm_10.weight] had the following issues when converted to FP16:
 - Subnormal FP16 values detected.
If this is not the desired behavior, please modify the weights or retrain with regularization to reduce the magnitude of the weights.
Weights [name=Gemm_10.weight] had the following issues when converted to FP16:
 - Subnormal FP16 values detected.
If this is not the desired behavior, please modify the weights or retrain with regularization to reduce the magnitude of the weights.
Weights [name=Gemm_10.weight] had the following issues when converted to FP16:
 - Subnormal FP16 values detected.
If this is not the desired behavior, please modify the weights or retrain with regularization to reduce the magnitude of the weights.
Weights [name=Gemm_10.weight] had the following issues when converted to FP16:
 - Subnormal FP16 values detected.
If this is not the desired behavior, please modify the weights or retrain with regularization to reduce the magnitude of the weights.
Weights [name=Gemm_10.weight] had the following issues when converted to FP16:
 - Subnormal FP16 values detected.
If this is not the desired behavior, please modify the weights or retrain with regularization to reduce the magnitude of the weights.
Weights [name=Gemm_10.weight] had the following issues when converted to FP16:
 - Subnormal FP16 values detected.
If this is not the desired behavior, please modify the weights or retrain with regularization to reduce the magnitude of the weights.
Weights [name=Conv_3 + Relu_4.weight] had the following issues when converted to FP16:
 - Subnormal FP16 values detected.
 - Values less than smallest positive FP16 Subnormal value detected. Converting to FP16 minimum subnormalized value.
If this is not the desired behavior, please modify the weights or retrain with regularization to reduce the magnitude of the weights.
Weights [name=Gemm_8 + Relu_9.weight] had the following issues when converted to FP16:
 - Subnormal FP16 values detected.
 - Values less than smallest positive FP16 Subnormal value detected. Converting to FP16 minimum subnormalized value.
If this is not the desired behavior, please modify the weights or retrain with regularization to reduce the magnitude of the weights.
Weights [name=Gemm_8 + Relu_9.bias] had the following issues when converted to FP16:
 - Subnormal FP16 values detected.
If this is not the desired behavior, please modify the weights or retrain with regularization to reduce the magnitude of the weights.
Weights [name=Gemm_10.weight] had the following issues when converted to FP16:
 - Subnormal FP16 values detected.
If this is not the desired behavior, please modify the weights or retrain with regularization to reduce the magnitude of the weights.
The getMaxBatchSize() function should not be used with an engine built from a network created with NetworkDefinitionCreationFlag::kEXPLICIT_BATCH flag. This function will always return 1.
The getMaxBatchSize() function should not be used with an engine built from a network created with NetworkDefinitionCreationFlag::kEXPLICIT_BATCH flag. This function will always return 1.
Export success, saved as: ../weights/model.engine

D:\VS_project\onnx2TensorRT\x64\Debug\onnx2TensorRT.exe (进程 17936)已退出，代码为 0。
要在调试停止时自动关闭控制台，请启用“工具”->“选项”->“调试”->“调试停止时自动关闭控制台”。
按任意键关闭此窗口. . .

从最后两句话可以看出

The getMaxBatchSize() function should not be used with an engine built from a network created with NetworkDefinitionCreationFlag::kEXPLICIT_BATCH flag. This function will always return 1.
Export success, saved as: ../weights/model.engine

D:\VS_project\onnx2TensorRT\x64\Debug\onnx2TensorRT.exe (进程 17936)已退出，代码为 0。
要在调试停止时自动关闭控制台，请启用“工具”->“选项”->“调试”->“调试停止时自动关闭控制台”。
按任意键关闭此窗口. . .

程序已经运行成功，没有报错。

7.注意：如果运行时出现 如下警告

问题 — 缺少 zlibwapi.dll 文件

若出现缺少zlibwapi.dll，需要下载，

若出现缺少zlibwapi.dll，需要下载，
链接：https://pan.baidu.com/s/12sVdiDH-NOOZNI9QqJoZuA?pwd=a0n0

 链接：百度网盘 请输入提取码百度网盘为您提供文件的网络备份、同步和分享服务。空间大、速度快、安全稳固，支持教育网加速，支持手机端。注册使用百度网盘即可享受免费存储空间https://pan.baidu.com/s/12sVdiDH-NOOZNI9QqJoZuA?pwd=a0n0
提取码：a0n0
下载后解压，将zlibwapi.dll 放入 …\NVIDIA GPU Computing Toolkit\CUDA\v11.2\bin(我的路径)
重新运行程序。如果没有错误，就忽略这个注意。

8. 生成engine文件

选择Release模式，然后重新配置一遍

点击，更换为Release

结束

欢迎   点赞   收藏   加   关注

参考：

(56条消息) Win10系统C++将ONNX转换为TensorRT_田小草呀的博客-CSDN博客_c++ onnx转tensorrt

Vertira
关注
专栏目录
PoolingNd层（Pooling层）1
08-03
PoolingNd 层（Pooling 层）参数及例（括号中的层名和参数名适于 TensorRT8 及之前版本，TensorRT9 及之后被废弃）初始例代码三维
onnx-＞trt F16:Subnormal FP16 value detected
Zhou_yongzhe的博客
10-12 2838
onnx转trt
tensorrt plugin
最新发布
xihuanniNI的博客
09-02 753
对于enqueue函数，由于都是不确定的数值，需要输入输出的描述。静态shape的隐式batch意思是，这个batch的数值是enqueue传递进来的，剩下的维度都是确定，batch是动态的。静态shape中，TRT的推理中，batch是拿不到的，getOutputDimensions的inputs参数只会有CHW，是一个明确数值和维度的数组。默认写的.cu是fp32的，TensorRT在fp16运行模式下，运行到不支持fp16的插件op时，会自动切换到fp32模式，等插件op运行完再切换回来。
onnx模型转trt模型报错TensorRT was linked against cuDNN 8.4.1 but loaded cuDNN 8.2.1
blink182007的博客
05-24 2331
(16条消息) 【TensorRT】TensorRT was linked against cudnn 8.6.0 but loaded cudnn 8.3.2_半路转行的水博的博客-CSDN博客。(16条消息) win10下 yolov8 tensorrt模型加速部署【实战】_FeiYull_的博客-CSDN博客。(16条消息) Win10环境下yolov8快速配置与测试_FeiYull_的博客-CSDN博客。具体报错：Results saved to D:\todesk\yolov8model。
TensorRT ubuntu18.04 安装过程记录
先说一下，就简单地写点儿氵文，不是博客，只是我的笔记本/工具书
07-29 3695
.
TensorRT(4)：NvInferRuntime.h接口头文件分析
hjxu2016的博客
10-26 7523
文章目录一、ICudaEngine类 记录NvInferRuntime.h头文件中的几个重要的接口类以及类中各接口函数的作用 头文件中都有英文注释，这里只是简单中文翻译一下，如有误，欢迎讨论。业余时间，看多少记录多少，纯属笔记。 NvInferRuntime.h是运行TensorRT的最重要的接口头文件之一，NV都说了 This is the top-level API file for TensorRT extended runtime library 介绍几个重要的类 1、ICudaEngine 2、I
tensorrt c++ 推理项目
07-03
tensorrt c++ 推理示例项目，支持分类网络，比如alexnet...转换流程：pytorch转onnx，onnx转tensorrt引擎 python生成tensorrt引擎步骤，可以查看我的博客： https://blog.csdn.net/jacke121/article/details/125382721
yolov7 的win10 tensorrt推理 c++ 版本
01-27
tensorrtx/yolov7 at master · wang-xinyu/tensorrtx · GitHub yolov7的win10 cuda11.4.3 cudnn8.2 tensorrt8.2.1.8 推理
onnxruntime v1.14.0 win7操作系统64位dll
12-11
onnxruntime1.15.0版本以后不支持win...版本：onnxruntime1.14.0操作系统：win7编译器：VS2019动态链接库为64位，release版本，已经win7 64位操作系统下VS2019 C++调用dll联调测试通过，现在共享出来给有需要的小伙伴。
yolov5.zip c++推理yolov5 onnx模型
01-02
本项目主要涉及将YOLOv5的PyTorch模型转换为ONNX（Open Neural Network Exchange）格式，并在Python、C++以及Java环境下进行推理。 首先，我们需要理解YOLOv5的工作原理。YOLO系列模型的核心思想是将图像分割为网格...
使用C++部署RTMPose的onnx模型或者tensorrt模型源码+使用说明.zip
05-15
RTMPose-Deploy是一个不使用mmdeploy进行RTMPose本地化部署C++示例。 目前我们在Windows系统上使用OnnxRuntime CPU SDK对...后续会考虑添加在Windows系统上使用C++ TensorRT SDK对RTMDetnano和RTMPose进行部署。
tensorRT_Pro 项目实践
博观约取，厚积薄发
08-12 1016
使用 tensorrt 部署模型
把onnx模型转TensorRT模型的trt模型报错：Your ONNX model has been generated with INT64 weights. while TensorRT
热门推荐
求真、务实
07-31 2万+
欢迎大家关注笔者，你的关注是我持续更博的最大动力 原创文章，转载告知，盗版必究 把onnx模型转TensorRT模型的trt模型报错：[TRT] onnx2trt_utils.cpp:198: Your ONNX model has been generated with INT64 weights, while TensorRT does not natively support INT64. Attempting to cast down to INT32. 文章目录：1 错误原因分析2 错误..
onnx模型转TensorRT模型时出错
weixin_48192326的博客
06-08 1367
【代码】把onnx模型转TensorRT模型出错。
TensorRt（1）安装和命令行测试
热爱生活，忠于自己
11-06 5507
根据机器硬件配置，选择TensorRt版本，安装对应的cuda、cudnn版本，命令行trtexec进行简单测试。
TensorRTX反序列fp32模型量化失败，原因是因为numclasses数量不对。当前为v8s_seg分割模型，但是报错，生成的engine文件为0kb故为错误的。
blink182007的博客
04-10 1021
Convolution]:kernel weights has count 3840 but 32640 was expected，yolov5s生成engine模型失败_yolov5s.engine-CSDN博客。当前分割背景的类别不要算进去，训练的时候，是几个类别，这里就是几个，不要额外加一。先附上解决方案：参考修改config.h文件。修改这个config.h文件，然后继续修改。
yolov5模型转换为TensorRT模型时报错Your ONNX model has been generated with INT64 weights
liguiyuan的博客
03-14 3723
因为我们的onnx模型参数类型是INT64, 然后TensorRT是不支持 INT64的。这就需要我们先把ONNX模型的精度降到 INT32，然后再进行转换为 TensorRT模型。
onnx | 把onnx模型转trt报错：Your ONNX model has been generated with INT64 weights. while TensorRT
Mopes__的博客
03-25 2148
onnx | 把onnx模型转trt报错：Your ONNX model has been generated with INT64 weights. while TensorRT
onnx-simplifier 模型简化中报错处理
yyc1820252161的博客
04-18 2252
报错1：Your ONNX model has been generated with INT64 weights, while TensorRT does not natively support INT64. Attempting to cast down to INT32 报错2：RuntimeError: The shape of input "input_1" has dynamic size "[0, 0, 0, 3]"
提供一下c++端调用onnx模型并通过tensorRT加速推理的代码案例并注释
03-31
以下是一个简单的C++代码示例，演示如何使用ONNX模型进行推理，并使用TensorRT进行加速。该示例使用TensorRT 5.1.5和CUDA 10.0进行编译。 首先，我们需要加载ONNX模型并创建推理引擎： ```c++ #include #include ...
评论 2
添加红包

请填写红包祝福语或标题

个

红包个数最小为10个

元

红包金额最低5元

当前余额3.43元 前往充值 >
需支付：10.00元
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付元
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明：

1.余额是钱包充值的虚拟货币，按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载，可以购买VIP、付费专栏及课程。

余额充值