TensorRT-官方例程SampleOnnxMNIST代码研读（草稿）

研究TensorRT，以期弄懂

0. 官方例程SampleOnnxMNIST简述

官方例程位置在安装TensorRT所在文件夹下：/TensorRT-8.6.1.6/samples/sampleOnnxMNIST/
其中含有Makefile、README.md、sampleOnnxMNIST.cpp三个文件

1. README文件

1.1 Description

This sample, sampleOnnxMNIST, converts a model trained on the MNIST dataset in Open Neural Network Exchange (ONNX) format to a TensorRT network and runs inference on the network.

ONNX is a standard for representing deep learning models that enables models to be transferred between frameworks.

1.2 How does this sample work?

This sample creates and runs the TensorRT engine from an ONNX model of the MNIST network. It demonstrates how TensorRT can consume an ONNX model as input to create a network.

1.2.1 Converting the ONNX model to a TensorRT network

The model file can be converted to a TensorRT network using the ONNX parser. The parser can be initialized with the network definition that the parser will write to and the logger object.

auto parser = nvonnxparser::createParser(*network, sample::gLogger.getTRTLogger());

The ONNX model file is then passed onto the parser along with the logging level

if (!parser->parseFromFile(model_file, static_cast<int>(sample::gLogger.getReportableSeverity())))
{
   
	  string msg("failed to parse onnx file");
	  sample::gLogger->log(nvinfer1::ILogger::Severity::kERROR, msg.c_str());
	  exit(EXIT_FAILURE);
}

After the TensorRT network is constructed by parsing the model, the TensorRT engine can be built to run inference.

1.2.2 Building the engine

To build the engine, create the builder and pass a logger created for TensorRT which is used for reporting errors, warnings and informational messages in the network:

IBuilder* builder = createInferBuilder(sample::gLogger);

To build the engine from the generated TensorRT network, issue the following call:

SampleUniquePtr<IHostMemory> plan{
   builder->buildSerializedNetwork(*network, *config)};

After you build the engine, verify that the engine is running properly by confirming the output is what you expected. The output format of this sample should be the same as the output of sampleMNIST.

1.2.3 Running inference

To run inference using the created engine, see Performing Inference In C++.

Note: It’s important to preprocess the data and convert it to the format accepted by the network. In this example, the sample input is in PGM (portable graymap) format. The model expects an input of image 1x28x28 scaled to between [0,1].

1.2.4 TensorRT API layers and ops

In this sample, the following layers are used. For more information about these layers, see the TensorRT Developer Guide: Layers documentation.
Activation layer
The Activation layer implements element-wise activation functions. Specifically, this sample uses the Activation layer with the type kRELU.
Convolution layer
The Convolution layer computes a 2D (channel, height, and width) convolution, with or without bias.
MatrixMultiplyLayer
The MatrixMultiply layer implements a matrix multiplication operation.
(The FullyConnected layer is deprecated since 8.4.
The bias of a FullyConnected layer can be added with an
ElementwiseLayer of SUM operation.)
Pooling layer
The Pooling layer implements pooling within a channel. Supported pooling types are maximum, average and maximum-average blend.
Scale layer
The Scale layer implements a per-tensor, per-channel, or per-ele