Tensorrt 自定义插件的调用顺序及过程

一个自定义插件需要实现两个类，分别继承于

class MyPluginDynamic : public IPluginV2DynamicEx

class MyPluginDynamicCreator : public IPluginCreator

然后使用

REGISTER_TENSORRT_PLUGIN(MyPluginDynamicCreator);

看其详细定义，也就是将这个插件加入到一个全局链表中，方便trtexec根据op name来查找该插件，这个实现跟ffmpeg中注册编解码器是一个思路

#define REGISTER_TENSORRT_PLUGIN(name)                                                                         

    static nvinfer1::PluginRegistrar<name> pluginRegistrar##name {}

#endif // NV_INFER_RUNTIME_H

下面是method详细的调用过程：

Creator:

MyPluginDynamicCreator

getPluginName （连续被调用两次）

getPluginNamespace

getPluginVersion

================================

Plugin:

getPluginVersion
getPluginNamespace
getPluginNamespace
getFieldNames
createPlugin
MyPluginDynamic
getNbOutputs
getOutputDataType
getNbOutputs
clone
MyPluginDynamic
setPluginNamespace
getPluginType
destroy
~MyPluginDynamic
getOutputDimensions
getOutputDataType
getOutputDimensions
 

=====================

一个有意义的参考

GitHub - eweill-nv/dcnv2_trt: TensorRT plugin forDCNv2 layer in ONNX model

IPluginV2DynamicExt

The first thing that we want to point out is the we are going to base our Plugin off of the IPluginV2DynamicExt class which will give us the ability to use alot of the functionality that TensorRT already has built in. You can see where we built our plugin class around the IPluginV2DynamicExt class here.

The first thing we want to do is to create our constructor and destructor for our TensorRT plugin (in this case, DCNv2PluginDynamic). You can see an example of that here:

DCNv2PluginDynamic();

DCNv2PluginDynamic(const void* data, size_t length, const std::string& name);
			
DCNv2PluginDynamic(DCNv2Parameters param, const std::string& name);
			
~DCNv2PluginDynamic() override;

Note that here, we have 2 different ways that a DCNv2PluginDynamic can be created: either passing in the data as a pointer and reading each value separately, or simply passing it in as a mParam structure with all of the data already included in the right format.
Methods

We have a few methods that are part of the IPluginV2DynamicExt class that we want to override so that we can modify if necessary:

    clone(): copies over all internal plugin parameters and returns a new plugin object with these parameters
    getOutputDimensions(): computes the dimensions of the output tensor from dimensions of the input tensor
    supportsFormatCombination(): determines supported data types
    configurePlugin(): configure the layer
    getWorkspaceSize(): find the workspace size required by the layer (it is still necessary to provide the --workspace flag to trtexec as well)
    enqueue(): execute the layer

Next, we have a few of the methods that are part of IPluginv2Ext that we want to override as well for our functionality:

    getOutputDataType(): returns the datatype of the plugin output (in this case, either kFLOAT or kHALF)
    attachToContext(): attach the plugin to an execution context and graph plugin access to context resources (use of cuBLAS/cuDNN/etc.)
    detatchFromContext(): detach the plugin from its execution context

Lastly, we have a few of the methods that are part of IPluginV2 that we want to override for the same reason:

    getPluginType(): return the type for the plugin (matches the plugin name returned by the plugin creator)
    getPluginVersion(): returns the plugin version (should also match the plugin version returned by the plugin creator)
    getNbOutputs(): returns number of outputs for the layer
    initialize(): initialize the layer for execution (called when the engine is created)
    terminate(): releases resources aqcuired during plugin layer initialization (called when engine is destroyed)
    getSerializationSize(): returns size of serialization buffer necessary
    serialize(): serialize the layer
    destroy(): destroy the plugin object
    setPluginNamespace(): set the namespace for the plugin object
    getPluginNamespace(): return the namespace for the plugin object

More information about these plugins can be found here in the TensorRT documentation.
Members

As part of our IPluginV2DynamicExt instantiated class, we also want to create a few member variables that will help us with our plugin implementation. These can be found here

We have the following variables defined here:

    mLayerName: given name for the layer (how it shows up in the graph)
    mNamespace: namespace in which the layer resides
    cublasHandle_: handle to the cuBlas context
    cudnnHandle_: handle to the cuDNN context
    mType: layer type (in this case, either kFLOAT or kHALF)
    input1_shape: shape of first input to DCNv2 layer (from Add layer)
    input2_shape: shape of second input to DCNv2 layer (from Conv layer)
    weights_shape: shape of weights for DCNv2 layer
    output_shape: shape of output for DCNv2 layer
    mParam: structure container attributes for DCNv2 layer
    mDeviceWeights: variable for weights on GPU for DCNv2 layer
    mDeviceBiases: variable for biases on GPU for DCNv2 layer

IPluginCreator

For user implemented layers, we need to also instantiate another class (i.e. DCNv2PluginDynamicCreator) which is going to be part of the IPluginCreator class with the following methods:

The first thing we want to do (as before) is create our constructor and destructor for our TensorRT plugin creator (in this case, DCNv2PluginDynamicCreator). An example of that can be found here with the declaration here.
Methods

    getTensorRTVersion(): return version of API the plugin creator was compiled with
    getPluginName(): return plugin name
    getPluginVersion(): return plugin version
    getFieldNames(): return list of fields to be passed to createPlugin
    createPlugin(): return plugin object
    deserializePlugin(): called during deserialization of plugin layer
    setPluginNamespace(): set namespace for plugin creator based on plugin library
    getPluginNamespace(): return namespace of plugin creator object

Members

    mFC: contains information about the PluginFieldCollection
    mPluginAttributes: contains information about attributes of the plugin
    mNamespace: namespace in which the layer reside

More information about the plugin creator can be found here in the TensorRT documentation.