torch.onnx.export

官方文档

https://pytorch.org/docs/1.10/onnx.html#torch-autograd-functions

 torch.onnx.export

torch.onnx.export(model, args, f, export_params=True, verbose=False, training=False, input_names=None, output_names=None, aten=False, export_raw_ir=False, operator_export_type=None, opset_version=None, _retain_param_name=True, do_constant_folding=False, example_outputs=None, strip_doc_string=True, dynamic_axes=None, keep_initializers_as_inputs=None)

x = torch.onnx.export(model,  # 网络模型
                torch.randn(1, 3, 224, 224), # 用于确定输入大小和类型，其中的值可以是随机的。
                your_onnx_name.onnx,  # 输出onnx的名称
                verbose=False,      # 是否以字符串的形式显示计算图
                input_names=["input"],  # 输入节点的名称，可以是一个list
                output_names=["output"], # 输出节点的名称
                opset_version=12,   # onnx 支持采用的operator set
                do_constant_folding=True, # 是否压缩常量
                #设置动态维度，此处指明input节点的第0维度可变，命名为batch_size
                dynamic_axes={"input":{0: "batch_size", 2: "h", 3: "w"}, "output":{0: "batch_size"}} 
                )

Export a model into ONNX format.  This exporter runs your model
    once in order to get a trace of its execution to be exported;
    at the moment, it supports a limited set of dynamic models (e.g., RNNs.)
    See also: :ref:`onnx-export`
    Arguments:
        model (torch.nn.Module): the model to be exported.
        args (tuple of arguments): the inputs to
            the model, e.g., such that ``model(*args)`` is a valid
            invocation of the model.  Any non-Tensor arguments will
            be hard-coded into the exported model; any Tensor arguments
            will become inputs of the exported model, in the order they
            occur in args.  If args is a Tensor, this is equivalent
            to having called it with a 1-ary tuple of that Tensor.
            (Note: passing keyword arguments to the model is not currently
            supported.  Give us a shout if you need it.)
        f: a file-like object (has to implement fileno that returns a file descriptor)
            or a string containing a file name.  A binary Protobuf will be written
            to this file.
        export_params (bool, default True): if specified, all parameters will
            be exported.  Set this to False if you want to export an untrained model.
            In this case, the exported model will first take all of its parameters
            as arguments, the ordering as specified by ``model.state_dict().values()``
        verbose (bool, default False): if specified, we will print out a debug
            description of the trace being exported.
        training (bool, default False): export the model in training mode.  At
            the moment, ONNX is oriented towards exporting models for inference
            only, so you will generally not need to set this to True.
        input_names(list of strings, default empty list): names to assign to the
            input nodes of the graph, in order
        output_names(list of strings, default empty list): names to assign to the
            output nodes of the graph, in order
        aten (bool, default False): [DEPRECATED. use operator_export_type] export the
            model in aten mode. If using aten mode, all the ops original exported
            by the functions in symbolic_opset<version>.py are exported as ATen ops.
        export_raw_ir (bool, default False): [DEPRECATED. use operator_export_type]
            export the internal IR directly instead of converting it to ONNX ops.
        operator_export_type (enum, default OperatorExportTypes.ONNX):
            OperatorExportTypes.ONNX: all ops are exported as regular ONNX ops.
            OperatorExportTypes.ONNX_ATEN: all ops are exported as ATen ops.
            OperatorExportTypes.ONNX_ATEN_FALLBACK: if symbolic is missing,
                                                    fall back on ATen op.
            OperatorExportTypes.RAW: export raw ir.
        opset_version (int, default is 9): by default we export the model to the
            opset version of the onnx submodule. Since ONNX's latest opset may
            evolve before next stable release, by default we export to one stable
            opset version. Right now, supported stable opset version is 9.
            The opset_version must be _onnx_master_opset or in _onnx_stable_opsets
            which are defined in torch/onnx/symbolic_helper.py
        do_constant_folding (bool, default False): If True, the constant-folding
            optimization is applied to the model during export. Constant-folding
            optimization will replace some of the ops that have all constant
            inputs, with pre-computed constant nodes.
        example_outputs (tuple of Tensors, default None): example_outputs must be provided
            when exporting a ScriptModule or TorchScript Function.
        strip_doc_string (bool, default True): if True, strips the field
            "doc_string" from the exported model, which information about the stack
            trace.
        example_outputs: example outputs of the model that is being exported.
        dynamic_axes (dict<string, dict<int, string>> or dict<string, list(int)>, default empty dict):
            a dictionary to specify dynamic axes of input/output, such that:
            - KEY:  input and/or output names
            - VALUE: index of dynamic axes for given key and potentially the name to be used for
            exported dynamic axes. In general the value is defined according to one of the following
            ways or a combination of both:
            (1). A list of integers specifiying the dynamic axes of provided input. In this scenario
            automated names will be generated and applied to dynamic axes of provided input/output
            during export.
            OR (2). An inner dictionary that specifies a mapping FROM the index of dynamic axis in
            corresponding input/output TO the name that is desired to be applied on such axis of
            such input/output during export.
            Example. if we have the following shape for inputs and outputs:
                shape(input_1) = ('b', 3, 'w', 'h')
                and shape(input_2) = ('b', 4)
                and shape(output)  = ('b', 'd', 5)

            Then dynamic axes can be defined either as:
                (a). ONLY INDICES:
                    dynamic_axes = {'input_1':[0, 2, 3], 'input_2':[0], 'output':[0, 1]}

                    where automatic names will be generated for exported dynamic axes

                (b). INDICES WITH CORRESPONDING NAMES:
                    dynamic_axes = {'input_1':{0:'batch', 1:'width', 2:'height'},
                    'input_2':{0:'batch'},
                    'output':{0:'batch', 1:'detections'}

                    where provided names will be applied to exported dynamic axes

                (c). MIXED MODE OF (a) and (b)
                    dynamic_axes = {'input_1':[0, 2, 3], 'input_2':{0:'batch'}, 'output':[0,1]}
        keep_initializers_as_inputs (bool, default None): If True, all the initializers
            (typically corresponding to parameters) in the exported graph will also be
            added as inputs to the graph. If False, then initializers are not added as
            inputs to the graph, and only the non-parameter inputs are added as inputs.
            This may allow for better optimizations (such as constant folding etc.) by
            backends/runtimes that execute these graphs. If unspecified (default None),
            then the behavior is chosen automatically as follows. If operator_export_type
            is OperatorExportTypes.ONNX, the behavior is equivalent to setting this
            argument to False. For other values of operator_export_type, the behavior is
            equivalent to setting this argument to True.

        model - 要导出的模型.
        args (tuple of arguments) – 模型的输入, 任何非Tensor参数都将硬编码到导出的模型中；任何Tensor参数都将成为导出的模型的输入，并按照他们在args中出现的顺序输入。因为export运行模型，所以我们需要提供一个输入张量x。只要是正确的类型和大小，其中的值就可以是随机的。请注意，除非指定为动态轴，否则输入尺寸将在导出的ONNX图形中固定为所有输入尺寸。在此示例中，我们使用输入batch_size 1导出模型，但随后dynamic_axes 在torch.onnx.export()。因此，导出的模型将接受大小为[batch_size，3、100、100]的输入，其中batch_size可以是可变的。

        f - onnx名字
        export_params (bool, default True) – 如果指定为True或默认, 参数也会被导出. 如果你要导出一个没训练过的就设为 False.

        opset_version 表示 ONNX 算子集的版本
        verbose (bool, default False) - 如果指定，我们将打印出一个导出轨迹的调试描述。
        training (bool, default False) - 在训练模式下导出模型。目前，ONNX导出的模型只是为了做推断，所以你通常不需要将其设置为True。
        input_names (list of strings, default empty list) – 按顺序分配名称到图中的输入节点
        output_names (list of strings, default empty list) –按顺序分配名称到图中的输出节点
        dynamic_axes – {‘input’ : {0 : ‘batch_size’}, ‘output’ : {0 : ‘batch_size’}}

onnx 转为engine

TensorRT-7.2.3.4/bin/trtexec

trtexec --onnx=conv.onnx

         --saveEngine=flame_sim.engine

         --workspace=1024

         --minShapes=inputx:1x3x224x224

         --optShapes=inputx:1x3x2224x224

         --maxShapes=inputx:1x3x224x224

构建engine一般有两种方式。

方式1：torch模型->wts(序列化网络)->engine->推理

方式2：torch模型->onnx->engine->推理

验证一下模型文件是否正确

import onnx
onnx_model = onnx.load("your.onnx")
try:
    onnx.checker.check_model(onnx_model)
except Exception:
    print("onnx error")
else:
    print("onnx correct")

 可视化ONNX 模型结构 netron

netron your.onnx

查看模型输入输出的名字

torch.onnx.is_in_onnx_export()

该函数仅在执行 torch.onnx.export()时为真

 onnx INT64 weights error

TensorRT解析PyTorch导出的onnx模型时:

Your ONNX model has been generated with INT64 weights, while TensorRT does not natively support INT64. Attempting to cast down to INT32.

pip install onnx-simplifier
python -m onnxsim model_old.onnx model_sim_new.onnx