移动端推理引擎通用性小测
推理引擎
测试模型
类别 | 模型 | 地址 | 输入尺寸 | |
---|---|---|---|---|
视觉 | 分类 | MobileNet-V2 | https://github.com/onnx/models/blob/main/vision/classification/mobilenet/model/mobilenetv2-10.onnx | 1,3,224,224 |
ResNet50-V1 | https://github.com/onnx/models/blob/main/vision/classification/resnet/model/resnet50-v1-7.onnx | 1,3,224,224 | ||
SqueezeNet-V1.1 | https://github.com/onnx/models/blob/main/vision/classification/squeezenet/model/squeezenet1.1-7.onnx | 1,3,224,224 | ||
检测分割 | FasterRCNN-12 | https://github.com/onnx/models/blob/main/vision/object_detection_segmentation/faster-rcnn/model/FasterRCNN-12.onnx | 3,224,224 | |
超分辨率 | SuperResolution-10 | https://github.com/onnx/models/blob/main/vision/super_resolution/sub_pixel_cnn_2016/model/super-resolution-10.onnx | 1,1,224,224 | |
文本 | GPT | GPT-2 | https://github.com/onnx/models/blob/main/text/machine_comprehension/gpt-2/model/gpt2-10.onnx | 1,8,56 |
Bert | Bertsquad-12 | https://github.com/onnx/models/blob/main/text/machine_comprehension/bert-squad/model/bertsquad-12.onnx | input_ids:0=1,256 input_mask:0=1,256 segment_ids:0=1,256 unique_ids_raw_output___9:0=1 |
模型支持情况
CPU 运行
模型 | MNN | TensorFlow Lite | Paddle Lite | TNN | MindSpore Lite | NCNN |
---|---|---|---|---|---|---|
MobileNet-V2 | O | O | O | X | O | O |
ResNet50-V1 | O | O | O | O | O | O |
SqueezeNet-V1.1 | O | O | O | O | O | O |
FasterRCNN-12 | O | X | X | X | X | X |
SuperResolution-10 | O | O | O | O | O | O |
GPT-2 | O | X | X | X | O | X |
Bertsquad-12 | O | X | X | X | X | X |
GPU 运行 (OpenCL 或 Vulkan)
模型 | MNN | TensorFlow Lite | Paddle Lite | TNN | MindSpore Lite | NCNN |
---|---|---|---|---|---|---|
MobileNet-V2 | O | X | O | X | O | O |
ResNet50-V1 | O | X | O | O | O | O |
SqueezeNet-V1.1 | O | O | O | O | O | O |
FasterRCNN-12 | X | X | X | X | X | X |
SuperResolution-10 | O | O | O | X | O | O |
GPT-2 | O | X | X | X | X | X |
Bertsquad-12 | O | X | X | X | X | X |
结论与后续
CPU | GPU | 视觉 | 文本 | |
---|---|---|---|---|
MNN | 7/7 | 6/7 | 5/5 | 2/2 |
Paddle-Lite | 4/7 | 4/7 | 4/5 | 0/2 |
TNN | 3/7 | 3/7 | 3/5 | 0/2 |
Mindspore-Lite | 5/7 | 3/7 | 4/5 | 1/2 |
NCNN | 4/7 | 4/7 | 4/5 | 0/2 |
Tensorflow-Lite | 4/7 | 2/7 | 4/5 | 0/2 |
- MNN 全部模型均支持CPU运行,但GPU 跑 FasterRCNN12 出现错误(已于 2.6.2 修正)
- 其他推理引擎对 视觉类模型支持较好,文本类模型支持较弱。
- 带控制流的ONNX模型仅 MNN 转换运行成功