openvino直接使用TensorFlow的模型

opencv里面dnn模块可以读取TensorFlow等深度学习模型，我在这篇文章里面实践过。

opencv的dnn模块还可以直接结合openvino使用深度学习模型进行推理，虽然openvino可以使用自己的方式加载模型，但是直接使用opencv::dnn相对来说简单直接。

重点是我们可以先不将TF的模型转成openvino的模型，也能直接运行，不过亲测好像TF2以上的办法没法运行。

这里我们还是使用的MobileNetSSD

模型下载链接:GitHub - Qengineering/MobileNet_SSD_OpenCV_TensorFlowContribute to Qengineering/MobileNet_SSD_OpenCV_TensorFlow development by creating an account on GitHub.https://github.com/Qengineering/MobileNet_SSD_OpenCV_TensorFlow

代码如下：

#include<iostream>
#include<opencv2/core.hpp>
#include<opencv2/highgui.hpp>
#include<opencv2/imgproc.hpp>
#include<opencv2/opencv.hpp>
//#include<inference_engine.hpp>
//#include<ie_extension.h>
//#include<ie_blob.h>

using namespace std;
//using namespace InferenceEngine;
using namespace cv;
using namespace cv::dnn;

string model = "./frozen_inference_graph.pb";
string config = "./graph.pbtxt";


int main() {
    Mat src=cv::imread("51.jpg");
    Net net = readNetFromTensorflow(model, config);
    net.setPreferableBackend(DNN_BACKEND_INFERENCE_ENGINE);//使用openvino作为推理引擎
    net.setPreferableTarget(DNN_TARGET_CPU);
    Mat blob = blobFromImage(src, 1.0, Size(300, 300), Scalar(), true, false, 5);
    net.setInput(blob);
    float confidenceThreshold = 0.5;
    Mat detection = net.forward();
    vector<double> layerTimings;
    double freq = getTickFrequency() / 1000;
    double time = net.getPerfProfile(layerTimings) / freq;
    cout<<"openvino模型推理时间为："<<time<<" ms"<<endl;
    net.setPreferableBackend(DNN_BACKEND_OPENCV);//使用opencv作为推理引擎
    Mat detection2 = net.forward();
    vector<double> layerTimings2;
    double freq2 = getTickFrequency() / 1000;
    double time2 = net.getPerfProfile(layerTimings2) / freq2;
    cout<<"opencv模型推理时间为："<<time2<<" ms"<<endl;

    int h = src.size().height;
    int w = src.size().width;
    cv::Mat dectetionMat(detection.size[2], detection.size[3], CV_32F,detection.ptr<float>());
    for (int i = 0; i < dectetionMat.rows;i++) {
        float confidence = dectetionMat.at<float>(i, 2);
        // cout << confidence << endl;
        if (confidence> confidenceThreshold) {      
            int idx= dectetionMat.at<float>(i, 1);
            // cout << "idx is " << idx << endl;
            int left= static_cast<int>(dectetionMat.at<float>(i, 3) * w);
            int top = static_cast<int>(dectetionMat.at<float>(i, 4) * h);
            int right = static_cast<int>(dectetionMat.at<float>(i, 5) * w);
            int bottom = static_cast<int>(dectetionMat.at<float>(i, 6) * h);
            cv::rectangle(src,Rect(left,top,right-left,bottom-top),Scalar(255,0,0),2);            
        }
        }
        cv::imwrite("3.jpg",src);
    return 0;
}

运行结果如下：

[E:] [BSL] found 0 ioexpander device
[E:] [BSL] found 0 ioexpander device
openvino模型推理时间为：50.6836 ms
opencv模型推理时间为：97.7752 ms

我们可以发现使用openvino比opencv推理速度快很多

我们还是使用的这篇文章的图，我们可以看一下结果：

运行环境：

openvino_2021.4.582

操作系统：

Distributor ID:    Ubuntu
Description:    Ubuntu 20.04.3 LTS
Release:    20.04
Codename:    focal