YOLOv8-Openvino和ONNXRuntime推理【CPU】

你的陈某某

已于 2024-03-12 10:16:16 修改

阅读量1.4k

点赞数 22

分类专栏： Openvino和ONNXRuntime推理文章标签： YOLO openvino 人工智能 yolov8 目标检测

于 2024-02-19 11:26:55 首次发布

本文链接：https://blog.csdn.net/weixin_45679938/article/details/136164740

版权

Openvino和ONNXRuntime推理专栏收录该内容

11 篇文章 1 订阅

订阅专栏

注：YOLOv5、YOLOv6和YOLOv7代码内容基本一致！YOLOv8和YOLOv9代码内容基本一致！
全部代码Github：https://github.com/Bigtuo/YOLOv8_Openvino

1 环境：

CPU：i5-12500
Python：3.8.18

2 安装Openvino和ONNXRuntime

2.1 Openvino简介

Openvino是由Intel开发的专门用于优化和部署人工智能推理的半开源的工具包，主要用于对深度推理做优化。

Openvino内部集成了Opencv、TensorFlow模块，除此之外它还具有强大的Plugin开发框架，允许开发者在Openvino之上对推理过程做优化。

Openvino整体框架为：Openvino前端→ Plugin中间层→ Backend后端
Openvino的优点在于它屏蔽了后端接口，提供了统一操作的前端API，开发者可以无需关心后端的实现，例如后端可以是TensorFlow、Keras、ARM-NN，通过Plugin提供给前端接口调用，也就意味着一套代码在Openvino之上可以运行在多个推理引擎之上，Openvino像是类似聚合一样的开发包。

2.2 ONNXRuntime简介

ONNXRuntime是微软推出的一款推理框架，用户可以非常便利的用其运行一个onnx模型。ONNXRuntime支持多种运行后端包括CPU，GPU，TensorRT，DML等。可以说ONNXRuntime是对ONNX模型最原生的支持。

虽然大家用ONNX时更多的是作为一个中间表示，从pytorch转到onnx后直接喂到TensorRT或MNN等各种后端框架，但这并不能否认ONNXRuntime是一款非常优秀的推理框架。而且由于其自身只包含推理功能（最新的ONNXRuntime甚至已经可以训练），通过阅读其源码可以解深度学习框架的一些核心功能原理（op注册，内存管理，运行逻辑等）
总体来看，整个ONNXRuntime的运行可以分为三个阶段，Session构造，模型加载与初始化和运行。和其他所有主流框架相同，ONNXRuntime最常用的语言是python，而实际负责执行框架运行的则是C++。

2.3 安装

pip install openvino -i  https://pypi.tuna.tsinghua.edu.cn/simple
pip install onnxruntime -i  https://pypi.tuna.tsinghua.edu.cn/simple

3 准备YOLOv8s.onnx文件

YOLOv8官网
 YOLOv8原理
.pt文件转.onnx文件示例代码【注意自己转需要安装YOLOv8的环境】：

from ultralytics import YOLO
model = YOLO("yolov8s.pt")  # load a pretrained model
path = model.export(format="onnx", dynamic=True)  # export the mode l to ONNX format

4 Openvino和ONNXRuntime推理脚本

下面代码整个处理过程主要包括：预处理—>推理—>后处理—>画图。
假设图像resize为640×640，
预处理输出结果维度：(1, 3, 640, 640)；
推理输出结果维度：(1, 84, 8400)，其中84表示4个box坐标信息+80个类别概率，8400表示80×80+40×40+20×20；
后处理输出结果维度：(5, 6)，其中第一个5表示图bus.jpg检出5个目标，第二个维度6表示(x, y, w, h, conf, cls)。

4.1 预处理

注：其中pad部分去除能减少预处理时间，且推理精度几乎一致。

def preprocess(image, img_h, img_w):
    '''
    Yolo系列算法通用预处理
    '''
    image = cv2.cvtColor(image, cv2.COLOR_BGR2RGB)
    scale = max(image.shape[0] / img_h, image.shape[1] / img_w)
    image = cv2.resize(image, (int(image.shape[1] / scale), int(image.shape[0] / scale)))
   
    wpad = img_w - image.shape[1]
    hpad = img_h - image.shape[0]
    image_pad = np.ones((image.shape[0]+hpad, image.shape[1]+wpad, 3)) * 114.0
    image_pad[:image.shape[0], :image.shape[1], :] = image
    image_array = image_pad
    
    image_array = image_array / 255.0
    image_array = image_array.transpose((2, 0, 1))
    image_array = image_array.astype(np.float32)
    input_array = np.ascontiguousarray(np.expand_dims(image_array, 0))
    return input_array, scale, image.shape[0], image.shape[1]

4.2 后处理

注：尝试多种后处理写法，该种写法速度最快。

def postprocess(pred, conf_thres, iou_thres, img_w, img_h):
    """
    Args:
        pred: np.array([(x, y, w, h, cls1_conf, cls2_conf, cls3_conf, ...), ...]), shape=(-1, 4 + num_cls)
        conf_thres: 置信度阈值
        iou_thres: IOU阀值,若两个box的交并比大于该值，则置信度较小的box将会被抑制
        img_w: 原图w大小
        img_h: 原图h大小
        Returns:
        out: 经过NMS后的值，np.array([(x, y, w, h, conf, cls), ...]), shape=(-1, 4 + 1 + 1)
    """
    pred = np.squeeze(pred).transpose((1, 0))  # (1, 80+4, -1) -> (80+4, -1) -> (-1, 80+4)
    # 按置信度过滤
    conf = np.max(pred[..., 4:], axis=-1)
    mask = conf >= conf_thres

    # Where the score larger than score_threshold
    box = pred[mask][..., :4]
    confidences = conf[mask]
  	clsid = np.argmax(pred[mask][..., 4:], axis=1)  
    
    # 下面进行非极大抑制NMS处理
    # 对box进行转换，以及对不同类别分不同区间处理
    bounding_boxes = np.zeros_like(box)
    bounding_boxes[:, 0] = (box[:, 0] - box[:, 2] / 2) + clsid * img_w  # xmin + 每个类别分不同区间
    bounding_boxes[:, 1] = (box[:, 1] - box[:, 3] / 2) + clsid * img_h  # ymin + 每个类别分不同区间
    bounding_boxes[:, 2] = box[:, 2]  # w
    bounding_boxes[:, 3] = box[:, 3]  # h
    # xywh2xyxy
    bounding_boxes[:, 2] += bounding_boxes[:, 0]
    bounding_boxes[:, 3] += bounding_boxes[:, 1]
    if bounding_boxes.shape[0] != confidences.shape[0]:
        raise ValueError("Bounding box 与 Confidence 的数量不一致")
    if bounding_boxes.shape[0] == 0:
        return []
    bounding_boxes, confidences = bounding_boxes.astype(np.float32), np.array(confidences)
    x1, y1, x2, y2 = bounding_boxes[:, 0], bounding_boxes[:, 1], bounding_boxes[:, 2], bounding_boxes[:, 3]
    areas = (x2 - x1 + 1) * (y2 - y1 + 1)
    idxs = np.argsort(confidences)

    pick = []
   	while len(idxs) > 0:
        # 因为idxs是从小到大排列的，last_idx相当于idxs最后一个位置的索引
        last_idx = len(idxs) - 1
        # 取出最大值在数组上的索引
        max_value_idx = idxs[last_idx]
        # 将这个添加到相应索引上
      	pick.append(max_value_idx)

        xx1 = np.maximum(x1[max_value_idx], x1[idxs[: last_idx]])
        yy1 = np.maximum(y1[max_value_idx], y1[idxs[: last_idx]])
        xx2 = np.minimum(x2[max_value_idx], x2[idxs[: last_idx]])
        yy2 = np.minimum(y2[max_value_idx], y2[idxs[: last_idx]])
        w, h = np.maximum(0, xx2 - xx1 + 1), np.maximum(0, yy2 - yy1 + 1)
        iou = w * h / areas[idxs[: last_idx]]

        # 删除最大的value,并且删除iou > threshold的bounding boxes
        idxs = np.delete(idxs, np.concatenate(([last_idx], np.where(iou > iou_thres)[0])))
    out = np.concatenate([box[pick], confidences[pick].reshape(-1, 1), clsid[pick].reshape(-1, 1)], axis=1)
    return out

4.3 全部代码

import os
import time

# openvino速度比onnxruntime快一倍
from openvino.runtime import Core  # pip install openvino -i  https://pypi.tuna.tsinghua.edu.cn/simple
import onnxruntime as rt  # 使用onnxruntime推理用上，pip install onnxruntime
import numpy as np
import cv2

def preprocess(image, img_h, img_w):
    '''
    Yolo系列算法通用预处理
    '''
    image = cv2.cvtColor(image, cv2.COLOR_BGR2RGB)
    scale = max(image.shape[0] / img_h, image.shape[1] / img_w)
    image = cv2.resize(image, (int(image.shape[1] / scale), int(image.shape[0] / scale)))
   
    wpad = img_w - image.shape[1]
    hpad = img_h - image.shape[0]
    image_pad = np.ones((image.shape[0]+hpad, image.shape[1]+wpad, 3)) * 114.0
    image_pad[:image.shape[0], :image.shape[1], :] = image
    image_array = image_pad
    
    image_array = image_array / 255.0
    image_array = image_array.transpose((2, 0, 1))
    image_array = image_array.astype(np.float32)
    input_array = np.ascontiguousarray(np.expand_dims(image_array, 0))
    return input_array, scale, image.shape[0], image.shape[1]

def postprocess(pred, conf_thres, iou_thres, img_w, img_h):
    """
    Args:
        pred: np.array([(x, y, w, h, cls1_conf, cls2_conf, cls3_conf, ...), ...]), shape=(-1, 4 + num_cls)
        conf_thres: 置信度阈值
        iou_thres: IOU阀值,若两个box的交并比大于该值，则置信度较小的box将会被抑制
        img_w: 原图w大小
        img_h: 原图h大小
        Returns:
        out: 经过NMS后的值，np.array([(x, y, w, h, conf, cls), ...]), shape=(-1, 4 + 1 + 1)
    """
    pred = np.squeeze(pred).transpose((1, 0))  # (1, 80+4, -1) -> (80+4, -1) -> (-1, 80+4)
    # 按置信度过滤
    conf = np.max(pred[..., 4:], axis=-1)
    mask = conf >= conf_thres

    # Where the score larger than score_threshold
    box = pred[mask][..., :4]
    confidences = conf[mask]
  	clsid = np.argmax(pred[mask][..., 4:], axis=1)  
    
    # 下面进行非极大抑制NMS处理
    # 对box进行转换，以及对不同类别分不同区间处理
    bounding_boxes = np.zeros_like(box)
    bounding_boxes[:, 0] = (box[:, 0] - box[:, 2] / 2) + clsid * img_w  # xmin + 每个类别分不同区间
    bounding_boxes[:, 1] = (box[:, 1] - box[:, 3] / 2) + clsid * img_h  # ymin + 每个类别分不同区间
    bounding_boxes[:, 2] = box[:, 2]  # w
    bounding_boxes[:, 3] = box[:, 3]  # h
    # xywh2xyxy
    bounding_boxes[:, 2] += bounding_boxes[:, 0]
    bounding_boxes[:, 3] += bounding_boxes[:, 1]
    if bounding_boxes.shape[0] != confidences.shape[0]:
        raise ValueError("Bounding box 与 Confidence 的数量不一致")
    if bounding_boxes.shape[0] == 0:
        return []
    bounding_boxes, confidences = bounding_boxes.astype(np.float32), np.array(confidences)
    x1, y1, x2, y2 = bounding_boxes[:, 0], bounding_boxes[:, 1], bounding_boxes[:, 2], bounding_boxes[:, 3]
    areas = (x2 - x1 + 1) * (y2 - y1 + 1)
    idxs = np.argsort(confidences)

    pick = []
   	while len(idxs) > 0:
        # 因为idxs是从小到大排列的，last_idx相当于idxs最后一个位置的索引
        last_idx = len(idxs) - 1
        # 取出最大值在数组上的索引
        max_value_idx = idxs[last_idx]
        # 将这个添加到相应索引上
      	pick.append(max_value_idx)

        xx1 = np.maximum(x1[max_value_idx], x1[idxs[: last_idx]])
        yy1 = np.maximum(y1[max_value_idx], y1[idxs[: last_idx]])
        xx2 = np.minimum(x2[max_value_idx], x2[idxs[: last_idx]])
        yy2 = np.minimum(y2[max_value_idx], y2[idxs[: last_idx]])
        w, h = np.maximum(0, xx2 - xx1 + 1), np.maximum(0, yy2 - yy1 + 1)
        iou = w * h / areas[idxs[: last_idx]]

        # 删除最大的value,并且删除iou > threshold的bounding boxes
        idxs = np.delete(idxs, np.concatenate(([last_idx], np.where(iou > iou_thres)[0])))
    out = np.concatenate([box[pick], confidences[pick].reshape(-1, 1), clsid[pick].reshape(-1, 1)], axis=1)
    return out

def draw(img, xscale, yscale, pred, color=(255, 0, 0), tmp=True):
    img_ = img.copy()
    if len(pred):
        for detect in pred:
            caption = str('{:.2f}_{}'.format(detect[4], int(detect[5])))
            detect = [int((detect[0] - detect[2] / 2) * xscale), int((detect[1] - detect[3] / 2) * yscale),
                      int((detect[0] + detect[2] / 2) * xscale), int((detect[1] + detect[3] / 2) * yscale)]
            img_ = cv2.rectangle(img, (detect[0], detect[1]), (detect[2], detect[3]), color, 2)

            # 是否显示置信度类别
            if tmp:
                cv2.putText(img, caption, (detect[0], detect[1] - 5), 0, 1, color, 2, 16)
            
    return img_

class OpenvinoInference(object):
    def __init__(self, onnx_path):
        self.onnx_path = onnx_path
        ie = Core()
        self.model_onnx = ie.read_model(model=self.onnx_path)
        self.compiled_model_onnx = ie.compile_model(model=self.model_onnx, device_name="CPU")
        self.output_layer_onnx = self.compiled_model_onnx.output(0)

    def predict(self, datas):
        predict_data = self.compiled_model_onnx([datas])[self.output_layer_onnx]
        return predict_data

if __name__ == '__main__':
    
    height, width = 640, 640  # 修改1：图像resize大小
    conf, nms_iou = 0.25, 0.45  # 修改2：置信度阈值与nms的iou阈值
    openvino_tmp = True  # 修改3：是否进行openvino推理，False为onnxruntime推理

    onnx_path = 'D:\\C++\\yolov8s.onnx'  # 修改4：onnx文件路径
    input_path = 'D:\\C++\\bus.jpg'  # 修改5：原图路径
    output_path = 'D:\\C++\\out.jpg'  # 修改6：图像保存路径

    img = cv2.imread(input_path)
    
    if openvino_tmp:
        model = OpenvinoInference(onnx_path)
    else:
    	sess = rt.InferenceSession(onnx_path)

    t1 = time.time()
    data, scale, img_w, img_h = preprocess(img, height, width)  # resize_img
    print('预处理时间：{:.3f}s'.format(time.time() - t1))

	t2 = time.time()
    if openvino_tmp:
        pred = model.predict(data)
    else:
        input_name = sess.get_inputs()[0].name
        label_name = sess.get_outputs()[0].name
        pred = sess.run([label_name], {input_name: data.astype(np.float32)})[0]
    print('推理时间：{:.3f}s'.format(time.time() - t2))

	t3 = time.time()
    result = postprocess(pred, conf, nms_iou, img_w, img_h)
    print('后处理时间：{:.3f}s'.format(time.time() - t3))

    ret_img = draw(img, scale, scale, result, color=(0, 255, 0), tmp=True)
    cv2.imwrite(output_path, ret_img)

4.4 结果

在这里插入图片描述

具体时间消耗：

预处理时间：0.014s（预处理无Pad为0.007s）
推理时间：0.09~0.10s
后处理时间：0.001s
注：640×640下，Openvino和ONNXRuntime推理速度相差不大，1280×1280下，Openvino速度更快。

5 结合官方版本

5.1 全部代码

注：后处理输出结果维度：(5, 6)，其中第一个5表示图bus.jpg检出5个目标，第二个维度6表示(x1, y1, x2, y2, conf, cls)。
与YOLOv9输出维度一致，可通用！！！

import argparse
import time 
import cv2
import numpy as np
from openvino.runtime import Core  # pip install openvino -i  https://pypi.tuna.tsinghua.edu.cn/simple
import onnxruntime as ort  # 使用onnxruntime推理用上，pip install onnxruntime，默认安装CPU


# COCO默认的80类
CLASSES = ['person', 'bicycle', 'car', 'motorcycle', 'airplane', 'bus', 'train', 'truck', 'boat', 'traffic light',
            'fire hydrant', 'stop sign', 'parking meter', 'bench', 'bird', 'cat', 'dog', 'horse', 'sheep', 'cow',
              'elephant', 'bear', 'zebra', 'giraffe', 'backpack', 'umbrella', 'handbag', 'tie', 'suitcase', 'frisbee',
                'skis', 'snowboard', 'sports ball', 'kite', 'baseball bat', 'baseball glove', 'skateboard', 'surfboard',
                  'tennis racket', 'bottle', 'wine glass', 'cup', 'fork', 'knife', 'spoon', 'bowl', 'banana', 'apple', 'sandwich',
                    'orange', 'broccoli', 'carrot', 'hot dog', 'pizza', 'donut', 'cake', 'chair', 'couch', 'potted plant', 'bed',
                      'dining table', 'toilet', 'tv', 'laptop', 'mouse', 'remote', 'keyboard', 'cell phone', 'microwave', 'oven',
                        'toaster', 'sink', 'refrigerator', 'book', 'clock', 'vase', 'scissors', 'teddy bear', 'hair drier', 'toothbrush']


class OpenvinoInference(object):
    def __init__(self, onnx_path):
        self.onnx_path = onnx_path
        ie = Core()
        self.model_onnx = ie.read_model(model=self.onnx_path)
        self.compiled_model_onnx = ie.compile_model(model=self.model_onnx, device_name="CPU")
        self.output_layer_onnx = self.compiled_model_onnx.output(0)

    def predict(self, datas):
        predict_data = self.compiled_model_onnx([datas])[self.output_layer_onnx]
        return predict_data

class YOLOv8:
    """YOLOv8 object detection model class for handling inference and visualization."""

    def __init__(self, onnx_model, imgsz=(640, 640), infer_tool='openvino'):
        """
        Initialization.

        Args:
            onnx_model (str): Path to the ONNX model.
        """
        self.infer_tool = infer_tool
        if self.infer_tool == 'openvino':
            # 构建openvino推理引擎
            self.openvino = OpenvinoInference(onnx_model)
            self.ndtype = np.single
        else:
            # 构建onnxruntime推理引擎
            self.ort_session = ort.InferenceSession(onnx_model,
                                                providers=['CUDAExecutionProvider', 'CPUExecutionProvider']
                                                if ort.get_device() == 'GPU' else ['CPUExecutionProvider'])

            # Numpy dtype: support both FP32 and FP16 onnx model
            self.ndtype = np.half if self.ort_session.get_inputs()[0].type == 'tensor(float16)' else np.single
       
        self.classes = CLASSES  # 加载模型类别
        self.model_height, self.model_width = imgsz[0], imgsz[1]  # 图像resize大小
        self.color_palette = np.random.uniform(0, 255, size=(len(self.classes), 3))  # 为每个类别生成调色板

    def __call__(self, im0, conf_threshold=0.4, iou_threshold=0.45):
        """
        The whole pipeline: pre-process -> inference -> post-process.

        Args:
            im0 (Numpy.ndarray): original input image.
            conf_threshold (float): confidence threshold for filtering predictions.
            iou_threshold (float): iou threshold for NMS.

        Returns:
            boxes (List): list of bounding boxes.
        """
        # 前处理Pre-process
        t1 = time.time()
        im, ratio, (pad_w, pad_h) = self.preprocess(im0)
        print('预处理时间：{:.3f}s'.format(time.time() - t1))
        
        # 推理 inference
        t2 = time.time()
        if self.infer_tool == 'openvino':
            preds = self.openvino.predict(im)
        else:
            preds = self.ort_session.run(None, {self.ort_session.get_inputs()[0].name: im})[0]
        print('推理时间：{:.2f}s'.format(time.time() - t2))

        # 后处理Post-process
        t3 = time.time()
        boxes = self.postprocess(preds,
                                im0=im0,
                                ratio=ratio,
                                pad_w=pad_w,
                                pad_h=pad_h,
                                conf_threshold=conf_threshold,
                                iou_threshold=iou_threshold,
                                )
        print('后处理时间：{:.3f}s'.format(time.time() - t3))

        return boxes
        
    # 前处理，包括：resize, pad, HWC to CHW，BGR to RGB，归一化，增加维度CHW -> BCHW
    def preprocess(self, img):
        """
        Pre-processes the input image.

        Args:
            img (Numpy.ndarray): image about to be processed.

        Returns:
            img_process (Numpy.ndarray): image preprocessed for inference.
            ratio (tuple): width, height ratios in letterbox.
            pad_w (float): width padding in letterbox.
            pad_h (float): height padding in letterbox.
        """
        # Resize and pad input image using letterbox() (Borrowed from Ultralytics)
        shape = img.shape[:2]  # original image shape
        new_shape = (self.model_height, self.model_width)
        r = min(new_shape[0] / shape[0], new_shape[1] / shape[1])
        ratio = r, r
        new_unpad = int(round(shape[1] * r)), int(round(shape[0] * r))
        pad_w, pad_h = (new_shape[1] - new_unpad[0]) / 2, (new_shape[0] - new_unpad[1]) / 2  # wh padding
        if shape[::-1] != new_unpad:  # resize
            img = cv2.resize(img, new_unpad, interpolation=cv2.INTER_LINEAR)
        top, bottom = int(round(pad_h - 0.1)), int(round(pad_h + 0.1))
        left, right = int(round(pad_w - 0.1)), int(round(pad_w + 0.1))
        img = cv2.copyMakeBorder(img, top, bottom, left, right, cv2.BORDER_CONSTANT, value=(114, 114, 114))  # 填充

        # Transforms: HWC to CHW -> BGR to RGB -> div(255) -> contiguous -> add axis(optional)
        img = np.ascontiguousarray(np.einsum('HWC->CHW', img)[::-1], dtype=self.ndtype) / 255.0
        img_process = img[None] if len(img.shape) == 3 else img
        return img_process, ratio, (pad_w, pad_h)
    
    # 后处理，包括：阈值过滤与NMS
    def postprocess(self, preds, im0, ratio, pad_w, pad_h, conf_threshold, iou_threshold):
        """
        Post-process the prediction.

        Args:
            preds (Numpy.ndarray): predictions come from ort.session.run().
            im0 (Numpy.ndarray): [h, w, c] original input image.
            ratio (tuple): width, height ratios in letterbox.
            pad_w (float): width padding in letterbox.
            pad_h (float): height padding in letterbox.
            conf_threshold (float): conf threshold.
            iou_threshold (float): iou threshold.

        Returns:
            boxes (List): list of bounding boxes.
        """
        x = preds  # outputs: predictions (1, 84, 8400)
        # Transpose the first output: (Batch_size, xywh_conf_cls, Num_anchors) -> (Batch_size, Num_anchors, xywh_conf_cls)
        x = np.einsum('bcn->bnc', x)  # (1, 8400, 84)
   
        # Predictions filtering by conf-threshold
        x = x[np.amax(x[..., 4:], axis=-1) > conf_threshold]

        # Create a new matrix which merge these(box, score, cls) into one
        # For more details about `numpy.c_()`: https://numpy.org/doc/1.26/reference/generated/numpy.c_.html
        x = np.c_[x[..., :4], np.amax(x[..., 4:], axis=-1), np.argmax(x[..., 4:], axis=-1)]

        # NMS filtering
        # 经过NMS后的值, np.array([[x, y, w, h, conf, cls], ...]), shape=(-1, 4 + 1 + 1)
        x = x[cv2.dnn.NMSBoxes(x[:, :4], x[:, 4], conf_threshold, iou_threshold)]
       
        # 重新缩放边界框，为画图做准备
        if len(x) > 0:
            # Bounding boxes format change: cxcywh -> xyxy
            x[..., [0, 1]] -= x[..., [2, 3]] / 2
            x[..., [2, 3]] += x[..., [0, 1]]

            # Rescales bounding boxes from model shape(model_height, model_width) to the shape of original image
            x[..., :4] -= [pad_w, pad_h, pad_w, pad_h]
            x[..., :4] /= min(ratio)

            # Bounding boxes boundary clamp
            x[..., [0, 2]] = x[:, [0, 2]].clip(0, im0.shape[1])
            x[..., [1, 3]] = x[:, [1, 3]].clip(0, im0.shape[0])

            return x[..., :6]  # boxes
        else:
            return []

    # 绘框
    def draw_and_visualize(self, im, bboxes, vis=False, save=True):
        """
        Draw and visualize results.

        Args:
            im (np.ndarray): original image, shape [h, w, c].
            bboxes (numpy.ndarray): [n, 6], n is number of bboxes.
            vis (bool): imshow using OpenCV.
            save (bool): save image annotated.

        Returns:
            None
        """
        # Draw rectangles 
        for (*box, conf, cls_) in bboxes:
            # draw bbox rectangle
            cv2.rectangle(im, (int(box[0]), int(box[1])), (int(box[2]), int(box[3])),
                          self.color_palette[int(cls_)], 1, cv2.LINE_AA)
            cv2.putText(im, f'{self.classes[int(cls_)]}: {conf:.3f}', (int(box[0]), int(box[1] - 9)),
                        cv2.FONT_HERSHEY_SIMPLEX, 0.7, self.color_palette[int(cls_)], 2, cv2.LINE_AA)
    
        # Show image
        if vis:
            cv2.imshow('demo', im)
            cv2.waitKey(0)
            cv2.destroyAllWindows()

        # Save image
        if save:
            cv2.imwrite('demo.jpg', im)


if __name__ == '__main__':
    # Create an argument parser to handle command-line arguments
    parser = argparse.ArgumentParser()
    parser.add_argument('--model', type=str, default='yolov8s.onnx', help='Path to ONNX model')
    parser.add_argument('--source', type=str, default=str('bus.jpg'), help='Path to input image')
    parser.add_argument('--imgsz', type=tuple, default=(640, 640), help='Image input size')
    parser.add_argument('--conf', type=float, default=0.25, help='Confidence threshold')
    parser.add_argument('--iou', type=float, default=0.45, help='NMS IoU threshold')
    parser.add_argument('--infer_tool', type=str, default='openvino', choices=("openvino", "onnxruntime"), help='选择推理引擎')
    args = parser.parse_args()

    # Build model
    model = YOLOv8(args.model, args.imgsz, args.infer_tool)

    # Read image by OpenCV
    img = cv2.imread(args.source)

    # Inference
    boxes = model(img, conf_threshold=args.conf, iou_threshold=args.iou)

    # Visualize
    if len(boxes) > 0:
        model.draw_and_visualize(img, boxes, vis=False, save=True)

5.2 结果

在这里插入图片描述

具体时间消耗：

预处理时间：0.005s（包含Pad）
推理时间：0.09~0.10s
后处理时间：0.001s
注：640×640下，Openvino和ONNXRuntime推理速度相差不大，1280×1280下，Openvino速度更快。

YOLOv8s.onnx下载链接

你的陈某某

关注

22
点赞
踩
24

收藏

觉得还不错? 一键收藏
0
评论
YOLOv8-Openvino和ONNXRuntime推理【CPU】

Openvino是由Intel开发的专门用于优化和部署人工智能推理的半开源的工具包，主要用于对深度推理做优化。Openvino内部集成了Opencv、TensorFlow模块，除此之外它还具有强大的Plugin开发框架，允许开发者在Openvino之上对推理过程做优化。Openvino整体框架为：Openvino前端→ Plugin中间层→ Backend后端。
复制链接

扫一扫