YOLOv8在PyTorch、ONNXRuntime、OpenVINO框架下的推理性能对比

简介

YOLOv8是一个广泛使用的目标检测、图像分割工具，笔者在机器人感知任务中也曾有所应用。目前深度学习的部署框架也有很多，各有优势，这里针对三个常用的部署框架：PyTorch、ONNXRuntime、OpenVINO对YOLOv8的推理进行一个资源占用和推理效率的比较。

依赖

PyTorch2.0、ONNXRuntime、OpenVINO
笔者的算力较低：NVIDIA GeForce MX250，2G显存
YOLOv8项目直接提供PyTorch、ONNXRuntime、OpenVINO推理引擎文件

代码

PyTorch推理

import time
import os
from ultralytics import YOLO

# Load a pretrained YOLOv8n model
model = YOLO('/home/lz/yolov5/runs/train/best_20230921switch.pt')

inference_time = []

# Define path to the image file
source_dir = '/home/lz/yolov5/data/robot_arm_vision/fps_test'
sources = [f for f in os.listdir(source_dir)]

for source in sources:
    image = os.path.join(source_dir, source)
    start_time = time.time()
    results = model(image, imgsz=1280)  # list of Results objects
    end_time = time.time()
    inference_time.append(end_time-start_time)
average_inference_time = sum(inference_time) / len(inference_time)
print(f"Inference time is {
     average_inference_time} seconds")
fps = 1.0 / average_inference_time
print