边缘计算边缘计算edge_Edge AI-边缘上的计算机视觉推理

边缘计算边缘计算edge

Carsten Moenning, Ph.D.

卡斯滕·莫宁(Carsten Moenning)博士

This is the final part of a two-part series on running AI applications on the edge in general and Computer Vision inference on an Intel Neural Compute Stick 2-accelerated Raspberry Pi 3, in particular.

Ť他是在对英特尔的神经计算棒2加速树莓派3一般与计算机视觉推理边缘延伸AI的应用，特别是两部分系列的最后一部分。

Have a look at part 1 to obtain:

看一下第1部分 ，获得：

1. an overview of edge AI hardware accelerator and development board options,
   边缘AI硬件加速器和开发板选项的概述，
2. a guide as to how to configure a Raspberry Pi with an on-board camera as an edge AI-ready device,
   有关如何使用车载摄像头将Raspberry Pi配置为边缘AI就绪设备的指南，
3. a guide as to how to install the Intel OpenVINO toolkit on the Raspberry Pi and, for example, your laptop, to get ready for performing Computer Vision inference on the Raspberry Pi.
   有关如何在Raspberry Pi上(例如您的笔记本电脑)安装Intel OpenVINO工具包的指南，以准备在Raspberry Pi上执行计算机视觉推理。

This second part builds on steps 2 and 3. Whilst part 1 finished with an OpenVINO face detection demo application running on a Raspberry Pi, this post deals with developing your very own Computer Vision custom solution on such an edge device. More specifically, I will walk you through the custom implementation of a YOLOv3-tiny object detection model on a Raspberry Pi pre-configured as described in part 1. (Details on the YOLOv3 object detection model can be found here.) But, first, let’s have a closer look at the open source Intel OpenVINO toolkit in general and the OpenVINO toolkit for Raspbian OS, in particular. 🗹

第二部分基于第2步和第3步。第1部分完成了在Raspberry Pi上运行的OpenVINO人脸检测演示应用程序，而本文则涉及在这种边缘设备上开发自己的Computer Vision定制解决方案。 更具体地说，我将引导您完成在第1部分中所述预先配置的Raspberry Pi上的YOLOv3-tiny对象检测模型的自定义实现。(有关YOLOv3对象检测模型的详细信息，请参见此处 。)但是，首先，让我们仔细了解一下开放源代码的Intel OpenVINO工具包 ，特别是Raspbian OS的OpenVINO工具包 。 🗹

The Intel OpenVINO toolkit

英特尔OpenVINO工具包

To quote the official OpenVINO toolkit documentation: “The OpenVINO™ toolkit is a comprehensive toolkit for quickly developing applications and solutions that emulate human vision. Based on Convolutional Neural Networks (CNNs), the toolkit extends CV workloads across Intel® hardware, maximizing performance.”

引用官方的OpenVINO工具包文档：“ OpenVINO™工具包是一种综合工具包，用于快速开发可模拟人类视觉的应用程序和解决方案。 该工具包基于卷积神经网络(CNN)，可在英特尔®硬件上扩展CV工作负载，从而最大限度地提高性能。”

OpenVINO) OpenVINO )

To quote further, the toolkit:

进一步引用该工具包：

• “enables CNN-based deep learning inference on the edge
  “在边缘启用基于CNN的深度学习推理
• supports heterogeneous execution across an Intel® CPU, Intel® Integrated Graphics, Intel® FPGA, Intel® Neural Compute Stick 2 and Intel® Vision Accelerator Design with Intel® Movidius™ VPUs
  支持通过英特尔®Movidius™VPU在英特尔®CPU，英特尔®集成显卡，英特尔®FPGA，英特尔®神经计算棒2和英特尔®视觉加速器设计之间进行异构执行
• speeds time-to-market via an easy-to-use library of computer vision functions and pre-optimized kernels
  通过易于使用的计算机视觉功能和预优化内核库，加快了产品上市时间
• includes optimized calls for computer vision standards, including OpenCV and OpenCL™.”
  包括对计算机视觉标准(包括OpenCV和OpenCL™)的优化呼吁。”

Amongst other things, this implies that the toolkit is particularly well-suited for the development of Computer Vision inference applications on edge devices such as a Raspberry Pi enhanced by an Intel Neural Compute Stick 2 USB accelerator stick featuring the Intel Movidius VPU.

除其他外，这意味着该工具包特别适合在边缘设备(例如通过具有Intel Movidius VPU的Intel Neural Compute Stick 2 USB加速棒增强的Raspberry Pi)上开发计算机视觉推理应用程序。

⚡ In case you might be wondering: “OpenVINO” stands for “Open Visual Inference and Neural Network Optimization”, indicating its Deep Learning Computer Vision focus. ⚡

⚡也许您会感到奇怪：“ OpenVINO”代表“开放式视觉推理和神经网络优化”，表示其深度学习计算机视觉的重点。 ⚡

The main elements of the OpenVINO toolkit consist of:

OpenVINO工具包的主要元素包括：

• the Model Optimizer,
  模型优化器
• the Inference Engine and
  推理引擎和
• the Open Model Zoo.
  开放模型动物园。

The Model Optimizer takes Deep Learning Computer Vision models pre-trained in the Caffe, TensorFlow, MXNet, Kaldi or ONNX frameworks and converts them into the OpenVINO Intermediate Representation (IR), a streamlined model representation optimised for model execution using the Inference Engine running on an edge device. The Inference Engine loads and infers these IR files using a common API for both CPU and GPU as well as VPU hardware.

模型优化器采用在Caffe，TensorFlow，MXNet，Kaldi或ONNX框架中预先训练的深度学习计算机视觉模型，并将其转换为OpenVINO中间表示(IR)，这是一种简化的模型表示，可使用运行在以下平台上的推理引擎进行模型执行优化。边缘设备。 推理引擎使用针对CPU和GPU以及VPU硬件的通用API加载和推断这些IR文件。

The Open Model Zoo provides pre-trained models for free download that can be used to speed up the development process by not having to train one’s own model first, which would then need to be converted into the Intermediate Representation using the Model Optimizer next before being ready for use with the Inference Engine on the edge device under consideration.

Open Model Zoo提供了免费的预训练模型，可以免费下载，无需先训练自己的模型即可用于加速开发过程，然后需要先使用Model Optimizer将其转换为中间表示形式，然后再进行建模准备与正在考虑的边缘设备上的推理引擎一起使用。

When not using a pre-trained model from the Open Model Zoo, the typical workflow for deploying a deep learning model on the edge using the OpenVINO toolkit is shown below.

当不使用Open Model Zoo中的预训练模型时，使用OpenVINO工具包在边缘部署深度学习模型的典型工作流程如下所示。

OpenVINO) OpenVINO )

So, Open Model Zoo models come with the advantage of already having been converted into the OpenVINO Intermediate Representation, but what is the Intermediate Representation?

因此，Open Model Zoo模型的优点是已经转换为OpenVINO中间表示，但是什么是中间表示？

The IR describes the result of an input model, which has been optimised for edge inference with the help of one or multiple of the following techniques:

IR描述了输入模型的结果，借助以下一种或多种技术，该模型已针对边沿推理进行了优化：

• quantization, i.e. the reduction in numerical precision of the model weights and bias,
  量化，即模型权重和偏差的数值精度降低，
• model layer fusion, i.e. the combination of multiple model layers into one,
  模型层融合，即将多个模型层组合成一个层，
• freezing, i.e. the removal of metadata and operations which were only useful for model training (in the case of TensorFlow models only).
  冻结，即删除仅对模型训练有用的元数据和操作(仅在TensorFlow模型的情况下)。

You are also perfectly free to use models from other sources, of course. These will have to go through step two of the OpenVINO workflow, though. That is, you will have to apply the Mode