g:profiler_Auptimizer推出的Profiler：为目标设备选择最佳的AI模型-无需部署

g:profiler

Profiler is a simulator for profiling the performance of Machine Learning (ML) model scripts. Profiler can be used during both the training and inference stages of the development pipeline. It is particularly useful for evaluating script performance and resource requirements for models and scripts being deployed to edge devices. Profiler is part of Auptimizer. You can get Profiler from the Auptimizer GitHub page or via pip install auptimizer.

Profiler 是用于对机器学习(ML)模型脚本的性能进行性能分析的模拟器。 Profiler可以在开发管道的训练和推断阶段中使用。 对于评估部署到边缘设备的模型和脚本的脚本性能和资源要求，此功能特别有用。 Profiler是 Auptimizer的 一部分 。 您可以从 Auptimizer GitHub页面 或通过pip install auptimizer 获取Profiler 。

为什么我们建立探查器 (Why we built Profiler)

The cost of training machine learning models in the cloud has dropped dramatically over the past few years. While this drop has pushed model development to the cloud, there are still important reasons for training, adapting, and deploying models to devices. Performance and security are the big two but cost-savings is also an important consideration as the cost of transferring and storing data, and building models for millions of devices tends to add up. Unsurprisingly, machine learning for edge devices or Edge AI as it is more commonly known continues to become mainstream even as cloud compute becomes cheaper.

在过去的几年中，在云中训练机器学习模型的成本已急剧下降。 尽管这种下降将模型开发推向了云端，但仍然有重要的原因需要训练，调整模型并将其部署到设备。 性能和安全性是主要的两个方面，但是节省成本也是一个重要的考虑因素，因为传输和存储数据的成本以及为数百万个设备构建模型的成本往往会增加。 毫不奇怪，即使云计算变得更便宜，针对边缘设备或Edge AI的机器学习也将继续成为主流，因为它众所周知。

Developing models for the edge opens up interesting problems for practitioners.

为边缘开发模型会为从业人员带来有趣的问题。

1. Model selection now involves taking into consideration the resource requirements of these models.

   模型选择现在涉及考虑这些模型的资源需求 。

2. The training-testing cycle becomes longer due to having a device in the loop because the model now needs to be deployed on the device to test its performance. This problem is only magnified when there are multiple target devices.
   由于设备处于循环中，因此训练测试周期变得更长，因为现在需要在设备上部署模型以测试其性能。 仅当存在多个目标设备时，此问题才会放大。

Currently, there are three ways to shorten the model selection/deployment cycle:

当前，有三种方法可以缩短模型选择/部署周期：

• The use of device-specific simulators that run on the development machine and preclude the need for deployment to the device. Caveat: Simulators are usually not generalizable across devices.

  使用在开发机器上运行的特定于设备的模拟器，从而无需部署到设备。 警告：模拟器通常无法在设备之间通用。

• The use of profilers that are native to the target device. Caveat: They need the model to be deployed to the target device for measurement.

  使用目标设备本地的探查器。 警告：他们需要将模型部署到目标设备进行测量。

• The use of measures like FLOPS or Multiply-Add (MAC) operations to give approximate measures of resource usage. Caveat: The model itself is only one (sometimes insignificant) part of the entire pipeline (which also includes data loading, augmentation, feature engineering, etc.)

  使用FLOPS或乘加(MAC)操作之类的度量来大致估算资源使用量。 注意：模型本身只是整个管道的一部分(有时不重要)(其中还包括数据加载，扩充，功能工程等)

In practice, if you want to pick a model that will run efficiently on your target devices but do not have access to a dedicated simulator, you have to test each model by deploying on all of the target devices.

实际上，如果您想选择一种可以在目标设备上有效运行但无法访问专用模拟器的模型，则必须通过在所有目标设备上进行部署来测试每种模型。

Profiler helps alleviate these issues. Profiler allows you to simulate, on your development machine, how your training or inference script will perform on a target device. With Profiler, you can understand CPU- and memory-usage as well as run-time for your model script on the target device.

Profiler有助于缓解这些问题。 Profiler允许您在开发计算机上模拟训练或推理脚本在目标设备上的执行方式。 使用Profiler，您可以了解目标设备上模型脚本的CPU和内存使用情况以及运行时间。

Profiler的工作方式 (How Profiler works)

Profiler encapsulates the model script, its requirements, and corresponding data into a Docker container. It uses user-inputs on compute-, memory-, and framework-constraints to build a corresponding Docker image so the script can run independently and without external dependencies. This image can then easily be scaled and ported to ease future development and deployment. As the model script is executed within the container, Profiler tracks and records various resource utilization statistics including Average CPU Utilization, Memory Usage, Network I/O, and Block I/O. The logger also supports setting the Sample Time to control how frequently Profiler samples utilization statistics from the Docker container.

Profiler将模型脚本，其要求以及相应的数据封装到Docker容器中。 它在计算，内存和框架约束上使用用户输入来构建相应的Docker映像，因此脚本可以独立运行，而无需外部依赖。 然后可以轻松缩放和移植此映像，以简化将来的开发和部署。 在容器中执行模型脚本时，Profiler跟踪并记录各种资源利用率统计信息，包括平均CPU利用率 ， 内存使用率 ， 网络I / O和块I / O。 记录器还支持设置采样时间，以控制Profiler从Docker容器中采样使用率统计信息的频率。

Get Profiler: Click here

获取分析器： 单击此处

Profiler如何提供帮助 (How Profiler helps)

Our results show that Profiler can help users build a good estimate of model runtime and memory usage for many popular image/video recognition models. We conducted over 300 experiments across a variety of models (InceptionV3, SqueezeNet, Resnet18, MobileNetV2–0.25x, -0.5x, -0.75x, -1.0x, 3D-SqueezeNet, 3D-ShuffleNetV2–0.25x, -0.5x, -1.0x, -1.5x, -2.0x, 3D-MobileNetV2–0.25x, -0.5x, -0.75x, -1.0x, -2.0x) on three different devices — LG G6 and Samsung S8 phones, and NVIDIA Jetson Nano. You can find the full set of experimental results and more information on how to conduct similar experiments on your devices here.

我们的结果表明，Profiler可以帮助用户为许多流行的图像/视频识别模型建立模型运行时和内存使用的良好估计。 我们在各种模型(InceptionV3，SqueezeNet，Resnet18，MobileNetV2-0.25x，-0.5x，-0.75x，-1.0x，3D-SqueezeNet，3D-ShuffleNetV2-0.25x，-0.5x，-在三种不同的设备(LG G6和Samsung S8手机以及NVIDIA Jetson Nano)上的1.0x，-1.5x，-2.0x，3D-MobileNetV2-0.25x，-0.5x，-0.75x，-1.0x，-2.0x) 。 您可以在此处找到全套实验结果，以及有关如何在设备上进行类似实验的更多信息。

The addition of Profiler brings Auptimizer closer to the vision of a tool that helps machine learning scientists and engineers build models for edge devices. The hyperparameter optimization (HPO) capabilities of Auptimizer help speed up model discovery. Profiler helps with choosing the right model for deployment. It is particularly useful in the following two scenarios:

Profiler的添加使Auptimizer更加接近工具的愿景，该工具可帮助机器学习科学家和工程师为边缘设备建立模型。 Auptimizer的超参数优化(HPO)功能有助于加速模型发现。 Profiler有助于选择正确的部署模型。 它在以下两种情况下特别有用：

1. Deciding between models — The ranking of the run-times and memory usages of the model scripts measured using Profiler on the development machine is indicative of their ranking on the target device. For instance, if Model1 is faster than Model2 when measured using Profiler on the development machine, Model1 will be faster than Model2 on the device. This ranking is valid only when the CPU’s are running at full utilization.
   在模型之间做出决定-在开发机器上使用Profiler测量的模型脚本的运行时和内存使用情况的等级表示它们在目标设备上的等级。 例如，如果在开发机器上使用Profiler进行测量时Model1比Model2快，则Model1将比设备上的Model2快。 该排名仅在CPU处于满负荷运行时才有效。
2. Predicting model script performance on the device — A simple linear relationship relates the run-times and memory usage measured using Profiler on the development machine with the usage measured using a native profiling tool on the target device. In other words, if a model runs in time x when measured using Profiler, it will run approximately in time (a*x+b) on the target device (where a and b can be discovered by profiling a few models on the device with a native profiling tool). The strength of this relationship depends on the architectural similarity between the models but, in general, the models designed for the same task are architecturally similar as they are composed of the same set of layers. This makes Profiler a useful tool for selecting the best suited model.
   预测设备上的模型脚本性能-一个简单的线性关系将使用开发机上的Profiler测量的运行时间和内存使用量与使用目标设备上的本机性能分析工具测量的使用量相关联。 换句话说，如果模型在使用Profiler进行测量时在时间x中运行，它将在目标设备上大约在时间(a * x + b)中运行(其中a和b可以通过使用本机性能分析工具)。 这种关系的强度取决于模型之间的架构相似性，但是通常来说，为同一任务设计的模型在架构上相似，因为它们由相同的图层集组成。 这使Profiler成为选择最合适模型的有用工具。

期待 (Looking forward)

Profiler continues to evolve. So far, we have tested its efficacy on select mobile- and edge-platforms for running popular image and video recognition models for inference, but there is much more to explore. Profiler might have limitations for certain models or devices and can potentially result in inconsistencies between Profiler outputs and on-device measurements. Our experiment page provides more information on how to best set up your experiment using Profiler and how to interpret potential inconsistencies in results. The exact use case varies from user to user but we believe that Profiler is relevant to anyone deploying models on devices. We hope that Profiler’s estimation capability can enable leaner and faster model development for resource-constrained devices. We’d love to hear (via github) if you use Profiler during deployment.

Profiler不断发展。 到目前为止，我们已经在选择的移动平台和边缘平台上测试了其在运行流行的图像和视频识别模型以进行推理方面的功效，但是还有很多事情要做。 Profiler可能对某些型号或设备有所限制，并有可能导致Profiler输出与设备上的测量结果不一致。 我们的实验页面提供了有关如何使用Profiler最佳设置实验以及如何解释结果中潜在不一致之处的更多信息。 确切的用例因用户而异，但我们认为Profiler与在设备上部署模型的任何人有关。 我们希望Profiler的估算功能能够为资源受限的设备实现更精简，更快的模型开发。 如果您在部署期间使用Profiler，我们很乐意(通过github)。

Authors: Samarth Tripathi, Junyao Guo, Vera Serdiukova, Unmesh Kurup, and Mohak Shah — Advanced AI, LG Electronics USA

作者：萨玛斯·特里帕蒂(Samarth Tripathi)，郭俊尧，薇拉·塞尔迪乌科娃(Vera Serdiukova)，Unmesh Kurup和Mohak Shah — LG电子美国公司的高级AI

翻译自: https://towardsdatascience.com/introducing-profiler-select-the-best-ai-model-for-your-target-device-no-deployment-required-91a396c4bfeb

g:profiler