MLPerf Benchmarks - MLPerf 基准测试

Yongqiang Cheng

已于 2022-10-22 10:53:11 修改

阅读量2.4k

点赞数 2

分类专栏： Refactoring 文章标签： MLPerf Benchmark 基准测试

于 2022-10-17 23:12:43 首次发布

世上没有白读的书，每一页都算数。

本文链接：https://blog.csdn.net/chengyq116/article/details/127377979

版权

Refactoring 专栏收录该内容

38 篇文章

订阅专栏

MLPerf Benchmarks - MLPerf 基准测试

Machine Learning Performance，MLPerf

1. MLCommons

https://mlcommons.org/en/
https://github.com/mlcommons

MLCommons aims to accelerate machine learning innovation to benefit everyone. Machine learning has tremendous potential to save lives in areas like healthcare and automotive safety and to improve information access and understanding through technologies like voice interfaces, automatic translation, and natural language processing. However, machine learning is completely unlike conventional software - developers train an application rather than program it - and requires a whole new set of techniques analogous to the breakthroughs in precision measurement, raw materials, and manufacturing that drove the industrial revolution.
MLCommons 旨在促进机器学习创新，以此惠及普罗大众。机器学习技术拥有巨大的潜力：它能在健康医疗和自动驾驶安全领域拯救生命，也能通过语音互动、自动翻译和自然语言处理等技术来提高人们获取信息和理解信息的能力。然而，机器学习和传统的软件有根本性的区别 - 开发者们要训练这个应用，而不是为它编写程序命令。因此，机器学习需要一套全新的技术来推动，这就好比过去的工业革命首先要有精密测量、原材料和制造方面的重大突破来支持。

tremendous [trəˈmendəs]：adj.	巨大的，极大的，极好的，精彩的，了不起的，非常的，惊人的，可怕的
analogous [əˈnæləɡəs]：adj. 相似的，类似的

MLPerf is a consortium of AI leaders from academia, research labs, and industry whose mission is to “build fair and useful benchmarks” that provide unbiased evaluations of training and inference performance for hardware, software, and services - all conducted under prescribed conditions. To stay on the cutting edge of industry trends, MLPerf continues to evolve, holding new tests at regular intervals and adding new workloads that represent the state of the art in AI.
MLPerf 是由来自学术界、研究实验室和相关行业的 AI 领导者组成的联盟，旨在“构建公平和有用的基准测试”，在规定的条件下，针对硬件、软件和服务的训练和推理性能提供公平的评估。为确保始终站在行业趋势的前沿，MLPerf 不断演进，定期进行新的测试并添加代表 AI 领域当前水平的新工作负载。

nascent [ˈnæsnt]：adj. 新生的，萌芽的，未成熟的
consortium [kənˈsɔːtiəm]：n. 联盟，财团，银团，联营企业
academia [ˌækəˈdiːmiə]：n. 学术界
prescribe [prɪˈskraɪb]：vt. 开药方，规定，给 ... 开，让 ... 采用，命令，指示
training and inference：训练和推理

MLPerf Training v2.0 is the sixth instantiation for training and consists of eight different workloads covering a broad diversity of use cases, including vision, language, recommenders, and reinforcement learning.
MLPerf Training v2.0 是第六个训练版本，由八个不同的工作负载组成，涵盖视觉、语言、推荐系统和强化学习等各种用例。

MLPerf Inference v2.1 is the sixth instantiation for inference and tested six different use cases across six different kinds of neural networks. Two of these use cases were for computer vision, one for recommender systems, two for language processing, and one for medical imaging.
MLPerf Inference v2.1 在六个不同种类的神经网络中测试了六个不同的用例。其中二个用例针对计算机视觉，一个用例针对推荐系统，两个用例针对语言处理，还有一个用例针对医学影像。

MLCommons aims to answer the needs of the nascent machine learning industry through open, collaborative engineering in three areas: Benchmarks, Datasets and Best Practices.

在这里插入图片描述

1.1 Benchmarks - Inference: Mobile

September 28, 2022 - Inference: Mobile
v2.1 Results: https://mlcommons.org/en/inference-mobile-21/
April 06, 2022 - Inference: Mobile
v2.0 Results: https://mlcommons.org/en/inference-mobile-20/

1.2 MLPerf™ Mobile App

https://github.com/mlcommons/mobile_app_open

This project contains the MLPerf mobile app, an app-based implementation of MLPerf Inference tasks.

Folder Structure

flutter - Contains the Flutter (cross-platform) version of the app
mobile_back_qti - QTI backend for Android
mobile_back_samsung - Samsung backend for Android
mobile_back_pixel - Google Pixel backend for Android
mobile_back_tflite - Combined TFLite / MediaTek backends for Android and TFLite backend for iOS
mobile_back_apple - Apple backend (Core ML) for iOS
datasets - Contains scripts to prepare test and calibration data used for accuracy evaluation and model quantization
docs - Contains documentation
tools - Contains miscellaneous tools (e.g. formatting commands, code scanner)