机器学习:贝叶斯和优化方法_Facebook如何使用贝叶斯优化在机器学习模型中进行更好的实验

机器学习:贝叶斯和优化方法

重点 (Top highlight)

I recently started a new newsletter focus on AI education. TheSequence is a no-BS( meaning no hype, no news etc) AI-focused newsletter that takes 5 minutes to read. The goal is to keep you up to date with machine learning projects, research papers and concepts. Please give it a try by subscribing below:

我最近开始了一份有关AI教育的新时事通讯。 TheSequence是无BS(意味着没有炒作，没有新闻等)，它是专注于AI的新闻通讯，需要5分钟的阅读时间。 目标是让您了解机器学习项目，研究论文和概念的最新动态。 请通过以下订阅尝试一下：

Hyperparameter optimization is a key aspect of the lifecycle of machine learning applications. While methods such as grid search are incredibly effective for optimizing hyperparameters for specific isolated models, they are very difficult to scale across large permutations of models and experiments. A company like Facebook operates thousands of concurrent machine learning models that need to be constantly tuned. To achieve that, Facebook engineering teams need to regularly conduct A/B tests in order to determine the right hyperparameter configuration. Data in those tests is difficult to collect and they are typically conducted in isolation of each other which end up resulting in very computationally expensive exercises. Tools for rapid experimentation and hyper parameter optimization are rapidly emerging in the deep learning but its hard to imagine how to apply it at large scale. From the studies in the market, one of my favorites is a research paper published bya team of AI researchers from Facebook proposing a method based on Bayesian optimization to adaptively design rounds of A/B tests based on the results of prior tests.

超参数优化是机器学习应用程序生命周期的关键方面。 尽管诸如网格搜索之类的方法对于优化特定隔离模型的超参数非常有效，但它们很难在较大的模型和实验排列范围内扩展。 像Facebook这样的公司运营着成千上万个需要不断调整的并发机器学习模型。 为此，Facebook工程团队需要定期进行A / B测试，以确定正确的超参数配置。 这些测试中的数据很难收集，它们通常彼此隔离地进行，最终导致计算量很大。 用于快速实验和超参数优化的工具在深度学习中Swift兴起，但是很难想象如何将其大规模应用。 从市场上的研究中，我最喜欢的一项研究是由来自Facebook的