python实现贝叶斯优化_贝叶斯优化的并行实现

python实现贝叶斯优化

The concept of ‘optimization’ is central to data science. We minimize loss by optimizing weights in a neural network. We optimize hyper-parameters in our gradient boosted trees to find the best bias-variance trade-off. We use A-B testing to optimize behavior on our websites. Whether our function is a neural network, consumer behavior, or something more sinister, we all have something we want to optimize.

“优化”的概念对于数据科学至关重要。 通过优化神经网络中的权重，我们将损失降至最低。 我们在梯度增强树中优化超参数，以找到最佳的偏差方差折衷方案。 我们使用AB测试来优化我们网站上的行为。 无论我们的功能是神经网络，消费者行为还是更险恶的事物，我们都有一些我们要优化的东西。

Sometimes the functions we are trying to optimize are expensive, and we wish to get to our destination in as few steps as possible. Sometimes we want to be confident that we find the best possible solution, and sometimes our functions don’t have a tractable gradient, so there is no nice arrow to point us in the right direction. Often, our functions have random elements to them, so we are really trying to optimize f(x) = y + e, where e is some random error element. Bayesian optimization is a function optimizer (maximizer) which thrives in these conditions.

有时，我们尝试优化的功能非常昂贵，并且我们希望以尽可能少的步骤到达目的地。 有时我们希望找到可能的最佳解决方案，有时我们的函数没有可控制的梯度，因此没有很好的箭头指向正确的方向。 通常，我们的函数具有随机元素，因此我们实际上是在尝试优化f(x)= y + e，其中e是一些随机误差元素。 贝叶斯优化是在这些条件下蓬勃发展的函数优化器(最大化器)。

目录 (Table of Contents)

1. What Is Bayesian Optimization
   什么是贝叶斯优化
2. Implementing From Scratch
   从头开始实施
3. Implementing In Parallel
   并行实施
4. Final Words
   最后的话

什么是贝叶斯优化(What Is Bayesian Optimization)

Let’s say we have a function f, and we want to find the x which maximizes (or minimizes) f(x). We have many, many options. However, if our function fits the description right above the table of contents, we will definitely want to consider Bayesian optimization.

比方说，我们有一个函数f，我们要找到它最大化X(或最小化)F(X)。 我们有很多选择。 但是，如果我们的函数适合目录上方的描述，则我们肯定会考虑贝叶斯优化。

There are several different methods for performing Bayesian optimization. All of them involve creating an assumption about how certain things are distributed, making a decision based on that assumption, and then updating the assumption.

有几种不同的方法可以执行贝叶斯优化。 所有这些都涉及创建关于某些事物如何分布的假设，基于该假设做出决定，然后更新该假设。

The method in this article uses Gaussian processes to create an assumption about how f(x) is distributed. These processes can be thought of as a distribution of functions — where drawing a random sample from a Gaussian distribution results in a number, drawing a random sample from a Gaussian process results in a function. If you are not familiar with Gaussian processes, this is a little hard to picture. I recommend this video, which is what made the concept click for me.

本文中的方法使用高斯过程来创建关于f(x)分布的假设。 可以将这些过程视为函数的分布-从高斯分布中抽取随机样本会产生一个数，从高斯过程中抽取随机样本会产生一个函数。 如果您不熟悉高斯过程，这很难想象。 我推荐此视频，这正是使这个概念吸引我的原因。