Hyperopt工具包 贝叶斯优化

Hyperopt工具包

一. Bayesian Optimization 贝叶斯优化四大步骤

1. Objective 目标函数
2. Domain space 指定参数空间
3. Hyperparameter optimizationfunction 可选择的采样算法,随机或者贝叶斯优化
4. Trials 记录结果的保存

1. Objective 自定义目标, 这里得有一个最小值

import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
import seaborn as sns
%matplotlib inline

def objective(x):
    f = np.poly1d([1, -3, -28, 29, 12, -22, 99]) 
    # np.poly1d生成多项式, 如np.poly1d([2,3,4])生成2x^2 + 3x + 4
    return f(x) * 0.05

x = np.linspace(-5, 7, 10000)
y = objective(x)

miny = min(y)
minx = x[np.argmin(y)]

plt.figure(figsize = (8, 6))
plt.style.use('fivethirtyeight')
plt.title('Objective Function')
plt.xlabel('x')
plt.ylabel('f(x)')
plt.vlines(minx, min(y)-50, max(y), linestyles='--', colors='r')
plt.plot(x, y)
print('Minimum of %0.4f occurs at %0.4f' % (miny, minx))

Minimum of -394.7324 occurs at 5.4266

2. Domain 参数取值范围

from hyperopt import hp
space = hp.uniform('x', -5, 7)

from hyperopt.pyll.stochastic import sample
samples = []
for _ in range(10000):
    samples.append(sample(space))

plt.hist(samples, bins = 20, edgecolor = 'black')
plt.xlabel('x'); plt.ylabel('Frequency'); plt.title('Domain Space')

3. 选择采样算法

Hyperparameter Optimization Algorithm 随机或者贝叶斯优化

from hyperopt import rand, tpe
tpe_algo = tpe.sugge