本文是tensorflow2.0版本下对tf.keras实现超参数搜索的代码笔记,使用sklearn对keras.model进行封装。代码实现是在jupyter notebook上,首先导入一些需要的库:
import matplotlib as mpl
import matplotlib.pyplot as plt
%matplotlib inline
import numpy as np
import sklearn
import pandas as pd
import os
import sys
import time
import tensorflow as tf
from tensorflow import keras
print(tf.__version__)
print(sys.version_info)
for module in mpl, np, pd, sklearn, tf, keras:
print(module.__name__, module.__version__)
1.10.0
sys.version_info(major=3, minor=6, micro=5, releaselevel='final', serial=0)
matplotlib 3.0.2
numpy 1.14.5
pandas 0.23.4
sklearn 0.21.2
tensorflow 1.10.0
tensorflow.keras 2.1.6-tf
从sklearn中导入加州房价数据:
from sklearn.datasets import fetch_california_housing
housing = fetch_california_housing()
# print(housing.DESCR)
# print(housing.data.shape)
# print(housing.target.shape)
对数据进行分割:
from sklearn.model_selection import train_test_split
x_train_all, x_test, y_train_all, y_test = train_test_split(
housing.data, housing.target, random_state = 7)
x_train, x_valid, y_train, y_valid = train_test_split(
x_train_all, y_train_all, random_state = 11)
print(x_train.shape, y_train.shape)
print(x_valid.shape, y_valid.shape)
print(x_test.shape, y_test.shape)
数据进行标准化处理,训练数据需要fit.trainform(),而验证与测试数据只需trainform():
from sklearn.preprocessing import StandardScaler
scaler = StandardScaler()
x_train_scaled = scaler.fit_transform(x_train)
x_valid_scaled = scaler.transform(x_valid)
x_test_scaled = scaler.transform(x_test)
将tf.keras中的model转化为sklearn_model,采用EarlyStoping回调函数在设定的最优值下终止训练,这里是简单的回归问题,使用tensorflow中keras.wrappers.scikit_learn.KerasRegressor()进行sklearn_model()创建:
def build_model(hidden_layers = 1,
layer_size = 30,
learning_rate = 3e-3):
model = keras.models.Sequential()
model.add(keras.layers.Dense(layer_size,activation='relu',
input_shape = x_train.shape[1:]))
for _ in range(hidden_layers-1):
model.add(keras.layers.Dense(layer_size,
activation = 'relu'))
model.add(keras.layers.Dense(1))
optimizer = keras.optimizers.SGD(learning_rate)
model.compile(loss = 'mse',optimizer = optimizer)
return model
sklearn_model = keras.wrappers.scikit_learn.KerasRegressor(build_model)
callbacks = [keras.callbacks.EarlyStopping(
patience=5, min_delta=1e-2)]
history = sklearn_model.fit(x_train_scaled,y_train,epochs = 100,
validation_data = (x_valid_scaled, y_valid),
callbacks = callbacks)
对训练损失进行可视化:
def plot_learning_curves(history):
pd.DataFrame(history.history).plot(figsize=(8, 5))
plt.grid(True)
plt.gca().set_ylim(0, 1)
plt.show()
plot_learning_curves(history)
****接下来进行sklearn超参数搜索,定义参数集合,实现sklearn对超参数搜索:
from scipy.stats import reciprocal
#reciprocal分布函数表达式f(x) = 1/(x*log(b/a)) a<=x<=b
param_distribution = {
"hidden_layers":[1,2,3,4],
"layer_size":np.arange(1,100),
"learning_rate":reciprocal(1e-4,1e-2),
}
from sklearn.model_selection import RandomizedSearchCV
random_search_cv = RandomizedSearchCV(sklearn_model,
param_distribution,
n_iter = 10,
n_jobs = 1)
random_search_cv.fit(x_train_scaled,y_train,epochs = 100,
validation_data = (x_valid_scaled,y_valid),
callbacks = callbacks)
可以对reciprocal分布参数进行查看,并输出训练后的最优参数:
from scipy.stats import reciprocal
reciprocal.rvs(1e-4,1e-2,size=10)
print(random_search_cv.best_params_)
print(random_search_cv.best_score_)
print(random_search_cv.best_estimator_)
最后进行对最优model的测试:
model = random_search_cv.best_estimator_.model
model.evaluate(x_test_scaled,y_test)