Ray.tune官方文档
调整超参数通常是机器学习工作流程中最昂贵的部分。 Tune专为解决此问题而设计,展示了针对此痛点的有效且可扩展的解决方案。 请注意,此示例取决于Tensorflow 2.0。
Code: ray/python/ray/tune at master · ray-project/ray · GitHub
Examples: https://github.com/ray-project/ray/tree/master/python/ray/tune/examples)
Documentation: Tune: Scalable Hyperparameter Tuning — Ray v1.6.0
Mailing List: https://groups.google.com/forum/#!forum/ray-dev
## If you are running on Google Colab, uncomment below to install the necessary dependencies
## before beginning the exercise.
# print("Setting up colab environment")
# !pip uninstall -y -q pyarrow
# !pip install -q https://s3-us-west-2.amazonaws.com/ray-wheels/latest/ray-0.8.0.dev5-cp36-cp36m-manylinux1_x86_64.whl
# !pip install -q ray[debug]
# # A hack to force the runtime to restart, needed to include the above dependencies.
# print("Done installing! Restarting via forced crash (this is not an issue).")
# import os
# os._exit(0)
## If you are running on Google Colab, please install TensorFlow 2.0 by uncommenting below..
# try:
# # %tensorflow_version only exists in Colab.
# %tensorflow_version 2.x
# except Exception:
# pass
本教程将逐步介绍使用Tune进行超参数调整的几个关键步骤。
- 可视化数据。
- 创建模型训练过程(使用Keras)。
- 通过调整上述模型训练过程以使用Tune来调整模型。
- 分析Tune创建的模型。
请注意,这使用了Tune的基于函数的API。 这主要是用于原型制作。 后面的教程将介绍Tune更加强大的基于类的可训练 API。
import numpy as np
np.random.seed(0)
import tensorflow as tf
try:
tf.get_logger().setLevel('INFO')
except Exception as exc:
print(exc)
import warnings
warnings.simplefilter("ignore")
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Dense
from tensorflow.keras.optimizers import SGD, Adam
from tensorflow.keras.callbacks import ModelCheckpoint
import ray
from ray import tune
from ray.tune.examples.utils import get_iris_data
import inspect
import pandas as pd
import matplotlib.pyplot as plt
plt.style.use('ggplot')
%matplotlib inline
Visualize your data
首先让我们看一下数据集的分布。
鸢尾花数据集由3种不同类型的鸢尾花(Setosa,Versicolour和Virginica)的花瓣和萼片长度组成,存储在150x4 numpy中。
行为样本,列为:隔片长度,隔片宽度,花瓣长度和花瓣宽度。
本教程的目标是提供一个模型,该模型可以准确地预测给定的萼片长度,萼片宽度,花瓣长度和花瓣宽度4元组的真实标签。
from sklearn.datasets import load_iris
iris = load_iris()
true_data = iris['data']
true_label = iris['target']
names = iris['target_names']
feature_names = iris['feature_names']
def plot_data(X, y):
# Visualize the data sets
plt.figure(figsize=(16, 6))
plt.subplot(1, 2, 1)
for target, target_name in enumerate(names):
X_plot = X[y == target]
plt.plot(X_plot[:, 0], X_plot[:, 1], linestyle='none', marker='o', label=target_name)
plt.xlabel(feature_names[0])
plt.ylabel(feature_names[1])
plt.axis('equal')
plt.legend();
plt.subplot(1, 2, 2)
for target, target_name in enumerate(names):
X_plot = X[y == target]
plt.plot(X_plot[:, 2], X_plot[:, 3], linestyle='none', marker='o', label=target_name)
plt.xlabel(feature_names[2])
plt.ylabel(feature_names[3])
plt.axis('equal')
plt.legend();
plot_data(true_data, true_label)
创建模型训练过程(使用Keras)
现在,让我们定义一个函数,该函数将包含一些超参数并返回一个可用于训练的模型。
def create_model(learning_rate, dense_1, dense_2):
assert learning_rate > 0 and dense_1 > 0 and dense_2 > 0, "Did you set the right configuration?"
model = Sequential()
model.add(Dense(int(dense_1), input_shape=(4,), activation='relu', name='fc1'))
model.add(Dense(int(dense_2), activation='relu', name='fc2'))
model.add(Dense(3, activation='softmax', name='output'))
optimizer = SGD(lr=learning_rate)
model.compile(optimizer, loss='categorical_crossentropy', metrics=['accuracy'])
return model
下面是一个使用create_model函数训练模型并返回训练后的模型的函数。
def train_on_iris():
train_x, train_y, test_x, test_y = get_iris_data()
model = create_model(learning_rate=0.1, dense_1=2, dense_2=2)
# This saves the top model. `acc