Keras Tuner 超参数优化

代码

# -*- coding: utf-8 -*-
"""
Created on  2020/11/20 14:04
@Author: CY
@email: 5844104706@qq.com
"""
# Hyperparameters
## Model hyperparameters
## Algorithm hyperparameters
import tensorflow as tf
from tensorflow import keras

import IPython
# !pip install -q -U keras-tuner
import kerastuner as kt

print("#1. 数据集 Fashion MNIST dataset.")
(img_train, label_train), (img_test, label_test) = keras.datasets.fashion_mnist.load_data()
# Normalize pixel values between 0 and 1
img_train = img_train.astype('float32') / 255.0
img_test = img_test.astype('float32') / 255.0

print("#2. 模型定义")


def model_builder(hp):
    model = keras.Sequential()
    model.add(keras.layers.Flatten(input_shape=(28, 28)))

    # Tune the number of units in the first Dense layer
    # Choose an optimal value between 32-512
    hp_units = hp.Int('units', min_value=32, max_value=512, step=32)
    model.add(keras.layers.Dense(units=hp_units, activation='relu'))
    model.add(keras.layers.Dense(10))

    # Tune the learning rate for the optimizer
    # Choose an optimal value from 0.01, 0.001, or 0.0001
    hp_learning_rate = hp.Choice('learning_rate', values=[1e-2, 1e-3, 1e-4])

    model.compile(optimizer=keras.optimizers.Adam(learning_rate=hp_learning_rate),
                  loss=keras.losses.SparseCategoricalCrossentropy(from_logits=True),
                  metrics=['accuracy'])

    return model


tuner = kt.Hyperband(model_builder,
                     objective='val_accuracy',
                     max_epochs=10,
                     factor=3,
                     directory='my_dir',
                     project_name='intro_to_kt')
class ClearTrainingOutput(tf.keras.callbacks.Callback):
    def on_train_end(*args, **kwargs):
        IPython.display.clear_output(wait= True)

tuner.search(img_train, label_train, epochs = 10, validation_data = (img_test, label_test), callbacks = [ClearTrainingOutput()])

# Get the optimal hyperparameters
best_hps = tuner.get_best_hyperparameters(num_trials = 1)[0]

print(f"""
The hyperparameter search is complete. The optimal number of units in the first densely-connected
layer is {best_hps.get('units')} and the optimal learning rate for the optimizer
is {best_hps.get('learning_rate')}.
""")

# Build the model with the optimal hyperparameters and train it on the data
model = tuner.hypermodel.build(best_hps)
model.fit(img_train, label_train, epochs = 10, validation_data = (img_test, label_test))

运行结果

Epoch 1/2
1875/1875 [==============================] - 4s 2ms/step - loss: 0.6146 - accuracy: 0.7994 - val_loss: 0.4803 - val_accuracy: 0.8346
Epoch 2/2
1875/1875 [==============================] - 4s 2ms/step - loss: 0.4323 - accuracy: 0.8503 - val_loss: 0.4366 - val_accuracy: 0.8492
[Trial complete]
[Trial summary]
 |-Trial ID: 191820c4c1db0f9e0887d266e14cbd00
 |-Score: 0.8492000102996826
 |-Best step: 0
 > Hyperparameters:
 |-learning_rate: 0.0001
 |-tuner/bracket: 2
 |-tuner/epochs: 2
 |-tuner/initial_epoch: 0
 |-tuner/round: 0
 |-units: 384
Epoch 1/2
1875/1875 [==============================] - 2s 875us/step - loss: 0.7257 - accuracy: 0.7698 - val_loss: 0.5402 - val_accuracy: 0.8172
Epoch 2/2
1875/1875 [==============================] - 2s 942us/step - loss: 0.4874 - accuracy: 0.8370 - val_loss: 0.4871 - val_accuracy: 0.8330
[Trial complete]
[Trial summary]
 |-Trial ID: 779b0ec8dbfceee7e4b7166b20aa3d8f
 |-Score: 0.8330000042915344
 |-Best step: 0
 > Hyperparameters:
 |-learning_rate: 0.0001
 |-tuner/bracket: 2
 |-tuner/epochs: 2
 |-tuner/initial_epoch: 0
 |-tuner/round: 0
 |-units: 96
Epoch 1/2
1875/1875 [==============================] - 5s 3ms/step - loss: 0.5946 - accuracy: 0.8058 - val_loss: 0.4797 - val_accuracy: 0.8346
Epoch 2/2
1875/1875 [==============================] - 5s 3ms/step - loss: 0.4261 - accuracy: 0.8531 - val_loss: 0.4372 - val_accuracy: 0.8464
[Trial complete]
[Trial summary]
 |-Trial ID: d3a96eb83cc740ec1e5db373d9fd55c3
 |-Score: 0.8464000225067139
 |-Best step: 0
 > Hyperparameters:
 |-learning_rate: 0.0001
 |-tuner/bracket: 2
 |-tuner/epochs: 2
 |-tuner/initial_epoch: 0
 |-tuner/round: 0
 |-units: 448
Epoch 1/2
1875/1875 [==============================] - 3s 2ms/step - loss: 0.6247 - accuracy: 0.7956 - val_loss: 0.4990 - val_accuracy: 0.8292
Epoch 2/2
1875/1875 [==============================] - 3s 1ms/step - loss: 0.4386 - accuracy: 0.8506 - val_loss: 0.4458 - val_accuracy: 0.8466
[Trial complete]
[Trial summary]
 |-Trial ID: 5e8585e74ca9cba4e8a4064cbc7bd668
 |-Score: 0.8465999960899353
 |-Best step: 0
 > Hyperparameters:
 |-learning_rate: 0.0001
 |-tuner/bracket: 2
 |-tuner/epochs: 2
 |-tuner/initial_epoch: 0
 |-tuner/round: 0
 |-units: 320
Epoch 1/2
1875/1875 [==============================] - 6s 3ms/step - loss: 0.5458 - accuracy: 0.8098 - val_loss: 0.4780 - val_accuracy: 0.8282
Epoch 2/2
1875/1875 [==============================] - 5s 3ms/step - loss: 0.4339 - accuracy: 0.8444 - val_loss: 0.4672 - val_accuracy: 0.8367
[Trial complete]
[Trial summary]
 |-Trial ID: 0d8ec7608c018b20fd9b4cad6e66a9b6
 |-Score: 0.8367000222206116
 |-Best step: 0
 > Hyperparameters:
 |-learning_rate: 0.01
 |-tuner/bracket: 2
 |-tuner/epochs: 2
 |-tuner/initial_epoch: 0
 |-tuner/round: 0
 |-units: 480
Epoch 1/2
1875/1875 [==============================] - 2s 983us/step - loss: 0.5315 - accuracy: 0.8114 - val_loss: 0.4696 - val_accuracy: 0.8292
Epoch 2/2
1875/1875 [==============================] - 2s 983us/step - loss: 0.4341 - accuracy: 0.8454 - val_loss: 0.4831 - val_accuracy: 0.8317
[Trial complete]
[Trial summary]
 |-Trial ID: e53c8d4b661b5520e392b34a65f0ca1d
 |-Score: 0.8317000269889832
 |-Best step: 0
 > Hyperparameters:
 |-learning_rate: 0.01
 |-tuner/bracket: 2
 |-tuner/epochs: 2
 |-tuner/initial_epoch: 0
 |-tuner/round: 0
 |-units: 160
Epoch 1/2
1875/1875 [==============================] - 2s 883us/step - loss: 0.5238 - accuracy: 0.8195 - val_loss: 0.4598 - val_accuracy: 0.8344
Epoch 2/2
1875/1875 [==============================] - 2s 842us/step - loss: 0.3932 - accuracy: 0.8602 - val_loss: 0.3968 - val_accuracy: 0.8580
[Trial complete]
[Trial summary]
 |-Trial ID: 57f46babcb66e2f2829177035445f65c
 |-Score: 0.8579999804496765
 |-Best step: 0
 > Hyperparameters:
 |-learning_rate: 0.001
 |-tuner/bracket: 2
 |-tuner/epochs: 2
 |-tuner/initial_epoch: 0
 |-tuner/round: 0
 |-units: 64
Epoch 1/2
1875/1875 [==============================] - 2s 1ms/step - loss: 0.4923 - accuracy: 0.8249 - val_loss: 0.4501 - val_accuracy: 0.8431
Epoch 2/2
1875/1875 [==============================] - 2s 983us/step - loss: 0.3672 - accuracy: 0.8670 - val_loss: 0.3996 - val_accuracy: 0.8564
[Trial complete]
[Trial summary]
 |-Trial ID: 59551e7ff2ab521be9ae98c3c76f764b
 |-Score: 0.8564000129699707
 |-Best step: 0
 > Hyperparameters:
 |-learning_rate: 0.001
 |-tuner/bracket: 2
 |-tuner/epochs: 2
 |-tuner/in