How to Save Your Models For Later With Serialization

Given that deep learning models can take hours, days and even weeks to train, it is important to know how to save and load them from disk. In this lesson you will discover how you can save your Keras models to file and load them up again to make predictions. After completing this lesson you will know:

• How to save and load Keras model weights to HDF5 formatted files.
• How to save and load Keras model structure to JSON files.
• How to save and load Keras model structure to YAML files.

1.1 Tutorial Overview

Keras separates the concerns of saving your model architecture and saving your model weights. Model weights are saved to HDF5 format. This is a grid format that is ideal for storing multi-dimensional arrays of numbers. The model structure can be described and saved (and loaded) using two di↵erent formats: JSON and YAML.

        Each example will also demonstrate saving and loading your model weights to HDF5 formatted files. The examples will use the same simple network trained on the Pima Indians onset of diabetes binary classification dataset.

1.1.1 HDF5 Format

The Hierarchical Data Format or HDF5 for short is a flexible data storage format and is convenient for storing large arrays of real values, as we have in the weights of neural networks. You may need to install Python support for the HDF5 file format. You can do this using your preferred Python package management system such as Pip:

# Install Python Support For the HDF5 File Format via Pip
sudo pip install h5py

1.2 Save Your Neural Network Model to JSON

JSON is a simple file format for describing data hierarchically. Keras provides the ability to describe any model using JSON format with a to json() function. This can be saved to file and later loaded via the model_from_json() function that will create a new model from the JSON specification. The weights are saved directly from the model using the save weights() function and later loaded using the symmetrical load weights() function. The example below trains and evaluates a simple model on the Pima Indians dataset. The model structure is then converted to JSON format and written to model.json in the local directory. The network weights are written to model.h5 in the local directory. The model and weight data is loaded from the saved files and a new model is created. It is important to compile the loaded model before it is used. This is so that predictions made using the model can use the appropriate efficient computation from the Keras backend. The model is evaluated in the  same way printing the same evaluation score.

# Serialize Model To JSON Format
# MLP for Pima Indian Dataset Serialize to JSON and HDFS
from keras.models import Sequential
from keras.layers import Dense
from keras.models import model_from_json
import numpy as np
import os

# fix random seed for reproducibility
seed = 7
np.random.seed(seed)

# load pima indians dataset
dataset = np.loadtxt("pima-indians-diabetes.csv",delimiter=',')

# split into input (X) and output (Y) variables
X = dataset[:,0:8]
Y = dataset[:,8]

# create model
model = Sequential()
model.add(Dense(12, input_dim=8, kernel_initializer='uniform',activation='relu'))
model.add(Dense(8, kernel_initializer='normal',activation='relu'))
model.add(Dense(1, kernel_initializer='normal',activation='sigmoid'))

# Compile model 
model.compile(loss='binary_crossentropy',optimizer='adam',metrics=['accuracy'])

# fit the model
model.fit(X, Y, epochs=150, batch_size=10, verbose=0)

# evaluate the model
scores = model.evaluate(X, Y, verbose=0)
print("%s: %.2f%%" %(model.metrics_names[1],scores[1]*100))

# serialize model to JSON
model_json = model.to_json()
with open("model.json","w") as json_file:
    json_file.write(model_json)
    
# serialize weights to HDFS
model.save_weights("model.h5")
print("Saved model to disk")

# later

# load json and create model
json_file = open('model.json','r')
loaded_model_json = json_file.read()
json_file.close()
loaded_model = model_from_json(loaded_model_json)

# load weights into new model
loaded_model.load_weights("model.h5")
print("Loaded model from disk")

# evaluate loaded model on test data
loaded_model.compile(loss='binary_crossentropy',optimizer='rmsprop', metrics=['accuracy'])
score = loaded_model.evaluate(X, Y, verbose=0)
print("%s: %.2f%%" %(loaded_model.metrics_names[1],score[1]*100))

     Running this example provides the output below. It shows first the accuracy of the trained model, the saving of the model to disk in JSON format, the loading of the model and finally the re-evaluation of the loaded model achieving the same accuracy.

accuracy: 79.95%
Saved model to disk
Loaded model from disk
accuracy: 79.95%

Sample Output From Serializing Model To JSON Format.

The JSON format of the model looks like the following:

{"class_name": "Sequential", "config": {"name": "sequential_2", "layers": [{"class_name": "InputLayer", "config": {"batch_input_shape": [null, 8], "dtype": "float32", "sparse": false, "ragged": false, "name": "dense_6_input"}}, {"class_name": "Dense", "config": {"name": "dense_6", "trainable": true, "batch_input_shape": [null, 8], "dtype": "float32", "units": 12, "activation": "relu", "use_bias": true, "kernel_initializer": {"class_name": "RandomUniform", "config": {"minval": -0.05, "maxval": 0.05, "seed": null}}, "bias_initializer": {"class_name": "Zeros", "config": {}}, "kernel_regularizer": null, "bias_regularizer": null, "activity_regularizer": null, "kernel_constraint": null, "bias_constraint": null}}, {"class_name": "Dense", "config": {"name": "dense_7", "trainable": true, "dtype": "float32", "units": 8, "activation": "relu", "use_bias": true, "kernel_initializer": {"class_name": "RandomNormal", "config": {"mean": 0.0, "stddev": 0.05, "seed": null}}, "bias_initializer": {"class_name": "Zeros", "config": {}}, "kernel_regularizer": null, "bias_regularizer": null, "activity_regularizer": null, "kernel_constraint": null, "bias_constraint": null}}, {"class_name": "Dense", "config": {"name": "dense_8", "trainable": true, "dtype": "float32", "units": 1, "activation": "sigmoid", "use_bias": true, "kernel_initializer": {"class_name": "RandomNormal", "config": {"mean": 0.0, "stddev": 0.05, "seed": null}}, "bias_initializer": {"class_name": "Zeros", "config": {}}, "kernel_regularizer": null, "bias_regularizer": null, "activity_regularizer": null, "kernel_constraint": null, "bias_constraint": null}}]}, "keras_version": "2.8.0", "backend": "tensorflow"}

1.3 Save Your Neural Network Model to YAML

This example is much the same as the above JSON example, except the YAML format is used for the model specification. The model is described using YAML, saved to file model.yaml and later loaded into a new model via the model_from_yaml() function. Weights are handled in the same way as above in HDF5 format as model.h5.

# Serailize Model To YAML Format
# MLP for Pima Indians Dataset serialize to YAML and HDF5
from keras.models import Sequential
from keras.layers import Dense
from keras.models import model_from_yaml
import numpy as np
import os

# fix random seed for reproducibility
seed = 7
np.random.seed(seed)

# load pima indians dataset
dataset = np.loadtxt("pima-indians-diabetes.csv",delimiter=",")

# split into input(X) and output(Y) variables
X = dataset[:,0:8]
Y = dataset[:,8]

# create model
model = Sequential()
model.add(Dense(12, input_dim=8, kernel_initializer='normal',activation='relu'))
model.add(Dense(8, kernel_initializer='normal', activation='relu'))
model.add(Dense(1, kernel_initializer='uniform',activation='sigmoid'))

# compile model

model.compile(loss='binary_crossentropy', optimizer='adam',metrics=['accuracy'])

# Fit the model

model.fit(X, Y, epochs=150, batch_size=10, verbose=0)

# evaluate the model

scores = model.evaluate(X, Y, verbose=0)
print("%s: %.2f%%" % (model.metrics_names[1], scores[1]*100))

# serialize model to YAML
model_yaml = model.to_yaml()
with open("model.yaml","w") as yaml_file:
    yaml_file.write(model_yaml)

# serialize weights to HDF5
model.save_weights("model.h5")
print("Saved model to disk")

# later ...

# load YAML and create model
yaml_file = open('model.yaml','r')
loaded_model_yaml = yaml_file.read()
yaml_file.close()
loaded_model = model_from_yaml(loaded_model_yaml)

# load weights into new model
loaded_model.load_weights("model.h5")
print("Loaded model from disk")

# evaluate loaded model on test data
loaded_model.compile(loss='binary_crossentropy',optimizer='rmsprop',metrics=['accuracy'])
score = loaded_model.evaluate(X, Y, verbose=0)
print("%s: %.2f%%" % (loaded_model.metrics_names[1],score[1]*100))

1.4 Summary

Saving and loading models is an important capability for transplanting a deep learning model from research and development to operations. In this lesson you discovered how to serialize your Keras deep learning models. You learned:

• How to save model weights to HDF5 formatted files and load them again later.
• How to save Keras model definitions to JSON files and load them again.
• How to save Keras model definitions to YAML files and load them again.(Method `model.to_yaml()` has been removed due to security risk of arbitrary code execution. Please use `model.to_json()` instead.)

1.4.1 Next

You now know how to serialize your deep learning models in Keras. Next you will discover the importance of checkpointing your models during long training periods and how to load those checkpointed models in order to make predictions.