保存和序列化模型

准备

!pip3 install tensorflow==2.0.0a0
%matplotlib inline
from tensorflow import keras
import tensorflow as tf
import numpy as np

Looking in indexes: https://pypi.tuna.tsinghua.edu.cn/simple
Requirement already satisfied: tensorflow==2.0.0a0 in /usr/local/lib/python3.7/site-packages (2.0.0a0)
Requirement already satisfied: google-pasta>=0.1.2 in /usr/local/lib/python3.7/site-packages (from tensorflow==2.0.0a0) (0.1.4)
Requirement already satisfied: astor>=0.6.0 in /Users/fei/Library/Python/3.7/lib/python/site-packages (from tensorflow==2.0.0a0) (0.7.1)
Requirement already satisfied: keras-preprocessing>=1.0.5 in /usr/local/lib/python3.7/site-packages (from tensorflow==2.0.0a0) (1.0.9)
Requirement already satisfied: gast>=0.2.0 in /Users/fei/Library/Python/3.7/lib/python/site-packages (from tensorflow==2.0.0a0) (0.2.2)
Requirement already satisfied: tb-nightly<1.14.0a20190302,>=1.14.0a20190301 in /usr/local/lib/python3.7/site-packages (from tensorflow==2.0.0a0) (1.14.0a20190301)
Requirement already satisfied: six>=1.10.0 in /Users/fei/Library/Python/3.7/lib/python/site-packages (from tensorflow==2.0.0a0) (1.12.0)
Requirement already satisfied: absl-py>=0.7.0 in /Users/fei/Library/Python/3.7/lib/python/site-packages (from tensorflow==2.0.0a0) (0.7.0)
Requirement already satisfied: numpy<2.0,>=1.14.5 in /usr/local/lib/python3.7/site-packages (from tensorflow==2.0.0a0) (1.16.2)
Requirement already satisfied: protobuf>=3.6.1 in /usr/local/lib/python3.7/site-packages (from tensorflow==2.0.0a0) (3.7.0)
Requirement already satisfied: grpcio>=1.8.6 in /usr/local/lib/python3.7/site-packages (from tensorflow==2.0.0a0) (1.19.0)
Requirement already satisfied: tf-estimator-nightly<1.14.0.dev2019030116,>=1.14.0.dev2019030115 in /usr/local/lib/python3.7/site-packages (from tensorflow==2.0.0a0) (1.14.0.dev2019030115)
Requirement already satisfied: wheel>=0.26 in /usr/local/lib/python3.7/site-packages (from tensorflow==2.0.0a0) (0.33.1)
Requirement already satisfied: keras-applications>=1.0.6 in /usr/local/lib/python3.7/site-packages (from tensorflow==2.0.0a0) (1.0.7)
Requirement already satisfied: termcolor>=1.1.0 in /Users/fei/Library/Python/3.7/lib/python/site-packages (from tensorflow==2.0.0a0) (1.1.0)
Requirement already satisfied: markdown>=2.6.8 in /Users/fei/Library/Python/3.7/lib/python/site-packages (from tb-nightly<1.14.0a20190302,>=1.14.0a20190301->tensorflow==2.0.0a0) (3.0.1)
Requirement already satisfied: werkzeug>=0.11.15 in /Users/fei/Library/Python/3.7/lib/python/site-packages (from tb-nightly<1.14.0a20190302,>=1.14.0a20190301->tensorflow==2.0.0a0) (0.14.1)
Requirement already satisfied: setuptools in /usr/local/lib/python3.7/site-packages (from protobuf>=3.6.1->tensorflow==2.0.0a0) (40.8.0)
Requirement already satisfied: h5py in /Users/fei/Library/Python/3.7/lib/python/site-packages (from keras-applications>=1.0.6->tensorflow==2.0.0a0) (2.9.0)

保存序列模型和函数API模型

考虑下面的模型：

inputs = keras.Input(shape=(784, ), name='digits')
x = keras.layers.Dense(64, activation='relu', name='dense_1')(inputs)
x = keras.layers.Dense(64, activation='relu', name='dense_2')(x)
outputs = keras.layers.Dense(10, activation='softmax', name='predictions')(x)

model = keras.Model(inputs=inputs, outputs=outputs, name='3_layer_mlp')
model.summary()

Model: "3_layer_mlp"
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
digits (InputLayer)          [(None, 784)]             0         
_________________________________________________________________
dense_1 (Dense)              (None, 64)                50240     
_________________________________________________________________
dense_2 (Dense)              (None, 64)                4160      
_________________________________________________________________
predictions (Dense)          (None, 10)                650       
=================================================================
Total params: 55,050
Trainable params: 55,050
Non-trainable params: 0
_________________________________________________________________

为了保存的时候可以有权重和优化器的状态，我们使用数据对该模型进行训练，当然也可以保存没有任何训练的模型，但是这样没啥意义。

(x_train, y_train), (x_test, y_test) = keras.datasets.mnist.load_data()
x_train = x_train.reshape(60000, 784) / 255.0
x_test = x_test.reshape(10000, 784) / 255.0

model.compile(loss='sparse_categorical_crossentropy', optimizer=keras.optimizers.RMSprop())
model.fit(x_train, y_train, batch_size=64, epochs=1)

60000/60000 [==============================] - 3s 52us/sample - loss: 0.3065





<tensorflow.python.keras.callbacks.History at 0x13310e320>

predictions = model.predict(x_test) # 后面对比使用

保存所有状态

可以使用API函数将模型的所有信息保存到一个文件中，这样即使后面没有创建模型的代码，也可以从保存的文件中恢复模型。
保存的文件中包含模型的如下信息：

1. 模型网络结构
2. 模型的权重(训练获取的)
3. 模型训练的配置信息(compile函数设置的)
4. 优化器的设置和状态(训练获取的)

model.save('save_my_model.h5')# 保存模型
new_model = keras.models.load_model('save_my_model.h5') #从文件中加载模型，和原始模型一模一样

# 使用新加载的模型进行推断，同时和上面的推断结果进行对比
new_predictions = new_model.predict(x_test)
np.testing.assert_allclose(new_predictions, predictions, atol=1e-6)  # 差距小于1e-6就认为是相同的

保存为tf格式

也可以将模型保存为tensorflow的格式，该格式受到tensorflow的支持和维护。

keras.experimental.export_saved_model(model, 'path_to_saved_model')
new_model = keras.experimental.load_from_saved_model('path_to_saved_model')
new_predictions = new_model.predict(x_test)
np.testing.assert_allclose(predictions, new_predictions, atol=1e-6)

WARNING: Logging before flag parsing goes to stderr.
W0415 08:57:40.228030 4583425472 deprecation.py:323] From /usr/local/lib/python3.7/site-packages/tensorflow/python/saved_model/signature_def_utils_impl.py:253: build_tensor_info (from tensorflow.python.saved_model.utils_impl) is deprecated and will be removed in a future version.
Instructions for updating:
This function will only be available through the v1 compatibility library as tf.compat.v1.saved_model.utils.build_tensor_info or tf.compat.v1.saved_model.build_tensor_info.
W0415 08:57:40.229865 4583425472 tf_logging.py:161] Export includes no default signature!
W0415 08:57:40.842535 4583425472 tf_logging.py:161] Export includes no default signature!

tensorflow格式的文件包含了下面的信息：

1. 模型的快照
2. 包含了图的结构，包括训练图、验证图和推断图。如果模型时没有训练的，那么仅仅包含推断图
3. 模型结构的配置

仅保存模型结构

有时候我们仅仅对模型的结构感兴趣，并不需要模型的权重和优化器状态。此时就可以通过get_config方法从模型中获取其结构，这是一个python的字典，后续可以通过该字典从新创建模型(新的模型中不包含任何之前模型训练的信息)。

config = model.get_config()
reinitialized_model = keras.Model.from_config(config)

# 新的模型由于没有训练，因此输出肯定不会相同
new_predictions = reinitialized_model.predict(x_test)
assert abs(np.sum(predictions - new_predictions)) > 0.

当然也可以使用to_json()/from_json()和to_yaml()/from_yaml()将模型字典转换为json和yaml类型的字符串，这样可以很方便的保存到磁盘中。

# json
json_config = model.to_json()
reinitialized_model = keras.models.model_from_json(json_config)

# yaml
yaml_config = model.to_yaml()
reinitialized_model = keras.models.model_from_yaml(yaml_config)

/usr/local/lib/python3.7/site-packages/tensorflow/python/keras/saving/model_config.py:76: YAMLLoadWarning: calling yaml.load() without Loader=... is deprecated, as the default Loader is unsafe. Please read https://msg.pyyaml.org/load for full details.
  config = yaml.load(yaml_string)

仅保存权重

当然有时也可能仅仅对模型的状态(权重)感兴趣，那么就可以通过get_weights()获取numpy格式的模型权重数据，也可以通过set_weights()将权重赋予模型。

weights = model.get_weights()
model.set_weights(weights)

通过搭配to_json()/from_json()和get_weights()/set_weights()就可以将模型恢复到保存时的状态，但是此时模型并不包含训练参数的配置(比如：优化器、loss等)，此时就需要重新调用compile方法设置训练参数。
权重保存到文件和从文件加载所对应的方法是save_weights()和load_weights()

权重保存为tf格式

通过指定文件格式，save_weights()可以将权重保存为h5格式或者tf的格式。

model.save_weights('path_to_my_tf_savedmodel')    # h5格式
model.save_weights('path_to_my_tf_savedmodel', save_format='tf')  # tf格式

子类模型的保存

序列模型和功能模型保存的是层的DGA结构，因此可以安全的进行保存和序列化。但是对于模型的子类，这一点并不能进行保证。因此加载此类模型，就需要原始模型的创建代码，当然也可以将代码序列化，但是这并不是安全和不可移植的。
考虑下面的子类模型：

class ThreeLayerMLP(keras.Model):

    def __init__(self, name=None):
        super(ThreeLayerMLP, self).__init__(name=name)
        self.dense_1 = keras.layers.Dense(64, activation='relu', name='dense_1')
        self.dense_2 = keras.layers.Dense(64, activation='relu', name='dense_2')
        self.pred_layer = keras.layers.Dense(10, activation='softmax', name='predictions')

    def call(self, inputs):
        x = self.dense_1(inputs)
        x = self.dense_2(x)
        return self.pred_layer(x)

def get_model():
    return ThreeLayerMLP(name='3_layer_mlp')

model = get_model()

首先，一个模型的子类在没有使用的情况下是不能进行保存的。这是因为这种模型需要在一些数据上进行操作才可以创建权重参数。
模型被使用之前，我们是不能获取到模型的数据类型以及权重的尺寸的，这是因为之前介绍过的权重延迟创建的原因。直到模型在数据上进行使用，我们才能够获取到权重的数据类型以及权重的尺寸。
这里我们先对模型进行训练：

(x_train, y_train), (x_test, y_test) = keras.datasets.mnist.load_data()
x_train = x_train.reshape(60000, 784).astype('float32') / 255
x_test = x_test.reshape(10000, 784).astype('float32') / 255

model.compile(loss='sparse_categorical_crossentropy',
              optimizer=keras.optimizers.RMSprop())
history = model.fit(x_train, y_train,
                    batch_size=64,
                    epochs=1)

60000/60000 [==============================] - 3s 53us/sample - loss: 0.3167

推荐使用save_weights方法将权重、优化器状态以及metrics等状态以快照的方式保存为tf类型的文件。

model.save_weights('path_to_my_weights', save_format='tf')

如果需要恢复保存的权重，那么首先需要获取原始模型结构的代码。
其次，在原始模型创建出来后，不能直接进行load_weights操作，同样是由于权重延迟创建的原因。需要先调用compile配置模型，然后使用任意的数据调用模型，才可以加载权重。

new_model = get_model()        # 生成新模型
new_model.compile(loss='sparse_categorical_crossentropy', optimizer=keras.optimizers.RMSprop()) # 为新模型配置参数
history = new_model.train_on_batch(x_train[:1], y_train[:1]) # 在任意数据上调用模型

# 加载权重
new_model.load_weights('path_to_my_weights')

<tensorflow.python.training.tracking.util.CheckpointLoadStatus at 0x134d2f240>