准备
!pip3 install tensorflow==2.0.0a0
%matplotlib inline
from tensorflow import keras
import tensorflow as tf
import numpy as np
Looking in indexes: https://pypi.tuna.tsinghua.edu.cn/simple
Requirement already satisfied: tensorflow==2.0.0a0 in /usr/local/lib/python3.7/site-packages (2.0.0a0)
Requirement already satisfied: google-pasta>=0.1.2 in /usr/local/lib/python3.7/site-packages (from tensorflow==2.0.0a0) (0.1.4)
Requirement already satisfied: astor>=0.6.0 in /Users/fei/Library/Python/3.7/lib/python/site-packages (from tensorflow==2.0.0a0) (0.7.1)
Requirement already satisfied: keras-preprocessing>=1.0.5 in /usr/local/lib/python3.7/site-packages (from tensorflow==2.0.0a0) (1.0.9)
Requirement already satisfied: gast>=0.2.0 in /Users/fei/Library/Python/3.7/lib/python/site-packages (from tensorflow==2.0.0a0) (0.2.2)
Requirement already satisfied: tb-nightly<1.14.0a20190302,>=1.14.0a20190301 in /usr/local/lib/python3.7/site-packages (from tensorflow==2.0.0a0) (1.14.0a20190301)
Requirement already satisfied: six>=1.10.0 in /Users/fei/Library/Python/3.7/lib/python/site-packages (from tensorflow==2.0.0a0) (1.12.0)
Requirement already satisfied: absl-py>=0.7.0 in /Users/fei/Library/Python/3.7/lib/python/site-packages (from tensorflow==2.0.0a0) (0.7.0)
Requirement already satisfied: numpy<2.0,>=1.14.5 in /usr/local/lib/python3.7/site-packages (from tensorflow==2.0.0a0) (1.16.2)
Requirement already satisfied: protobuf>=3.6.1 in /usr/local/lib/python3.7/site-packages (from tensorflow==2.0.0a0) (3.7.0)
Requirement already satisfied: grpcio>=1.8.6 in /usr/local/lib/python3.7/site-packages (from tensorflow==2.0.0a0) (1.19.0)
Requirement already satisfied: tf-estimator-nightly<1.14.0.dev2019030116,>=1.14.0.dev2019030115 in /usr/local/lib/python3.7/site-packages (from tensorflow==2.0.0a0) (1.14.0.dev2019030115)
Requirement already satisfied: wheel>=0.26 in /usr/local/lib/python3.7/site-packages (from tensorflow==2.0.0a0) (0.33.1)
Requirement already satisfied: keras-applications>=1.0.6 in /usr/local/lib/python3.7/site-packages (from tensorflow==2.0.0a0) (1.0.7)
Requirement already satisfied: termcolor>=1.1.0 in /Users/fei/Library/Python/3.7/lib/python/site-packages (from tensorflow==2.0.0a0) (1.1.0)
Requirement already satisfied: markdown>=2.6.8 in /Users/fei/Library/Python/3.7/lib/python/site-packages (from tb-nightly<1.14.0a20190302,>=1.14.0a20190301->tensorflow==2.0.0a0) (3.0.1)
Requirement already satisfied: werkzeug>=0.11.15 in /Users/fei/Library/Python/3.7/lib/python/site-packages (from tb-nightly<1.14.0a20190302,>=1.14.0a20190301->tensorflow==2.0.0a0) (0.14.1)
Requirement already satisfied: setuptools in /usr/local/lib/python3.7/site-packages (from protobuf>=3.6.1->tensorflow==2.0.0a0) (40.8.0)
Requirement already satisfied: h5py in /Users/fei/Library/Python/3.7/lib/python/site-packages (from keras-applications>=1.0.6->tensorflow==2.0.0a0) (2.9.0)
保存序列模型和函数API模型
考虑下面的模型:
inputs = keras.Input(shape=(784, ), name='digits')
x = keras.layers.Dense(64, activation='relu', name='dense_1')(inputs)
x = keras.layers.Dense(64, activation='relu', name='dense_2')(x)
outputs = keras.layers.Dense(10, activation='softmax', name='predictions')(x)
model = keras.Model(inputs=inputs, outputs=outputs, name='3_layer_mlp')
model.summary()
Model: "3_layer_mlp"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
digits (InputLayer) [(None, 784)] 0
_________________________________________________________________
dense_1 (Dense) (None, 64) 50240
_________________________________________________________________
dense_2 (Dense) (None, 64) 4160
_________________________________________________________________
predictions (Dense) (None, 10) 650
=================================================================
Total params: 55,050
Trainable params: 55,050
Non-trainable params: 0
_________________________________________________________________
为了保存的时候可以有权重和优化器的状态,我们使用数据对该模型进行训练,当然也可以保存没有任何训练的模型,但是这样没啥意义。
(x_train, y_train), (x_test, y_test) = keras.datasets.mnist.load_data()
x_train = x_train.reshape(60000, 784) / 255.0
x_test = x_test.reshape(10000, 784) / 255.0
model.compile(loss='sparse_categorical_crossentropy', optimizer=keras.optimizers.RMSprop())
model.fit(x_train, y_train, batch_size=64, epochs=1)
60000/60000 [==============================] - 3s 52us/sample - loss: 0.3065
<tensorflow.python.keras.callbacks.History at 0x13310e320>
predictions = model.predict(x_test) # 后面对比使用
保存所有状态
可以使用API函数将模型的所有信息保存到一个文件中,这样即使后面没有创建模型的代码,也可以从保存的文件中恢复模型。
保存的文件中包含模型的如下信息:
- 模型网络结构
- 模型的权重(训练获取的)
- 模型训练的配置信息(
compile
函数设置的) - 优化器的设置和状态(训练获取的)
model.save('save_my_model.h5')# 保存模型
new_model = keras.models.load_model('save_my_model.h5') #从文件中加载模型,和原始模型一模一样
# 使用新加载的模型进行推断,同时和上面的推断结果进行对比
new_predictions = new_model.predict(x_test)
np.testing.assert_allclose(new_predictions, predictions, atol=1e-6) # 差距小于1e-6就认为是相同的
保存为tf格式
也可以将模型保存为tensorflow
的格式,该格式受到tensorflow
的支持和维护。
keras.experimental.export_saved_model(model, 'path_to_saved_model')
new_model = keras.experimental.load_from_saved_model('path_to_saved_model')
new_predictions = new_model.predict(x_test)
np.testing.assert_allclose(predictions, new_predictions, atol=1e-6)
WARNING: Logging before flag parsing goes to stderr.
W0415 08:57:40.228030 4583425472 deprecation.py:323] From /usr/local/lib/python3.7/site-packages/tensorflow/python/saved_model/signature_def_utils_impl.py:253: build_tensor_info (from tensorflow.python.saved_model.utils_impl) is deprecated and will be removed in a future version.
Instructions for updating:
This function will only be available through the v1 compatibility library as tf.compat.v1.saved_model.utils.build_tensor_info or tf.compat.v1.saved_model.build_tensor_info.
W0415 08:57:40.229865 4583425472 tf_logging.py:161] Export includes no default signature!
W0415 08:57:40.842535 4583425472 tf_logging.py:161] Export includes no default signature!
tensorflow
格式的文件包含了下面的信息:
- 模型的快照
- 包含了图的结构,包括训练图、验证图和推断图。如果模型时没有训练的,那么仅仅包含推断图
- 模型结构的配置
仅保存模型结构
有时候我们仅仅对模型的结构感兴趣,并不需要模型的权重和优化器状态。此时就可以通过get_config
方法从模型中获取其结构,这是一个python
的字典,后续可以通过该字典从新创建模型(新的模型中不包含任何之前模型训练的信息)。
config = model.get_config()
reinitialized_model = keras.Model.from_config(config)
# 新的模型由于没有训练,因此输出肯定不会相同
new_predictions = reinitialized_model.predict(x_test)
assert abs(np.sum(predictions - new_predictions)) > 0.
当然也可以使用to_json()/from_json()
和to_yaml()/from_yaml()
将模型字典转换为json
和yaml
类型的字符串,这样可以很方便的保存到磁盘中。
# json
json_config = model.to_json()
reinitialized_model = keras.models.model_from_json(json_config)
# yaml
yaml_config = model.to_yaml()
reinitialized_model = keras.models.model_from_yaml(yaml_config)
/usr/local/lib/python3.7/site-packages/tensorflow/python/keras/saving/model_config.py:76: YAMLLoadWarning: calling yaml.load() without Loader=... is deprecated, as the default Loader is unsafe. Please read https://msg.pyyaml.org/load for full details.
config = yaml.load(yaml_string)
仅保存权重
当然有时也可能仅仅对模型的状态(权重)感兴趣,那么就可以通过get_weights()
获取numpy
格式的模型权重数据,也可以通过set_weights()
将权重赋予模型。
weights = model.get_weights()
model.set_weights(weights)
通过搭配to_json()/from_json()
和get_weights()/set_weights()
就可以将模型恢复到保存时的状态,但是此时模型并不包含训练参数的配置(比如:优化器、loss
等),此时就需要重新调用compile
方法设置训练参数。
权重保存到文件和从文件加载所对应的方法是save_weights()
和load_weights()
权重保存为tf
格式
通过指定文件格式,save_weights()
可以将权重保存为h5
格式或者tf
的格式。
model.save_weights('path_to_my_tf_savedmodel') # h5格式
model.save_weights('path_to_my_tf_savedmodel', save_format='tf') # tf格式
子类模型的保存
序列模型和功能模型保存的是层的DGA结构,因此可以安全的进行保存和序列化。但是对于模型的子类,这一点并不能进行保证。因此加载此类模型,就需要原始模型的创建代码,当然也可以将代码序列化,但是这并不是安全和不可移植的。
考虑下面的子类模型:
class ThreeLayerMLP(keras.Model):
def __init__(self, name=None):
super(ThreeLayerMLP, self).__init__(name=name)
self.dense_1 = keras.layers.Dense(64, activation='relu', name='dense_1')
self.dense_2 = keras.layers.Dense(64, activation='relu', name='dense_2')
self.pred_layer = keras.layers.Dense(10, activation='softmax', name='predictions')
def call(self, inputs):
x = self.dense_1(inputs)
x = self.dense_2(x)
return self.pred_layer(x)
def get_model():
return ThreeLayerMLP(name='3_layer_mlp')
model = get_model()
首先,一个模型的子类在没有使用的情况下是不能进行保存的。这是因为这种模型需要在一些数据上进行操作才可以创建权重参数。
模型被使用之前,我们是不能获取到模型的数据类型以及权重的尺寸的,这是因为之前介绍过的权重延迟创建的原因。直到模型在数据上进行使用,我们才能够获取到权重的数据类型以及权重的尺寸。
这里我们先对模型进行训练:
(x_train, y_train), (x_test, y_test) = keras.datasets.mnist.load_data()
x_train = x_train.reshape(60000, 784).astype('float32') / 255
x_test = x_test.reshape(10000, 784).astype('float32') / 255
model.compile(loss='sparse_categorical_crossentropy',
optimizer=keras.optimizers.RMSprop())
history = model.fit(x_train, y_train,
batch_size=64,
epochs=1)
60000/60000 [==============================] - 3s 53us/sample - loss: 0.3167
推荐使用save_weights
方法将权重、优化器状态以及metrics
等状态以快照的方式保存为tf
类型的文件。
model.save_weights('path_to_my_weights', save_format='tf')
如果需要恢复保存的权重,那么首先需要获取原始模型结构的代码。
其次,在原始模型创建出来后,不能直接进行load_weights
操作,同样是由于权重延迟创建的原因。需要先调用compile
配置模型,然后使用任意的数据调用模型,才可以加载权重。
new_model = get_model() # 生成新模型
new_model.compile(loss='sparse_categorical_crossentropy', optimizer=keras.optimizers.RMSprop()) # 为新模型配置参数
history = new_model.train_on_batch(x_train[:1], y_train[:1]) # 在任意数据上调用模型
# 加载权重
new_model.load_weights('path_to_my_weights')
<tensorflow.python.training.tracking.util.CheckpointLoadStatus at 0x134d2f240>