keras训练和评估

!pip3 install tensorflow==2.0.0a0
%matplotlib inline

Looking in indexes: https://pypi.tuna.tsinghua.edu.cn/simple
Requirement already satisfied: tensorflow==2.0.0a0 in /usr/local/lib/python3.7/site-packages (2.0.0a0)
Requirement already satisfied: protobuf>=3.6.1 in /usr/local/lib/python3.7/site-packages (from tensorflow==2.0.0a0) (3.7.0)
Requirement already satisfied: termcolor>=1.1.0 in ./Library/Python/3.7/lib/python/site-packages (from tensorflow==2.0.0a0) (1.1.0)
Requirement already satisfied: tb-nightly<1.14.0a20190302,>=1.14.0a20190301 in /usr/local/lib/python3.7/site-packages (from tensorflow==2.0.0a0) (1.14.0a20190301)
Requirement already satisfied: grpcio>=1.8.6 in /usr/local/lib/python3.7/site-packages (from tensorflow==2.0.0a0) (1.19.0)
Requirement already satisfied: absl-py>=0.7.0 in ./Library/Python/3.7/lib/python/site-packages (from tensorflow==2.0.0a0) (0.7.0)
Requirement already satisfied: wheel>=0.26 in /usr/local/lib/python3.7/site-packages (from tensorflow==2.0.0a0) (0.33.1)
Requirement already satisfied: numpy<2.0,>=1.14.5 in /usr/local/lib/python3.7/site-packages (from tensorflow==2.0.0a0) (1.16.2)
Requirement already satisfied: astor>=0.6.0 in ./Library/Python/3.7/lib/python/site-packages (from tensorflow==2.0.0a0) (0.7.1)
Requirement already satisfied: keras-preprocessing>=1.0.5 in /usr/local/lib/python3.7/site-packages (from tensorflow==2.0.0a0) (1.0.9)
Requirement already satisfied: tf-estimator-nightly<1.14.0.dev2019030116,>=1.14.0.dev2019030115 in /usr/local/lib/python3.7/site-packages (from tensorflow==2.0.0a0) (1.14.0.dev2019030115)
Requirement already satisfied: google-pasta>=0.1.2 in /usr/local/lib/python3.7/site-packages (from tensorflow==2.0.0a0) (0.1.4)
Requirement already satisfied: keras-applications>=1.0.6 in /usr/local/lib/python3.7/site-packages (from tensorflow==2.0.0a0) (1.0.7)
Requirement already satisfied: gast>=0.2.0 in ./Library/Python/3.7/lib/python/site-packages (from tensorflow==2.0.0a0) (0.2.2)
Requirement already satisfied: six>=1.10.0 in ./Library/Python/3.7/lib/python/site-packages (from tensorflow==2.0.0a0) (1.12.0)
Requirement already satisfied: setuptools in /usr/local/lib/python3.7/site-packages (from protobuf>=3.6.1->tensorflow==2.0.0a0) (40.8.0)
Requirement already satisfied: markdown>=2.6.8 in ./Library/Python/3.7/lib/python/site-packages (from tb-nightly<1.14.0a20190302,>=1.14.0a20190301->tensorflow==2.0.0a0) (3.0.1)
Requirement already satisfied: werkzeug>=0.11.15 in ./Library/Python/3.7/lib/python/site-packages (from tb-nightly<1.14.0a20190302,>=1.14.0a20190301->tensorflow==2.0.0a0) (0.14.1)
Requirement already satisfied: h5py in ./Library/Python/3.7/lib/python/site-packages (from keras-applications>=1.0.6->tensorflow==2.0.0a0) (2.9.0)

使用tf.keras进行训练和评估

本教程包含了两种方式使用Tensorflow 2.0对模型进行训练、评估和推理。

• 使用内建的API函数进行训练和验证，如如model.fit()、model.evaluate()和model.predict()，该部分包含在“使用内建的方法训练和评估”目录下

• 使用GradientTape和eager执行器从头开始编写自定义的循环，该部分包含在“从头开始编写训练和评估循环”目录下

无论你采用上述的那种方式，无论对于Sequential模型、功能API创建的模型亦或是编写的继承的模型，模型的训练和评估都是严格按照相同的方式进行。

该教程不包含分布式训练 。

使用内建的方法训练和评估

当使用模型内建的方法进行训练的时候，你必须为模型提供Numpy类型的数据或者tf.data.Dataset类。在下面的例子中，我们会使用mnist的数据来示范如何使用optimizers，losses和metrics。

API概览：一个端到端的模型

我们使用下面的模型(这里使用的是功能API创建的，也可以使用Sequential或者子类继承的方式创建)

from tensorflow import keras
import tensorflow as tf
inputs = keras.Input(shape=(784, ), name='digits')
x = keras.layers.Dense(64, activation='relu', name='dense_1')(inputs)
x = keras.layers.Dense(64, activation='relu', name='dense_2')(x)
outputs = keras.layers.Dense(10, activation='softmax', name='predictions')(x)

model = keras.Model(inputs=inputs, outputs=outputs)

这是一个典型的端到端的模型，包含训练、对训练数据进行分割用于验证以及使用测试数据进行评估。

# 加载mnist数据
(x_train, y_train), (x_test, y_test) = keras.datasets.mnist.load_data()
# 对数据进行处理(这里是numpy类型的数据)
x_train = x_train.reshape(60000, 784).astype('float32')/255.0
x_test = x_test.reshape(10000, 784).astype('float32')/255.0
# 拆分出验证集
x_val = x_train[-10000:]
y_val = y_train[-10000:]
x_train = x_train[:-10000]
y_train = y_train[:-10000]
# 配置训练参数(包括optimizer, loss, metrics)
model.compile(optimizer=keras.optimizers.RMSprop(), 
                     loss=keras.losses.SparseCategoricalCrossentropy(), 
                     metrics=[keras.metrics.SparseCategoricalAccuracy()])
# 训练，输出会保留在history中
print("Fit model on training data")
history = model.fit(x_train, y_train, batch_size=64, epochs=3, 
                           # 使用验证数据进行验证
                           validation_data=(x_val, y_val))
# 输出
print('history dict: ', history.history)

# 测试
print('\nEvaluate on test data')
results = model.evaluate(x_test, y_test, batch_size=128)
print('test loss, test acc: ', results)

# 推理
print('\nGenerate predictions for 3 samples')
predictions = model.predict(x_test[:3])
print('predictions shape: ', predictions.shape)

Fit model on training data
Train on 50000 samples, validate on 10000 samples
Epoch 1/3
50000/50000 [==============================] - 3s 51us/sample - loss: 0.3450 - sparse_categorical_accuracy: 0.9020 - val_loss: 0.1878 - val_sparse_categorical_accuracy: 0.9446
Epoch 2/3
50000/50000 [==============================] - 2s 42us/sample - loss: 0.1640 - sparse_categorical_accuracy: 0.9505 - val_loss: 0.1512 - val_sparse_categorical_accuracy: 0.9560
Epoch 3/3
50000/50000 [==============================] - 2s 43us/sample - loss: 0.1200 - sparse_categorical_accuracy: 0.9630 - val_loss: 0.1267 - val_sparse_categorical_accuracy: 0.9611
history dict:  {'loss': [0.34502086482048033, 0.16395757224321367, 0.12004149877429009], 'sparse_categorical_accuracy': [0.902, 0.95052, 0.963], 'val_loss': [0.1877518826007843, 0.1512176885008812, 0.12673389554023742], 'val_sparse_categorical_accuracy': [0.9446, 0.956, 0.9611]}

Evaluate on test data
10000/10000 [==============================] - 0s 14us/sample - loss: 0.1220 - sparse_categorical_accuracy: 0.9625
test loss, test acc:  [0.12199614572525025, 0.9625]

Generate predictions for 3 samples
predictions shape:  (3, 10)

指定optimizer、loss和metrics

你必须指定优化器、损失函数以及用于观测的量(可选)才能够对模型进行训练。这些都是在compile方法中进行指定。

model.compile(optimizer=keras.optimizers.RMSprop(lr=1e-3), 
                     loss=keras.losses.SparseCategoricalCrossentropy(), 
                     metrics=[keras.metrics.SparseCategoricalAccuracy()])

metrics必须是一个列表，你可以为模型指定任意多个metrics。
如果你的模型有多个输出，那么你可以为每个输出指定不同的loss和metrics，以及每个输出的loss所占的权重，你可以在多输入多输出模型的数据载入那一段找到更多关于此的细节。
值得注意的是，很多情况下，loss和metrics可以通过简化的字符型名字来指定。

model.compile(optimizer=keras.optimizers.RMSprop(lr=1e-3), 
                     loss='sparse_categorical_crossentropy', 
                     metrics=['sparse_categorical_accuracy'])

为了后面简化代码，我这里把模型的定义和配置写作函数。

def get_uncompiled_model():
    inputs = keras.Input(shape=(784, ), name='digits')
    x = keras.layers.Dense(64, activation='relu', name='dense_1')(inputs)
    x = keras.layers.Dense(64, activation='relu', name='dense_2')(x)
    outputs = keras.layers.Dense(10, activation='softmax', name='predictions')(x)
    return keras.Model(inputs=inputs, outputs=outputs)

def get_compiled_model():
    model = get_uncompiled_model()
    model.compile(optimizer=keras.optimizers.RMSprop(lr=1e-3), 
                         loss='sparse_categorical_crossentropy', 
                         metrics=['sparse_categorical_accuracy'])
    return model

内建的optimizer、loss和metrics

一般情况下，你无需自己从头开始编写optimizer、loss和metrics，因为tf.keras已经内建了这些函数。
Optimizers：SGD()(包含动量或者不包含)、RMSprop()、Adam()等。
Losses：MeanSquaredError()、KLDIvergence()、CosineSimilarity()等。
Metrics：AUC()、Precision()、Recall()等。

自定义loss和metrics

如果你需要自己编写metrics，那么你可以通过继承Metric类来实现，你必须实现下面4个方法：

• __init__(self)，该函数中你可以创建相应的状态变量
• update_state(self, y_true, y_pred, sample_weight=None)，使用真实值和预测值来更新状态变量
• result(self)，通过状态变量来计算最终的输出结果
• reset_states(self)，来为metrics进行复位

状态的更新和计算结果需要分开进行(update_state函数和result函数分别负责)，这是因为在有些例子中，计算结果的代价十分昂贵，只能周期的进行。
下面是一个实现CategoricalTruePositives的例子，用于统计多少样本被正确分类了：

class CategoricalTruePositives(keras.metrics.Metric):
    def __init__(self, name='binary_true_positives', **kwargs):
        super(CategoricalTruePositives, self).__init__(name=name, **kwargs)
        self.true_positives = self.add_weight(name='tp', initializer='zeros')
    
    def update_state(self, y_true, y_pred, sample_weight=None):
        y_pred = tf.argmax(y_pred)
        values = tf.cast(tf.equal(tf.cast(y_pred, tf.int32), tf.cast(y_true, tf.int32)), tf.float32)
        if sample_weight is not None:
            sample_weight = tf.cast(sample_weight, tf.float32)
            values = tf.multiply(values, sample_weight)
        return self.true_positives.assign_add(tf.reduce_sum(values))
    
    def result(self):
        return tf.identity(self.true_positives