深度学习实战(二) 模型重用及冻结层

Transfer learning.

Create a new DNN that reuses all the pretrained hidden layers of the previous model, freezes them, and replaces the softmax output layer with a fresh new one.

Train this new DNN on digits 5 to 9, using only 100 images per digit, and time how long it takes. Despite this small number of examples, can you achieve high precision?

import tensorflow as tf
import numpy as np
from datetime import datetime
import os
import time

def shuffle_batch(X, y, batch_size):
    rnd_idx = np.random.permutation(len(X))
    n_batches = len(X) // batch_size
    for batch_idx in np.array_split(rnd_idx, n_batches):
        X_batch, y_batch = X[batch_idx], y[batch_idx]
        yield X_batch, y_batch

(X_train, y_train), (X_test, y_test) = tf.keras.datasets.mnist.load_data()
X_train = X_train.astype(np.float32).reshape(-1, 28*28) / 255.0
X_test = X_test.astype(np.float32).reshape(-1, 28*28) / 255.0
y_train = y_train.astype(np.int32)
y_test = y_test.astype(np.int32)
X_valid, X_train = X_train[:5000], X_train[5000:]
y_valid, y_train = y_train[:5000], y_train[5000:]

X_train = X_train[y_train > 4]
y_train = y_train[y_train > 4]-5
X_valid = X_valid[y_valid > 4]
y_valid = y_valid[y_valid > 4]-5
X_test = X_test[y_test > 4]
y_test = y_test[y_test > 4]-5

rnd_idx = np.random.permutation(len(X_train))
rnd_idx = rnd_idx[0:100]
X_train = X_train[rnd_idx,:]
y_t