文章目录
特别说明
本篇博客继承上篇 Tensorflow 笔记 Ⅳ——mnist手写数字识别,所用的文件都在上一篇博客下载文件中
mnist 全连接神经网络 TensorFlow 1.x 实现
全连接神经网络建模实现
使用多层神经元进行模型构建
本次将使用四隐藏层神经元进行模型构建,每层神经元个数为 256、128、64、32,其中我们将首次引入 dropout 层来防止多层神经元带来的过拟合问题,dropout 层是通过随机放弃部分神经元参与模型训练,让每次神经元之间的连接不同,关于具体的 drop out层将在博客中详细讲解
采用的激活函数仍是 softmax
o
u
t
p
u
t
=
f
(
z
)
=
f
(
∑
i
=
1
n
x
i
∗
w
i
+
b
)
output = f(z) = f(\sum_{i=1}^nx_i*w_i+b)
output=f(z)=f(i=1∑nxi∗wi+b)
导入必要模块
import tensorflow as tf
import numpy as np
from tensorflow.examples.tutorials.mnist import input_data
import math
import matplotlib.pyplot as plt
import time
import os
mnist = input_data.read_data_sets('./mnist_dataset/', one_hot=True)
Extracting ./mnist_dataset/train-images-idx3-ubyte.gz
Extracting ./mnist_dataset/train-labels-idx1-ubyte.gz
Extracting ./mnist_dataset/t10k-images-idx3-ubyte.gz
Extracting ./mnist_dataset/t10k-labels-idx1-ubyte.gz
构建输入层
x = tf.placeholder(tf.float32, [None, 784], name='X')
y = tf.placeholder(tf.float32, [None, 10], name='Y')
第 1 ~ 4 隐藏层
注意,多层神经网络中,生成权值不适用完全随机数,使用截断的方法 tf.truncated_normal() 生成随机数,否则容易造成 Loss = Nan 情况
H1_NN = 256
W1 = tf.Variable(tf.truncated_normal([784, H1_NN], stddev=0.1))
b1 = tf.Variable(tf.zeros([H1_NN]))
Y1 = tf.nn.relu(tf.matmul(x, W1) + b1)
Y1 = tf.nn.dropout(Y1, rate=0.2)
H2_NN = 128
W2 = tf.Variable(tf.truncated_normal([H1_NN, H2_NN], stddev=0.1))
b2 = tf.Variable(tf.zeros([H2_NN]))
Y2 = tf.nn.relu(tf.matmul(Y1, W2) + b2)
Y2 = tf.nn.dropout(Y2, rate=0.2)
H3_NN = 64
W3 = tf.Variable(tf.truncated_normal([H2_NN, H3_NN], stddev=0.1))
b3 = tf.Variable(tf.zeros([H3_NN]))
Y3 = tf.nn.relu(tf.matmul(Y2, W3) + b3)
Y3 = tf.nn.dropout(Y3, rate=0.2)
H4_NN = 32
W4 = tf.Variable(tf.truncated_normal([H3_NN, H4_NN], stddev=0.1))
b4 = tf.Variable(tf.zeros([H4_NN]))
Y4 = tf.nn.relu(tf.matmul(Y3, W4) + b4)
Y4 = tf.nn.dropout(Y4, rate=0.2)
构建输出层
W_out = tf.Variable(tf.truncated_normal([H4_NN, 10], stddev=0.1))
b_out = tf.Variable(tf.zeros([10]))
forward = tf.matmul(Y4, W_out) + b_out
pred = tf.nn.softmax(forward)
定义损失函数
原始定义方式:
loss_function = tf.reduce_mean(-tf.reduce_sum(y * tf.log(pred), reduction_indices=1))
为了避免
lim
x
→
0
l
o
g
(
x
)
\lim_{x\to0}log(x)
limx→0log(x) 造成值趋于无穷大,使得最终的 loss 值超出认可范围,将使用 TensorFlow 1.x 自带的 Softmax 交叉熵损失函数定义方法
loss_function = tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits_v2(logits=forward, labels=y))
设置训练参数
epochs = 150
batch_size = 100
total_batch = int(mnist.train.num_examples / batch_size)
display_step = 1
learning_rate = 0.01
选择优化器
optimizer = tf.train.GradientDescentOptimizer(learning_rate).minimize(loss_function)
定义准确率
correct_prediction = tf.equal(tf.argmax(pred, 1), tf.argmax(y, 1))
accuracy = tf.reduce_mean(tf.cast(correct_prediction, tf.float32))
开始训练
startTime = time.time()
sess = tf.Session()
train_loss_list = []
train_acc_list = []
val_loss_list = []
val_acc_list = []
sess.run(tf.global_variables_initializer())
for epoch in range(epochs):
if epoch > 99:
learning_rate = 0.001
for batch in range(total_batch):
xs, ys = mnist.train.next_batch(batch_size)
sess.run(optimizer, feed_dict={x:xs, y:ys})
train_loss, train_acc = sess.run([loss_function, accuracy],
feed_dict={x:mnist.train.images, y:mnist.train.labels})
val_loss, val_acc = sess.run([loss_function, accuracy],
feed_dict={x:mnist.validation.images, y:mnist.validation.labels})
train_loss_list.append(train_loss)
train_acc_list.append(train_acc)
val_loss_list.append(val_loss)
val_acc_list.append(val_acc)
if (epoch + 1) % display_step == 0:
print('Epoch: %3d' % (epoch + 1),
'train_loss=%6f' % train_loss,
'train_accuracy=%4f' % train_acc,
'val_loss=%6f' % val_loss,
'val_accuracy=%4f' % val_acc,
'learning_rate=%.4f' % learning_rate)
duration = time.time() - startTime
print('INFO:Train Finished!\nTaking for %d seconds!' % duration)
部分训练如下
Epoch: 120 train_loss=0.065860 train_accuracy=0.981527 val_loss=0.132282 val_accuracy=0.965000 learning_rate=0.0010
Epoch: 121 train_loss=0.064610 train_accuracy=0.982327 val_loss=0.125659 val_accuracy=0.966600 learning_rate=0.0010
Epoch: 122 train_loss=0.068193 train_accuracy=0.980200 val_loss=0.128001 val_accuracy=0.964800 learning_rate=0.0010
Epoch: 123 train_loss=0.062760 train_accuracy=0.982182 val_loss=0.132810 val_accuracy=0.969000 learning_rate=0.0010
Epoch: 124 train_loss=0.065039 train_accuracy=0.981309 val_loss=0.132855 val_accuracy=0.967800 learning_rate=0.0010
Epoch: 125 train_loss=0.063242 train_accuracy=0.981782 val_loss=0.127155 val_accuracy=0.971200 learning_rate=0.0010
Epoch: 126 train_loss=0.064617 train_accuracy=0.981655 val_loss=0.130808 val_accuracy=0.968800 learning_rate=0.0010
Epoch: 127 train_loss=0.062583 train_accuracy=0.982545 val_loss=0.124542 val_accuracy=0.967800 learning_rate=0.0010
Epoch: 128 train_loss=0.061034 train_accuracy=0.982873 val_loss=0.114629 val_accuracy=0.971200 learning_rate=0.0010
Epoch: 129 train_loss=0.062726 train_accuracy=0.982455 val_loss=0.124787 val_accuracy=0.969800 learning_rate=0.0010
Epoch: 130 train_loss=0.059900 train_accuracy=0.982818 val_loss=0.129608 val_accuracy=0.967800 learning_rate=0.0010
Epoch: 131 train_loss=0.059676 train_accuracy=0.982400 val_loss=0.128246 val_accuracy=0.970400 learning_rate=0.0010
Epoch: 132 train_loss=0.060196 train_accuracy=0.982491 val_loss=0.117495 val_accuracy=0.970600 learning_rate=0.0010
Epoch: 133 train_loss=0.059927 train_accuracy=0.982473 val_loss=0.125890 val_accuracy=0.971400 learning_rate=0.0010
Epoch: 134 train_loss=0.061151 train_accuracy=0.982855 val_loss=0.127866 val_accuracy=0.969600 learning_rate=0.0010
Epoch: 135 train_loss=0.059084 train_accuracy=0.982327 val_loss=0.136047 val_accuracy=0.965600 learning_rate=0.0010
Epoch: 136 train_loss=0.059557 train_accuracy=0.983018 val_loss=0.126029 val_accuracy=0.968600 learning_rate=0.0010
Epoch: 137 train_loss=0.054957 train_accuracy=0.984564 val_loss=0.119716 val_accuracy=0.970000 learning_rate=0.0010
Epoch: 138 train_loss=0.058328 train_accuracy=0.983000 val_loss=0.126019 val_accuracy=0.969600 learning_rate=0.0010
Epoch: 139 train_loss=0.054724 train_accuracy=0.984727 val_loss=0.140184 val_accuracy=0.967400 learning_rate=0.0010
Epoch: 140 train_loss=0.057109 train_accuracy=0.984309 val_loss=0.117697 val_accuracy=0.966800 learning_rate=0.0010
Epoch: 141 train_loss=0.054068 train_accuracy=0.984127 val_loss=0.130676 val_accuracy=0.970800 learning_rate=0.0010
Epoch: 142 train_loss=0.055419 train_accuracy=0.984455 val_loss=0.131557 val_accuracy=0.968200 learning_rate=0.0010
Epoch: 143 train_loss=0.053236 train_accuracy=0.984491 val_loss=0.120054 val_accuracy=0.971600 learning_rate=0.0010
Epoch: 144 train_loss=0.055174 train_accuracy=0.984273 val_loss=0.128828 val_accuracy=0.969400 learning_rate=0.0010
Epoch: 145 train_loss=0.051997 train_accuracy=0.985164 val_loss=0.128098 val_accuracy=0.970000 learning_rate=0.0010
Epoch: 146 train_loss=0.052802 train_accuracy=0.984127 val_loss=0.133567 val_accuracy=0.969400 learning_rate=0.0010
Epoch: 147 train_loss=0.052215 train_accuracy=0.984691 val_loss=0.131717 val_accuracy=0.970400 learning_rate=0.0010
Epoch: 148 train_loss=0.050656 train_accuracy=0.985600 val_loss=0.138663 val_accuracy=0.968600 learning_rate=0.0010
Epoch: 149 train_loss=0.051594 train_accuracy=0.985455 val_loss=0.130331 val_accuracy=0.967800 learning_rate=0.0010
Epoch: 150 train_loss=0.049943 train_accuracy=0.985691 val_loss=0.125346 val_accuracy=0.970800 learning_rate=0.0010
INFO:Train Finished!
Taking for 254 seconds!
test_acc = sess.run(accuracy, feed_dict={x:mnist.test.images, y:mnist.test.labels})
print('Test dataset accuracy: %f' % test_acc)
Test dataset accuracy: 0.969800
损失与准确率可视化
fig = plt.gcf()
fig.set_size_inches(10, 5)
ax1 = fig.add_subplot(111)
ax1.set_title('Train and Validation Picture')
ax1.set_ylabel('Loss value')
line1, = ax1.plot(train_loss_list, color=(0.5, 0.5, 1.0), label='Loss train')
line2, = ax1.plot(val_loss_list, color=(0.5, 1.0, 0.5), label='Loss valid')
ax2 = ax1.twinx()
ax2.set_ylabel('Accuracy value')
line3, = ax2.plot(train_acc_list, color=(0.5, 0.5, 0.5), label='Accuracy train')
line4, = ax2.plot(val_acc_list, color=(1, 0, 0), label='Accuracy valid')
plt.legend(handles=(line1, line2, line3, line4), loc='best')
plt.show()
sess.close()
定义全连接函数简化网络定义代码并保存模型
def fcn_layer(inputs,
input_dim,
output_dim,
activation=None):
W = tf.Variable(tf.truncated_normal([input_dim, output_dim], stddev=0.1))
b = tf.Variable(tf.zeros([output_dim]))
XWb = tf.matmul(inputs, W) + b
if activation is None:
outputs = XWb
else:
outputs = activation(XWb)
outputs = tf.nn.dropout(outputs, rate=0.2)
return outputs
x = tf.placeholder(tf.float32, [None, 784], name='X')
h1 = fcn_layer(x, 784, 256, tf.nn.relu)
h2 = fcn_layer(h1, 256, 128, tf.nn.relu)
h3 = fcn_layer(h2, 128, 64, tf.nn.relu)
h4 = fcn_layer(h3, 64, 32, tf.nn.relu)
forward = fcn_layer(h4, 32, 10, None)
pred = tf.nn.softmax(forward)
epochs = 201
batch_size = 100
total_batch = int(mnist.train.num_examples / batch_size)
display_step = 1
learning_rate = 0.01
save_step = 5
ckpt_dir = './mnist_ckpt'
if not os.path.exists(ckpt_dir):
os.makedirs(ckpt_dir)
loss_function = tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits_v2(logits=forward, labels=y))
optimizer = tf.train.GradientDescentOptimizer(learning_rate).minimize(loss_function)
correct_prediction = tf.equal(tf.argmax(pred, 1), tf.argmax(y, 1))
accuracy = tf.reduce_mean(tf.cast(correct_prediction, tf.float32))
startTime = time.time()
saver = tf.train.Saver()
sess = tf.Session()
train_loss_list = []
train_acc_list = []
val_loss_list = []
val_acc_list = []
sess.run(tf.global_variables_initializer())
for epoch in range(epochs):
if epoch > 99:
learning_rate = 0.001
for batch in range(total_batch):
xs, ys = mnist.train.next_batch(batch_size)
sess.run(optimizer, feed_dict={x:xs, y:ys})
train_loss, train_acc = sess.run([loss_function, accuracy],
feed_dict={x:mnist.train.images, y:mnist.train.labels})
val_loss, val_acc = sess.run([loss_function, accuracy],
feed_dict={x:mnist.validation.images, y:mnist.validation.labels})
train_loss_list.append(train_loss)
train_acc_list.append(train_acc)
val_loss_list.append(val_loss)
val_acc_list.append(val_acc)
if (epoch + 1) % display_step == 0:
print('Epoch: %3d' % (epoch + 1),
'train_loss=%6f' % train_loss,
'train_accuracy=%4f' % train_acc,
'val_loss=%6f' % val_loss,
'val_accuracy=%4f' % val_acc,
'learning_rate=%.4f' % learning_rate)
if (epoch + 1) % save_step == 0 or (epoch + 1) == epochs:
saver.save(sess, os.path.join(ckpt_dir, 'mnist_model_{0:03d}.ckpt'.format(epoch + 1)))
print('INFO:mnist_model_{0:03d}.ckpt saved'.format(epoch + 1))
duration = time.time() - startTime
print('INFO:Model saved\n', 'INFO:Train Finished!\nTaking for %d seconds!' % duration)
部分训练过程如下
Epoch: 180 train_loss=0.044205 train_accuracy=0.987273 val_loss=0.141515 val_accuracy=0.969000 learning_rate=0.0010
INFO:mnist_model_180.ckpt saved
Epoch: 181 train_loss=0.042262 train_accuracy=0.987473 val_loss=0.129795 val_accuracy=0.972400 learning_rate=0.0010
Epoch: 182 train_loss=0.040721 train_accuracy=0.988127 val_loss=0.141154 val_accuracy=0.969200 learning_rate=0.0010
Epoch: 183 train_loss=0.041534 train_accuracy=0.987800 val_loss=0.135395 val_accuracy=0.969800 learning_rate=0.0010
Epoch: 184 train_loss=0.041826 train_accuracy=0.987873 val_loss=0.133711 val_accuracy=0.971600 learning_rate=0.0010
Epoch: 185 train_loss=0.042363 train_accuracy=0.988218 val_loss=0.137348 val_accuracy=0.970800 learning_rate=0.0010
INFO:mnist_model_185.ckpt saved
Epoch: 186 train_loss=0.041273 train_accuracy=0.988145 val_loss=0.139761 val_accuracy=0.968400 learning_rate=0.0010
Epoch: 187 train_loss=0.041105 train_accuracy=0.987873 val_loss=0.131214 val_accuracy=0.972200 learning_rate=0.0010
Epoch: 188 train_loss=0.038462 train_accuracy=0.988927 val_loss=0.116528 val_accuracy=0.972000 learning_rate=0.0010
Epoch: 189 train_loss=0.040760 train_accuracy=0.988255 val_loss=0.136059 val_accuracy=0.969800 learning_rate=0.0010
Epoch: 190 train_loss=0.039955 train_accuracy=0.988545 val_loss=0.130248 val_accuracy=0.970600 learning_rate=0.0010
INFO:mnist_model_190.ckpt saved
Epoch: 191 train_loss=0.040176 train_accuracy=0.988982 val_loss=0.136247 val_accuracy=0.971000 learning_rate=0.0010
Epoch: 192 train_loss=0.036810 train_accuracy=0.989509 val_loss=0.139191 val_accuracy=0.970000 learning_rate=0.0010
Epoch: 193 train_loss=0.039340 train_accuracy=0.988782 val_loss=0.139221 val_accuracy=0.967800 learning_rate=0.0010
Epoch: 194 train_loss=0.038176 train_accuracy=0.988709 val_loss=0.119204 val_accuracy=0.972800 learning_rate=0.0010
Epoch: 195 train_loss=0.039080 train_accuracy=0.988709 val_loss=0.138383 val_accuracy=0.968600 learning_rate=0.0010
INFO:mnist_model_195.ckpt saved
Epoch: 196 train_loss=0.039176 train_accuracy=0.988945 val_loss=0.127481 val_accuracy=0.972000 learning_rate=0.0010
Epoch: 197 train_loss=0.038120 train_accuracy=0.988545 val_loss=0.128279 val_accuracy=0.973200 learning_rate=0.0010
Epoch: 198 train_loss=0.035479 train_accuracy=0.989873 val_loss=0.138994 val_accuracy=0.967800 learning_rate=0.0010
Epoch: 199 train_loss=0.037645 train_accuracy=0.989036 val_loss=0.134254 val_accuracy=0.967200 learning_rate=0.0010
Epoch: 200 train_loss=0.035901 train_accuracy=0.989291 val_loss=0.133777 val_accuracy=0.968600 learning_rate=0.0010
INFO:mnist_model_200.ckpt saved
Epoch: 201 train_loss=0.036870 train_accuracy=0.989473 val_loss=0.128872 val_accuracy=0.969800 learning_rate=0.0010
INFO:mnist_model_201.ckpt saved
INFO:Model saved
INFO:Train Finished!
Taking for 335 seconds!
test_acc = sess.run(accuracy, feed_dict={x:mnist.test.images, y:mnist.test.labels})
print('Test dataset accuracy: %f' % test_acc)
Test dataset accuracy: 0.970000
fig = plt.gcf()
fig.set_size_inches(10, 5)
ax1 = fig.add_subplot(111)
ax1.set_title('Train and Validation Picture')
ax1.set_ylabel('Loss value')
line1, = ax1.plot(train_loss_list, color=(0.5, 0.5, 1.0), label='Loss train')
line2, = ax1.plot(val_loss_list, color=(0.5, 1.0, 0.5), label='Loss valid')
ax2 = ax1.twinx()
ax2.set_ylabel('Accuracy value')
line3, = ax2.plot(train_acc_list, color=(0.5, 0.5, 0.5), label='Accuracy train')
line4, = ax2.plot(val_acc_list, color=(1, 0, 0), label='Accuracy valid')
plt.legend(handles=(line1, line2, line3, line4), loc='best')
plt.show()
sess.close()
恢复模型进行预测
模型仅仅保存着训练出的权重参数,并未定义网络结构,需要定义网络结构,读取模型,在这里由于前面已运行网络结构,这里就不再重新定义
saver = tf.train.Saver()
sess = tf.Session()
sess.run(tf.global_variables_initializer())
ckpt_dir = './mnist_ckpt'
ckpt = tf.train.get_checkpoint_state(ckpt_dir)
if ckpt and ckpt.model_checkpoint_path:
saver.restore(sess, ckpt.model_checkpoint_path)
print('Restore model from ' + ckpt.model_checkpoint_path)
INFO:tensorflow:Restoring parameters from ./mnist_ckpt\mnist_model_201.ckpt
Restore model from ./mnist_ckpt\mnist_model_201.ckpt
准确率不同缘由
因为只保存了模型权重,并未保存网络结构,网络含有 dropout 层,每次读入的权重一样,但网络结构会因为 dropout 层不同,去掉 dropout 层,准确率就会跟前面测试集准确率一样,或者将网络结构一并保存
test_acc = sess.run(accuracy, feed_dict={x:mnist.test.images, y:mnist.test.labels})
print('Test dataset accuracy: %f' % test_acc)
Test dataset accuracy: 0.969300
随机可视化显示预测值与标签值
def plot_apply_images(num):
if num < 1:
print('INFO:The number of input pictures must be greater than zero!')
else:
choose_list = []
for i in range(num):
choose_n = np.random.randint(len(mnist.test.images))
choose_list.append(choose_n)
fig = plt.gcf()
fig.set_size_inches(18, 5 * math.ceil(num / 3))
prediction_result = sess.run(tf.argmax(pred, 1), feed_dict={x:mnist.test.images})
for i in range(num):
ax_img = plt.subplot(math.ceil(num / 3), 3, i + 1)
plt_img = mnist.test.images[choose_list[i]].reshape(28, 28)
ax_img.imshow(plt_img, cmap='binary')
ax_img.set_title('Original label:' \
+ str(np.argmax(mnist.test.labels[choose_list[i]])) \
+ ' Predict label:' \
+ str(prediction_result[choose_list[i]]),
fontsize=10)
ax_img.set_xticks([])
ax_img.set_yticks([])
plt.show()
plot_apply_images(9)
随机从预测错误中图像
prediction_result = sess.run(tf.argmax(pred, 1), feed_dict={x:mnist.test.images})
predict_bool = (prediction_result == np.argmax(mnist.test.labels, 1))
predict_bool_list = list(predict_bool)
err_list = []
for i in range(len(predict_bool_list)):
if not predict_bool_list[i]:
err_list.append(i)
print('There are %d images that are misrecongnized' % len(err_list))
There are 313 images that are misrecongnized
def plot_wrong_images(num):
prediction_result = sess.run(tf.argmax(pred, 1), feed_dict={x:mnist.test.images})
predict_bool = (prediction_result == np.argmax(mnist.test.labels, 1))
predict_bool_list = list(predict_bool)
err_list = []
for i in range(len(predict_bool_list)):
if not predict_bool_list[i]:
err_list.append(i)
print('There are %d images that are misrecongnized' % len(err_list))
if num < 1:
print('INFO:The number of input pictures must be greater than zero!')
else:
choose_list = []
for i in range(num):
choose_n = np.random.randint(len(err_list))
choose_list.append(err_list[choose_n])
fig = plt.gcf()
fig.set_size_inches(18, 5 * math.ceil(num / 3))
for i in range(num):
ax_img = plt.subplot(math.ceil(num / 3), 3, i + 1)
plt_img = mnist.test.images[choose_list[i]].reshape(28, 28)
ax_img.imshow(plt_img, cmap='binary')
ax_img.set_title('Original label:' \
+ str(np.argmax(mnist.test.labels[choose_list[i]])) \
+ ' Predict label:' \
+ str(prediction_result[choose_list[i]]),
fontsize=10)
ax_img.set_xticks([])
ax_img.set_yticks([])
plt.show()
plot_wrong_images(9)
There are 282 images that are misrecongnized
mnist 全连接神经网络 TensorFlow 2.x 实现
TensorFlow 低阶 API
导入必要的包
import tensorflow as tf
import numpy as np
import matplotlib.pyplot as plt
tf.__version__
'2.0.0'
多层神经网络
数据预处理
mnist = tf.keras.datasets.mnist
(train_images, train_labels), (test_images, test_labels) = mnist.load_data()
total_num = len(train_images)
valid_split = 0.2
train_num = int(total_num * (1 - valid_split))
train_x = train_images[:train_num]
train_y = train_labels[:train_num]
valid_x = train_images[train_num:]
valid_y = train_labels[train_num:]
test_x = test_images
test_y = test_labels
train_x = train_x.reshape(-1, 784)
valid_x = valid_x.reshape(-1, 784)
test_x = test_x.reshape(-1, 784)
train_x = tf.cast(train_x / 255.0, tf.float32)
valid_x = tf.cast(valid_x / 255.0, tf.float32)
test_x = tf.cast(test_x / 255.0, tf.float32)
train_y = tf.one_hot(train_y, depth=10)
valid_y = tf.one_hot(valid_y, depth=10)
test_y = tf.one_hot(test_y, depth=10)
创建变量
Input_dim = 784
H1_NN = 64
W1 = tf.Variable(tf.random.normal([Input_dim, H1_NN], mean=0.0, stddev=1.0, dtype=tf.float32))
B1 = tf.Variable(tf.zeros([H1_NN]), dtype=tf.float32)
H2_NN = 32
W2 = tf.Variable(tf.random.normal([H1_NN, H2_NN], mean=0.0, stddev=1.0, dtype=tf.float32))
B2 = tf.Variable(tf.zeros([H2_NN]), dtype=tf.float32)
H3_NN = 10
W3 = tf.Variable(tf.random.normal([H2_NN, H3_NN], mean=0.0, stddev=1.0, dtype=tf.float32))
B3 = tf.Variable(tf.zeros([10]), dtype=tf.float32)
建立待优化列表
W = [W1, W2, W3]
B = [B1, B2, B3]
模型建立
def model(x, w, b):
x = tf.matmul(x, w[0] + b[0])
x = tf.nn.relu(x)
x = tf.matmul(x, w[1] + b[1])
x = tf.nn.relu(x)
x = tf.matmul(x, w[2] + b[2])
pred = tf.nn.softmax(x)
return pred
定义损失函数
def loss(x, y, w, b):
pred = model(x, w, b)
loss_ = tf.keras.losses.categorical_crossentropy(y_true=y, y_pred=pred)
return tf.reduce_mean(loss_)
定义梯度下降
def grad(x, y, w, b):
var_list = w + b
with tf.GradientTape() as tape:
loss_ = loss(x, y, w, b)
return tape.gradient(loss_, var_list)
设置训练参数
epochs = 400
batch_size = 50
learning_rate = 0.001
定义优化器
optimizer = tf.keras.optimizers.Adam(learning_rate=learning_rate)
定义准确率
def accuracy(x, y, w, b):
pred = model(x, w, b)
correct_prediction = tf.equal(tf.argmax(pred, 1), tf.argmax(y, 1))
return tf.reduce_mean(tf.cast(correct_prediction, tf.float32))
total_step = int(train_num / batch_size)
loss_list_train = []
loss_list_valid = []
acc_list_train = []
acc_list_valid = []
for epoch in range(epochs):
if epoch > 10:
learning_rate = 0.00001
for step in range(total_step):
xs = train_x[step * batch_size:(step + 1) * batch_size]
ys = train_y[step * batch_size:(step + 1) * batch_size]
grads = grad(xs, ys, W, B)
optimizer.apply_gradients(zip(grads, W + B))
loss_train = loss(train_x, train_y, W, B).numpy()
loss_valid = loss(valid_x, valid_y, W, B).numpy()
acc_train = accuracy(train_x, train_y, W, B).numpy()
acc_valid = accuracy(valid_x, valid_y, W, B).numpy()
loss_list_train.append(loss_train)
loss_list_valid.append(loss_valid)
acc_list_train.append(acc_train)
acc_list_valid.append(acc_valid)
print('Epoch: {:03}'.format(epoch + 1),
'train_loss= %.6f' % loss_train,
'train_acc=%.6f' % acc_train,
'val_loss= %.6f' % loss_valid,
'val_acc=%.6f' % acc_valid,
'learning_rate=%.6f' % learning_rate)
print('INFO:Train Finished!')
部分训练过程如下
Epoch: 380 train_loss= 0.163936 train_acc=0.989479 val_loss= 0.480753 val_acc=0.968667 learning_rate=0.000010
Epoch: 381 train_loss= 0.222664 train_acc=0.984979 val_loss= 0.571479 val_acc=0.962583 learning_rate=0.000010
Epoch: 382 train_loss= 0.193336 train_acc=0.987521 val_loss= 0.521553 val_acc=0.966417 learning_rate=0.000010
Epoch: 383 train_loss= 0.157278 train_acc=0.989875 val_loss= 0.493450 val_acc=0.967917 learning_rate=0.000010
Epoch: 384 train_loss= 0.149343 train_acc=0.990354 val_loss= 0.482244 val_acc=0.968583 learning_rate=0.000010
Epoch: 385 train_loss= 0.170983 train_acc=0.988792 val_loss= 0.516505 val_acc=0.966500 learning_rate=0.000010
Epoch: 386 train_loss= 0.179502 train_acc=0.988333 val_loss= 0.516758 val_acc=0.966417 learning_rate=0.000010
Epoch: 387 train_loss= 0.177936 train_acc=0.988396 val_loss= 0.536245 val_acc=0.965250 learning_rate=0.000010
Epoch: 388 train_loss= 0.156727 train_acc=0.989937 val_loss= 0.486693 val_acc=0.968417 learning_rate=0.000010
Epoch: 389 train_loss= 0.214938 train_acc=0.986083 val_loss= 0.537136 val_acc=0.965833 learning_rate=0.000010
Epoch: 390 train_loss= 0.149115 train_acc=0.990417 val_loss= 0.452357 val_acc=0.970583 learning_rate=0.000010
Epoch: 391 train_loss= 0.176035 train_acc=0.988583 val_loss= 0.553299 val_acc=0.963833 learning_rate=0.000010
Epoch: 392 train_loss= 0.173085 train_acc=0.988937 val_loss= 0.467770 val_acc=0.969833 learning_rate=0.000010
Epoch: 393 train_loss= 0.162568 train_acc=0.989604 val_loss= 0.455658 val_acc=0.970250 learning_rate=0.000010
Epoch: 394 train_loss= 0.212621 train_acc=0.986042 val_loss= 0.525348 val_acc=0.965917 learning_rate=0.000010
Epoch: 395 train_loss= 0.179633 train_acc=0.988521 val_loss= 0.499386 val_acc=0.967500 learning_rate=0.000010
Epoch: 396 train_loss= 0.157325 train_acc=0.989937 val_loss= 0.524120 val_acc=0.966417 learning_rate=0.000010
Epoch: 397 train_loss= 0.235230 train_acc=0.984333 val_loss= 0.562348 val_acc=0.963833 learning_rate=0.000010
Epoch: 398 train_loss= 0.154102 train_acc=0.990062 val_loss= 0.499408 val_acc=0.967917 learning_rate=0.000010
Epoch: 399 train_loss= 0.178979 train_acc=0.988333 val_loss= 0.535948 val_acc=0.965333 learning_rate=0.000010
Epoch: 400 train_loss= 0.153352 train_acc=0.990021 val_loss= 0.471192 val_acc=0.969750 learning_rate=0.000010
INFO:Train Finished!
损失率与准确率可视化
fig = plt.gcf()
fig.set_size_inches(10, 5)
ax1 = fig.add_subplot(111)
ax1.set_title('Train and Validation Picture')
ax1.set_ylabel('Loss value')
line1, = ax1.plot(loss_list_train, color=(0.5, 0.5, 1.0), label='Loss train')
line2, = ax1.plot(loss_list_valid, color=(0.5, 1.0, 0.5), label='Loss valid')
ax2 = ax1.twinx()
ax2.set_ylabel('Accuracy value')
line3, = ax2.plot(acc_list_train, color=(0.5, 0.5, 0.5), label='Accuracy train')
line4, = ax2.plot(acc_list_valid, color=(1, 0, 0), label='Accuracy valid')
plt.legend(handles=(line1, line2, line3, line4), loc='best')
plt.show()
模型评估
acc_test = accuracy(test_x, test_y, W, B).numpy()
print('Test Accuracy:', acc_test)
Test Accuracy: 0.9694
Keras 序列模型建模
基于 onehot 编码
import tensorflow as tf
import numpy as np
import matplotlib.pyplot as plt
import math
tf.__version__
'2.0.0'
mnist = tf.keras.datasets.mnist
(train_images, train_labels), (test_images, test_labels) = mnist.load_data()
train_images = train_images / 255.0
test_images = test_images / 255.0
train_labels_onehot = tf.one_hot(train_labels, depth=10).numpy()
test_labels_onehot = tf.one_hot(test_labels, depth=10).numpy()
模型建立
model = tf.keras.models.Sequential()
model.add(tf.keras.layers.Flatten(input_shape=(28, 28)))
model.add(tf.keras.layers.Dense(units=256,
kernel_initializer='normal',
activation='relu'))
model.add(tf.keras.layers.Dense(units=128,
kernel_initializer='normal',
activation='relu'))
model.add(tf.keras.layers.Dense(units=64,
kernel_initializer='normal',
activation='relu'))
model.add(tf.keras.layers.Dense(units=32,
kernel_initializer='normal',
activation='relu'))
model.add(tf.keras.layers.Dense(10, activation='softmax'))
模型摘要
model.summary()
Model: "sequential"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
flatten (Flatten) (None, 784) 0
_________________________________________________________________
dense (Dense) (None, 256) 200960
_________________________________________________________________
dense_1 (Dense) (None, 128) 32896
_________________________________________________________________
dense_2 (Dense) (None, 64) 8256
_________________________________________________________________
dense_3 (Dense) (None, 32) 2080
_________________________________________________________________
dense_4 (Dense) (None, 10) 330
=================================================================
Total params: 244,522
Trainable params: 244,522
Non-trainable params: 0
_________________________________________________________________
参数设置及训练
model.compile(optimizer='adam',
loss='categorical_crossentropy',
metrics=['accuracy'])
train_epochs = 10
batch_size = 40
train_history = model.fit(train_images, train_labels_onehot,
validation_split=0.2,
epochs=train_epochs,
batch_size=batch_size,
verbose=1)
Train on 48000 samples, validate on 12000 samples
Epoch 1/10
48000/48000 [==============================] - 145s 3ms/sample - loss: 0.3239 - accuracy: 0.9032 - val_loss: 0.1386 - val_accuracy: 0.9586
Epoch 2/10
48000/48000 [==============================] - 141s 3ms/sample - loss: 0.1158 - accuracy: 0.9654 - val_loss: 0.1302 - val_accuracy: 0.9592
Epoch 3/10
48000/48000 [==============================] - 142s 3ms/sample - loss: 0.0807 - accuracy: 0.9758 - val_loss: 0.1019 - val_accuracy: 0.9699
Epoch 4/10
48000/48000 [==============================] - 142s 3ms/sample - loss: 0.0575 - accuracy: 0.9824 - val_loss: 0.0994 - val_accuracy: 0.9714
Epoch 5/10
48000/48000 [==============================] - 141s 3ms/sample - loss: 0.0468 - accuracy: 0.9853 - val_loss: 0.1204 - val_accuracy: 0.9672
Epoch 6/10
48000/48000 [==============================] - 142s 3ms/sample - loss: 0.0386 - accuracy: 0.9883 - val_loss: 0.1029 - val_accuracy: 0.9723
Epoch 7/10
48000/48000 [==============================] - 143s 3ms/sample - loss: 0.0313 - accuracy: 0.9898 - val_loss: 0.1074 - val_accuracy: 0.9737
Epoch 8/10
48000/48000 [==============================] - 141s 3ms/sample - loss: 0.0298 - accuracy: 0.9906 - val_loss: 0.1065 - val_accuracy: 0.9740
Epoch 9/10
48000/48000 [==============================] - 142s 3ms/sample - loss: 0.0248 - accuracy: 0.9920 - val_loss: 0.0963 - val_accuracy: 0.9778
Epoch 10/10
48000/48000 [==============================] - 143s 3ms/sample - loss: 0.0221 - accuracy: 0.9929 - val_loss: 0.1000 - val_accuracy: 0.9753
train_history.history 保存了训练中,四种基本数据 loss,accuracy,val_loss,val_accuracy 的值,并以字典形式存储
train_history.history
{'loss': [0.32386956638656555,
0.11580678951577283,
0.08069420200908402,
0.057474158356781116,
0.046842153635103995,
0.03864645366907401,
0.03133942916045271,
0.02975376448806249,
0.02482963896606937,
0.02210053606606228],
'accuracy': [0.90316665,
0.9653958,
0.97577083,
0.982375,
0.9853333,
0.98833334,
0.9897708,
0.99058336,
0.992,
0.99291664],
'val_loss': [0.1386103215805876,
0.13024117694740805,
0.10186999143411715,
0.09937536970265985,
0.1203663934887542,
0.10294981063261123,
0.10740806087201539,
0.10651071226072721,
0.09627585080283704,
0.10002141902339645],
'val_accuracy': [0.95858335,
0.95925,
0.96991664,
0.97141665,
0.96716666,
0.9723333,
0.97375,
0.974,
0.97783333,
0.97533333]}
fig = plt.gcf()
fig.set_size_inches(10, 5)
ax1 = fig.add_subplot(111)
ax1.set_title('Train and Validation Picture')
ax1.set_ylabel('Loss value')
line1, = ax1.plot(train_history.history['loss'], color=(0.5, 0.5, 1.0), label='Loss train')
line2, = ax1.plot(train_history.history['val_loss'], color=(0.5, 1.0, 0.5), label='Loss valid')
ax2 = ax1.twinx()
ax2.set_ylabel('Accuracy value')
line3, = ax2.plot(train_history.history['accuracy'], color=(0.5, 0.5, 0.5), label='Accuracy train')
line4, = ax2.plot(train_history.history['val_accuracy'], color=(1, 0, 0), label='Accuracy valid')
plt.legend(handles=(line1, line2, line3, line4), loc='best')
plt.show()
模型评估
返回值为一个列表,分别赋值给 test_loss,test_acc
test_loss, test_acc = model.evaluate(test_images, test_labels_onehot, verbose=2)
print('test_loss:', test_loss,
'\ntest_acc:', test_acc,
'\nmetrics_names:', model.metrics_names)
10000/1 - 25s - loss: 0.0496 - accuracy: 0.9759
test_loss: 0.09910532588202041
test_acc: 0.9759
metrics_names: ['loss', 'accuracy']
模型预测
def plot_apply_images(num):
if num < 1:
print('INFO:The number of input pictures must be greater than zero!')
else:
test_pred = model.predict_classes(test_images)
choose_list = []
for i in range(num):
choose_n = np.random.randint(len(test_images))
choose_list.append(choose_n)
fig = plt.gcf()
fig.set_size_inches(18, 5 * math.ceil(num / 3))
for i in range(num):
ax_img = plt.subplot(math.ceil(num / 3), 3, i + 1)
plt_img = test_images[choose_list[i]]
ax_img.imshow(plt_img, cmap='binary')
ax_img.set_title('Original label:' \
+ str(test_labels[choose_list[i]]) \
+ ' Predict label:' \
+ str(test_pred[choose_list[i]]),
fontsize=10)
ax_img.set_xticks([])
ax_img.set_yticks([])
plt.show()
plot_apply_images(9)
注意
以下两个 cell 等价,predict_classes 能直接预测整数分类值,predict返回 onehot 预测值
test_pred = model.predict(test_images)
print('test_pred.shape', test_pred.shape,
'\ntest_pred[0]', np.argmax(test_pred[0]))
test_pred = model.predict_classes(test_images)
print('test_pred.shape', test_pred.shape,
'\ntest_pred[0]', test_pred[0],
'\ntest_labels[0]', test_labels[0])
test_pred.shape (10000, 10)
test_pred[0] 7
test_pred.shape (10000,)
test_pred[0] 7
test_labels[0] 7
基于整数型编码
mnist = tf.keras.datasets.mnist
(train_images, train_labels), (test_images, test_labels) = mnist.load_data()
train_images = train_images / 255.0
test_images = test_images / 255.0
model = tf.keras.models.Sequential([
tf.keras.layers.Flatten(input_shape=(28, 28)),
tf.keras.layers.Dense(units=64,
kernel_initializer='normal',
activation='relu'),
tf.keras.layers.Dense(units=32,
kernel_initializer='normal',
activation='relu'),
tf.keras.layers.Dense(10, activation='softmax')
])
model.summary()
Model: "sequential_1"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
flatten_1 (Flatten) (None, 784) 0
_________________________________________________________________
dense_5 (Dense) (None, 64) 50240
_________________________________________________________________
dense_6 (Dense) (None, 32) 2080
_________________________________________________________________
dense_7 (Dense) (None, 10) 330
=================================================================
Total params: 52,650
Trainable params: 52,650
Non-trainable params: 0
_________________________________________________________________
model.compile(optimizer='adam',
loss='sparse_categorical_crossentropy',
metrics=['accuracy'])
train_epochs = 10
batch_size = 40
train_history = model.fit(train_images, train_labels,
validation_split=0.2,
epochs=train_epochs,
batch_size=batch_size,
verbose=1)
Train on 48000 samples, validate on 12000 samples
Epoch 1/10
48000/48000 [==============================] - 5s 99us/sample - loss: 0.3811 - accuracy: 0.8907 - val_loss: 0.1879 - val_accuracy: 0.9455
Epoch 2/10
48000/48000 [==============================] - 4s 88us/sample - loss: 0.1703 - accuracy: 0.9496 - val_loss: 0.1588 - val_accuracy: 0.9517
Epoch 3/10
48000/48000 [==============================] - 4s 85us/sample - loss: 0.1252 - accuracy: 0.9621 - val_loss: 0.1207 - val_accuracy: 0.9653
Epoch 4/10
48000/48000 [==============================] - 4s 85us/sample - loss: 0.1004 - accuracy: 0.9698 - val_loss: 0.1209 - val_accuracy: 0.9644
Epoch 5/10
48000/48000 [==============================] - 4s 85us/sample - loss: 0.0806 - accuracy: 0.9752 - val_loss: 0.1053 - val_accuracy: 0.9693
Epoch 6/10
48000/48000 [==============================] - 4s 85us/sample - loss: 0.0673 - accuracy: 0.9789 - val_loss: 0.1070 - val_accuracy: 0.9668
Epoch 7/10
48000/48000 [==============================] - 4s 87us/sample - loss: 0.0572 - accuracy: 0.9825 - val_loss: 0.1037 - val_accuracy: 0.9701
Epoch 8/10
48000/48000 [==============================] - 4s 85us/sample - loss: 0.0481 - accuracy: 0.9851 - val_loss: 0.1038 - val_accuracy: 0.9701
Epoch 9/10
48000/48000 [==============================] - 4s 85us/sample - loss: 0.0440 - accuracy: 0.9861 - val_loss: 0.1091 - val_accuracy: 0.9694
Epoch 10/10
48000/48000 [==============================] - 4s 85us/sample - loss: 0.0351 - accuracy: 0.9884 - val_loss: 0.1005 - val_accuracy: 0.9722
训练可视化
fig = plt.gcf()
fig.set_size_inches(10, 5)
ax1 = fig.add_subplot(111)
ax1.set_title('Train and Validation Picture')
ax1.set_ylabel('Loss value')
line1, = ax1.plot(train_history.history['loss'], color=(0.5, 0.5, 1.0), label='Loss train')
line2, = ax1.plot(train_history.history['val_loss'], color=(0.5, 1.0, 0.5), label='Loss valid')
ax2 = ax1.twinx()
ax2.set_ylabel('Accuracy value')
line3, = ax2.plot(train_history.history['accuracy'], color=(0.5, 0.5, 0.5), label='Accuracy train')
line4, = ax2.plot(train_history.history['val_accuracy'], color=(1, 0, 0), label='Accuracy valid')
plt.legend(handles=(line1, line2, line3, line4), loc='best')
plt.show()
模型预测
def plot_apply_images(num):
if num < 1:
print('INFO:The number of input pictures must be greater than zero!')
else:
test_pred = model.predict_classes(test_images)
choose_list = []
for i in range(num):
choose_n = np.random.randint(len(test_images))
choose_list.append(choose_n)
fig = plt.gcf()
fig.set_size_inches(18, 5 * math.ceil(num / 3))
for i in range(num):
ax_img = plt.subplot(math.ceil(num / 3), 3, i + 1)
plt_img = test_images[choose_list[i]]
ax_img.imshow(plt_img, cmap='binary')
ax_img.set_title('Original label:' \
+ str(test_labels[choose_list[i]]) \
+ ' Predict label:' \
+ str(test_pred[choose_list[i]]),
fontsize=10)
ax_img.set_xticks([])
ax_img.set_yticks([])
plt.show()
plot_apply_images(9)
mnist TensorFlow 1.x Tensorboard 应用
以 mnist 手写数字识别为例
为了简单方便,我们以 单个神经元为例进行
import tensorflow as tf
import numpy as np
from tensorflow.examples.tutorials.mnist import input_data
import math
import matplotlib.pyplot as plt
import os
mnist = input_data.read_data_sets('./mnist_dataset/', one_hot=True)
Extracting ./mnist_dataset/train-images-idx3-ubyte.gz
Extracting ./mnist_dataset/train-labels-idx1-ubyte.gz
Extracting ./mnist_dataset/t10k-images-idx3-ubyte.gz
Extracting ./mnist_dataset/t10k-labels-idx1-ubyte.gz
模型建立与训练
x = tf.placeholder(tf.float32, [None, 784], name='X')
y = tf.placeholder(tf.float32, [None, 10], name='Y')
image_shape_input = tf.reshape(x, [-1, 28, 28, 1])
tf.summary.image('input', image_shape_input, 9)
W = tf.Variable(tf.random_normal([784, 10]), name='W')
b = tf.Variable(tf.zeros([10]), name='b')
forward = tf.matmul(x, W) + b
pred = tf.nn.softmax(forward)
epochs = 100
batch_size = 100
total_batch = int(mnist.train.num_examples / batch_size)
display_step = 1
learning_rate = 0.01
save_step = 5
ckpt_dir = './mnist_Tensorboard_tutorial_ckpt'
if not os.path.exists(ckpt_dir):
os.makedirs(ckpt_dir)
loss_function = tf.reduce_mean(-tf.reduce_sum(y * tf.log(pred), reduction_indices=1))
optimizer = tf.train.GradientDescentOptimizer(learning_rate).minimize(loss_function)
correct_prediction = tf.equal(tf.argmax(pred, 1), tf.argmax(y, 1))
accuracy = tf.reduce_mean(tf.cast(correct_prediction, tf.float32))
tf.summary.histogram('forward', forward)
tf.summary.scalar('loss', loss_function)
tf.summary.scalar('accuracy', accuracy)
saver = tf.train.Saver()
sess = tf.Session()
sess.run(tf.global_variables_initializer())
merged_summary_op = tf.summary.merge_all()
writer = tf.summary.FileWriter('./Tensorboard_tutorial_log', sess.graph)
for epoch in range(epochs):
for batch in range(total_batch):
xs, ys = mnist.train.next_batch(batch_size)
summary_str = sess.run(merged_summary_op, feed_dict={x:xs, y:ys})
writer.add_summary(summary_str, epoch)
sess.run(optimizer, feed_dict={x:xs, y:ys})
val_loss, val_acc = sess.run([loss_function, accuracy],
feed_dict={x:mnist.validation.images, y:mnist.validation.labels})
if (epoch + 1) % display_step == 0:
print('Epoch:{:03d}'.format(epoch + 1),
'val_loss=%6f' % val_loss,
'val_accuracy=%4f' % val_acc,
'learning_rate=%.4f' % learning_rate)
if (epoch + 1) % save_step == 0 or (epoch + 1) == epochs:
saver.save(sess, os.path.join(ckpt_dir, 'mnist_model_{0:03d}.ckpt'.format(epoch + 1)))
print('INFO:mnist_model_{0:03d}.ckpt saved'.format(epoch + 1))
print('INFO:Train Finished!')
部分训练如下
Epoch:080 val_loss=0.552384 val_accuracy=0.880600 learning_rate=0.0100
INFO:mnist_model_080.ckpt saved
Epoch:081 val_loss=0.550444 val_accuracy=0.881000 learning_rate=0.0100
Epoch:082 val_loss=0.548490 val_accuracy=0.881600 learning_rate=0.0100
Epoch:083 val_loss=0.546378 val_accuracy=0.882200 learning_rate=0.0100
Epoch:084 val_loss=0.544601 val_accuracy=0.882400 learning_rate=0.0100
Epoch:085 val_loss=0.542755 val_accuracy=0.882800 learning_rate=0.0100
INFO:mnist_model_085.ckpt saved
Epoch:086 val_loss=0.540885 val_accuracy=0.883800 learning_rate=0.0100
Epoch:087 val_loss=0.539172 val_accuracy=0.882800 learning_rate=0.0100
Epoch:088 val_loss=0.537330 val_accuracy=0.883600 learning_rate=0.0100
Epoch:089 val_loss=0.535444 val_accuracy=0.884400 learning_rate=0.0100
Epoch:090 val_loss=0.533857 val_accuracy=0.884600 learning_rate=0.0100
INFO:mnist_model_090.ckpt saved
Epoch:091 val_loss=0.532277 val_accuracy=0.883800 learning_rate=0.0100
Epoch:092 val_loss=0.530567 val_accuracy=0.885800 learning_rate=0.0100
Epoch:093 val_loss=0.528905 val_accuracy=0.885400 learning_rate=0.0100
Epoch:094 val_loss=0.527362 val_accuracy=0.885600 learning_rate=0.0100
Epoch:095 val_loss=0.525661 val_accuracy=0.884000 learning_rate=0.0100
INFO:mnist_model_095.ckpt saved
Epoch:096 val_loss=0.524464 val_accuracy=0.886400 learning_rate=0.0100
Epoch:097 val_loss=0.522694 val_accuracy=0.886400 learning_rate=0.0100
Epoch:098 val_loss=0.521264 val_accuracy=0.887600 learning_rate=0.0100
Epoch:099 val_loss=0.519479 val_accuracy=0.886800 learning_rate=0.0100
Epoch:100 val_loss=0.518293 val_accuracy=0.887600 learning_rate=0.0100
INFO:mnist_model_100.ckpt saved
INFO:Train Finished!
test_acc = sess.run(accuracy, feed_dict={x:mnist.test.images, y:mnist.test.labels})
print('Test dataset accuracy: %f' % test_acc)
Test dataset accuracy: 0.877300
模型读取恢复
saver = tf.train.Saver()
sess = tf.Session()
sess.run(tf.global_variables_initializer())
ckpt_dir = './mnist_Tensorboard_tutorial_ckpt'
ckpt = tf.train.get_checkpoint_state(ckpt_dir)
if ckpt and ckpt.model_checkpoint_path:
saver.restore(sess, ckpt.model_checkpoint_path)
print('Restore model from ' + ckpt.model_checkpoint_path)
INFO:tensorflow:Restoring parameters from ./mnist_Tensorboard_tutorial_ckpt\mnist_model_100.ckpt
Restore model from ./mnist_Tensorboard_tutorial_ckpt\mnist_model_100.ckpt
test_acc = sess.run(accuracy, feed_dict={x:mnist.test.images, y:mnist.test.labels})
print('Test dataset accuracy: %f' % test_acc)
Test dataset accuracy: 0.877300
如何打开 Tensorboard 并查看日志文件查看 Tensorflow 笔记 Ⅰ——TensorFlow 编程基础
完结
TensorFlow1.x 与 TensorFlow2.x 在读入数据的方式不同,这篇仅涉及单个神经元的搭建,下篇将使用多层神经网络进行搭建,整个文件已经上传在该页面,土豪任性下载,我将在整个系列完成后,上传至我的 GitHub