tensorflow 2.0 keras基础框架 以fashion_minst入门项目为例

图像识别项目 fashion_minst

来源于mooc网Google工程师亲授 Tensorflow 2.0入门到进阶

第二章 wide-deep网络之前的视频课内容整理

代码来源于课程，其余内容为本人根据课程内容整理。

#图像识别项目   fashion_minst

#导入头文件
import matplotlib as mpl
import matplotlib.pyplot as plt
import numpy as np
import tensorflow as tf
from tensorflow import keras   #这个是与TensorFlow集成的？
#import keras 这是不同的，这个是基于tensorflow的keras
import pandas as pd
import os
import sys
import sklearn
import matplotlib as mpl
import matplotlib.pyplot as plt
%matplotlib inline
import numpy as np
import sklearn
import pandas as pd
import os
import sys
import time
import tensorflow as tf

from tensorflow import keras

print(tf.__version__)
print(sys.version_info)
for module in mpl, np, pd, sklearn, tf, keras:
    print(module.__name__, module.__version__)

fashion_mnist = keras.datasets.fashion_mnist
(x_train_all, y_train_all), (x_test, y_test) = fashion_mnist.load_data()
x_valid, x_train = x_train_all[:5000], x_train_all[5000:]
y_valid, y_train = y_train_all[:5000], y_train_all[5000:]

print(x_valid.shape, y_valid.shape)
print(x_train.shape, y_train.shape)
print(x_test.shape, y_test.shape)

# x = (x - u) / std

from sklearn.preprocessing import StandardScaler

scaler = StandardScaler()
# x_train: [None, 28, 28] -> [None, 784]
x_train_scaled = scaler.fit_transform(
    x_train.astype(np.float32).reshape(-1, 1)).reshape(-1, 28, 28)
x_valid_scaled = scaler.transform(
    x_valid.astype(np.float32).reshape(-1, 1)).reshape(-1, 28, 28)
x_test_scaled = scaler.transform(
    x_test.astype(np.float32).reshape(-1, 1)).reshape(-1, 28, 28)

# tf.keras.models.Sequential()

model = keras.models.Sequential()
model.add(keras.layers.Flatten(input_shape=[28, 28]))
for _ in range(20):
    model.add(keras.layers.Dense(100, activation="selu"))
model.add(keras.layers.AlphaDropout(rate=0.5))
# AlphaDropout: 1. 均值和方差不变 2. 归一化性质也不变
# model.add(keras.layers.Dropout(rate=0.5))
model.add(keras.layers.Dense(10, activation="softmax"))

model.compile(loss="sparse_categorical_crossentropy",
              optimizer = "sgd",
              metrics = ["accuracy"])

model.summary()

# Tensorboard, earlystopping, ModelCheckpoint
logdir = './dnn-selu-dropout-callbacks'
if not os.path.exists(logdir):
    os.mkdir(logdir)
output_model_file = os.path.join(logdir,
                                 "fashion_mnist_model.h5")

callbacks = [
    keras.callbacks.TensorBoard(logdir),
    keras.callbacks.ModelCheckpoint(output_model_file,
                                    save_best_only = True),
    keras.callbacks.EarlyStopping(patience=5, min_delta=1e-3),
]
history = model.fit(x_train_scaled, y_train, epochs=10,
                    validation_data=(x_valid_scaled, y_valid),
                    callbacks = callbacks)

def plot_learning_curves(history):
    pd.DataFrame(history.history).plot(figsize=(8, 5))
    plt.grid(True)
    plt.gca().set_ylim(0, 1)
    plt.show()

plot_learning_curves(history)

# 1. 参数众多，训练不充分
# 2. 梯度消失 -> 链式法则 -> 复合函数f(g(x))
#    selu缓解梯度消失

model.evaluate(x_test_scaled, y_test)

---

---

---

知识点：敲黑板

1.数据导入

'''
分为训练集（60000）和测试集（10000）。测试集又分为训练（55000）和验证集（5000）。

数据归一化：z-score x-avg/std, 其中，avg为平均值，std为方差。

                     max-min (x-max)/(max-min)

数据归一化的作用：加快收敛速度，缓解梯度消失。
对象：输入的x需要进行归一化
'''
from sklearn.preprocessing import StandardScaler

scaler=StandatdScaler()
x_train_scaled=scaler.fit_transform(x_train.astype(np.float32).reshape(-1,1)).reshape(-1,28,28)
#x_validation,x_test 同理。但是他俩归一化用 transform（）即可。
#因为fit 的意思是通过训练集得出（avg,std）,在验证和测试集中也用训练集得到的这两个值。
#transform 需要的输入是二维的，因此，需要将[None,28,28]--->[None,784]再变回原来大小，经过了两次reshape。

2.建立模型

1.建立顺序连接的模型
model.keras.models.Sequential()
#-------------------------第一种，激活函数在批归一化之前---------------------------------
2.加入网络层 
model.add(...)
keras.layers.Flatten(变平)/Dense（全连接）/BatchNormalization(批归一化)/Activation(激活函数层)/Dropout(AlphaDropout)
1)输入层
keras.layers.Flatten(input_shape=[,])
2)全连接层
keras.layers.Dense(num,activation='relu')
3)批归一化  作用 提高训练速度，缓解梯度消失。 在每层的输出/激活函数之后进行批归一化。
keras.layers.BatchNormalization()
4)Dropout  防止过拟合，一般用在后面几层
keras.layers.Dropout(rate=0.5)  基本用AlphaDropout，自带批归一化的激活函数
#------------------------第二种，激活函数在批归一化之后---------------------------------
1)输入层
keras.layers.Flatten(input_shape=[,])
2)全连接层
keras.layers.Dense(num)
3)批归一化
keras.layers.BatchNormalization()
4)激活函数层
keras.layers.Activation('relu')
5)Dropout
keras.layers.Dropout(rate=0.5)
#-------------------------第三种，特殊的激活函数selu----------------------------------
1)输入层
keras.layers.Flatten(input_shape=[,])
2)全连接层
keras.layers.Dense(num，activation='selu')  #selu已经做了批归一化的工作。
3)Dropout
keras.layers.Dropout(rate=0.5)
#--------------------------AlphaDropout--------------------------------------------
Droput层一般用AlphaDropout
1)均值和方差不变  2)归一化性质也不变  

!!!最后一层给写丢了
输出层
keras.layers.Dense(10,activation='softmax')
激活函数
tanh; relu max(0,x); leaky relu max(0.1x,x); sigmoid; 
softmax(一般用于输出层)
（e^x1/sum,e^x2/sum,e^x3/sum）（sum=e^x1/sum+e^x2/sum+e^x3/sum）
softmax的作用是把输出变成概率（0-1）

3.compile

model.compile(loss='sparse_categorical_crossentropy',optimizer='sgd',metrics=['accuracy'])
1)loss值
sparse 的意思是y是数，非向量，加入sparse会进行one-hot操作。如果y已经是向量，则不需要sparse
Attention，please
损失函数
a.分类
平方差损失，1/n ×（sigma（求和符号） 1/2 × (y-Model(x))的平方）
交叉熵损失，1/n × y × In(Model(x))
b.回归
平方差损失，同上
绝对值损失  1/n × sigma |y-Model(x)|
2)optimizer 优化器
sgd 随机梯度下降
3) metrics=['accuracy']  to be continued

4.fit  训练

logdir='.../'
#判断是否存在，不存在则创建
if not os.path.exists(logdir)
    os.mkdir(logdir)
output_model_file=os.path.join(logdir,'fashion_model.h5') 
callbacks=[keras.callbacks.TensorBoard(logdir)
           keras.callbacks.ModelCheckpoint(output_model_file)
           keras.callbacks.EarlyStopping(patience=5,min_delta=1e-3)]
#min_delta 为上次和本次的损失查值，若连续＜此值patience次，则停止训练。
model.fit(x_train_scaled,y_train,epochs=10,validation_data=(x_validation_scaled,y_validation),callbacks=callbacks)
1)x_train_scaled,y_train
2)epochs=10 训练轮数，将所有图片遍历10轮
3)validation_data 用于验证的数据，且会计算准确率和误差
4)callbacks 多用TensorBoard、EarlyStopping、ModelCheckpoint
TensorBoard需要路径，ModelCheckPoint需要文件
在使用之前先建立路径和文件.

5.验证

model.evaluate(x_test_scaled,y_test)

6.七七八八

1.绘制loss图
history=model.fit(...)
def learning_curves(history):
    pd.DataFrame(history.history).plot(figsize=(8,5))
    plt.grid('off')
    plt.gca().set_ylim(0,1)
    plt.show()
learning_curves(history)

2.model.summary() 能看模型的构成及输出，参数个数
Model: "sequential"
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
flatten (Flatten)            (None, 784)               0         
_________________________________________________________________
dense (Dense)                (None, 100)               78500     
_________________________________________________________________
dense_1 (Dense)              (None, 100)               10100     
_________________________________________________________________
dense_2 (Dense)              (None, 100)               10100     
_________________________________________________________________
dense_3 (Dense)              (None, 100)               10100     
_________________________________________________________________
dense_4 (Dense)              (None, 100)               10100     
_________________________________________________________________
dense_5 (Dense)              (None, 100)               10100     
_________________________________________________________________
dense_6 (Dense)              (None, 100)               10100     
_________________________________________________________________
dense_7 (Dense)              (None, 100)               10100     
_________________________________________________________________
dense_8 (Dense)              (None, 100)               10100     
_________________________________________________________________
dense_9 (Dense)              (None, 100)               10100     
_________________________________________________________________
dense_10 (Dense)             (None, 100)               10100     
_________________________________________________________________
dense_11 (Dense)             (None, 100)               10100     
_________________________________________________________________
dense_12 (Dense)             (None, 100)               10100     
_________________________________________________________________
dense_13 (Dense)             (None, 100)               10100     
_________________________________________________________________
dense_14 (Dense)             (None, 100)               10100     
_________________________________________________________________
dense_15 (Dense)             (None, 100)               10100     
_________________________________________________________________
dense_16 (Dense)             (None, 100)               10100     
_________________________________________________________________
dense_17 (Dense)             (None, 100)               10100     
_________________________________________________________________
dense_18 (Dense)             (None, 100)               10100     
_________________________________________________________________
dense_19 (Dense)             (None, 100)               10100     
_________________________________________________________________
alpha_dropout (AlphaDropout) (None, 100)               0         
_________________________________________________________________
dense_20 (Dense)             (None, 10)                1010      
=================================================================
Total params: 271,410
Trainable params: 271,410
Non-trainable params: 0
3.tensorboard的使用
本文：
tensorboard --logdir= callbacks
logdir就写文件路径就行吧