tensorflow--2--神经网络与全连接层

一、数据加载

• keras.datadets
• tf.data.Dataset.from_tensor_slices ：迭代器
  shuffle：打散图片顺序
  map：数据预处理
  batch：同时迭代几个
  repeat（）：重复迭代几次
  • db4=db3.repeat(2)
    

import tensorflow as tf
from tensorflow import keras
from tensorflow.keras import datasets

def prepare_mnist_features_and_labels(x,y):
    x=tf.cast(x,tf.float32)/255.
    y=tf.cast(y,tf.int64)

    return x,y

def mnist_dataset():
    (x,y),(x_test,y_test)=datasets.fashion_mnist.load_data()

    y=tf.one_hot(y,depth=10)
    y_val=tf.one_hot(y_val, depth=10)

    ds=tf.data.Dataset.from_tensor_slices((x,y))
    #预处理
    ds=ds.map(prepare_mnist_features_and_labels)
    #打散，同时迭代25个
    ds=ds.shuffle(1000).batch(25)

    ds_val = tf.data.Dataset.from_tensor_slices((x_test, y_test))
    ds = ds.map(prepare_mnist_features_and_labels)
    ds = ds.shuffle(1000).batch(25)
    return ds,ds_val

二、全连接层

• net（）自动调用了build（）创建了b和w

import tensorflow as tf
x=tf.random.normal([4,784])

net=tf.keras.layers.Dense(512)
out=net(x)

out.shape
（out:TensorShape([4, 512])）

#w和b的维度
net.kernel.shape,net.bias.shape
(out:(TensorShape([784, 512]), TensorShape([512])))

• build（）创建b,w

net=tf.keras.layers.Dense(10)
net.get_weights()
net.weights()

net.build(input_shape=(None,4))
net.kernel.shape,net.bias.shape
(out:(TensorShape([4, 10]), TensorShape([10])))

net.build(input_shape=(None,20))
net.kernel.shape,net.bias.shape
(out:(TensorShape([20, 10]), TensorShape([10])))

net.build(input_shape=(2,20))
net.kernel.shape,net.bias.shape
(out:(out:(TensorShape([20, 10]), TensorShape([10]))))

• keras.Sequential([layer1,layer2, layer3])

三、输出方法

• sigmoid 将范围缩到【0，1】

• softmax将概率和为1
  
  
  
  

四、误差计算

4.1MSE

import tensorflow as tf
y=tf.constant([1,2,3,0,2])
y=tf.one_hot(y,depth=4)
y=tf.cast(y,dtype=tf.float32)

out = tf.random.normal([5,4])

loss1=tf.reduce_mean(tf.square(y-out))

loss2=tf.square(tf.norm(y-out))/(5*4)

loss3=tf.reduce_mean(tf.losses.MSE(y,out))

print(loss1,loss2,loss3)
（out:tf.Tensor(1.8936198, shape=(), dtype=float32)
 tf.Tensor(1.8936198, shape=(), dtype=float32) 
 tf.Tensor(1.8936198, shape=(), dtype=float32)）

4.2Entropy(信息熵)

熵越小，信息越多，越不稳定

import tensorflow as tf
a=tf.fill([4],0.25)
a*tf.math.log(a)/tf.math.log(2.)
（out：<tf.Tensor: shape=(4,), dtype=float32, numpy=array([-0.5, -0.5, -0.5, -0.5], dtype=float32)>）
-tf.reduce_sum(a*tf.math.log(a)/tf.math.log(2.))
(out:<tf.Tensor: shape=(), dtype=float32, numpy=2.0>)

a=tf.constant([0.1,0.1,0.1,0.7])
-tf.reduce_sum(a*tf.math.log(a)/tf.math.log(2.))
(out:<tf.Tensor: shape=(), dtype=float32, numpy=1.3567797>)

14.3cross entropy

14.4 classification

H([0,1,0],[p0,p1,p2]) = 0+Dkl(p/g) = -1logq1

14.5 categorical Cross Entropy

tf.losses.categorical_crossentropy(y_true,prob,from_logits= )

• from_logits=True时更稳定，并且包含softmax和cross entropy两种操作

 tf.losses.categorical_crossentropy([0,1,0,0],[0.25,0.25,0.25,0.25])
 (out:<tf.Tensor: shape=(), dtype=float32, numpy=1.3862944>)

import tensorflow as tf
x=tf.random.normal([1,784])
w=tf.random.normal([784,2])
b=tf.zeros([2])

logits=x@w+b
prob=tf.math.softmax(logits,axis=1)

tf.losses.categorical_crossentropy([[0,1]],logits,from_logits=True)
（out：<tf.Tensor: shape=(1,), dtype=float32, numpy=array([18.918505], dtype=float32)>）

tf.losses.categorical_crossentropy([[0,1]],prob)
（out:<tf.Tensor: shape=(1,), dtype=float32, numpy=array([16.118095], dtype=float32)>）