Tensorflow(十八) —— 误差计算
1. 主要方式
1、MSE
2、cross entropy loss
3、Hinge loss
Σmax(0,1-y*h(x))
2. MSE
# ******************* MSE
y = tf.constant([0,1,3,2,1])
y = tf.cast(tf.one_hot(y,depth = 4),dtype=tf.float32)
logits = tf.random.normal([5,4])
loss1 = tf.reduce_mean(tf.square(y - logits))
print("loss1:",loss1.numpy())
loss2 = tf.square(tf.norm(y - logits))/(5*4)
print("loss2:",loss2.numpy())
loss3 = tf.losses.MSE(y,logits)
print("loss3:",loss3.shape)
loss4 = tf.reduce_mean(tf.losses.MSE(y,logits))
print("loss4:",loss4.numpy())
loss5 = tf.reduce_mean(tf.losses.MeanSquaredError()(y,logits))
print("loss5:",loss5.numpy())
"""
小写为函数
大写为类,使用前需实例化,再调用函数.__call__方法,该方法可省略不写
"""
loss6 = tf.reduce_mean(tf.losses.MeanSquaredError().__call__(y,logits))
print("loss6:",loss6.numpy())
3. cross entropy loss
# ********************* cross entropy loss
"""
Binary Classification: single output
multi output
"""
"""
小写为函数
大写为类
"""
# 对于二分类问题单输出的loss
y = tf.constant([1])
p = tf.constant([0.6])
loss1 = tf.losses.binary_crossentropy(y,p)
loss1_ = tf.losses.BinaryCrossentropy()(y,p)
loss1_2 = tf.losses.BinaryCrossentropy().__call__([[1],[1]],[[0.6],[0.6]])
print("loss1:",loss1.numpy(),loss1_.numpy(),loss1_2.numpy())
# 对于多输出的loss计算
loss2 = tf.losses.categorical_crossentropy([0,1,0,0,0],[0.5,0.1,0.1,0.1,0.1])
print("loss2:",loss2.numpy())
loss3 = tf.losses.CategoricalCrossentropy().__call__([0,1,0,0,0],[0.1,0.5,0.1,0.1,0.1])
print("loss3:",loss3.numpy())
loss4 = tf.losses.CategoricalCrossentropy()([0,1,0,0,0],[0.2,0.2,0.2,0.2,0.2])
print("loss4:",loss4.numpy())
loss5 = tf.losses.categorical_crossentropy([0,1,0,0,0],[0.01,0.96,0.01,0.01,0.01])
print("loss5:",loss5.numpy())
4. MSE的缺点
# ******************** why not MES
"""
1、sigmoid + MSE => gradient vanish
2、converge slower
"""
import matplotlib.pyplot as plt
import numpy as np
x = np.linspace(0.001,1,100)
y = -np.log2(x)
plt.plot(x,y)
plt.show()
5. 直接对logits计算交叉熵损失函数
"""
input => logits => softmax => crossentropy
最后两部步容易出现数据不稳定的情况 因而可以将其合为一步
"""
logits = tf.random.normal([5,4])
y = tf.one_hot([0,3,2,1,3],depth=4)
loss = tf.reduce_mean(tf.losses.categorical_crossentropy(y,logits,from_logits = True))
print("loss:",loss.numpy())
"""
推荐使用from_logits
"""
本文为参考龙龙老师的“深度学习与TensorFlow 2入门实战“课程书写的学习笔记
by CyrusMay 2022 04 17