损失函数

损失函数（loss）：预测值（y）与已知答案（y_）的差距

                                             均方误差

NN优化目标： loss最小=》  自定义

                                              交叉熵

均方误差    (y-y_)的平方求平均数

loss_mse = tf.reduce_mean(tf.square(y_-y))

import tensorflow as tf
import numpy as np

SEED = 23455

rdm = np.random.RandomState(seed=SEED)  # 生成[0,1)之间的随机数
x = rdm.rand(32, 2) # 生成32组，每组两个数值，都在0-1之间的
y_ = [[x1 + x2 + (rdm.rand() / 10.0 - 0.05)] for (x1, x2) in x]  # 生成噪声[0,1)/10=[0,0.1); [0,0.1)-0.05=[-0.05,0.05)
x = tf.cast(x, dtype=tf.float32)

w1 = tf.Variable(tf.random.normal([2, 1], stddev=1, seed=1))

epoch = 15000
lr = 0.002

for epoch in range(epoch):
    with tf.GradientTape() as tape:
        y = tf.matmul(x, w1)
        loss_mse = tf.reduce_mean(tf.square(y_ - y))

    grads = tape.gradient(loss_mse, w1)
    w1.assign_sub(lr * grads)

    if epoch % 500 == 0:
        print("After %d training steps,w1 is " % (epoch))
        print(w1.numpy(), "\n")
print("Final w1 is: ", w1.numpy())

自定义损失函数

loss_zdy = ft.reduce_sum(tf where :(tf.greater(y,y_),cost(y-y_),PROF(y_-y))

 loss = tf.reduce_sum(tf.where(tf.greater(y, y_), (y - y_) * COST, (y_ - y) * PROFIT))

import tensorflow as tf
import numpy as np

SEED = 23455
COST = 1
PROFIT = 99

rdm = np.random.RandomState(SEED)
x = rdm.rand(32, 2)
y_ = [[x1 + x2 + (rdm.rand() / 10.0 - 0.05)] for (x1, x2) in x]  # 生成噪声[0,1)/10=[0,0.1); [0,0.1)-0.05=[-0.05,0.05)
x = tf.cast(x, dtype=tf.float32)

w1 = tf.Variable(tf.random.normal([2, 1], stddev=1, seed=1))

epoch = 10000
lr = 0.002

for epoch in range(epoch):
    with tf.GradientTape() as tape:
        y = tf.matmul(x, w1)
        loss = tf.reduce_sum(tf.where(tf.greater(y, y_), (y - y_) * COST, (y_ - y) * PROFIT))

    grads = tape.gradient(loss, w1)
    w1.assign_sub(lr * grads)

    if epoch % 500 == 0:
        print("After %d training steps,w1 is " % (epoch))
        print(w1.numpy(), "\n")
print("Final w1 is: ", w1.numpy())

# 自定义损失函数
# 酸奶成本1元， 酸奶利润99元
# 成本很低，利润很高，人们希望多预测些，生成模型系数大于1，往多了预测

 

交叉熵损失函数CE（Cross Entropy）:表征两个概率分布之间的距离 H（y_,y）= - Σy_ * lny

eg.二分类已知答案y_=(1,0) 预测 y1=（0.6,0.4） y2=（0.8,0.2）哪个更接近标准答案？

H1((1,0),(0.6,0.4))=-(1*ln0.6+0*0.4) ≈ -(-0.511+0) = 0.511

H2((1,0),(0.8,0.2))=-(1*ln0.8+0*0.2) ≈ -(-0.223+0) = 0.223

因为H1>H2,所以y2预测更准

tf.losses.categorical_crossentropy(y_,y)

import tensorflow as tf

loss_ce1 = tf.losses.categorical_crossentropy([1, 0], [0.6, 0.4])
loss_ce2 = tf.losses.categorical_crossentropy([1, 0], [0.8, 0.2])
print("loss_ce1:", loss_ce1)
print("loss_ce2:", loss_ce2)

# 交叉熵损失函数

 

 

ln为 a=e的情况

a的y次方等于x
那么 loga x = y

loge x = y

e的y次方 = x

e的0次方 等于1

 

实际使用的时候先把输出结果变为和为1的占比。然后在通过交叉熵计算哪个离真实结果最近。

softmax与交叉熵结合
   输出先过softmax函数，再计算y与y_的交叉熵损失函数。
tf.nn.softmax_cross_entropy_with_logits(y_,y)
y_ = np.array([[1,0,0],[0,1,0],[0,0,1],[1,0,0],[0,1,0]])
y  = np.array([[12,3,2],[3,10,1],[1,2,5],[4,6.5,1.2],[3,6,1]])

y_pro = tf.nn.softmax(y)
loss_ce1 = tf.losses.categorical_crossentropy(y_,y_pro)
loss_ce2 = tf.nn.softmax_cross_entropy_with_logits(y_,y)