softmax函数原理
softmax函数又称为归一化指数函数,是逻辑函数的一种推广。它能将一个含任意实数的K维向量z的“压缩”到另一个K维实向量f(z)中,使得每一个元素的范围都是在(0,1)之间,并且所有元素的和为1。公式为:
softmax函数实现
import tensorflow as tf
import numpy as np
#softmax函数
#测试的三组数据
x = tf.convert_to_tensor(np.array([[1001, 1002], [3, 4], [-1001, -1003]]), dtype=tf.float32)
#求每组数据的最大值
u = tf.reduce_max(x, reduction_indices=[1])
#扩展维度
maxes = tf.expand_dims(u, 1)
#减去最大值,防止数据值过大溢出
x_red = x - maxes
#e指数次方
x_exp = tf.exp(x_red)
#每组数据求和
sums = tf.expand_dims(tf.reduce_sum(x_exp, reduction_indices=[1]), 1)
#归一化 用分量除以和
out = x_exp / sums
#tensorflow中的softmax函数
out1 = tf.nn.softmax(x)
with tf.Session():
print(x.eval())
print('-------')
print(u.eval())
print('-------')
print(maxes.eval())
print('-------')
print(x_red.eval())
print('-------')
print(x_exp.eval())
print('-------')
print(sums.eval())
print('This is my softmax:')
print(out.eval())
print('This is tf.nn.softmax:')
print(out1.eval())
#结果:
[[ 1001. 1002.]
[ 3. 4.]
[-1001. -1003.]]
-------
[ 1002. 4. -1001.]
-------
[[ 1002.]
[ 4.]
[-1001.]]
-------
[[-1. 0.]
[-1. 0.]
[ 0. -2.]]
-------
[[ 0.36787945 1. ]
[ 0.36787945 1. ]
[ 1. 0.13533528]]
-------
[[ 1.36787939]
[ 1.36787939]
[ 1.13533533]]
This is my softmax:
[[ 0.26894143 0.7310586 ]
[ 0.26894143 0.7310586 ]
[ 0.88079703 0.11920292]]
This is tf.nn.softmax:
[[ 0.26894143 0.7310586 ]
[ 0.26894143 0.7310586 ]
[ 0.88079703 0.11920291]]
交叉熵损失函数实现
#交叉熵损失函数
#数据
y = np.array([[0, 1], [1, 0], [1, 0]])
yhat = np.array([[.5, .5], [.5, .5], [.5, .5]])
#转换为张量
y_tensor = tf.convert_to_tensor(y, dtype=tf.int32)
yhat_tensor = tf.convert_to_tensor(yhat, dtype=tf.float32)
#求交叉熵
y_tensor_float = tf.to_float(y_tensor)
out = -tf.reduce_sum(y_tensor_float * tf.log(yhat_tensor))
with tf.Session():
print(y)
print(yhat)
print(y_tensor.eval())
print(yhat_tensor.eval())
print(y_tensor_float.eval())
print(out.eval())
print(-3 * np.log(.5))
print(tf.reduce_sum(tf.nn.softmax_cross_entropy_with_logits
(labels=y_tensor_float, logits=yhat_tensor)).eval())
#结果:
[[0 1]
[1 0]
[1 0]]
[[ 0.5 0.5]
[ 0.5 0.5]
[ 0.5 0.5]]
[[0 1]
[1 0]
[1 0]]
[[ 0.5 0.5]
[ 0.5 0.5]
[ 0.5 0.5]]
[[ 0. 1.]
[ 1. 0.]
[ 1. 0.]]
2.07944
2.07944154168
2.07944
通过这两个函数的实现,了解了tensorflow中相关函数的功能。