1 tf.keras.losses.sparse_categorical_crossentropy
是一个函数。
返回每个样本的损失。
等价于:
tf.keras.backend.sparse_categorical_crossentropy
2 tf.keras.losses.SparseCategoricalCrossentropy
是一个类。
可以返回每个样本的损失,可以返回所有样本的累加损失,可以返回样本的平均损失。
参数 reduction:
reduction=tf.keras.losses.Reduction.NONE时,返回每个样本的损失。
reduction=tf.keras.losses.Reduction.SUM时,返回所有样本的累加损失。
reduction=tf.keras.losses.Reduction.SUM_OVER_BATCH_SIZE时,返回平均损失。
reduction=tf.keras.losses.Reduction.AUTO时,几乎和reduction=tf.keras.losses.Reduction.SUM_OVER_BATCH_SIZE效果一样。
文档:
* `AUTO`: Indicates that the reduction option will be determined by the usage context. For almost all cases this defaults to `SUM_OVER_BATCH_SIZE`. When used with `tf.distribute.Strategy`, outside of built-in training loops such as `tf.keras` `compile` and `fit`, we expect reduction value to be `SUM` or `NONE`. Using `AUTO` in that case will raise an error.
3 程序验证
import tensorflow as tf
import random
import numpy as np
seed = 42
random.seed(seed)
np.random.seed(seed)
tf.random.set_seed(42)
gpus = tf.config.experimental.list_physical_devices(device_type='GPU')
print(gpus)
for gpu in gpus:
tf.config.experimental.set_memory_growth(gpu, True)
strategy = tf.distribute.get_strategy()
with strategy.scope():
labels = tf.constant(np.random.randint(0, 2, (3,)))
print(labels) # tf.Tensor([0 1 0], shape=(3,), dtype=int64)
print(labels.shape) # (3,)
pred = tf.constant(np.random.randn(3, 10))
# print(pred)
print(pred.shape) # (3, 10)
# tf.Tensor([2.1810578 2.85535032 2.57512316], shape=(3,), dtype=float64)
print(tf.keras.backend.sparse_categorical_crossentropy(labels, pred, from_logits=True))
loss_obj = tf.keras.losses.SparseCategoricalCrossentropy(reduction=tf.keras.losses.Reduction.SUM_OVER_BATCH_SIZE,
from_logits=True)
# tf.Tensor(2.537177085876465, shape=(), dtype=float64)
print(loss_obj(labels, pred))
loss_obj = tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True)
# tf.Tensor(2.537177085876465, shape=(), dtype=float64)
print(loss_obj(labels, pred))
loss_obj = tf.keras.losses.SparseCategoricalCrossentropy(reduction=tf.keras.losses.Reduction.NONE, from_logits=True)
# tf.Tensor([2.18105769 2.85535026 2.57512307], shape=(3,), dtype=float64)
print(loss_obj(labels, pred))
loss_obj = tf.keras.losses.SparseCategoricalCrossentropy(reduction=tf.keras.losses.Reduction.SUM, from_logits=True)
# tf.Tensor(7.6115312576293945, shape=(), dtype=float64)
print(loss_obj(labels, pred))
loss_obj = tf.keras.losses.SparseCategoricalCrossentropy(reduction=tf.keras.losses.Reduction.AUTO, from_logits=True)
# tf.Tensor(2.537177085876465, shape=(), dtype=float64)
print(loss_obj(labels, pred))
# tf.Tensor([2.1810578 2.85535032 2.57512316], shape=(3,), dtype=float64)
print(tf.keras.losses.sparse_categorical_crossentropy(labels, pred, from_logits=True))
4 其他
tf.keras.losses.binary_crossentropy返回平均损失。
tf.keras.backend.binary_crossentropy返回每个样本的损失。