tf.keras.losses.sparse_categorical_crossentropy和tf.keras.losses.SparseCategoricalCrossentropy

1 tf.keras.losses.sparse_categorical_crossentropy

是一个函数。

返回每个样本的损失。

等价于:

tf.keras.backend.sparse_categorical_crossentropy

2  tf.keras.losses.SparseCategoricalCrossentropy

是一个类。

可以返回每个样本的损失，可以返回所有样本的累加损失，可以返回样本的平均损失。

参数 reduction:

reduction=tf.keras.losses.Reduction.NONE时,返回每个样本的损失。

reduction=tf.keras.losses.Reduction.SUM时，返回所有样本的累加损失。

reduction=tf.keras.losses.Reduction.SUM_OVER_BATCH_SIZE时，返回平均损失。

reduction=tf.keras.losses.Reduction.AUTO时，几乎和reduction=tf.keras.losses.Reduction.SUM_OVER_BATCH_SIZE效果一样。

文档:

* `AUTO`: Indicates that the reduction option will be determined by the usage
   context. For almost all cases this defaults to `SUM_OVER_BATCH_SIZE`. When
   used with `tf.distribute.Strategy`, outside of built-in training loops such
   as `tf.keras` `compile` and `fit`, we expect reduction value to be
   `SUM` or `NONE`. Using `AUTO` in that case will raise an error.

3 程序验证

import tensorflow as tf
import random
import numpy as np

seed = 42
random.seed(seed)
np.random.seed(seed)
tf.random.set_seed(42)
gpus = tf.config.experimental.list_physical_devices(device_type='GPU')
print(gpus)
for gpu in gpus:
    tf.config.experimental.set_memory_growth(gpu, True)

strategy = tf.distribute.get_strategy()
with strategy.scope():
    labels = tf.constant(np.random.randint(0, 2, (3,)))
    print(labels)  # tf.Tensor([0 1 0], shape=(3,), dtype=int64)
    print(labels.shape)  # (3,)
    pred = tf.constant(np.random.randn(3, 10))
    # print(pred)
    print(pred.shape)  # (3, 10)

    # tf.Tensor([2.1810578  2.85535032 2.57512316], shape=(3,), dtype=float64)
    print(tf.keras.backend.sparse_categorical_crossentropy(labels, pred, from_logits=True))

    loss_obj = tf.keras.losses.SparseCategoricalCrossentropy(reduction=tf.keras.losses.Reduction.SUM_OVER_BATCH_SIZE,
                                                             from_logits=True)
    # tf.Tensor(2.537177085876465, shape=(), dtype=float64)
    print(loss_obj(labels, pred))

    loss_obj = tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True)
    # tf.Tensor(2.537177085876465, shape=(), dtype=float64)
    print(loss_obj(labels, pred))

    loss_obj = tf.keras.losses.SparseCategoricalCrossentropy(reduction=tf.keras.losses.Reduction.NONE, from_logits=True)
    # tf.Tensor([2.18105769 2.85535026 2.57512307], shape=(3,), dtype=float64)
    print(loss_obj(labels, pred))

    loss_obj = tf.keras.losses.SparseCategoricalCrossentropy(reduction=tf.keras.losses.Reduction.SUM, from_logits=True)
    # tf.Tensor(7.6115312576293945, shape=(), dtype=float64)
    print(loss_obj(labels, pred))

    loss_obj = tf.keras.losses.SparseCategoricalCrossentropy(reduction=tf.keras.losses.Reduction.AUTO, from_logits=True)
    # tf.Tensor(2.537177085876465, shape=(), dtype=float64)
    print(loss_obj(labels, pred))

    # tf.Tensor([2.1810578  2.85535032 2.57512316], shape=(3,), dtype=float64)
    print(tf.keras.losses.sparse_categorical_crossentropy(labels, pred, from_logits=True))

 

4 其他

tf.keras.losses.binary_crossentropy返回平均损失。

tf.keras.backend.binary_crossentropy返回每个样本的损失。