语法结构:
tf.nn.sampled_softmax_loss(
weights, biases, labels, inputs, num_sampled, num_classes, num_true=1,
sampled_values=None, remove_accidental_hits=True, seed=None,
name='sampled_softmax_loss'
)
weights
ATensor
of shape[num_classes,dim]
, or a list ofTensor
objects whose concatenation along dimension 0 has shape [num_classes, dim]. The (possibly-sharded) class embeddings.
biases
ATensor
of shape[num_classes]
. The class biases.
labels
ATensor
of typeint64
and shape[batch_size,num_true]
. The target classes. Note that this format differs from thelabels
argument ofnn.softmax_cross_entropy_with_logits
.
inputs
ATensor
of shape[batch_size,dim]
. The forward activations of the input network.
num_sampled
Anint
. The number of classes to randomly sample per batch.
num_classes
Anint
. The number of possible classes.
num_true
Anint
. The number of target classes per training example.
sampled_values
a tuple of (sampled_candidates
,true_expected_count
,sampled_expected_count
) returned by a*_candidate_sampler
function. (if None, we default tolog_uniform_candidate_sampler
)
remove_accidental_hits
Abool
. whether to remove "accidental hits" where a sampled class equals one of the target classes. Default is True.
seed
random seed for candidate sampling. Default to None, which doesn't set the op-level random seed for candidate sampling.
name
A name for the operation (optional).
示例:
import tensorflow as tf
a = tf.nn.sampled_softmax_loss(weights=tf.constant([[1, 2], [3, 4], [5, 6], [7, 8], [9, 10], [11, 12], [13, 14], [15, 16], [17, 18], [19, 20]]),
# [num_classes, dim] = [10, 2]
biases=tf.constant([0., 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9]),
# [num_classes] = [10]
labels=tf.constant([[2], [3], [5]]),
# [batch_size, num_true] = [3, 1]
inputs=tf.constant([[0.2, 0.1], [0.4, 0.1], [0.22, 0.12]]),
# [batch_size, dim] = [3, 2]
num_sampled=3,
num_classes=10,
num_true = 1,
seed = 2020,
name = "sampled_softmax_loss"
)
结果:
with tf.compat.v1.Session() as sess:
print(sess.run(a))
# [0.37478584 0.2859666 0.0703702 ]
在源码的介绍中,train的方式和eval的方式是分开的:
if mode == "train":
loss = tf.nn.sampled_softmax_loss(
weights=weights,
biases=biases,
labels=labels,
inputs=inputs,
...)
elif mode == "eval":
logits = tf.matmul(inputs, tf.transpose(weights))
logits = tf.nn.bias_add(logits, biases)
labels_one_hot = tf.one_hot(labels, n_classes)
loss = tf.nn.softmax_cross_entropy_with_logits(
labels=labels_one_hot,
logits=logits)
train步骤中除了选择一批label作为负样本外,实现方式与eval中描述的一致。
参考:
https://tensorflow.google.cn/api_docs/python/tf/nn/sampled_softmax_loss?hl=zh-cn
https://github.com/tensorflow/tensorflow/blob/v2.2.0/tensorflow/python/ops/nn_impl.py#L2066-L2154