2020-12-19 nn.CrossEntropyLoss()

nn.CrossEntropyLoss()实例理解：

针对PICA的具体理解：

以下可理解为K*K的PUI中的某一行所对应的损失：

其中x可以理解为K*K的PUI中的某一行；cluster_index即指代在该行中所对应的元素；分母部分即为该行的所以元素进行累加和。

CrossEntropyLoss(input, target)

1.

input:
entroy=nn.CrossEntropyLoss()
input=torch.Tensor([[-0.7715, -0.6205, -0.2562],
                    [-0.7715, -0.6205,  -0.2562],
                    [-0.7715, -0.6205,  -0.2562]])
target = torch.tensor([0, 0, 0])
# target = torch.arange(3)
output = entroy(input, target)
print(output)

output : tensor(1.3447)

target对应某所得特征向量中第某个待求元素。

(1)
-x[0] + log(exp(x[0]), exp(x[1]), exp(x[2])) =
0.7715 + log(exp(-0.7715) + exp(-0.6205) + exp(-0.2562)) = 1.3447
(2)
-x[0] + log(exp(x[0]), exp(x[1]), exp(x[2])) =
0.7715 + log(exp(-0.7715) + exp(-0.6205) + exp(-0.2562)) = 1.3447
(3)
-x[0] + log(exp(x[0]), exp(x[1]), exp(x[2])) =
0.7715 + log(exp(-0.7715) + exp(-0.6205) + exp(-0.2562)) = 1.3447

loss = [(1) + (2) + (3)] /3 = 1.3447

2.

input:
entroy=nn.CrossEntropyLoss()
input=torch.Tensor([[-0.7715, -0.6205, -0.2562],
                    [-0.7715, -0.6205,  -0.2562],
                    [-0.7715, -0.6205,  -0.2562]])
target = torch.tensor([1, 1, 1])
# target = torch.arange(3)
output = entroy(input, target)
print(output)

output : tensor(1.1937)

(1)
-x[1] + log(exp(x[0]), exp(x[1]), exp(x[2])) =
0.6205 + log(exp(-0.7715) + exp(-0.6205) + exp(-0.2562)) = 1.1937
(2)
-x[1] + log(exp(x[0]), exp(x[1]), exp(x[2])) =
0.6205 + log(exp(-0.7715) + exp(-0.6205) + exp(-0.2562)) = 1.1937
(3)
-x[1] + log(exp(x[0]), exp(x[1]), exp(x[2])) =
0.6205 + log(exp(-0.7715) + exp(-0.6205) + exp(-0.2562)) = 1.1937

loss = [(1) + (2) + (3)] / 3 = 1.1937

3.

input:
entroy=nn.CrossEntropyLoss()
input=torch.Tensor([[-0.7715, -0.6205, -0.2562],
                    [-0.7715, -0.6205,  -0.2562],
                    [-0.7715, -0.6205,  -0.2562]])
target = torch.tensor([2, 2, 2])
# target = torch.arange(3)
output = entroy(input, target)
print(output)

output ：tensor(0.8294)

(1)
-x[2] + log(exp(x[0]), exp(x[1]), exp(x[2])) =
0.2562 + log(exp(-0.7715) + exp(-0.6205) + exp(-0.2562)) = 0.8294
(2)
-x[2] + log(exp(x[0]), exp(x[1]), exp(x[2])) =
0.2562 + log(exp(-0.7715) + exp(-0.6205) + exp(-0.2562)) = 0.8294
(3)
-x[2] + log(exp(x[0]), exp(x[1]), exp(x[2])) =
0.2562 + log(exp(-0.7715) + exp(-0.6205) + exp(-0.2562)) = 0.8294

loss = [(1) + (2) + (3)] / 3 = 0.8294

4.

input:
entroy=nn.CrossEntropyLoss()
input=torch.Tensor([[-0.7715, -0.6205, -0.2562],
                    [-0.7715, -0.6205,  -0.2562],
                    [-0.7715, -0.6205,  -0.2562]])
target = torch.tensor([0, 1, 2])   # 或 target = torch.arange(3)
# target = torch.arange(3)
output = entroy(input, target)
print(output)

output ：tensor(1.1226)

(1)
-x[0] + log(exp(x[0]), exp(x[1]), exp(x[2])) =
0.7715 + log(exp(-0.7715) + exp(-0.6205) + exp(-0.2562)) = 1.3447
(2)
-x[1] + log(exp(x[0]), exp(x[1]), exp(x[2])) =
0.6205+ log(exp(-0.7715) + exp(-0.6205) + exp(-0.2562)) = 1.1937
(3)
-x[2] + log(exp(x[0]), exp(x[1]), exp(x[2])) =
0.2562 + log(exp(-0.7715) + exp(-0.6205) + exp(-0.2562)) = 0.8294

loss = [(1) + (2) + (3)] / 3 = 1.1226