softmax函数的公式如下:
Softmax( x i x_i xi)= e x p ( x i ) ∑ j e x p ( x j ) \frac{exp(x_i)}{\sum_j exp(x_j)} ∑jexp(xj)exp(xi)
在pytorch中导入softmax函数的如下:
import torch.nn.functional as F
F.softmax()
在深度学习中,我们一般要对图像(三维(C, W, H)或四维(N, C, W, H))的分类层做作用于softmax函数.
pytorch中如果input=3维数据,则默认维度为0, 如果input=4维数据,则默认维度为1,但是一般情况下还是最好自己加上维度,如:output = softmax(input, dim=1)
import torch as t
input = t.randn(2,3,3)
print(input):
tensor([[[-0.5103, -0.1621, -0.1131],
[ 0.4883, -0.7009, -0.8431],
[-1.5509, -1.4812, -0.9600]],
[[ 1.6368, 0.8301, -0.6083],
[-0.4552, -0.0908, 1.8140],
[ 0.5065, 0.5957, 0.1602]]])
output = F.softmax(input)
print(output):
tensor([[[0.1046, 0.2705, 0.6213],
[0.7198, 0.3520, 0.0656],
[0.1133, 0.1114, 0.2460]],
[[0.8954, 0.7295, 0.3787],
[0.2802, 0.6480, 0.9344],
[0.8867, 0.8886, 0.7540]]])
output = F.softmax(input, dim=0)
print(output):
tensor([[[0.1046, 0.2705, 0.6213],
[0.7198, 0.3520, 0.0656],
[0.1133, 0.1114, 0.2460]],
[[0.8954, 0.7295, 0.3787],
[0.2802, 0.6480, 0.9344],
[0.8867, 0.8886, 0.7540]]])
#可见对于三维数据,默认softmax在零维
#现在尝试四位数据
input = t.randn(2,3,2,2)
print(input):
tensor([[[[-0.1269, -0.0440],
[-0.1551, 0.0342]],
[[-0.7264, -0.4489],
[ 0.4556, -1.1491]],
[[-1.0470, 0.4808],
[ 1.7501, 0.8382]]],
[[[-0.1387, 0.7927],
[ 0.2578, -0.8865]],
[[ 0.5880, 1.1675],
[-0.7048, 0.2471]],
[[ 0.0237, -0.9734],
[-0.6788, 1.6315]]]])
output = F.softmax(input)
print(output):
tensor([[[[0.5135, 0.2979],
[0.1046, 0.2824]],
[[0.2819, 0.1987],
[0.1926, 0.0865]],
[[0.2046, 0.5034],
[0.7028, 0.6311]]],
[[[0.2356, 0.3809],
[0.5637, 0.0606]],
[[0.4873, 0.5540],
[0.2153, 0.1882]],
[[0.2772, 0.0651],
[0.2210, 0.7513]]]])
output = F.softmax(input, dim=1)
print(output):
tensor([[[[0.5135, 0.2979],
[0.1046, 0.2824]],
[[0.2819, 0.1987],
[0.1926, 0.0865]],
[[0.2046, 0.5034],
[0.7028, 0.6311]]],
[[[0.2356, 0.3809],
[0.5637, 0.0606]],
[[0.4873, 0.5540],
[0.2153, 0.1882]],
[[0.2772, 0.0651],
[0.2210, 0.7513]]]])