softmax loss:
softmax: softmax的作用,将fc的输出映射成为(0,1)的概率,并将其差距拉大。
cross entropy loss: y是样本的真实标签,为1,y'是样本的分数。单个样本的交叉熵计算方法。
softmax loss: 计算n个样本的交叉熵,然后取均值。
pytorch 的crossentropy loss的测试demo:
#coding=utf-8 import torch import torch.nn as nn import math import numpy as np # output = torch.randn(1, 5, requires_grad = True) #假设是网络的最后一层,5分类 output=np.array([[-0.3625, -0.4523, 1.7358, 0.6512, -0.8057]],dtype='float') output=torch.from_numpy(output) # label = torch.empty(1, dtype=torch.long).random_(5) # 0 - 4, 任意选取一个分类 label=np.array([2],dtype='int64') label=torch.from_numpy(label) print ('Network Output is: ', output) print ('Ground Truth Label is: ', label) score = output [0,label.item()].item() # label对应的class的logits(得分) print ('Score for the ground truth class = ', label) first = - score second = 0 for i in range(5): second += math.exp(output[0,i]) second = math.log(second) loss = first + second print ('-' * 20) print ('my loss = ', loss) loss = nn.CrossEntropyLoss() print ('pytorch loss = ', loss(output, label))
输出:
在计算方式上:
cross entropy loss:
sigmoid:
sigmoid的导数: