【目录】
-
nn.L1Loss、nn.MSELoss
-
nn.SmoothL1Loss
SmoothLoss是对L1Loss的一个平滑
-
nn.PoissonNLLLoss
泊松分布
如果输入已经是对数形式，则直接取指数
如果输入不是对数形式，则需要求取对数，为了防止log结果为nan，选取一个很小的数避免底数为0
# ---------------------------------- 8 Poisson NLL Loss ------------------------------
flag = 0
# flag = 1
if flag:
inputs = torch.randn((2, 2))
target = torch.randn((2, 2))
loss_f = nn.PoissonNLLLoss(log_input=True, full=False, reduction='none')
loss = loss_f(inputs, target)
print("input:{}\ntarget:{}\nPoisson NLL loss:{}".format(inputs, target, loss))
# --------------------------------- compute by hand
flag = 0
# flag = 1
if flag:
idx = 0
loss_1 = torch.exp(inputs[idx, idx]) - target[idx, idx]*inputs[idx, idx]
print("第一个元素loss:", loss_1)-
nn.KLDivLoss
在pytorch中需要提前计算模型预测值的信息熵
# ------------------------------ 9 KL Divergence Loss ------------------------------
flag = 0
# flag = 1
if flag:
inputs = torch.tensor([[0.5, 0.3, 0.2], [0.2, 0.3, 0.5]])#batch为2,两组输入
# pytorch函数计算公式不是原始定义公式，其对输入默认已经取log了，在损失函数计算中比公式定义少了一个log（input）的操作
# 因此公式定义里有一个log(y_i / x_i)，在pytorch变为了 log(y_i) - x_i，
inputs = F.log_softmax(inputs, 1)
target = torch.tensor([[0.9, 0.05, 0.05], [0.1, 0.7, 0.2]], dtype=torch.float)
loss_f_none = nn.KLDivLoss(reduction='none')
loss_f_mean = nn.KLDivLoss(reduction='mean')#平均值除以6
loss_f_bs_mean = nn.KLDivLoss(reduction='batchmean')#batch平均值除以2
loss_none = loss_f_none(inputs, target)
loss_mean = loss_f_mean(inputs, target)
loss_bs_mean = loss_f_bs_mean(inputs, target)
print("loss_none:\n{}\nloss_mean:\n{}\nloss_bs_mean:\n{}".format(loss_none, loss_mean, loss_bs_mean))
# --------------------------------- compute by hand
flag = 0
# flag = 1
if flag:
idx = 0
loss_1 = target[idx, idx] * (torch.log(target[idx, idx]) - inputs[idx, idx])
print("第一个元素loss:", loss_1)-
nn.MarginRankingLoss
拿x1的第一个元素分别与x2的各个元素，依据target进行比较
y=1时，若x1>x2，则Loss=0；x1<x2，则Loss=x2-x1+margin(大-小)
y=-1时，若x2>x1，则Loss=0；x2<x1，则Loss=x1-x2+margin(大-小)
# --------------------------- 10 Margin Ranking Loss --------------------------
flag = 0
# flag = 1
if flag:
x1 = torch.tensor([[1], [2], [3]], dtype=torch.float)
x2 = torch.tensor([[2], [2], [2]], dtype=torch.float)
target = torch.tensor([1, 1, -1], dtype=torch.float)#希望前两个x2比x1大，后一个希望x1比x2小
loss_f_none = nn.MarginRankingLoss(margin=0, reduction='none')
loss = loss_f_none(x1, x2, target)#拿x1的第一个元素分别与x2的各个元素，依据target进行比较
print(loss)
#输出tensor([[1.,1.,0.],
#[0.,0.,0],
#[0.,0.,1.]])
-
nn.MultiLabelMarginLoss
功能：多标签边界损失函数
举例：四分类任务，样本x属于0类和3类， 标签：[0, 3, -1, -1] , 不是[1, 0, 0, 1]
主要参数： • reduction ：计算模式
多标签：一张图片对应好几个标签（一张图片中既有树，又有草和云等）
标签所在神经元 - 非标签所在神经元 ÷ （输出的神经元的个数）
要标签所在神经元的输出比非标签所在神经元的输出大，如果不够大都会产生Loss
# ----------------------------- 11 Multi Label Margin Loss ------------------------------
flag = 0
# flag = 1
if flag:
x = torch.tensor([[0.1, 0.2, 0.4, 0.8]])
y = torch.tensor([[0, 3, -1, -1]], dtype=torch.long)#标签不用0101的形式，而是用类别的数组，然后用-1占位
#标签长度要与输出向量的长度是一样的
loss_f = nn.MultiLabelMarginLoss(reduction='none')
loss = loss_f(x, y)
print(loss)
# --------------------------------- compute by hand
flag = 0
# flag = 1
if flag:
x = x[0]
item_1 = (1-(x[0] - x[1])) + (1 - (x[0] - x[2])) # [0]用第0类神经元输出减去第1类和第2类神经元的输出
item_2 = (1-(x[3] - x[1])) + (1 - (x[3] - x[2])) # [3]用第3类神经元输出减去第1类和第2类神经元的输出
loss_h = (item_1 + item_2) / x.shape[0]
print(loss_h)-
nn.SoftMarginLoss
y为标签值有两种模式，一种是1的模式，一种是-1的模式
x为神经元输出的预测值
上面是logistic损失函数的计算公式 ÷ (x.nelement为神经元输出的)平均值
# --------------------------------- 12 SoftMargin Loss ----------------------------------
flag = 0
# flag = 1
if flag:
inputs = torch.tensor([[0.3, 0.7], [0.5, 0.5]])#因为是二分类任务，所以有两个神经元
target = torch.tensor([[-1, 1], [1, -1]], dtype=torch.float)#需要对每个神经元设置标签
loss_f = nn.SoftMarginLoss(reduction='none')
loss = loss_f(inputs, target)
print("SoftMargin: ", loss)
# --------------------------------- compute by hand
flag = 0
# flag = 1
if flag:
idx = 0
inputs_i = inputs[idx, idx]
target_i = target[idx, idx]
loss_h = np.log(1 + np.exp(-target_i * inputs_i))
print(loss_h)
-
nn.MultiLabelSoftMarginLoss
C指标签类别的数量
这里的标签y必须是01两种模式，是标签的神经元采用前面一项计算Loss，不是标签的神经元采用后面一项计算Loss
# ---------------------------- 13 MultiLabel SoftMargin Loss ----------------------------
flag = 0
# flag = 1
if flag:
#对每个神经元的输出根据标签计算Loss
inputs = torch.tensor([[0.3, 0.7, 0.8]])#假设现在有个三分类任务
target = torch.tensor([[0, 1, 1]], dtype=torch.float)#根据标签，此数据属于第1类也属于第2类
loss_f = nn.MultiLabelSoftMarginLoss(reduction='none')
loss = loss_f(inputs, target)
print("MultiLabel SoftMargin: ", loss)#输出值为tensor([0.5429])，一组向量输出一个值
# --------------------------------- compute by hand
flag = 0
# flag = 1
if flag:
i_0 = torch.log(torch.exp(-inputs[0, 0]) / (1 + torch.exp(-inputs[0, 0])))#标签为0
i_1 = torch.log(1 / (1 + torch.exp(-inputs[0, 1])))#标签为1
i_2 = torch.log(1 / (1 + torch.exp(-inputs[0, 2])))#标签为1
loss_h = (i_0 + i_1 + i_2) / -3
print(loss_h)-
nn.MultiMarginLoss
要理解i的取值范围，i为标签值[0,1,2,...]，i不能取当前标签值y
x[y]为标签值神经元所在的输出，x[i]为非标签神经元所在的输出
x.size[0]为标签值的数量
# -------------------------------- 14 Multi Margin Loss ---------------------------------
flag = 0
# flag = 1
if flag:
x = torch.tensor([[0.1, 0.2, 0.7], [0.2, 0.5, 0.3]])
y = torch.tensor([1, 2], dtype=torch.long)#第一个标签1对应中间的神经元，输出为0.2；第二个标签3对应最后面的神经元，输出为0.3；
loss_f = nn.MultiMarginLoss(reduction='none')
loss = loss_f(x, y)
print("Multi Margin Loss: ", loss)
# --------------------------------- compute by hand
flag = 0
# flag = 1
if flag:
x = x[0]
margin = 1
i_0 = margin - (x[1] - x[0])#标签类别为1，则用标签所在神经元的输出减去非标签所在神经元的输出
# i_1 = margin - (x[1] - x[1])
i_2 = margin - (x[1] - x[2])
loss_h = (i_0 + i_2) / x.shape[0]#相加除以类别数
print(loss_h)-
nn.TripleMarginLoss
希望Positive到Anchor之间的距离要比Negative到Anchor之间的距离要小
意思就是，a到p的距离要比a到n的距离要小，比边界值小margin才不会有loss
# ------------------------------ 15 Triplet Margin Loss ---------------------------
flag = 0
# flag = 1
if flag:
anchor = torch.tensor([[1.]])
pos = torch.tensor([[2.]])
neg = torch.tensor([[0.5]])#|pos-anchor| = 1;|neg-anchor| = 0.5；1-0.5+1(margin) = 1.5
loss_f = nn.TripletMarginLoss(margin=1.0, p=1)
loss = loss_f(anchor, pos, neg)
print("Triplet Margin Loss", loss)#输出为1.5
# --------------------------------- compute by hand
flag = 0
# flag = 1
if flag:
margin = 1
a, p, n = anchor[0], pos[0], neg[0]
d_ap = torch.abs(a-p)
d_an = torch.abs(a-n)
loss = d_ap - d_an + margin
print(loss)-
nn.HingeEmbeddingLoss
常用于非线性嵌入学习和半监督学习
△为边界值margin，默认值为1
# ---------------------------- 16 Hinge Embedding Loss ------------------------------
flag = 0
# flag = 1
if flag:
inputs = torch.tensor([[1., 0.8, 0.5]])#输入为两个输入之差的绝对值，需要手动计算两组数值之差的绝对值
target = torch.tensor([[1, 1, -1]])#当标签为-1时，用margin-0.5
loss_f = nn.HingeEmbeddingLoss(margin=1, reduction='none')
loss = loss_f(inputs, target)
print("Hinge Embedding Loss", loss)#输出为tensor([1.,0.8,0.5])
# --------------------------------- compute by hand
flag = 0
# flag = 1
if flag:
margin = 1.
loss = max(0, margin - inputs.numpy()[0, 2])
print(loss)-
nn.CosineEmbeddingLoss
关注两个数据在方向上的差异，角度
cos分子是两个向量的点积，分母是两个向量的模长相乘
margin有取值范围，因为cosθ的值域范围为[-1,1]
# ----------------------------- 17 Cosine Embedding Loss --------------------------------
flag = 0
# flag = 1
if flag:
x1 = torch.tensor([[0.3, 0.5, 0.7], [0.3, 0.5, 0.7]])
x2 = torch.tensor([[0.1, 0.3, 0.5], [0.1, 0.3, 0.5]])
target = torch.tensor([[1, -1]], dtype=torch.float)
loss_f = nn.CosineEmbeddingLoss(margin=0., reduction='none')
loss = loss_f(x1, x2, target)
print("Cosine Embedding Loss", loss)
# --------------------------------- compute by hand
flag = 0
# flag = 1
if flag:
margin = 0.
def cosine(a, b):
numerator = torch.dot(a, b)#两个向量点积
denominator = torch.norm(a, 2) * torch.norm(b, 2)#两个向量模长乘积
return float(numerator/denominator)
l_1 = 1 - (cosine(x1[0], x2[0]))
l_2 = max(0, cosine(x1[0], x2[0]))
print(l_1, l_2)-
nn.CTCLoss
可了解，时序数据的分类
扫一扫
80
被折叠的 条评论
为什么被折叠?
到【灌水乐园】发言
