目标检测 YOLOv5 - loss for objectness and classification

目标检测 YOLOv5 - loss for objectness and classification

flyfish

下面两句位于utils/loss.py的ComputeLoss类中

BCEcls = nn.BCEWithLogitsLoss(pos_weight=torch.tensor([h['cls_pw']], device=device))
BCEobj = nn.BCEWithLogitsLoss(pos_weight=torch.tensor([h['obj_pw']], device=device))

BCEWithLogitsLoss的公式是

ℓ ( x , y ) = L = { l 1 , … , l N } ⊤ , l n = − w n [ y n ⋅ log ⁡ σ ( x n ) + ( 1 − y n ) ⋅ log ⁡ ( 1 − σ ( x n ) ) ] , \ell(x, y) = L = \{l_1,\dots,l_N\}^\top, \quad l_n = - w_n \left[ y_n \cdot \log \sigma(x_n) + (1 - y_n) \cdot \log (1 - \sigma(x_n)) \right], ℓ(x,y)=L={l1​,…,lN​}⊤,ln​=−wn​[yn​⋅logσ(xn​)+(1−yn​)⋅log(1−σ(xn​))],

使用BCEWithLogitsLoss代码示例

import torch
target = torch.ones([10, 64], dtype=torch.float32)  # 64 classes, batch size = 10
output = torch.full([10, 64], 1.5)  # A prediction (logit)
pos_weight = torch.ones([64])  # All weights are equal to 1
criterion = torch.nn.BCEWithLogitsLoss(pos_weight=pos_weight)
loss=criterion(output, target)  # -log(sigmoid(1.5))
print(loss) #tensor(0.2014)

BCEWithLogitsLoss = Sigmoid + BCELoss

代码示例

import torch
target = torch.ones([10, 64], dtype=torch.float32)  # 64 classes, batch size = 10
output = torch.full([10, 64], 1.5)  # A prediction (logit)
a=torch.nn.Sigmoid()
criterion=torch.nn.BCELoss()
a=torch.nn.Sigmoid()
loss = criterion(a(output), target)
print(loss)#tensor(0.2014)

调用binary_cross_entropy的实现

import torch
import torch.nn.functional as F
target = torch.ones([10, 64], dtype=torch.float32,requires_grad=False)  # 64 classes, batch size = 10
output = torch.full([10, 64], 1.5,requires_grad=True)  # A prediction (logit)
a=torch.nn.Sigmoid()
loss = F.binary_cross_entropy(a(output), target)
loss.backward()
print(loss) #tensor(0.2014, grad_fn=<BinaryCrossEntropyBackward>)

推荐使用BCEWithLogitsLoss，而不是其他方式的实现，因为BCEWithLogitsLoss内部使用了log-sum-exp技巧
这里有是如何实现的log-sum-exp，包含整个推理过程 。

以一个实际的例子来理解BCEWithLogitsLoss内部的计算过程

sigmoid的公式是
$$\text{Sigmoid}(x)=\sigma (x)=\frac{1}{1+\exp (-x)}$$
BCELoss的公式是
ℓ ( x , y ) = L = { l 1 , … , l N } ⊤ , l n = − w n [ y n ⋅ log ⁡ x n + ( 1 − y n ) ⋅ log ⁡ ( 1 − x n ) ] \ell(x, y) = L = \{l_1,\dots,l_N\}^\top, \quad l_n = - w_n \left[ y_n \cdot \log x_n + (1 - y_n) \cdot \log (1 - x_n) \right] ℓ(x,y)=L={l1​,…,lN​}⊤,ln​=−wn​[yn​⋅logxn​+(1−yn​)⋅log(1−xn​)]
预测值分别是 0.7,0.2,0.1
类别标签分别是 1 , 0, 0
预测值经过sigmoid结果是0.6682, 0.5498, 0.5250
代入sigmoid公式

print(1 /(1+math.exp(-1 * 0.7)))
print(1 /(1+math.exp(-1 * 0.2)))
print(1 /(1+math.exp(-1 * 0.1)))

math.exp就是e的多少次方
e约等于2.718281828459045，上面第一个式子与print(1/(1+2.718281828459045 ** (-0.7)))相同

在经过BCELoss结果是0.648557191505494

代入BCELoss公式

from math import log
a=0.6682
b=0.5498
c=0.5250
print(((1 * log (a)+(1-1) * log (1-a))+(0 * log (b)+(1-0) * log (1-b))+(0 * log (c)+(1-0) * log (1-c))) /(-3))

最终得到的结果是0.648557191505494

上例如果用BCEWithLogitsLoss代码写是

import torch
pred=torch.tensor([0.7,0.2,0.1],dtype=torch.float)
target=torch.tensor([1,0,0],dtype=torch.float)
criterion = torch.nn.BCEWithLogitsLoss()
loss=criterion(pred, target)
print(loss)#tensor(0.6486)