nn.TripletMarginLoss
这个是最正宗的 Triplet Loss的实现,它的输入是anchor,positive,negative三个B*N的张量(表示batch_szie个N维向量),输出triplet loss的值
定义为:
criterion = torch.nn.TripletMarginLoss(margin, reduction = 'mean')
- margin: 距离
- reduction: 批损失值(mean),批损失和(sum)
计算方法为:
L
(
a
,
p
,
n
)
=
m
a
x
(
d
(
a
,
p
)
−
d
(
a
,
n
)
+
m
a
r
g
i
n
,
0
)
L(a, p, n) = max(d(a, p) - d(a, n) + margin, 0)
L(a,p,n)=max(d(a,p)−d(a,n)+margin,0)
其中
d
(
x
,
y
)
=
∥
x
−
y
∥
p
d(x, y) = \parallel x-y \parallel~p~
d(x,y)=∥x−y∥ p
举例:
anchor = torch.tensor([[1, 0, 1], [1, 1, 1]])
positive = torch.tensor([[4, 1, 5], [2, 2, 2]])
negative = torch.tensor([[3, 1, 2], [1, 1, 2]])
criterion = nn.TripletMarginLoss(margin=0.3)
loss = criterion(anchor, positive, negative)
>>>
tensor(1.9908)
上述代码的具体计算流程如下:
对于第一组向量
a = [1, 0, 1]; p = [4, 1, 5]; n = [3, 1, 2]
d
a
p
=
(
(
1
−
4
)
2
+
(
0
−
1
)
2
+
(
1
−
5
)
2
)
1
/
2
=
5.099
d_{ap} = ((1-4)^2 + (0-1)^2 + (1-5)^2)^{1/2} = 5.099
dap=((1−4)2+(0−1)2+(1−5)2)1/2=5.099
d
a
n
=
(
(
1
−
3
)
2
+
(
0
−
1
)
2
+
(
1
−
2
)
2
)
1
/
2
=
2.4495
d_{an} = ((1-3)^2 + (0-1)^2 + (1-2)^2)^{1/2} = 2.4495
dan=((1−3)2+(0−1)2+(1−2)2)1/2=2.4495
m
a
x
(
d
a
p
−
d
a
n
+
m
a
r
g
i
n
,
0
)
=
2.9495
max(d_{ap}-d_{an}+margin, 0) = 2.9495
max(dap−dan+margin,0)=2.9495
对于第二组向量
a = [1, 1, 1]; p = [2, 2, 2]; n = [1, 1, 2]
d
a
p
=
(
(
1
−
2
)
2
+
(
1
−
2
)
2
+
(
1
−
2
)
2
)
1
/
2
=
1.732
d_{ap} = ((1-2)^2 + (1-2)^2 + (1-2)^2)^{1/2} = 1.732
dap=((1−2)2+(1−2)2+(1−2)2)1/2=1.732
d
a
n
=
(
(
1
−
1
)
2
+
(
1
−
1
)
2
+
(
1
−
2
)
2
)
1
/
2
=
1
d_{an} = ((1-1)^2 + (1-1)^2 + (1-2)^2)^{1/2} = 1
dan=((1−1)2+(1−1)2+(1−2)2)1/2=1
m
a
x
(
d
a
p
−
d
a
n
+
m
a
r
g
i
n
,
0
)
=
1.032
max(d_{ap}-d_{an}+margin, 0) = 1.032
max(dap−dan+margin,0)=1.032
求平均:
l
o
s
s
=
(
2.9495
+
1.032
)
/
2
=
1.9908
loss = (2.9495 + 1.032) / 2 = 1.9908
loss=(2.9495+1.032)/2=1.9908
nn.MarginRankingLoss
它的输入是三个长度为B的一维向量
定义为:
criterion = nn.MarginRankingLoss(margin=0.3, reduction='mean')
计算方法:
l
o
s
s
(
x
,
y
)
=
m
a
x
(
−
y
∗
(
x
1
−
x
2
)
+
m
a
r
g
i
n
,
0
)
loss(x, y) = max(-y * (x1 - x2) + margin, 0)
loss(x,y)=max(−y∗(x1−x2)+margin,0)
如果y是一个全1的向量, 那么计算公式就变成
l
o
s
s
(
x
,
y
)
=
m
a
x
(
x
2
−
x
1
+
m
a
r
g
i
n
,
0
)
loss(x, y) = max(x2 - x1 + margin, 0)
loss(x,y)=max(x2−x1+margin,0),这刚好就是triplet loss的公式
举例:(仍用上面的那个例子来比较两者使用方式的不同)
# 根据上面的计算, 可以得到如下的距离
dist_ap = torch.tensor([5.099, 1.732])
dist_an = torch.tensor([2.4495, 1.])
y = torch.ones_like(dist_an)
# 这里要注意dist_an在前面
loss = criterion(dist_an, dist_ap, y)
print(loss)
>>>
tensor(1.9908)
这里补充一个算两个向量之间的距离矩阵的函数:
def cal_dist_mat(x1, x2):
"""
Args:
x1: [m, f]
x2: [n, f]
Returns:
dist: [m, n]
"""
assert len(x1.shape) == 2
assert len(x2.shape) == 2
assert x1.shape[1] == x2.shape[1]
m, n = x1.shape[0], x2.shape[0]
dist = torch.pow(x1, 2).sum(dim=1, keepdim=True).expand(m, n) + \
torch.pow(x2, 2).sum(dim=1, keepdim=True).expand(n, m).T
dist = torch.addmm(dist, mat1=x1, mat2=x2.T, beta=1, alpha=-2)
dist = torch.clamp(dist.sqrt(), min=1e-12)
return dist