三个创新点
- 将人脸检测和跟踪结合起来得到高质量的人脸轨迹
- 提出一种基于Erdos-Renyi图理论的聚类算法(It is based on the observations that large clusters can be fully connected by joining just a small fraction of their point pairs, while just a single connection between two different people can lead to poor clustering results.)。提出新的验证方法rank-1 counts verification,用误报率小的验证系统修正召回率。
- 定义一种端到端的人脸检测和聚类评价标准。
检测和跟踪
- 检测用Faster R-CNN,在WIDER face data数据集重新训练
- distribution field tracker,不用再训练
- 检测和跟踪的结合方法(感觉就是Sort啊^ ^):
- 每帧都检测
- IoU阈值0.3
- 匈牙利算法进行多目标匹配
- 检测与跟踪未匹配上,新建一个tracklet
- 跟踪与检测未匹配上,保留α=10帧
Erdos-Renyi Clustering 和 rank-1 counts verification
- Erdos-Renyi图论
- 设图G,有n个顶点,每条边出现的概率p,则期望的边数为 ( n 2 ) p \left ( \frac{n}{2} \right )p (2n)p,当 ϵ > 0 \epsilon >0 ϵ>0且n足够大时,若 p > ( 1 + ϵ ) l n n n p>\frac{\left ( 1+\epsilon \right )ln n}{n} p>n(1+ϵ)lnn则G一定是连通图。
- 人脸验证系统(focus on low false positives rather than high recall)
- 提取fc7 layer 的4096维特征
- 参考集G=50张图,对应特征: R 1 , R 2 , R 3 , . . . , R G R^{1},R^{2},R^{3},...,R^{G} R1,R2,R3,...,RG
- R = ∑ i = 1 4096 I [ ∣ A i − B i ∣ < min j ∣ ∣ A i − R i j ∣ ] R=\sum_{i=1}^{4096}I\left [ \left |A _{i}-B_{i} \right |<{\min_{j}}\left | |A _{i}-R_{i}^{j} \right | \right ] R=i=1∑4096I[∣Ai−Bi∣<jmin∣∣∣∣Ai−Rij∣∣∣]I[.]是指示函数(二值0或1)
- 自动测定rank-1 counts阈值。(We do this by fitting the left half of the LFW distribution to the left half of the clustering distribution using scale and location parameters)
- 每条轨迹随机选取10张图片计算R值,取最大值
- 参考图库:G=1000,g=50
- do-not-link 限制(硬制约)
- 同一帧的两个人不是同一个人
- 两条轨迹在时间上有任何的重叠,则为两条不同的轨迹
实验数据集
- the big bang theory(BBT) s01e01-s01e06 5-8人
- buffy the vampire slayer(Buffy) s05e01-s05e06 6-11人 有遮挡
- hannah and her sisters(hannah)235人 有遮挡
- LFW 5730 subjects
评价标准
-
annotations: { a 1 , a 2 , . . . , a A } \left \{ a_1,a_2,...,a_A\right \} {a1,a2,...,aA},detections: { d 1 , d 2 , . . . , d D } \left \{ d_1,d_2,...,d_D\right \} {d1,d2,...,dD}
-
false positives resulting from unannotated face detections { d i , ϕ } \left \{ d_i,\phi \right \} {di,ϕ}检测到未标记的数据
-
valid face detections { d i , a j } \left \{ d_i,a_j \right \} {di,aj}有效的检测
-
false negatives resulting from unmatched annotations { ϕ , a j } \left \{ \phi,a_j \right \} {ϕ,aj}未检测到标记的数据
-
----------- { d i , ϕ } \left \{ d_i,\phi \right \} {di,ϕ} false positive ------------------------ { d i , a j } \left \{ d_i,a_j \right \} {di,aj}True positive------------------ { ϕ , a j } \left \{ \phi,a_j \right \} {ϕ,aj}false nagtive - ------- 1 0 ------- 1 green white cyan red 0 green magenta blue red -
unified pairwise precision(UPP)
- { d i , a j } \left \{ d_i,a_j \right \} {di,aj} within all clusters with matching identities
- U P P = w h i t e w h i t e + c y a n + g r e e n UPP=\frac{white}{white+cyan+green} UPP=white+cyan+greenwhite
-
unified pairwise recall(UPR)
- within all identities that have been properly clustered
- U P P = w h i t e w h i t e + m a g e n t a + r e d UPP=\frac{white}{white+magenta+red} UPP=white+magenta+redwhite
-
加权调和平均数 F-measure: F α = 1 α U P P + 1 − α U P R F_\alpha =\frac{1}{\frac{\alpha}{UPP}+\frac{1-\alpha}{UPR}} Fα=UPPα+UPR1−α1