DINO&PAWS

Self-supervised learning 之DINO和PAWS

一、DINO(self-distillation with no labels)

讲解链接：https://sh-tsang.medium.com/review-dino-emerging-properties-in-self-supervised-vision-transformers-cfddbb4d3549

1.1 整体框架：

• DINO is inspired from BYOL.
• In DINO, the model passes two kind of random transformations of an input image to the student network $g_{{\theta }_s}$ and the teacher network $g_{{\theta }_t}$.
• Both student and teacher networks have the same architecture but different parameters.
• The output of the teacher network is centered with a mean computed over the batch. Each networks outputs a K dimensional feature denoted by $P_s$ and P t P_t