论文阅读 | Dual Contrastive Prediction for Incomplete Multi-view Representation Learning

文章介绍

彭玺团队的一篇发表于PAMI上的文章，附源码github-TPAMI-DCP

文章主要贡献

• We provide a novel insight to the community that the cross-view consistency learning and data recovery are with intrinsic connections in the framework of information theory. Such a theoretical framework is remarkably different from existing MvRL studies which treat the consistency learning and data recovery as two separate problems. （把consistency 和 data recovery 联系到了一块）
• Under our information theoretical framework, we propose DCP which achieves the information consistency and data recoverability through a dual contrastive loss and a dualprediction loss, respectively.（基于信息论框架，实现了infomation consistency 和data recovery）
• To utilize available label information, DCP designs and utilizes the instance- and category-level contrastive loss to enhance the separability of representations. （该框架利用了实现了不同表示之间的区分程度（separability of representations） 利用实例间的对比和类别之间的对比）
• We theoretically and experimentally prove that DCP could learn a sufficient and minimal representation for three tasks, i.e., clustering, classification, and human action recognition. （可以具体实现三个任务，聚类、分类、人类行为的辨别）

文章框架

分三个主要模块:
1、Whin-view Reconstruction : 每个视图的数据分别投影到一个低维的子空间，从而学习每个视图的特定表示。
2、Dual Prediction : 作用是数据恢复作用。
3、Contrastive Learning : 分两个子模块，一个是Instance-Level Contrastive Learning（作用是通过最大化不同视图表示之间的互信息，该损失函数旨在增强跨视图的一致性），一个是Category-level Contrastive Learing(作用是使来自同一类别的样本在表示空间中更接近，而来自不同类别的样本更远)

损失函数

1、总损失函数

2、对比学习损失
contrastive loss

instance-contrastive-learning loss

category-contrastive-learning loss

3、对比预测损失

实验

Table 1 是聚类性能，Table 2 是分类性能

总结