AAAI2021 蒸馏论文review

简介

• aaai论文列表：https://aaai.org/Conferences/AAAI-21/wp-content/uploads/2020/12/AAAI-21_Accepted-Paper-List.Main_.Technical.Track_.pdf
• 总体：feature map的蒸馏很普遍，之前很多手动选择feature map的蒸馏方法，这次aaai2021的论文中，有3篇关于feature map的蒸馏论文，都是基于自动挑选feature map/attention的思路去做的，这也可以看出一个趋势：中间层特征是大家研究的热点，但是怎样去自动挖掘中间层特征的信息，还有待进一步的挖掘。

蒸馏相关论文列表

• PSSM-Distil: Protein Secondary Structure Prediction (PSSP) on Low-Quality PSSM by Knowledge Distillation with Contrastive Learning

• LRC-BERT: Latent-Representation Contrastive Knowledge Distillation for Natural Language Understanding

• Peer Collaborative Learning for Online Knowledge Distillation

• Cross-Layer Distillation with Semantic Calibration

• Few-Shot Class-Incremental Learning via Relation Knowledge Distillation

• Harmoniz