蒸馏论文一(knowledge distillation):利用soft target
蒸馏论文二(Attention Transfer):利用注意力图
蒸馏论文三(Similarity-Preserving):利用特征图内在相似性
蒸馏论文四(Relational Knowledge Distillation):利用样本间距离/角度差异
蒸馏论文五(Neuron Selectivity Transfer):利用教师网络中间层神经元的激活分布
蒸馏论文六(Probabilistic Knowledge Transfer):利用特征向量分布概率信息
蒸馏论文七(Variational Information Distillation):利用互信息
- Be Your Own Teacher: Improve the Performance of Convolutional Neural Networks via Self Distillation
- MSD: MULTI-SELF-DISTILLATION LEARNING VIA MULTI-CLASSIFIERS WITHIN DEEP NEURAL NET- WORKS
- FEED: Feature-level Ensemble for Knowledge Distillation
- Adaptive Multi-Teacher Multi-level Knowledge Distillation
- Deep Mutual Learning
- Training convolutional neural networks with cheap convolutions and online distillation
- Cascaded channel pruning using hierarchical self-distillation
- Cascaded channel pruning using hierarchical self-distillation
- Mimicking Very Efficient Network for Object Detection
- General Instance Distillation for Object Detection (2021.4)
- Multi-Scale Aligned Distillation for Low-Resolution Detection