论文阅读- Paying More Attention to Attention: Improving the Performance of Convolutional Neural Networks

Paying More Attention to Attention: Improving the Performance of Convolutional Neural Networks via Attention Transfer

论文地址：https://arxiv.org/abs/1612.03928
github地址：https://github.com/szagoruyko/attention-transfer
published: ICLR 2017

Abstract

student CNN 通过模仿teacher CNN的attention map 来提高性能

Introduction

One of popular hypothesis: There are non-attentional and attentional perception processes.
当进入一个未知场景后，首先是non-attention，Non-attentional processes help to observe a scene in general and gather high-level information。然后， control the attention processes and navigate to a certain part of the scene. This implies that different observers with different knowledge, different goals, and therefore different attentional strategies can literally see the same scene differently.

本文创新点

1）activation-based patial attention maps

这个方法能够对网络性能带来一定提升。

2）gradient-based patial attention maps

这个方法的效果不大