动态生成卷积核加速卷积神经网络Dynet

利用动态生成的卷积核加速卷积神经网络
ICLR2020文章：
DYNET: DYNAMIC CONVOLUTION FOR ACCELERATING CONVOLUTION NEURAL NETWORKS

（一）文章的核心思想：
adaptively generate convolution kernels based on image contents
通过 image的内容动态的生成卷积核
经过在mobilenet shufflenet resnet18 resnet50上对比试验，在相同的精度下可以减少40.0%, 56.7%, 68.2% 72.4%的 FLOPs
加速效果非常明显

（二）
For example, Liu
et al. (2019) compress MobileNetV2 to 124M, but the accuracy drops by 5.4% on ImageNet. We
theoretically analyze above observation, and find that this phenomenon is caused by the nature of
static convolution, where correlated kernels are cooperated to extract noise-irrelevant features. Thus
it is hard to compress the fixed convolution kernels without information loss. We also find that if we
linearly fuse several convolution kernels to generate one dynamic kernel based on the input, we can
obtain the noise-irrelevant features without the cooperation of multiple kernels, and further reduce
the computation cost of convolution layer remarkably
卷积核之间协同提取可以抑制噪声的特征，如果进行压缩，会丢失信息，造成准确度下降；同时，如果基于输入的image去线性加权合并现有的卷积核来生成新的动态卷积核，可以在保证准确性的同时，进一步减小计算量

cofficient prediction 是根据输入的特征来生成权值，通过对fixed conv kernel 进行加权来生成动态的卷积核（明显的，动态生成的卷积核数量少很多，可以减小计算量）

上图就是生成加权的权重过程
GAP是 global avg pooling
fc是以softmax为激活函数的全连接层（之所以选择softmax，是因为输出权值得和为1）