非权重压缩
摘要:将网络分为张量层(conv,FC)和非张量层(LRN,pool,dropout,activity等),并对模型进行非张量层的修改,并对修改后的网络进行fine-tune。
论文下载见我的资源:https://download.csdn.net/download/qq_23546067/12045193
下面为非张量层在不同平台下的推理运行时间百分比。
Network | Intel x86 | Arm | Titan X |
---|---|---|---|
AlexNet | 32.08% | 25.08% | 22.37% |
GoogLeNet | 62.03% | 37.81% | 26.14% |
ResNet-50 | 55.66% | 36.61% | 47.87% |
ResNet-152 | 49.77% | N/A | 44.49% |
Average | 49.89% | 33.17% | 35.22% |
DEEPREBIRTH
STREAMLINE MERGING
BRANCH MERGING
使DEEPREBIRTH适应整体模式
新生成的层(即如Yosinski et al.(2014)所述,需要使用finetuning学习新的参数;Razavian et al.(2014)。我们使用标准的初始化方法来(例如,Xavier Glorot &Bengio(2010)初始化:在保持其他层的权值不变的情况下,初始化新层中的参数。在我们的优化过程中,我们将新学习层的学习率设置为其他层的10倍。将所提出的优化方案从底层应用到顶层。It is also possible to learn multiple rebirth layers at the same time (we merge and fine-tune 3 sequential inception layers 4b-4d together for GoogLeNet) or merge layers in orders other than bottom-to-top.
四组对比试验
Thus, we have 4 models to compare, namely GoogLeNet,
GoogLeNet-Merge, GoogLeNet-Tucker and GoogLeNet-Merge-Tucker.