【2】Model Compression As Constrained Optimization, with Application to Neural Nets. 2017

文章地址: https://arxiv.org/pdf/1707.01209.pdf

Part I: general framework.

We give a general formulation of model compression as constrained optimization.

Related work.

Four categories of model compression.

1. Direct learning: ${\min }_{\Theta }L(h(x;\Theta ))$: find the small model with the best loss regardless of the reference.
2. Direct compression (DC): min ⁡ Θ ∥ w − ∆ ( Θ ) ∥ 2 \min_\Theta ∥w − ∆(\Theta)∥^2 </