数据压缩算法方面相关文章整理【转】

Parameter pruning and sharing

1. Quantization and Binarization

• Compressing deep convolutional networks using vector quantization
• Quantized convolutional neural networks for mobile devices
• Improving the speed of neural networks on cpus
• Deep learning with limited numerical precision
• Deep compression: Compressing deep neural networks with pruning, trained quantization and huffman coding
• Towards the limit of network quantization
• Binaryconnect: Training deep neural networks with binary weights during propagations
• Binarynet: Training deep neural networks with weights and activations constrained to +1 or -1
• Xnor-net: Imagenet classification using binary convolutional neural networks
• Deep neural networks are robust to weight binarization and other non- linear distortions
• Loss-aware binarization of deep networks
• Neural networks with few multiplications

2. Pruning and Sharing

• Comparing biases for minimal network construction with back-propagation
• Advances in neural information processing systems 2
• Second order derivatives for network pruning: Optimal brain surgeon
• Data-free parameter pruning for deep neural networks
• Learning both weights and connections for efficient neural networks
• Com- pressing neural networks with the hashing trick
• Soft weight-sharing for neural network compression
• Fast convnets using group-wise brain damage
• Less is more: Towards compact cnns
• Learning structured sparsity in deep neural networks
• Pruning filters for efficient convnets

3. Designing Structural Matrix

• An exploration of parameter redundancy in deep networks with circulant projections
• Fast neural networks with circulant projections
• Deep fried convnets

Low rank factorization and sparsity

• Learning separable filters
• Exploiting linear structure within convolutional networks for efficient evaluation
• Speeding up convolutional neural networks with low rank expansions
• Speeding-up convolutional neural networks using fine-tuned cp-decomposition
• Convolutional neural networks with low-rank regularization
• Predicting parameters in deep learning
• Low-rank matrix factorization for deep neural network training with high-dimensional output targets

Transferred/compact convolution filters

• Group equivariant convolutional net- works
• Doubly convolutional neural networks
• Understanding and improving convolutional neural networks via concatenated rectified linear units
• Multi-bias non-linear activation in deep neural networks
• Exploiting cyclic symmetry in convolutional neural networks
• Inception-v4, inception-resnet and the impact of residual connections on learning
• Squeezedet: Unified, small, low power fully convolutional neural networks for real-time object detection for autonomous driving
• SQUEEZENET: ALEXNET-LEVEL ACCURACY WITH 50X FEWER PARAMETERS AND <0.5MB MODEL SIZE
• MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications

Knowledge distillation

• Model compression
• Do deep nets really need to be deep?
• Distilling the knowledge in a neural network
• Fitnets: Hints for thin deep nets
• Bayesian dark knowledge
• Face model compression by distilling knowledge from neurons
• Net2net: Accelerating learning via knowledge transfer
• Paying more attention to attention: Improving the performance of convolutional neural networks via attention transfer

Other

• Dynamic capacity networks
• Outrageously large neural networks: The sparsely- gated mixture-of-experts layer
• Deep dynamic neural networks for multimodal gesture segmentation and recognition
• Deep Networks with Stochastic Depth
• Deep pyramidal residual networks with separated stochastic depth
• Fast training of convolutional networks through FFTs
• Fast algorithms for convolutional neural networks
• S3pool: Pooling with stochastic spatial sampling

Survey

• A Survey of Model Compression and Acceleration for Deep Neural Networks