About "Model capacity"--有关模型容量的解释

Model Capacity Simple Explanation

This usually refers to the complexity of the underlying pattern that the neural network is able to learn. Usually going deep increases the capacity of the neural network by increasing the number of model parameters which means it can fit more complex functions. However, increasing capacity by going deep also increases your chance of over-fitting and issues such as regularisation becomes important. It usually also means you need to increase the number of training samples.

    简单来讲所谓模型的容量，主要着重于强调模型的表征能力，对相应任务特征空间的刻画能力。而人们有时会为了增大模型的表征能力（容量）而采用更深的结构，从而带来参数量的增加，所以不能简单地将模型容量大理解为模型参数多。一方面因为有时候浅层模型的效果并不一定比深层模型要差；另一方面，模型越深，参数量更多，会带来更多过拟合的风险。

    而一般机器学习模型的容量的大小一般用VC 维来进行刻画，但是在神经网络中传统 VC 维的概念就与模型的容量有较大的出入。后面我会写专门的章节来介绍 VC 维的一些概念。

    平时我们会经常听到一些相关领域的大佬在讲模型容量之类的东西，其实有时候他们的使用也不是很准确，总之我们了解它是个啥东西就好。

References

1.https://stats.stackexchange.com/questions/312424/what-is-the-capacity-of-a-machine-learning-model
2.https://stackoverflow.com/questions/40337510/what-is-the-definition-of-high-capacity-cnn-or-high-capacity-architecture