关于CNN网络架构的思考

3个影响网络准确性的重要参数

• Depth 网络深度，即层数
• Width 特征个数，即卷积核个数
• Resolution 图像分辨率，即经过各层的size大小

1 why depth scaling?

The intuition is that a deeper network can capture richer and more complex features, and
generalizes well on new tasks.

Theoretically, with more layers, the network performance should improve but practically it doesn’t follow.
Vanishing gradients is one of the most common problems that arises as we go deep

2 width scaling?

This is commonly used when we want to keep our model small. Wider networks tend to be able to capture more fine-grained features. Also, smaller models are easier to train.

3 Resolution

Intuitively, we can say that in a high-resolution image, the features are more fine-grained and hence high-res images should work better.
But this doesn’t scale linearly.

Conclusion

1 Scaling up any dimension of network (width, depth or resolution) improves accuracy, but the accuracy gain diminishes for bigger models.

2 As the resolution of the images is increased, depth and width of the network should be increased as well.

As the depth is increased, larger receptive fields can capture similar features that include more pixels in an image. Also, as the width is increased, more fine-grained features will be captured.

Proposed Compound Scaling 缩放这些参数的方法