ResNet--Deep Residual Learning for Image Recognition

Key question

• Vanishing/exploding gradients hamper convergence from the beginning, as the network becomes more deeper.
• with the network depth increasing, accuracy gets saturated (which might be unsurprising) and then degrades rapidly.
  

Methods

• skip connections
  
• The form of the residual function F is flexible
  
• The function F(x; fWig) can represent multiple convolutional layers. The element-wise addition is performed on two feature maps, channel by channel.

Architecture

• 

• Architectures for ImageNet
  

Experiments

• 
  (Training on ImageNet. Thin curves denote training error, and bold curves denote validation error of the center crops. Left: plain networks of 18 and 34 layers. Right: ResNets of 18 and 34 layers. In this plot, the residual networks have no extra parameter compared to their plain counterparts.)

• 

• 

• 

•