训练NN技巧

• 神经网络的训练依赖于问题和数据。Designing and training a network using backprop requires making many seemingly arbitrary choices such as the number and types of nodes, layers, learning rates, training and test sets, and so forth. These choices can be critical, yet there is no foolproof recipe for deciding them because they are largely problem and data dependent.

• When fitting a neural network model, these terms can be defined as:

  • Bias: A measure of how the network output averaged across all datasets differs from the desired function.
  • Variance: A measure of how much the network output varies across datasets.

• Early in training, the bias is large because the network output is far from the desired function. The variance is very small because the data has had little influence yet. Late in training, the bias is small because the network has learned the underly