训练网络时的设置如下
options = trainingOptions('sgdm', ...
'MaxEpochs', 5, ...
'MiniBatchSize', 1, ...
'InitialLearnRate', 1e-20,
'CheckpointPath', tempdir);
初始学习率'InitialLearnRate'已经设置到小数点后20位了。。。为什么训练完还是显示初始学习率可能过大?The training loss has NaN values的原因还与什么options有关?麻烦解答!谢谢!
%%%%%%%这是训练的step信息和报警信息%%%%%%%%%%%%%%%%%%%
Step 1 of 4: Training a Region Proposal Network (RPN).
Training on single CPU.
|=======================================================================================================|
| Epoch | Iteration | Time Elapsed | Mini-batch | Mini-batch | Mini-batch | Base Learning |
| | | (hh:mm:ss) | Loss | Accuracy | RMSE | Rate |
|=======================================================================================================|
| 1 | 1 | 00:00:03 | NaN | 0.00% | NaN | 1.0000e-20 |
| 5 | 5 | 00:01:44 | NaN | 0.00% | NaN | 1.0000e-20 |
|=======================================================================================================|
Step 2 of 4: Training a Fast R-CNN Network using the RPN from step 1.
--> Extracting region proposals from 16 training images...done.
Training on single CPU.
|=======================================================================================================|
| Epoch | Iteration | Time Elapsed | Mini-batch | Mini-batch | Mini-batch | Base Learning |
| | | (hh:mm:ss) | Loss | Accuracy | RMSE | Rate |
|=======================================================================================================|
| 1 | 1 | 00:00:03 | NaN | 100.00% | NaN | 1.0000e-20 |
| 5 | 5 | 00:01:46 | NaN | 100.00% | NaN | 1.0000e-20 |
|=======================================================================================================|
Step 3 of 4: Re-training RPN using weight sharing with Fast R-CNN.
Training on single CPU.
|=======================================================================================================|
| Epoch | Iteration | Time Elapsed | Mini-batch | Mini-batch | Mini-batch | Base Learning |
| | | (hh:mm:ss) | Loss | Accuracy | RMSE | Rate |
|=======================================================================================================|
| 1 | 1 | 00:00:01 | NaN | 0.00% | NaN | 1.0000e-20 |
| 5 | 5 | 00:01:22 | NaN | 0.00% | NaN | 1.0000e-20 |
|=======================================================================================================|
Step 4 of 4: Re-training Fast R-CNN using updated RPN.
--> Extracting region proposals from 16 training images...done.
Training on single CPU.
|=======================================================================================================|
| Epoch | Iteration | Time Elapsed | Mini-batch | Mini-batch | Mini-batch | Base Learning |
| | | (hh:mm:ss) | Loss | Accuracy | RMSE | Rate |
|=======================================================================================================|
| 1 | 1 | 00:00:01 | NaN | 100.00% | NaN | 1.0000e-20 |
| 5 | 5 | 00:01:36 | NaN | 100.00% | NaN | 1.0000e-20 |
|=======================================================================================================|
Detector training complete (with warnings):
警告: Step 1: The training loss has NaN values. The detector may not be properly trained. Try to lower the 'InitialLearnRate'
using the trainingOptions function and retrain the detector.
警告: Step 2: The training loss has NaN values. The detector may not be properly trained. Try to lower the 'InitialLearnRate'
using the trainingOptions function and retrain the detector.
警告: Step 3: The training loss has NaN values. The detector may not be properly trained. Try to lower the 'InitialLearnRate'
using the trainingOptions function and retrain the detector.
警告: Step 4: The training loss has NaN values. The detector may not be properly trained. Try to lower the 'InitialLearnRate'
using the trainingOptions function and retrain the detector.