报错FloatingPointError: Loss became infinite or NaN at iteration=1099!

报错FloatingPointError: Loss became infinite or NaN at iteration=1099!

[04/01 15:05:09] d2.engine.train_loop ERROR: Exception during training:
Traceback (most recent call last):
  File "/root/detectron2/detectron2/detectron2/engine/train_loop.py", line 138, in train
    self.run_step()
  File "/root/detectron2/detectron2/detectron2/engine/defaults.py", line 441, in run_step
    self._trainer.run_step()
  File "/root/detectron2/detectron2/detectron2/engine/train_loop.py", line 242, in run_step
    self._write_metrics(loss_dict, data_time)
  File "/root/detectron2/detectron2/detectron2/engine/train_loop.py", line 285, in _write_metrics
    f"Loss became infinite or NaN at iteration={self.iter}!\n"
FloatingPointError: Loss became infinite or NaN at iteration=1099!
loss_dict = {'loss_cls': nan, 'loss_box_reg': nan, 'loss_rpn_cls': 0.6423566341400146, 'loss_rpn_loc': 0.08039061725139618}

修改学习率为0.0001后不报错。