模型训练时报错Failed to allocate 12192768 bytes in function ‘cv::OutOfMemoryError‘

目录

报错信息:

查找网上解决方法:

改进思路:

改进方法:


报错信息:

D:\Programs\miniconda3\envs\python311\python.exe D:\python\project\VisDrone2019-DET-MOT\train.py 
Ultralytics YOLOv8.1.9 🚀 Python-3.11.7 torch-2.2.0 CUDA:0 (NVIDIA GeForce GTX 1650, 4096MiB)
engine\trainer: task=detect, mode=train, model=yolov8n.pt, data=D:\python\project\VisDrone2019-DET-MOT\class.yaml, epochs=100, time=None, patience=50, batch=2, imgsz=900, save=True, save_period=-1, cache=False, device=0, workers=8, project=None, name=train4, exist_ok=False, pretrained=True, optimizer=auto, verbose=True, seed=0, deterministic=True, single_cls=False, rect=False, cos_lr=False, close_mosaic=10, resume=False, amp=True, fraction=1.0, profile=False, freeze=None, multi_scale=False, overlap_mask=True, mask_ratio=4, dropout=0.0, val=True, split=val, save_json=False, save_hybrid=False, conf=None, iou=0.7, max_det=300, half=False, dnn=False, plots=True, source=None, vid_stride=1, stream_buffer=False, visualize=False, augment=False, agnostic_nms=False, classes=None, retina_masks=False, embed=None, show=False, save_frames=False, save_txt=False, save_conf=False, save_crop=False, show_labels=True, show_conf=True, show_boxes=True, line_width=None, format=torchscript, keras=False, optimize=False, int8=False, dynamic=False, simplify=False, opset=None, workspace=4, nms=False, lr0=0.01, lrf=0.01, momentum=0.937, weight_decay=0.0005, warmup_epochs=3.0, warmup_momentum=0.8, warmup_bias_lr=0.1, box=7.5, cls=0.5, dfl=1.5, pose=12.0, kobj=1.0, label_smoothing=0.0, nbs=64, hsv_h=0.015, hsv_s=0.7, hsv_v=0.4, degrees=0.0, translate=0.1, scale=0.5, shear=0.0, perspective=0.0, flipud=0.0, fliplr=0.5, mosaic=1.0, mixup=0.0, copy_paste=0.0, auto_augment=randaugment, erasing=0.4, crop_fraction=1.0, cfg=None, tracker=botsort.yaml, save_dir=runs\detect\train4
Overriding model.yaml nc=80 with nc=12

                   from  n    params  module                                       arguments                     
  0                  -1  1       464  ultralytics.nn.modules.conv.Conv             [3, 16, 3, 2]                 
  1                  -1  1      4672  ultralytics.nn.modules.conv.Conv             [16, 32, 3, 2]                
  2                  -1  1      7360  ultralytics.nn.modules.block.C2f             [32, 32, 1, True]             
  3                  -1  1     18560  ultralytics.nn.modules.conv.Conv             [32, 64, 3, 2]                
  4                  -1  2     49664  ultralytics.nn.modules.block.C2f             [64, 64, 2, True]             
  5                  -1  1     73984  ultralytics.nn.modules.conv.Conv             [64, 128, 3, 2]               
  6                  -1  2    197632  ultralytics.nn.modules.block.C2f             [128, 128, 2, True]           
  7                  -1  1    295424  ultralytics.nn.modules.conv.Conv             [128, 256, 3, 2]              
  8                  -1  1    460288  ultralytics.nn.modules.block.C2f             [256, 256, 1, True]           
  9                  -1  1    164608  ultralytics.nn.modules.block.SPPF            [256, 256, 5]                 
 10                  -1  1         0  torch.nn.modules.upsampling.Upsample         [None, 2, 'nearest']          
 11             [-1, 6]  1         0  ultralytics.nn.modules.conv.Concat           [1]                           
 12                  -1  1    148224  ultralytics.nn.modules.block.C2f             [384, 128, 1]                 
 13                  -1  1         0  torch.nn.modules.upsampling.Upsample         [None, 2, 'nearest']          
 14             [-1, 4]  1         0  ultralytics.nn.modules.conv.Concat           [1]                           
 15                  -1  1     37248  ultralytics.nn.modules.block.C2f             [192, 64, 1]                  
 16                  -1  1     36992  ultralytics.nn.modules.conv.Conv             [64, 64, 3, 2]                
 17            [-1, 12]  1         0  ultralytics.nn.modules.conv.Concat           [1]                           
 18                  -1  1    123648  ultralytics.nn.modules.block.C2f             [192, 128, 1]                 
 19                  -1  1    147712  ultralytics.nn.modules.conv.Conv             [128, 128, 3, 2]              
 20             [-1, 9]  1         0  ultralytics.nn.modules.conv.Concat           [1]                           
 21                  -1  1    493056  ultralytics.nn.modules.block.C2f             [384, 256, 1]                 
 22        [15, 18, 21]  1    753652  ultralytics.nn.modules.head.Detect           [12, [64, 128, 256]]          
Model summary: 225 layers, 3013188 parameters, 3013172 gradients, 8.2 GFLOPs

Transferred 319/355 items from pretrained weights
Freezing layer 'model.22.dfl.conv.weight'
AMP: running Automatic Mixed Precision (AMP) checks with YOLOv8n...
AMP: checks passed ✅
WARNING ⚠️ imgsz=[900] must be multiple of max stride 32, updating to [928]
train: Scanning E:\DeepLearning\AI\VisDrone2019\VisDrone2019-DET-MOT\train\labels... 30669 images, 0 backgrounds, 0 corrupt: 100%|██████████| 30669/30669 [00:35<00:00, 875.04it/s] 
train: WARNING ⚠️ E:\DeepLearning\AI\VisDrone2019\VisDrone2019-DET-MOT\train\images\0000137_02220_d_0000163.jpg: 1 duplicate labels removed
train: WARNING ⚠️ E:\DeepLearning\AI\VisDrone2019\VisDrone2019-DET-MOT\train\images\0000140_00118_d_0000002.jpg: 1 duplicate labels removed
train: WARNING ⚠️ E:\DeepLearning\AI\VisDrone2019\VisDrone2019-DET-MOT\train\images\9999945_00000_d_0000114.jpg: 1 duplicate labels removed
train: WARNING ⚠️ E:\DeepLearning\AI\VisDrone2019\VisDrone2019-DET-MOT\train\images\9999987_00000_d_0000049.jpg: 1 duplicate labels removed
train: WARNING ⚠️ E:\DeepLearning\AI\VisDrone2019\VisDrone2019-DET-MOT\train\images\9999998_00219_d_0000175.jpg: 1 duplicate labels removed
train: New cache created: E:\DeepLearning\AI\VisDrone2019\VisDrone2019-DET-MOT\train\labels.cache
val: Scanning E:\DeepLearning\AI\VisDrone2019\VisDrone2019-DET-MOT\val\labels... 3394 images, 1 backgrounds, 0 corrupt: 100%|██████████| 3394/3394 [00:02<00:00, 1257.53it/s]
val: New cache created: E:\DeepLearning\AI\VisDrone2019\VisDrone2019-DET-MOT\val\labels.cache
Plotting labels to runs\detect\train4\labels.jpg... 
  0%|          | 0/15335 [00:00<?, ?it/s]optimizer: 'optimizer=auto' found, ignoring 'lr0=0.01' and 'momentum=0.937' and determining best 'optimizer', 'lr0' and 'momentum' automatically... 
optimizer: SGD(lr=0.01, momentum=0.9) with parameter groups 57 weight(decay=0.0), 64 weight(decay=0.0005), 63 bias(decay=0.0)
Image sizes 928 train, 928 val
Using 8 dataloader workers
Logging results to runs\detect\train4
Starting training for 100 epochs...

      Epoch    GPU_mem   box_loss   cls_loss   dfl_loss  Instances       Size
      1/100      2.56G        nan        nan        nan         77        928: 100%|██████████| 15335/15335 [1:25:43<00:00,  2.98it/s]
                 Class     Images  Instances      Box(P          R      mAP50  mAP50-95): 100%|██████████| 849/849 [01:50<00:00,  7.67it/s]
                   all       3394     158168    0.00413   1.32e-05    0.00208   0.000415

      Epoch    GPU_mem   box_loss   cls_loss   dfl_loss  Instances       Size
      2/100      3.19G        nan        nan        nan         44        928: 100%|██████████| 15335/15335 [1:25:33<00:00,  2.99it/s]
                 Class     Images  Instances      Box(P          R      mAP50  mAP50-95): 100%|██████████| 849/849 [01:50<00:00,  7.69it/s]
                   all       3394     158168    0.00401   1.32e-05    0.00203   0.000406

      Epoch    GPU_mem   box_loss   cls_loss   dfl_loss  Instances       Size
      3/100       2.1G        nan        nan        nan        134        928: 100%|██████████| 15335/15335 [1:25:26<00:00,  2.99it/s]
                 Class     Images  Instances      Box(P          R      mAP50  mAP50-95): 100%|██████████| 849/849 [01:50<00:00,  7.70it/s]
                   all       3394     158168    0.00154   1.14e-05   0.000779   7.79e-05

      Epoch    GPU_mem   box_loss   cls_loss   dfl_loss  Instances       Size
      4/100      3.26G        nan        nan        nan         99        928: 100%|██████████| 15335/15335 [1:26:01<00:00,  2.97it/s]
                 Class     Images  Instances      Box(P          R      mAP50  mAP50-95): 100%|██████████| 849/849 [01:52<00:00,  7.55it/s]
                   all       3394     158168    0.00409   1.32e-05    0.00206   0.000412

      Epoch    GPU_mem   box_loss   cls_loss   dfl_loss  Instances       Size
      5/100      3.16G        nan        nan        nan         90        928: 100%|██████████| 15335/15335 [1:25:27<00:00,  2.99it/s]
                 Class     Images  Instances      Box(P          R      mAP50  mAP50-95): 100%|██████████| 849/849 [01:50<00:00,  7.69it/s]
                   all       3394     158168    0.00412   1.32e-05    0.00207    0.00033
  0%|          | 0/15335 [00:00<?, ?it/s]
      Epoch    GPU_mem   box_loss   cls_loss   dfl_loss  Instances       Size
      6/100      2.25G        nan        nan        nan         85        928: 100%|██████████| 15335/15335 [1:25:42<00:00,  2.98it/s]
                 Class     Images  Instances      Box(P          R      mAP50  mAP50-95): 100%|██████████| 849/849 [01:52<00:00,  7.54it/s]
                   all       3394     158168     0.0043   1.32e-05    0.00216   0.000343

      Epoch    GPU_mem   box_loss   cls_loss   dfl_loss  Instances       Size
      7/100      3.53G        nan        nan        nan         39        928: 100%|██████████| 15335/15335 [1:25:29<00:00,  2.99it/s]
                 Class     Images  Instances      Box(P          R      mAP50  mAP50-95): 100%|██████████| 849/849 [01:50<00:00,  7.68it/s]
                   all       3394     158168    0.00401   1.32e-05    0.00202   0.000322

      Epoch    GPU_mem   box_loss   cls_loss   dfl_loss  Instances       Size
      8/100      2.61G        nan        nan        nan         28        928: 100%|██████████| 15335/15335 [1:25:25<00:00,  2.99it/s]
                 Class     Images  Instances      Box(P          R      mAP50  mAP50-95): 100%|██████████| 849/849 [01:50<00:00,  7.70it/s]
                   all       3394     158168     0.0016   1.14e-05    0.00081   0.000162

      Epoch    GPU_mem   box_loss   cls_loss   dfl_loss  Instances       Size
      9/100       3.4G        nan        nan        nan        109        928: 100%|██████████| 15335/15335 [1:25:51<00:00,  2.98it/s]
                 Class     Images  Instances      Box(P          R      mAP50  mAP50-95): 100%|██████████| 849/849 [01:52<00:00,  7.54it/s]
                   all       3394     158168    0.00419   1.32e-05    0.00211   0.000338

      Epoch    GPU_mem   box_loss   cls_loss   dfl_loss  Instances       Size
     10/100      2.79G        nan        nan        nan         35        928: 100%|██████████| 15335/15335 [1:25:25<00:00,  2.99it/s]
                 Class     Images  Instances      Box(P          R      mAP50  mAP50-95): 100%|██████████| 849/849 [01:50<00:00,  7.70it/s]
                   all       3394     158168     0.0016   1.14e-05    0.00081    8.1e-05
  0%|          | 0/15335 [00:00<?, ?it/s]
      Epoch    GPU_mem   box_loss   cls_loss   dfl_loss  Instances       Size
     11/100      3.68G        nan        nan        nan        193        928: 100%|██████████| 15335/15335 [1:25:28<00:00,  2.99it/s]
                 Class     Images  Instances      Box(P          R      mAP50  mAP50-95): 100%|██████████| 849/849 [01:50<00:00,  7.65it/s]
                   all       3394     158168    0.00163   1.14e-05   0.000826   8.26e-05
  0%|          | 0/15335 [00:00<?, ?it/s]
      Epoch    GPU_mem   box_loss   cls_loss   dfl_loss  Instances       Size
     12/100      3.38G        nan        nan        nan         41        928: 100%|██████████| 15335/15335 [1:25:48<00:00,  2.98it/s]
                 Class     Images  Instances      Box(P          R      mAP50  mAP50-95): 100%|██████████| 849/849 [01:52<00:00,  7.55it/s]
                   all       3394     158168    0.00419   1.32e-05    0.00211   0.000338
  0%|          | 0/15335 [00:00<?, ?it/s]
      Epoch    GPU_mem   box_loss   cls_loss   dfl_loss  Instances       Size
     13/100      3.09G        nan        nan        nan         72        928: 100%|██████████| 15335/15335 [1:25:25<00:00,  2.99it/s]
                 Class     Images  Instances      Box(P          R      mAP50  mAP50-95): 100%|██████████| 849/849 [01:50<00:00,  7.71it/s]
                   all       3394     158168    0.00427   1.32e-05    0.00215   0.000346

      Epoch    GPU_mem   box_loss   cls_loss   dfl_loss  Instances       Size
     14/100      2.11G        nan        nan        nan        135        928: 100%|██████████| 15335/15335 [1:25:26<00:00,  2.99it/s]
                 Class     Images  Instances      Box(P          R      mAP50  mAP50-95): 100%|██████████| 849/849 [01:50<00:00,  7.69it/s]
                   all       3394     158168    0.00412   1.32e-05    0.00207    0.00033
  0%|          | 0/15335 [00:00<?, ?it/s]
      Epoch    GPU_mem   box_loss   cls_loss   dfl_loss  Instances       Size
     15/100       3.5G        nan        nan        nan         35        928: 100%|██████████| 15335/15335 [1:26:21<00:00,  2.96it/s]
                 Class     Images  Instances      Box(P          R      mAP50  mAP50-95): 100%|██████████| 849/849 [01:52<00:00,  7.54it/s]
                   all       3394     158168    0.00412   1.32e-05    0.00207    0.00033

      Epoch    GPU_mem   box_loss   cls_loss   dfl_loss  Instances       Size
     16/100       3.1G        nan        nan        nan         29        928: 100%|██████████| 15335/15335 [1:25:27<00:00,  2.99it/s]
                 Class     Images  Instances      Box(P          R      mAP50  mAP50-95): 100%|██████████| 849/849 [01:50<00:00,  7.69it/s]
                   all       3394     158168    0.00405   1.32e-05    0.00204   0.000323

      Epoch    GPU_mem   box_loss   cls_loss   dfl_loss  Instances       Size
     17/100      2.52G        nan        nan        nan          9        928: 100%|██████████| 15335/15335 [1:25:25<00:00,  2.99it/s]
                 Class     Images  Instances      Box(P          R      mAP50  mAP50-95): 100%|██████████| 849/849 [01:50<00:00,  7.71it/s]
                   all       3394     158168    0.00405   1.32e-05    0.00204   0.000323
  0%|          | 0/15335 [00:00<?, ?it/s]
      Epoch    GPU_mem   box_loss   cls_loss   dfl_loss  Instances       Size
     18/100      3.05G        nan        nan        nan         46        928: 100%|██████████| 15335/15335 [1:25:26<00:00,  2.99it/s]
                 Class     Images  Instances      Box(P          R      mAP50  mAP50-95): 100%|██████████| 849/849 [01:50<00:00,  7.70it/s]
                   all       3394     158168    0.00412   1.32e-05    0.00207    0.00033
  0%|          | 0/15335 [00:00<?, ?it/s]
      Epoch    GPU_mem   box_loss   cls_loss   dfl_loss  Instances       Size
     19/100      2.82G        nan        nan        nan         74        928: 100%|██████████| 15335/15335 [1:25:25<00:00,  2.99it/s]
                 Class     Images  Instances      Box(P          R      mAP50  mAP50-95): 100%|██████████| 849/849 [01:50<00:00,  7.70it/s]
                   all       3394     158168    0.00427   1.32e-05    0.00215   0.000346

      Epoch    GPU_mem   box_loss   cls_loss   dfl_loss  Instances       Size
     20/100      2.88G        nan        nan        nan        108        928: 100%|██████████| 15335/15335 [1:25:24<00:00,  2.99it/s]
                 Class     Images  Instances      Box(P          R      mAP50  mAP50-95): 100%|██████████| 849/849 [01:50<00:00,  7.70it/s]
                   all       3394     158168    0.00423   1.32e-05    0.00213   0.000339
  0%|          | 0/15335 [00:00<?, ?it/s]
      Epoch    GPU_mem   box_loss   cls_loss   dfl_loss  Instances       Size
     21/100      3.52G        nan        nan        nan        102        928:  84%|████████▎ | 12806/15335 [1:11:20<14:05,  2.99it/s]
Traceback (most recent call last):
  File "D:\python\project\VisDrone2019-DET-MOT\train.py", line 20, in <module>
    results = model.train(data=r"D:\python\project\VisDrone2019-DET-MOT\class.yaml", imgsz=900, epochs=100, batch=2,
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "D:\Programs\miniconda3\envs\python311\Lib\site-packages\ultralytics\engine\model.py", line 601, in train
    self.trainer.train()
  File "D:\Programs\miniconda3\envs\python311\Lib\site-packages\ultralytics\engine\trainer.py", line 208, in train
    self._do_train(world_size)
  File "D:\Programs\miniconda3\envs\python311\Lib\site-packages\ultralytics\engine\trainer.py", line 358, in _do_train
    for i, batch in pbar:
  File "D:\Programs\miniconda3\envs\python311\Lib\site-packages\tqdm\std.py", line 1182, in __iter__
    for obj in iterable:
  File "D:\Programs\miniconda3\envs\python311\Lib\site-packages\ultralytics\data\build.py", line 49, in __iter__
    yield next(self.iterator)
          ^^^^^^^^^^^^^^^^^^^
  File "D:\Programs\miniconda3\envs\python311\Lib\site-packages\torch\utils\data\dataloader.py", line 631, in __next__
    data = self._next_data()
           ^^^^^^^^^^^^^^^^^
  File "D:\Programs\miniconda3\envs\python311\Lib\site-packages\torch\utils\data\dataloader.py", line 1346, in _next_data
    return self._process_data(data)
           ^^^^^^^^^^^^^^^^^^^^^^^^
  File "D:\Programs\miniconda3\envs\python311\Lib\site-packages\torch\utils\data\dataloader.py", line 1372, in _process_data
    data.reraise()
  File "D:\Programs\miniconda3\envs\python311\Lib\site-packages\torch\_utils.py", line 722, in reraise
    raise exception
cv2.error: Caught error in DataLoader worker process 2.
Original Traceback (most recent call last):
  File "D:\Programs\miniconda3\envs\python311\Lib\site-packages\torch\utils\data\_utils\worker.py", line 308, in _worker_loop
    data = fetcher.fetch(index)
           ^^^^^^^^^^^^^^^^^^^^
  File "D:\Programs\miniconda3\envs\python311\Lib\site-packages\torch\utils\data\_utils\fetch.py", line 51, in fetch
    data = [self.dataset[idx] for idx in possibly_batched_index]
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "D:\Programs\miniconda3\envs\python311\Lib\site-packages\torch\utils\data\_utils\fetch.py", line 51, in <listcomp>
    data = [self.dataset[idx] for idx in possibly_batched_index]
            ~~~~~~~~~~~~^^^^^
  File "D:\Programs\miniconda3\envs\python311\Lib\site-packages\ultralytics\data\base.py", line 251, in __getitem__
    return self.transforms(self.get_image_and_label(index))
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "D:\Programs\miniconda3\envs\python311\Lib\site-packages\ultralytics\data\base.py", line 257, in get_image_and_label
    label["img"], label["ori_shape"], label["resized_shape"] = self.load_image(index)
                                                               ^^^^^^^^^^^^^^^^^^^^^^
  File "D:\Programs\miniconda3\envs\python311\Lib\site-packages\ultralytics\data\base.py", line 157, in load_image
    im = cv2.imread(f)  # BGR
         ^^^^^^^^^^^^^
  File "D:\Programs\miniconda3\envs\python311\Lib\site-packages\ultralytics\utils\patches.py", line 26, in imread
    return cv2.imdecode(np.fromfile(filename, np.uint8), flags)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
cv2.error: OpenCV(4.9.0) D:\a\opencv-python\opencv-python\opencv\modules\core\src\alloc.cpp:73: error: (-4:Insufficient memory) Failed to allocate 12192768 bytes in function 'cv::OutOfMemoryError'



Process finished with exit code 1

查找网上解决方法:

改进思路:

报错OutOfMemoryError是说内存不足,上述方法本质上就是将图片缩小

改进方法:

模型训练时把epochs改小,这里下调到20,将图片大小imgsz改小

if __name__ == '__main__':
    os.environ["KMP_DUPLICATE_LIB_OK"] = "TRUE"
    # 加载模型
    # model = YOLO("yolov8n.yaml")  # 从头开始构建新模型
    model = YOLO("yolov8n.pt")  # 加载预训练模型(推荐用于训练)

    # Use the model
    results = model.train(data="class.yaml", imgsz=1120, epochs=20, batch=4, device=0)  # 训练模型

  • 2
    点赞
  • 0
    收藏
    觉得还不错? 一键收藏
  • 0
    评论

“相关推荐”对你有帮助么?

  • 非常没帮助
  • 没帮助
  • 一般
  • 有帮助
  • 非常有帮助
提交
评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值