报错信息是什么?
我在使用YOLOv8,在VisDrone数据集上训练的时候,在使用train.,py进行 数据下载和训练 的时候,会遇到下面这个报错,导致无法正常训练。
train.py里面写的内容是下面这些
from ultralytics import YOLO
model = YOLO("yolov8n.yaml")
model.train(data="VisDrone.yaml", epochs=1,batch=1)
完整的报错信息如下
File "电脑路径/ultralytics/yolo/data/dataset.py", line 114, in get_labels
raise FileNotFoundError(f"{self.prefix}No labels found in {cache_path}, can not start training. {HELP_URL}")
FileNotFoundError: train: No labels found in /root/m_coding/yolov3/datasets/VisDrone/datasets/VisDrone/VisDrone2019-DET-train/labels.cache, can not start training. See https://github.com/ultralytics/yolov5/wiki/Train-Custom-Data
更完整的报错信息
先是说,尝试进行了多次访问,都无法访问到那个下载链接
Dataset 'VisDrone.yaml' not found ⚠️, missing paths ['/root/m_coding/yolov3/datasets/VisDrone/datasets/VisDrone/VisDrone2019-DET-val/images']
Downloading https://github.com/ultralytics/yolov5/releases/download/v1.0/VisDrone2019-DET-train.zip to /root/m_coding/yolov3/datasets/VisDrone/datasets/VisDrone/VisDrone2019-DET-train.zip...
Downloading https://github.com/ultralytics/yolov5/releases/download/v1.0/VisDrone2019-DET-val.zip to /root/m_coding/yolov3/datasets/VisDrone/datasets/VisDrone/VisDrone2019-DET-val.zip...
Downloading https://github.com/ultralytics/yolov5/releases/download/v1.0/VisDrone2019-DET-test-dev.zip to /root/m_coding/yolov3/datasets/VisDrone/datasets/VisDrone/VisDrone2019-DET-test-dev.zip...
Downloading https://github.com/ultralytics/yolov5/releases/download/v1.0/VisDrone2019-DET-test-challenge.zip to /root/m_coding/yolov3/datasets/VisDrone/datasets/VisDrone/VisDrone2019-DET-test-challenge.zip...
curl: (56) Unexpected EOF
⚠️ Download failure, retrying 1/3 https://github.com/ultralytics/yolov5/releases/download/v1.0/VisDrone2019-DET-val.zip...
curl: (35) OpenSSL SSL_connect: SSL_ERROR_SYSCALL in connection to github.com:443
⚠️ Download failure, retrying 2/3 https://github.com/ultralytics/yolov5/releases/download/v1.0/VisDrone2019-DET-val.zip...
curl: (56) Unexpected EOF
⚠️ Download failure, retrying 1/3 https://github.com/ultralytics/yolov5/releases/download/v1.0/VisDrone2019-DET-train.zip...
Unzipping /root/m_coding/yolov3/datasets/VisDrone/datasets/VisDrone/VisDrone2019-DET-val.zip to /root/m_coding/yolov3/datasets/VisDrone/datasets/VisDrone...
curl: (56) Unexpected EOF
⚠️ Download failure, retrying 1/3 https://github.com/ultralytics/yolov5/releases/download/v1.0/VisDrone2019-DET-test-challenge.zip...
curl: (56) Unexpected EOF
⚠️ Download failure, retrying 2/3 https://github.com/ultralytics/yolov5/releases/download/v1.0/VisDrone2019-DET-train.zip...
curl: (56) Unexpected EOF
⚠️ Download failure, retrying 1/3 https://github.com/ultralytics/yolov5/releases/download/v1.0/VisDrone2019-DET-test-dev.zip...
curl: (7) Failed to connect to github.com port 443: Connection timed out
curl: (35) OpenSSL SSL_connect: SSL_ERROR_SYSCALL in connection to objects.githubusercontent.com:443
⚠️ Download failure, retrying 2/3 https://github.com/ultralytics/yolov5/releases/download/v1.0/VisDrone2019-DET-test-dev.zip...
curl: (56) Unexpected EOF
⚠️ Download failure, retrying 3/3 https://github.com/ultralytics/yolov5/releases/download/v1.0/VisDrone2019-DET-train.zip...
Unzipping /root/m_coding/yolov3/datasets/VisDrone/datasets/VisDrone/VisDrone2019-DET-test-challenge.zip to /root/m_coding/yolov3/datasets/VisDrone/datasets/VisDrone...
curl: (56) Unexpected EOF
⚠️ Download failure, retrying 3/3 https://github.com/ultralytics/yolov5/releases/download/v1.0/VisDrone2019-DET-test-dev.zip...
curl: (56) Unexpected EOF
然后说找不到labels.cache 标签文件
train: Scanning /root/m_coding/yolov3/datasets/VisDrone/datasets/VisDrone/VisDrone2019-DET-train/labels... 0 images, 6471 backgrounds, 0 corrup
train: WARNING ⚠️ No labels found in /root/m_coding/yolov3/datasets/VisDrone/datasets/VisDrone/VisDrone2019-DET-train/labels.cache. See https://github.com/ultralytics/yolov5/wiki/Train-Custom-Data
train: New cache created: /root/m_coding/yolov3/datasets/VisDrone/datasets/VisDrone/VisDrone2019-DET-train/labels.cache
Traceback (most recent call last):
File "train.py", line 3, in <module>
model.train(data="VisDrone.yaml", epochs=1,batch=1)
File "/root/m_coding/YOLOv8n+LADHead/ultralytics/yolo/engine/model.py", line 251, in train
self.trainer.train()
File "/root/m_coding/YOLOv8n+LADHead/ultralytics/yolo/engine/trainer.py", line 187, in train
self._do_train(RANK, world_size)
File "/root/m_coding/YOLOv8n+LADHead/ultralytics/yolo/engine/trainer.py", line 252, in _do_train
self._setup_train(rank, world_size)
File "/root/m_coding/YOLOv8n+LADHead/ultralytics/yolo/engine/trainer.py", line 237, in _setup_train
self.train_loader = self.get_dataloader(self.trainset, batch_size=batch_size, rank=rank, mode="train")
File "/root/m_coding/YOLOv8n+LADHead/ultralytics/yolo/v8/detect/train.py", line 42, in get_dataloader
build_dataloader(self.args, batch_size, img_path=dataset_path, stride=gs, rank=rank, mode=mode,
File "/root/m_coding/YOLOv8n+LADHead/ultralytics/yolo/data/build.py", line 71, in build_dataloader
dataset = YOLODataset(
File "/root/m_coding/YOLOv8n+LADHead/ultralytics/yolo/data/dataset.py", line 44, in __init__
super().__init__(img_path, imgsz, cache, augment, hyp, prefix, rect, batch_size, stride, pad, single_cls)
File "/root/m_coding/YOLOv8n+LADHead/ultralytics/yolo/data/base.py", line 49, in __init__
self.labels = self.get_labels()
File "/root/m_coding/YOLOv8n+LADHead/ultralytics/yolo/data/dataset.py", line 114, in get_labels
raise FileNotFoundError(f"{self.prefix}No labels found in {cache_path}, can not start training. {HELP_URL}")
FileNotFoundError: train: No labels found in /root/m_coding/yolov3/datasets/VisDrone/datasets/VisDrone/VisDrone2019-DET-train/labels.cache, can not start training. See https://github.com/ultralytics/yolov5/wiki/Train-Custom-Data
解释报错的原因
出现上面这种报错的情况下,你打开 训练集 验证集 测试集的文件夹会发现下面这种情况。只有image图片和annnotations标注,但是没有labels这个文件夹
正常情况下应该是下面这样,训练集、验证集、测试集 都是有 ultralytics加工过的labels文件夹和label.cache这个缓存文件。
labels这个文件夹,并不是天津大学VisDrone官方提供的。这个labels文件夹是你在运行上面train.py的这句话“model.train(data="VisDrone.yaml", epochs=1,batch=1) ”,执行了"VisDrone.yaml"里面 visdrone2yolo这个函数 下面这些加工处理annotations文件夹中的标注数据后得到的
def visdrone2yolo(dir):
from PIL import Image
from tqdm import tqdm
def convert_box(size, box):
# Convert VisDrone box to YOLO xywh box
dw = 1. / size[0]
dh = 1. / size[1]
return (box[0] + box[2] / 2) * dw, (box[1] + box[3] / 2) * dh, box[2] * dw, box[3] * dh
(dir / 'labels').mkdir(parents=True, exist_ok=True) # make labels directory
pbar = tqdm((dir / 'annotations').glob('*.txt'), desc=f'Converting {dir}')
for f in pbar:
img_size = Image.open((dir / 'images' / f.name).with_suffix('.jpg')).size
lines = []
with open(f, 'r') as file: # read annotation.txt
for row in [x.split(',') for x in file.read().strip().splitlines()]:
if row[4] == '0': # VisDrone 'ignored regions' class 0
continue
cls = int(row[5]) - 1
box = convert_box(img_size, tuple(map(int, row[:4])))
lines.append(f"{cls} {' '.join(f'{x:.6f}' for x in box)}\n")
with open(str(f).replace(os.sep + 'annotations' + os.sep, os.sep + 'labels' + os.sep), 'w') as fl:
fl.writelines(lines) # write label.txt
但是因为在白天使用AutoDL这个平台的时候,下载的过程里,AutoDL连接外网十分不稳定,一会可以连接github外网,一会又连不上github,下载时断时续,导致VisDrone.yaml文件里面的下载步骤被中止。
下载一旦被迫中止,后续的原始数据进行加工的步骤自然也没来得及运行。
当时当你,此时再次运行train.py的时候,代码会识别你是否已经有几个下载下来的文件。只要有一个文件被下载下来,就不会继续之前的下载,最关键的是代码会同时自动跳过数据加工这个步骤。
此后无论你执行“python train.py”这句代码多少次,都不会 把缺少的数据压缩包下载下来,同时也不会将 annotations文件夹进行再加工,也就没有labels这个文件夹,自然就会不停的报下面这个错,说找不到labels.cache这个文件。无论你执行“python train.py”这句代码多少次,都还是下面这个报错。我曾经为了这个报错,苦恼了一下午,怎么样debug都修不好。截图留念
train: Scanning /root/m_coding/yolov3/datasets/VisDrone/datasets/VisDrone/VisDrone2019-DET-train/labels.cache... 0 images, 6471 backgrounds, 0
Traceback (most recent call last):
File "train.py", line 3, in <module>
model.train(data="VisDrone.yaml", epochs=1,batch=1)
File "/root/m_coding/YOLOv8n+LADHead/ultralytics/yolo/engine/model.py", line 251, in train
self.trainer.train()
File "/root/m_coding/YOLOv8n+LADHead/ultralytics/yolo/engine/trainer.py", line 187, in train
self._do_train(RANK, world_size)
File "/root/m_coding/YOLOv8n+LADHead/ultralytics/yolo/engine/trainer.py", line 252, in _do_train
self._setup_train(rank, world_size)
File "/root/m_coding/YOLOv8n+LADHead/ultralytics/yolo/engine/trainer.py", line 237, in _setup_train
self.train_loader = self.get_dataloader(self.trainset, batch_size=batch_size, rank=rank, mode="train")
File "/root/m_coding/YOLOv8n+LADHead/ultralytics/yolo/v8/detect/train.py", line 42, in get_dataloader
build_dataloader(self.args, batch_size, img_path=dataset_path, stride=gs, rank=rank, mode=mode,
File "/root/m_coding/YOLOv8n+LADHead/ultralytics/yolo/data/build.py", line 71, in build_dataloader
dataset = YOLODataset(
File "/root/m_coding/YOLOv8n+LADHead/ultralytics/yolo/data/dataset.py", line 44, in __init__
super().__init__(img_path, imgsz, cache, augment, hyp, prefix, rect, batch_size, stride, pad, single_cls)
File "/root/m_coding/YOLOv8n+LADHead/ultralytics/yolo/data/base.py", line 49, in __init__
self.labels = self.get_labels()
File "/root/m_coding/YOLOv8n+LADHead/ultralytics/yolo/data/dataset.py", line 114, in get_labels
raise FileNotFoundError(f"{self.prefix}No labels found in {cache_path}, can not start training. {HELP_URL}")
FileNotFoundError: train: No labels found in /root/m_coding/yolov3/datasets/VisDrone/datasets/VisDrone/VisDrone2019-DET-train/labels.cache, can not start training. See https://github.com/ultralytics/yolov5/wiki/Train-Custom-Data
总结来说,使用YOLOv8的时候,下载数据集就要一气呵成,中间不能出现任何网络中断。如果下载中断了,删除掉所有你已经下载好的所有文件,从头开始下载。多次下载,直到你碰到下载整个过程中网络通畅,一口气下载下来,然后让代码把数据加工完。
至于外网github的连接时断时续,有可能是因为,白天的时候有许多人通过AutoDL的这个外网VPN访问github,但是AutoDL提供的VPN的带宽有限,导致你分配到的下载带宽特别窄。所以你可以尝试在半夜的时候运行训练程序,把数据下载和加工好。
讲解如何解决这个报错
方向1:删除掉你已经下载的数据集,从头下载,直到可以流畅下载并加工annotations数据
这个方法是最有效、最高效、最切实可用的方法。
重复这个方法多次,直到你也能像我下面这样,可以把下面四个文件流畅的下载下来,并解压好
VisDrone2019-DET-test-challenge, VisDrone2019-DET-test-dev,VisDrone2019-DET-train,VisDrone2019-DET-val
你的terminal里面一定要有下面这三行,这三行表示,代码成功将“annotations”文件夹里面的数据加工成了YOLOv8指定的数据格式,形成了labels文件夹
Converting /root/m_coding/yolov3/datasets/VisDrone/datasets/VisDrone/VisDrone2019-DET-train: 6471it [04:42, 22.88it/s]
Converting /root/m_coding/yolov3/datasets/VisDrone/datasets/VisDrone/VisDrone2019-DET-val: 548it [00:31, 17.59it/s]
Converting /root/m_coding/yolov3/datasets/VisDrone/datasets/VisDrone/VisDrone2019-DET-test-dev: 1610it [00:59, 27.17it/s]
Dataset download success ✅ (574.6s), saved to /root/m_coding/yolov3/datasets/VisDrone/datasets
方向2:拿现成的labels文件夹过来用
如果你之前保存了 traIn dev 和test三个数据的 制作加工好的label文件夹和label.cache缓存文件。你可以直接放到这三个文件夹里面,放好以后是下面这样
train
val 验证集
test-dev 公开label的测试集
方向3:搞定下载的网络
问客服,为什么github的网络连接多次无法访问?要求他修复网络,保证下载数据过程中,与github之间的连接是稳定的
如果github的网络连接修复好了,保证下载的时候不间断,完整的把所有数据在一次运行的时候下载下来,
再运行一次,看看能不能正常制作label
方向4:将本地数据集上传,然后自己制作label文件夹
如果你电脑上,没有VisDrone 训练 测试 验证 三个数据集的labels这个加工后的这个标注数据文件夹,你可以私信问我要。你也可以,绕开执行VisDrone.yaml这条途径,转而通过下面这个方法,自己制作出来。
通过这个路径“/ultralytics/yolo/data/datasets/VisDrone.yaml”可以找到YOLOv8的 VisDrone.yaml 数据集配置文件。
内容如下
# Ultralytics YOLO 🚀, GPL-3.0 license
# VisDrone2019-DET dataset https://github.com/VisDrone/VisDrone-Dataset by Tianjin University
# Example usage: python train.py --data VisDrone.yaml
# parent
# ├── yolov5
# └── datasets
# └── VisDrone ← downloads here (2.3 GB)
# Train/val/test sets as 1) dir: path/to/imgs, 2) file: path/to/imgs.txt, or 3) list: [path/to/imgs1, path/to/imgs2, ..]
path: ../datasets/VisDrone # dataset root dir
train: VisDrone2019-DET-train/images # train images (relative to 'path') 6471 images
val: VisDrone2019-DET-val/images # val images (relative to 'path') 548 images
test: VisDrone2019-DET-test-dev/images # test images (optional) 1610 images
# Classes
names:
0: pedestrian
1: people
2: bicycle
3: car
4: van
5: truck
6: tricycle
7: awning-tricycle
8: bus
9: motor
# Download script/URL (optional) ---------------------------------------------------------------------------------------
download: |
import os
from pathlib import Path
from ultralytics.yolo.utils.downloads import download
def visdrone2yolo(dir):
from PIL import Image
from tqdm import tqdm
def convert_box(size, box):
# Convert VisDrone box to YOLO xywh box
dw = 1. / size[0]
dh = 1. / size[1]
return (box[0] + box[2] / 2) * dw, (box[1] + box[3] / 2) * dh, box[2] * dw, box[3] * dh
(dir / 'labels').mkdir(parents=True, exist_ok=True) # make labels directory
pbar = tqdm((dir / 'annotations').glob('*.txt'), desc=f'Converting {dir}')
for f in pbar:
img_size = Image.open((dir / 'images' / f.name).with_suffix('.jpg')).size
lines = []
with open(f, 'r') as file: # read annotation.txt
for row in [x.split(',') for x in file.read().strip().splitlines()]:
if row[4] == '0': # VisDrone 'ignored regions' class 0
continue
cls = int(row[5]) - 1
box = convert_box(img_size, tuple(map(int, row[:4])))
lines.append(f"{cls} {' '.join(f'{x:.6f}' for x in box)}\n")
with open(str(f).replace(os.sep + 'annotations' + os.sep, os.sep + 'labels' + os.sep), 'w') as fl:
fl.writelines(lines) # write label.txt
# Download
dir = Path(yaml['path']) # dataset root dir
urls = ['https://github.com/ultralytics/yolov5/releases/download/v1.0/VisDrone2019-DET-train.zip',
'https://github.com/ultralytics/yolov5/releases/download/v1.0/VisDrone2019-DET-val.zip',
'https://github.com/ultralytics/yolov5/releases/download/v1.0/VisDrone2019-DET-test-dev.zip',
'https://github.com/ultralytics/yolov5/releases/download/v1.0/VisDrone2019-DET-test-challenge.zip']
download(urls, dir=dir, curl=True, threads=4)
# Convert
for d in 'VisDrone2019-DET-train', 'VisDrone2019-DET-val', 'VisDrone2019-DET-test-dev':
visdrone2yolo(dir / d) # convert VisDrone annotations to YOLO labels
这里面写了 数据集的下载链接。你可以先下载到本机
urls = ['https://github.com/ultralytics/yolov5/releases/download/v1.0/VisDrone2019-DET-train.zip',
'https://github.com/ultralytics/yolov5/releases/download/v1.0/VisDrone2019-DET-val.zip',
'https://github.com/ultralytics/yolov5/releases/download/v1.0/VisDrone2019-DET-test-dev.zip',
'https://github.com/ultralytics/yolov5/releases/download/v1.0/VisDrone2019-DET-test-challenge.zip']
然后将这些压缩包上传到AutoDL的服务器上
然后你可以将VisDrone.yaml文件里,加工标注文件的这一部分放到一个 .py 文件,自行对annotation文件夹里标注数据进行加工
def visdrone2yolo(dir):
from PIL import Image
from tqdm import tqdm
def convert_box(size, box):
# Convert VisDrone box to YOLO xywh box
dw = 1. / size[0]
dh = 1. / size[1]
return (box[0] + box[2] / 2) * dw, (box[1] + box[3] / 2) * dh, box[2] * dw, box[3] * dh
(dir / 'labels').mkdir(parents=True, exist_ok=True) # make labels directory
pbar = tqdm((dir / 'annotations').glob('*.txt'), desc=f'Converting {dir}')
for f in pbar:
img_size = Image.open((dir / 'images' / f.name).with_suffix('.jpg')).size
lines = []
with open(f, 'r') as file: # read annotation.txt
for row in [x.split(',') for x in file.read().strip().splitlines()]:
if row[4] == '0': # VisDrone 'ignored regions' class 0
continue
cls = int(row[5]) - 1
box = convert_box(img_size, tuple(map(int, row[:4])))
lines.append(f"{cls} {' '.join(f'{x:.6f}' for x in box)}\n")
with open(str(f).replace(os.sep + 'annotations' + os.sep, os.sep + 'labels' + os.sep), 'w') as fl:
fl.writelines(lines) # write label.txt
这种方法我也没用过。只是提供思路给你。你可以试试,成功的可能性很高