这个问题我在跑开源crnn网络时遇到了。
解决方案:
一般遇到这样的错误,都是因为在python里面自定义的class类别读取出错。这里的 Caught TypeError in DataLoader worker process 0.意思是多线程执行数据读取任务时,遇到了类型错误,
一般还会抛出异常如下:
TypeError: 'NoneType' object is not subscriptable
说明可能没有读取到数据,所以返回的数据是noneTYPE
该错误与datasetloader的迭代器相关:
所以解决该问题,
可以把自定义的dataset函数里面的def __len__(self):return self.nSamples + 1
改为def __len__(self):return self.nSamples - 1
这样虽然少读了一个数据,但是很好的规避了错误。
源码解读,注意注释部分
下面是dataset自定义的部分代码:
dataset.py
def __len__(self):
return self.nSamples + 1 # 这里有问题,长度应该改为-1,以规避此类错误
def __getitem__(self, index):
assert index <= len(self), 'index range error'
# 这里用到了 len(self),如果超出定义范围会导致严重的错误。
index += 1
with self.env.begin(write=False) as txn:
img_key = 'image-%09d' % index
imgbuf = txn.get(img_key.encode('utf-8'))
buf = six.BytesIO()
buf.write(imgbuf)
buf.seek(0)
try:
img = Image.open(buf).convert('L')
except IOError:
print('Corrupted image for %d' % index)
return self[index + 1]
if self.transform is not None:
img = self.transform(img)
label_key = 'label-%09d' % index
label = txn.get(label_key.encode())
if self.target_transform is not None:
label = self.target_transform(label)
return (img, label)
错误定位:
Traceback (most recent call last): File "/home/zero/blood_detect/PyTorch-YOLOv3/mTrain.py", line 96, in <module> for batch_i, info in
enumerate(dataloader): File "/home/zero/anaconda3/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 819, in __next__ return
self._process_data(data) File "/home/zero/anaconda3/lib/python3.7/site-
packages/torch/utils/data/dataloader.py", line 846, in _process_data data.reraise()
File "/home/zero/anaconda3/lib/python3.7/site-packages/torch/_utils.py", line 385, in
reraise raise self.exc_type(msg)TypeError: Caught TypeError in DataLoader
worker process 0.Original Traceback (most recent call last): File
"/home/zero/anaconda3/lib/python3.7/site-packages/torch/utils/data/_utils/worker.py", line 178, in _worker_loop data =
fetcher.fetch(index) File "/home/zero/anaconda3/lib/python3.7/site-
packages/torch/utils/data/_utils/fetch.py", line 44, in fetch data = [self.dataset[idx]
for idx in possibly_batched_index] File "/home/zero/anaconda3/lib/python3.7/site-
packages/torch/utils/data/_utils/fetch.py", line 44, in <listcomp> data =
[self.dataset[idx] for idx in possibly_batched_index] File
"/home/zero/blood_detect/PyTorch-YOLOv3/utils/datasets.py", line 130, in
__getitem__ img, targets = horisontal_flip(img, targets) File
"/home/zero/blood_detect/PyTorch-YOLOv3/utils/augmentations.py", line 8, in
horisontal_flip targets[:, 2] = 1 - targets[:, 2]TypeError: 'NoneType' object is not
subscriptable
来源source:
网上有一个博主解释了,他先把numworks改为了0
,改为0又报错,
其实numberworks是多线程个数
,改它不能解决问题。
但是他后面说的有道理,
这里我在他的基础上补充一下。