tfrecord 训练时报错 DataLossError corrupted record at 12

最新推荐文章于 2023-01-26 13:30:27 发布

大大蝌蚪

最新推荐文章于 2023-01-26 13:30:27 发布

阅读量520

点赞数 1

文章标签： python tensorflow 深度学习

本文链接：https://blog.csdn.net/weixin_41930058/article/details/126116605

版权

tfrecord 训练时报错 DataLossError corrupted record at 12

背景

图像分类，因为数据比较多所以转成了tfrecord进行训练，但是训练一个epoch之后总是报错。

报错信息

DataLossError (see above for traceback): corrupted record at 12

3/3 [==============================] - 73s 24s/step - loss: 0.6961 - accuracy: 0.5050
Traceback (most recent call last):
  File "debug.py", line 148, in <module>
    validation_data=eval_dataset)
  File "/root/anaconda3/envs/keke/lib/python3.6/site-packages/tensorflow_core/python/keras/engine/training.py", line 728, in fit
    use_multiprocessing=use_multiprocessing)
  File "/root/anaconda3/envs/keke/lib/python3.6/site-packages/tensorflow_core/python/keras/engine/training_v2.py", line 370, in fit
    total_epochs=1)
  File "/root/anaconda3/envs/keke/lib/python3.6/site-packages/tensorflow_core/python/keras/engine/training_v2.py", line 123, in run_one_epoch
    batch_outs = execution_function(iterator)
  File "/root/anaconda3/envs/keke/lib/python3.6/site-packages/tensorflow_core/python/keras/engine/training_v2_utils.py", line 86, in execution_function
    distributed_function(input_fn))
  File "/root/anaconda3/envs/keke/lib/python3.6/site-packages/tensorflow_core/python/eager/def_function.py", line 457, in __call__
    result = self._call(*args, **kwds)
  File "/root/anaconda3/envs/keke/lib/python3.6/site-packages/tensorflow_core/python/eager/def_function.py", line 526, in _call
    return self._concrete_stateful_fn._filtered_call(canon_args, canon_kwds)  # pylint: disable=protected-access
  File "/root/anaconda3/envs/keke/lib/python3.6/site-packages/tensorflow_core/python/eager/function.py", line 1141, in _filtered_call
    self.captured_inputs)
  File "/root/anaconda3/envs/keke/lib/python3.6/site-packages/tensorflow_core/python/eager/function.py", line 1224, in _call_flat
    ctx, args, cancellation_manager=cancellation_manager)
  File "/root/anaconda3/envs/keke/lib/python3.6/site-packages/tensorflow_core/python/eager/function.py", line 511, in call
    ctx=ctx)
  File "/root/anaconda3/envs/keke/lib/python3.6/site-packages/tensorflow_core/python/eager/execute.py", line 67, in quick_execute
    six.raise_from(core._status_to_exception(e.code, message), None)
  File "<string>", line 3, in raise_from
tensorflow.python.framework.errors_impl.DataLossError: 2 root error(s) found.
  (0) Data loss:  corrupted record at 2146170
         [[node IteratorGetNext (defined at /root/anaconda3/envs/keke/lib/python3.6/site-packages/tensorflow_core/python/framework/ops.py:1751) ]]
  (1) Data loss:  corrupted record at 2146170
         [[node IteratorGetNext (defined at /root/anaconda3/envs/keke/lib/python3.6/site-packages/tensorflow_core/python/framework/ops.py:1751) ]]
         [[IteratorGetNext/_2]]
0 successful operations.
0 derived errors ignored. [Op:__inference_distributed_function_89740]

Function call stack:
distributed_function -> distributed_function