在复现https://github.com/skumra/robotic-grasping时,使用MindSpore
加载数据出现异常,提示Dataset错误。很不明白MindSpore
中Dataset
的原理
[ERROR] MD(9471,7f0664347700,python):2021-12-28-01:08:07.456.946 [mindspore/ccsrc/minddata/dataset/core/data_type.cc:159] FromNpArray] Cannot convert from numpy type. Unknown data type is returned! [ERROR] MD(9471,7f0664347700,python):2021-12-28-01:08:07.456.992 [mindspore/ccsrc/minddata/dataset/core/data_type.cc:159] FromNpArray] Cannot convert from numpy type. Unknown data type is returned! [ERROR] MD(9471,7f0664347700,python):2021-12-28-01:08:07.717.762 [mindspore/ccsrc/minddata/dataset/util/task.cc:67] operator()] Task: GeneratorOp(ID:1) - thread(139665427691264) is terminated with err msg: Unexpected error. Invalid data type. Line of code : 93 File : /home/jenkins/agent-working-dir/workspace/Compile_GPU_X86_CentOS_Cuda11/mindspore/mindspore/ccsrc/minddata/dataset/core/tensor.cc [ERROR] MD(9471,7f0664347700,python):2021-12-28-01:08:07.717.820 [mindspore/ccsrc/minddata/dataset/util/task_manager.cc:217] InterruptMaster] Task is terminated with err msg(more detail in info level log):Unexpected error. Invalid data type. Line of code : 93 File : /home/jenkins/agent-working-dir/workspace/Compile_GPU_X86_CentOS_Cuda11/mindspore/mindspore/ccsrc/minddata/dataset/core/tensor.cc Traceback (most recent call last): File "/home/guoruiming/pycharm_project_232/GR-ConvNet/train.py", line 39, in <module> train(epoch, net, "GPU", train_data_iter, optimizer, 1000) File "/home/guoruiming/pycharm_project_232/GR-ConvNet/train.py", line 17, in train for data in train_data: File "/usr/local/python-3.7.5/lib/python3.7/site-packages/mindspore/dataset/engine/iterators.py", line 134, in __next__ data = self._get_next() File "/usr/local/python-3.7.5/lib/python3.7/site-packages/mindspore/dataset/engine/iterators.py", line 208, in _get_next return [self._transform_tensor(t) for t in self._iterator.GetNextAsList()] RuntimeError: Unexpected error. Invalid data type. Line of code : 93 File : /home/jenkins/agent-working-dir/workspace/Compile_GPU_X86_CentOS_Cuda11/mindspore/mindspore/ccsrc/minddata/dataset/core/tensor.cc Process finished with exit code 1
上述问题应该已解决,对应pr为: https://gitee.com/mindspore/mindspore/pulls/25805 - 主要原因应该是自定义Dataset的返回值,或者map接收的自定义函数的返回值,不是tuple of numpy array, 而是类似dict等 - 可以取从官网安装页面 取nightly版本验证下,看是否还有这个问题