python numpy dtype object_numpy - How to save a Python dtype('object') in an HDF5 file?

最新推荐文章于 2024-08-06 16:57:40 发布

weixin_39600291

最新推荐文章于 2024-08-06 16:57:40 发布

阅读量162

点赞数

文章标签： python numpy dtype object

在尝试将.mat文件中的数据加载到HDF5文件时遇到问题，因为scipy.io.loadmat返回的字典包含.object类型的数组。这导致在创建H5Py数据集时出现错误。解决方案是将.scipy multidimensional array转换为列表，使用ndarray.tolist()方法，然后可以成功创建数据集。

摘要由CSDN通过智能技术生成

I am trying to load data from .mat files into python, so that I can save it in an HDF5 file.

The problem is scipy.io.loadmat returns a dictionary containing arrays of dtype=[('counts', '|O4')])

Whenever I try to create a dataset with one of the arrays returned, I get the following error:

>> mat = scipy.io.loadmat('state-10.mat')

>> h = h5py.File('test.hdf5','w')

>> h.create_dataset('set', data=mat['ProteinComplex'])

Traceback (most recent call last):

File "", line 1, in

File "/usr/lib/python2.7/dist-packages/h5py/_hl/group.py", line 69, in create_dataset

dsid = dataset.make_new_dset(self, shape, dtype, data, **kwds)

File "/usr/lib/python2.7/dist-packages/h5py/_hl/dataset.py", line 88, in make_new_dset

tid = h5t.py_create(dtype, logical=1)

File "h5t.pyx", line 1332, in h5py.h5t.py_create (h5py/h5t.c:11600)

File "h5t.pyx", line 1373, in h5py.h5t.py_create (h5py/h5t.c:11209)

File "h5t.pyx", line 1311, in h5py.h5t._c_compound (h5py/h5t.c:10695)

File "h5t.pyx", line 1332, in h5py.h5t.py_create (h5py/h5t.c:11600)

File "h5t.pyx", line 1402, in h5py.h5t.py_create (h5py/h5t.c:11465)

TypeError: Object dtype dtype('object') has no native HDF5 equivalent

Is there any workaround from this?as to enter memory at some point anyway. My code, out of necessity, already run piecemeal over data from these cubes, pulling small bits into memory at a time. These functions are simplest if they simply iterate over the entirety of the dat

python

numpy

hdf5

this question asked Apr 17 '13 at 16:03

ncphillips 354 2 12

1 Answers

---Accepted---Accepted---Accepted---

It turns out the type of mat['ProteinComplex'] is a scipy multidimensional array (ndarray). It is possible to convert these to lists with the ndarray.tolist() method. If we change the nd array into a list it will work when creating the dataset with H5Py

>> mat = scipy.io.loadmat('state-10.mat')

>> h = h5py.File('test.hdf5','w')

>> protein_complex = mat['ProteinComplex'].tolist()

>> h.create_dataset('ProteinComplex', data=protein_complex)

this answer answered Apr 18 '13 at 16:32

ncphillips 354 2 12

I really need to fine a way to store my object. I have designed my file and already stored other datatypes into it and don't wanna move to other data strcuture. does anybody know a trick in order to store this datatype into h5py currently