python loadtxt,使用numpy.loadtxt（）将文本文件加载为字符串

I would like to load a big text file (around 1 GB with 3*10^6 rows and 10 - 100 columns) as a 2D np-array containing strings. However, it seems like numpy.loadtxt() only takes floats as default. Is it possible to specify another data type for the entire array? I've tried the following without luck:

loadedData = np.loadtxt(address, dtype=np.str)

I get the following error message:

/Library/Python/2.7/site-packages/numpy-1.8.0.dev_20224ea_20121123-py2.7-macosx-10.8-x86_64.egg/numpy/lib/npyio.pyc in loadtxt(fname, dtype, comments, delimiter, converters, skiprows, usecols, unpack, ndmin)

833 fh.close()

834

--> 835 X = np.array(X, dtype)

836 # Multicolumn data are returned with shape (1, N, M), i.e.

837 # (1, 1, M) for a single row - remove the singleton dimension there

ValueError: cannot set an array element with a sequence

Any ideas? (I don't know the exact number of columns in my file on beforehand.)

解决方案

Use genfromtxt instead. It's a much more general method than loadtxt:

import numpy as np

print np.genfromtxt('col.txt',dtype='str')

Using the file col.txt:

foo bar

cat dog

man wine

This gives:

[['foo' 'bar']

['cat' 'dog']

['man' 'wine']]

If you expect that each row has the same number of columns, read the first row and set the attribute filling_values to fix any missing rows.