python读取文本文档内容并存入列表内_从二进制文件读取/写入Python列表

According to Python Cookbook, below is how to write a list of tuple into binary file:

from struct import Struct

def write_records(records, format, f):

'''

Write a sequence of tuples to a binary file of structures.

'''

record_struct = Struct(format)

for r in records:

f.write(record_struct.pack(*r))

# Example

if __name__ == '__main__':

records = [ (1, 2.3, 4.5),

(6, 7.8, 9.0),

(12, 13.4, 56.7) ]

with open('data.b', 'wb') as f:

write_records(records, '

And it works well.

For reading (large amount of binary data), the author recommended the following:

>>> import numpy as np

>>> f = open('data.b', 'rb')

>>> records = np.fromfile(f, dtype='

>>> records

array([(1, 2.3, 4.5), (6, 7.8, 9.0), (12, 13.4, 56.7)],

dtype=[('f0', '

>>> records[0]

(1, 2.3, 4.5)

>>> records[1]

(6, 7.8, 9.0)

>>>

It is also good, but this record is not a normal numpy array. For instance, type(record[0]) will return . Even worse, I cannot extract the first column using X = record[:, 0].

Is there a way to efficiently load list(or any other types) from binary file into a normal numpy array?

Thx in advance.

解决方案In [196]: rec = np.fromfile('data.b', dtype='

In [198]: rec

Out[198]:

array([( 1, 2.3, 4.5), ( 6, 7.8, 9. ), (12, 13.4, 56.7)],

dtype=[('f0', '

This is a 1d structured array

In [199]: rec['f0']

Out[199]: array([ 1, 6, 12], dtype=int32)

In [200]: rec.shape

Out[200]: (3,)

In [201]: rec.dtype

Out[201]: dtype([('f0', '

Note that its tolist looks identical to your original records:

In [202]: rec.tolist()

Out[202]: [(1, 2.3, 4.5), (6, 7.8, 9.0), (12, 13.4, 56.7)]

In [203]: records

Out[203]: [(1, 2.3, 4.5), (6, 7.8, 9.0), (12, 13.4, 56.7)]

You could create a 2d array from either list with:

In [204]: arr2 = np.array(rec.tolist())

In [205]: arr2

Out[205]:

array([[ 1. , 2.3, 4.5],

[ 6. , 7.8, 9. ],

[ 12. , 13.4, 56.7]])

In [206]: arr2.shape

Out[206]: (3, 3)

There are other ways of converting a structured array to 'regular' array, but this is simplest and most consistent.

The tolist of a regular array uses nested lists. The tuples in the structured version are intended to convey a difference:

In [207]: arr2.tolist()

Out[207]: [[1.0, 2.3, 4.5], [6.0, 7.8, 9.0], [12.0, 13.4, 56.7]]

In the structured array the first field is integer. In the regular array the first column is same as the others, float.

If the binary file contained all floats, you could load it as a 1d of floats and reshape

In [208]: with open('data.f', 'wb') as f:

...: write_records(records, 'ddd', f)

In [210]: rec2 = np.fromfile('data.f', dtype='

In [211]: rec2

Out[211]: array([ 1. , 2.3, 4.5, 6. , 7.8, 9. , 12. , 13.4, 56.7])

But to take advantage of any record structure in the binary file, you have load by records as well, which means structured array:

In [213]: rec3 = np.fromfile('data.f', dtype='d,d,d')

In [214]: rec3

Out[214]:

array([( 1., 2.3, 4.5), ( 6., 7.8, 9. ), ( 12., 13.4, 56.7)],

dtype=[('f0', '