numpy的文件存储与处理

import numpy as np
from io import StringIO

第二部分 ndarray 文件存储与读取

一维二维数组 可以采用csv（Comma Separated Value 逗号分割值）

”’
1)np.savetxt np.loadtxt

numpy.savetxt(fname, X, fmt=’%.18e’, delimiter=’ ‘, newline=’\n’, header=”, footer=”, comments=’# ‘)

fname : filename or file handle 指定要存入的文件名,支持gzip （.gz .bz2）格式
If the filename ends in .gz, the file is automatically saved in compressed gzip format. loadtxt understands gzipped files transparently(易察觉的).

X : array_like 一个ndarray对象
Data to be saved to a text file.

fmt : str or sequence of strs, optional 制定每个元素要存入的格式
e.g. [‘%.3e + %.3ej’, ‘(%.15e%+.15ej)’] for 2 columns (复数可以这么写 多列)

delimiter : str, optional 每个元素之间的分割符
String or character separating columns.

newline : str, optional 每行的换行符
String or character separating lines.
New in version 1.5.0.
header : str, optional 文件开始时的文本
String that will be written at the beginning of the file.
New in version 1.7.0.
footer : str, optional 文件结束时的文本
String that will be written at the end of the file.
New in version 1.7.0.
comments : str, optional 将header and footer 标注为注释
String that will be prepended to the header and footer strings, to mark them as comments. Default: ‘# ‘, as expected by e.g. numpy.loadtxt.
New in version 1.7.0.
”’
x=np.array([[1,2,3,4],[2,3,4]])#不可存储，用loadtxt读取不了,因为loadtxt必须保证列数相同
np.savetxt(fname=”test1.out”,fmt=”%s”,header=”This is test1.txt”,X=x)
y=np.array([[1,2],[3,4]])
np.savetxt(fname=”test2.out”,fmt=”%d” ,footer=”This is test2.txt”,X=y)
”’

numpy.loadtxt(fname, dtype=float, comments=’#’, delimiter=None, converters=None, skiprows=0, usecols=None, unpack=False, ndmin=0)

Parameters:
fname : file, str, or pathlib.Path
File, filename, or generator to read. If the filename extension is .gz or .bz2, the file is first decompressed. Note that generators should return byte strings for Python 3k.

dtype : data-type, optional
Data-type of the resulting array; default: float. If this is a structured data-type, the resulting array will be 1-dimensional, and each row will be interpreted as an element of the array. In this case, the number of columns used must match the number of fields in the data-type.

comments : str or sequence, optional（注释类型 要和savetxt的相对应）
The characters or list of characters used to indicate the start of a comment; default: ‘#’.
delimiter : str, optional
The string used to separate values. By default, this is any whitespace.

converters : dict, optional(解析函数）
A dictionary mapping column number to a function that will convert that column to a float. E.g., if column 0 is a date string: converters = {0: datestr2num}. Converters can also be used to provide a default value for missing data (but see also genfromtxt): converters = {3: lambda s: float(s.strip() or 0)}. Default: None.
skiprows : int, optional
Skip the first skiprows lines; default: 0.
usecols : int or sequence, optional（要提取的列）
Which columns to read, with 0 being the first. For example, usecols = (1,4,5) will extract the 2nd, 5th and 6th columns. The default, None, results in all columns being read.
New in version 1.11.0.

Also when a single column has to be read it is possible to use an integer instead of a tuple. E.g usecols = 3 reads the fourth column the same way as usecols = (3,)` would.
unpack : bool, optional(打包？)
If True, the returned array is transposed, so that arguments may be unpacked using x, y, z = loadtxt(…). When used with a structured data-type, arrays are returned for each field. Default is False.
ndmin : int, optional
The returned array will have at least ndmin dimensions. Otherwise mono-dimensional axes will be squeezed. Legal values: 0 (default), 1 or 2.
New in version 1.6.0.

Returns:
out : ndarray
Data read from the text file.
”’

test1=np.loadtxt(fname=”test2.out”,dtype=int)
print(“test2:”,test1)
x1,x2=np.loadtxt(fname=”test2.out”,dtype=int,unpack=True)
print(“x1,x2:”,x1,x2)
str=StringIO(“Zhang 20 boy\nYu 20 girl”)
struct_dtype=np.dtype({‘names’:(‘name’,’age’,’gender’), ‘formats’:(‘S32’,’i4’,’S32’)},align=True)
str_array=np.loadtxt(fname=str,dtype=struct_dtype)
print(“str_array:”,str_array)

2)多维数组

”’

tofile(self, fid, sep=”“, format=”%s”)

fid : file or str
An open file object, or a string containing a filename.
sep : str 如果为空，则默认为二进制存储
Separator between array items for text output.
If “” (empty), a binary file is written, equivalent to
file.write(a.tobytes()).
format : str 元素类型
Format string for text file output.
Each entry in the array is formatted to text by first converting
it to the closest Python type, and then using “format” % item.

            This is a convenience function for quick storage(存储) of array data.
        Information on endianness(字节序) and precision(精确度) is lost, so this method is not a
        good choice for files intended to(打算) archive data or transport data between
        machines with different endianness. Some of these problems can be overcome
        by outputting the data as text files, at the expense of(在损失....的情况下) speed and file
        size.

”’
y.tofile(file=”tofile.txt”,format=”%d” ,sep=” “)
from_file_array=np.fromfile(file=”tofile.txt”,dtype=int,count=-1,sep=” “)#count=-1默认读取整个文件
print(“from_file_array:”,from_file_array)
reshape_from_file_array=from_file_array.reshape((2,2))
print(“reshape_from_file_array:”,reshape_from_file_array)

3)numpy 自定义格式

np.save(file=”np_save”,arr=y)#默认.npy后缀 其实这个文件打开看一看，第一行是明文的这个ndarray数组的信息，第二行开始是二进制的数据存储
print(“np.load:”,np.load(file=”np_save.npy”))
np.savez_compressed(file=”np_save_compressed”,Y=y)#默认为压缩的.npz格式 y的名字为Y
print(“np.load:”,np.load(file=”np_save_compressed.npz”)[‘Y’])#数组以字典方式进行存储
np.savez(file=”np_save_uncompressed”,Y=y)#默认为非压缩的.npz格式
print(“np.load:”,np.load(file=”np_save_uncompressed.npz”,)[‘Y’])