numpy

一、创建numpy数组

There are 6 general mechanisms for creating arrays:
1.Conversion from other Python structures (i.e. lists and tuples)
2. Intrinsic NumPy array creation functions (e.g. arange, ones, zeros, etc.)
3.Replicating, joining, or mutating existing arrays
4. Reading arrays from disk, either from standard or custom formats
5.Creating arrays from raw bytes through the use of strings or buffers
6. Use of special library functions (e.g., random)

第一种：creating arrays from python sequences
a1D = np.array([1, 2, 3, 4])
a2D = np.array([[1, 2], [3, 4]])
a3D = np.array([[[1, 2], [3, 4]],
[[5, 6], [7, 8]]])

可以使用dtype参数指定array数据类型

a = np.array([127, 128, 129], dtype=np.int8)

第二种：built-in functions for creating arrays
linspace, ones, zeros, arange, logspace, geomspace, diag

第三种：对array复制、join，mutate产生新的array

A = np.ones((2, 2))
B = np.eye((2, 2))
C = np.zeros((2, 2))
D = np.diag((-3, -4))
np.block([[A, B],
[C, D]])

二、numpy数据类型

array数据类型转化使用astype(new type)
例如z.astype(float)

三、Copies and Views

四、index

1. 单索引

2. slice index

x = np.arange(10)
x[2:5]
array([2, 3, 4])
x[:-7]
array([0, 1, 2])
x[1:7:2]
array([1, 3, 5])
y = np.arange(35).reshape(5,7)
y[1:5:2,::3]

3. Index arrays

x = np.arange(10,1,-1)
array([10, 9, 8, 7, 6, 5, 4, 3, 2])
x[np.array([3, 3, 1, 8])]
array([7, 7, 9, 2])
x[np.array([[1,1],[2,3]])]
array([[9, 9],
[8, 7]])

x = np.array([[[1],[2],[3]], [[4],[5],[6]]])
x.shape
(2, 3, 1)
x[1:2]
array([[[4],
[5],
[6]]])
x[1]
x[1,…]
比较区别

Indexing Multi-dimensional arrays

y[np.array([0,2,4]), np.array([0,1,2])]
array([ 0, 15, 30])

x = np.array([[1, 2], [3, 4], [5, 6]])
x[[0, 1, 2], [0, 1, 0]]
array([1, 4, 5])

3
x = np.array([[ 0, 1, 2],
[ 3, 4, 5],
[ 6, 7, 8],
[ 9, 10, 11]])
rows = np.array([[0, 0],
[3, 3]], dtype=np.intp)
columns = np.array([[0, 2],
[0, 2]], dtype=np.intp)
x[rows, columns]
array([[ 0, 2],
[ 9, 11]])

Boolean or “mask” index arrays

五、broadcasting

When operating on two arrays, NumPy compares their shapes element-wise. It starts with the trailing (i.e. rightmost) dimensions and works its way left. Two dimensions are compatible when

they are equal, or
one of them is 1

For example,
Image (3d array): 256 x 256 x 3
Scale (1d array): 3
Result (3d array): 256 x 256 x 3

The following example meet the condition above.
A (4d array): 8 x 1 x 6 x 1
B (3d array): 7 x 1 x 5
Result (4d array): 8 x 7 x 6 x 5
A (2d array): 5 x 4
B (1d array): 1
Result (2d array): 5 x 4

A (2d array): 5 x 4
B (1d array): 4
Result (2d array): 5 x 4

A (3d array): 15 x 3 x 5
B (3d array): 15 x 1 x 5
Result (3d array): 15 x 3 x 5

A (3d array): 15 x 3 x 5
B (2d array): 3 x 5
Result (3d array): 15 x 3 x 5

A (3d array): 15 x 3 x 5
B (2d array): 3 x 1
Result (3d array): 15 x 3 x 5

六、array attribute

七、array method

array convertion

tolist: convert array to a list
astype: convert the datype of array
reshape: convert shape of array
copy: copy array
view: generate a new view of array
resize: If the new array is larger than the original array, then the new
array is filled with repeated copies of a. Note that this behavior
is different from a.resize(new_shape) which fills with zeros instead
of repeated copies of a.
for example,
> a=np.array([[0,1],[2,3]])
>>> np.resize(a,(2,3))
array([[0, 1, 2],
[3, 0, 1]])

expand_dims： add the dimension, which is equivalent to np.newaxis
squeeze: remove the dimension
transpose: Returns a view of the array with axes transposed.
for example
a = np.array([[1, 2], [3, 4]])
a
array([[1, 2],
[3, 4]])
a.transpose()
array([[1, 3],
[2, 4]])
swapaxis: Interchange two axes of an array.

x = np.array([[[0,1],[2,3]],[[4,5],[6,7]]])
x
array([[[0, 1],
[2, 3]],
[[4, 5],
[6, 7]]])
np.swapaxes(x,0,2)
array([[[0, 4],
[2, 6]],
[[1, 5],
[3, 7]]])
flatten:
ravel: Return a copy of the array collapsed into one dimension.

item selection and manipulation

take:
select:
where:
put: Replaces specified elements of an array with given values.
repeat:
for example.
x = np.array([[1,2],[3,4]])
>>> np.repeat(x, 2)
array([1, 1, 2, 2, 3, 3, 4, 4])
>>> np.repeat(x, 3, axis=1)
array([[1, 1, 1, 2, 2, 2],
[3, 3, 3, 4, 4, 4]])
>>> np.repeat(x, [1, 2], axis=0)
array([[1, 2],
[3, 4],
[3, 4]])
delete:
insert:
append:
unqiue: Find the unique elements of an array.
np.unique([1, 1, 2, 2, 3, 3])
array([1, 2, 3])
a = np.array([[1, 1], [2, 3]])
np.unique(a)
array([1, 2, 3])
Return the unique rows of a 2D array

a = np.array([[1, 0, 0], [1, 0, 0], [2, 3, 4]])
np.unique(a, axis=0)
array([[1, 0, 0], [2, 3, 4]])
Return the indices of the original array that give the unique values:

a = np.array([‘a’, ‘b’, ‘b’, ‘c’, ‘a’])
u, indices = np.unique(a, return_index=True)
u
array([‘a’, ‘b’, ‘c’], dtype=’<U1’)
indices
array([0, 1, 3])
a[indices]
array([‘a’, ‘b’, ‘c’], dtype=’<U1’)
Reconstruct the input array from the unique values and inverse:

a = np.array([1, 2, 6, 4, 2, 3, 2])
u, indices = np.unique(a, return_inverse=True)
u
array([1, 2, 3, 4, 6])
indices
array([0, 1, 4, 3, 1, 2, 1])
u[indices]
array([1, 2, 6, 4, 2, 3, 2])
Reconstruct the input values from the unique values and counts:

a = np.array([1, 2, 6, 4, 2, 3, 2])
values, counts = np.unique(a, return_counts=True)
values
array([1, 2, 3, 4, 6])
counts
array([1, 3, 1, 1, 1])
np.repeat(values, counts)
array([1, 2, 2, 2, 3, 4, 6])

calculation

statistical calculation

sum, mean, std, max,min, argmin, argmax, sort, argsort, all, any, var,+,- dot,

numpy.lib.scimath

log

Linear algebra (numpy.linalg)

Logic functions

join and split

vstack:
hstack:
expand_dims:
squeeze:
concatenate:
stack: Join a sequence of arrays along a new axis. The stacked array has one more dimension than the input arrays. for example,

arrays = [np.random.randn(3, 4) for _ in range(10)]
np.stack(arrays, axis=0).shape
(10, 3, 4)
np.stack(arrays, axis=1).shape
(3, 10, 4)
np.stack(arrays, axis=2).shape
(3, 4, 10)
block:
Assemble an nd-array from nested lists of blocks.

Blocks in the innermost lists are concatenated (see concatenate) along the last dimension (-1), then these are concatenated along the second-last dimension (-2), and so on until the outermost list is reached.

dstack:
a = np.array((1,2,3))
b = np.array((2,3,4))
np.dstack((a,b))
array([[[1, 2],
[2, 3],
[3, 4]]])
a = np.array([[1],[2],[3]])
b = np.array([[2],[3],[4]])
np.dstack((a,b))
array([[[1, 2]],
[[2, 3]],
[[3, 4]]])

column_stack(tup): Stack 1-D arrays as columns into a 2-D array.
a = np.array((1,2,3))
b = np.array((2,3,4))
np.column_stack((a,b))
array([[1, 2],
[2, 3],
[3, 4]])
row_stack: Stack arrays in sequence vertically (row wise).

tile: Construct an array by repeating A the number of times given by reps.
a = np.array([0, 1, 2])
np.tile(a, 2)
array([0, 1, 2, 0, 1, 2])
np.tile(a, (2, 2))
array([[0, 1, 2, 0, 1, 2],
[0, 1, 2, 0, 1, 2]])
np.tile(a, (2, 1, 2))
array([[[0, 1, 2, 0, 1, 2]],
[[0, 1, 2, 0, 1, 2]]])
b = np.array([[1, 2], [3, 4]])
np.tile(b, 2)
array([[1, 2, 1, 2],
[3, 4, 3, 4]])
np.tile(b, (2, 1))
array([[1, 2],
[3, 4],
[1, 2],
[3, 4]])
c = np.array([1,2,3,4])
np.tile(c,(4,1))
array([[1, 2, 3, 4],
[1, 2, 3, 4],
[1, 2, 3, 4],
[1, 2, 3, 4]])

八、string operation

The numpy.char module provides a set of vectorized string operations for arrays of type numpy.str_ or numpy.bytes_. All of them are based on the string methods in the Python standard library.
add
multiply
mod
center
ljust
rjust
lstrip
rstrip
strip
partition
rpartition
split
captilize
lower
upper
join
swapcase
title
rfind
rindex
count
isnumerical
index
find
startswith
endswith

九 help

lookfor
info：When used interactively with an object, np.info(obj) is equivalent to help(obj) on the Python prompt or obj? on the IPython prompt. When using a string for object it is possible to get multiple results.
np.info(np.polyval)
polyval(p, x)
Evaluate the polynomial p at x.
np.info(‘fft’)
*** Found in numpy ***
Core FFT routines

 *** Found in numpy.fft ***

fft(a, n=None, axis=-1)

 *** Repeat reference found in numpy.fft.fftpack ***
 *** Total of 3 references found. ***

source: Print or write to a file the source code for a NumPy object.

np.source(np.interp)
In file: /usr/lib/python2.6/dist-packages/numpy/lib/function_base.py
def interp(x, xp, fp, left=None, right=None):
“”"… (full docstring printed)"""
if isinstance(x, (float, int, number)):
return compiled_interp([x], xp, fp, left, right).item()
else:
return compiled_interp(x, xp, fp, left, right)

十、Random sampling (numpy.random)¶

Numpy’s random number routines produce pseudo random numbers using combinations of a BitGenerator to create sequences and a Generator to use those sequences to sample from different statistical distributions:

十一、set routine

unique：
in1d: Test whether each element of a 1-D array is also present in a second array.
test = np.array([0, 1, 2, 5, 0])
states = [0, 2]
mask = np.in1d(test, states)
mask
array([ True, False, True, False, True])
test[mask]
array([0, 2, 0])
mask = np.in1d(test, states, invert=True)
mask
array([False, True, False, True, False])
test[mask]
array([1, 5])

intersect1d: Find the intersection of two arrays.

np.intersect1d([1, 3, 4, 3], [3, 1, 2, 1])
array([1, 3])
To intersect more than two arrays, use functools.reduce:

from functools import reduce
reduce(np.intersect1d, ([1, 3, 4, 3], [3, 1, 2, 1], [6, 3, 4, 2]))
array([3])
To return the indices of the values common to the input arrays along with the intersected values:

x = np.array([1, 1, 2, 3, 4])
y = np.array([2, 1, 4, 6])
xy, x_ind, y_ind = np.intersect1d(x, y, return_indices=True)
x_ind, y_ind
(array([0, 2, 4]), array([1, 0, 2]))
xy, x[x_ind], y[y_ind]
(array([1, 2, 4]), array([1, 2, 4]), array([1, 2, 4]))

isin: Calculates element in test_elements, broadcasting over element only. Returns a boolean array of the same shape as element that is True where an element of element is in test_elements and False otherwise.
element = 2*np.arange(4).reshape((2, 2))
element
array([[0, 2],
[4, 6]])
test_elements = [1, 2, 4, 8]
mask = np.isin(element, test_elements)
mask
array([[False, True],
[ True, False]])
element[mask]
array([2, 4])
The indices of the matched values can be obtained with nonzero:

np.nonzero(mask)
(array([0, 1]), array([1, 0]))
The test can also be inverted:

mask = np.isin(element, test_elements, invert=True)
mask
array([[ True, False],
[False, True]])
element[mask]
array([0, 6])
Because of how array handles sets, the following does not work as expected:

test_set = {1, 2, 4, 8}
np.isin(element, test_set)
array([[False, False],
[False, False]])
Casting the set to a list gives the expected result:

np.isin(element, list(test_set))
array([[False, True],
[ True, False]])

setdiff1d: Find the set difference of two arrays.
setxor1d: Find the set exclusive-or of two arrays.
union1d: Find the union of two arrays.

十二 inp input and output

binary input and output

load: Load arrays or pickled objects from .npy, .npz or pickled files.

If the file contains pickle data, then whatever object is stored in the pickle is returned.

If the file is a .npy file, then a single array is returned.

If the file is a .npz file, then a dictionary-like object is returned, containing {filename: array} key-value pairs, one for each file in the archive.

If the file is a .npz file, the returned value supports the context manager protocol in a similar fashion to the open function:

with load(‘foo.npz’) as data:
a = data[‘a’]
The underlying file descriptor is closed when exiting the ‘with’ block.
Examples

Store data to disk, and load it again:

np.save(’/tmp/123’, np.array([[1, 2, 3], [4, 5, 6]]))
np.load(’/tmp/123.npy’)
array([[1, 2, 3],
[4, 5, 6]])
Store compressed data to disk, and load it again:

a=np.array([[1, 2, 3], [4, 5, 6]])
b=np.array([1, 2])
np.savez(’/tmp/123.npz’, a=a, b=b)
data = np.load(’/tmp/123.npz’)
data[‘a’]
array([[1, 2, 3],
[4, 5, 6]])
data[‘b’]
array([1, 2])
data.close()
Mem-map the stored array, and then access the second row directly from disk:

X = np.load(’/tmp/123.npy’, mmap_mode=‘r’)
X[1, :]
memmap([4, 5, 6])

save: Save an array to a binary file in NumPy .npy format.
with open(‘test.npy’, ‘wb’) as f:
np.save(f, np.array([1, 2]))
np.save(f, np.array([1, 3]))
with open(‘test.npy’, ‘rb’) as f:
a = np.load(f)
b = np.load(f)
savez: Save several arrays into a single file in uncompressed .npz format.

outfile = TemporaryFile()
np.savez(outfile, x=x, y=y)
_ = outfile.seek(0)
npzfile = np.load(outfile)
sorted(npzfile.files)
[‘x’, ‘y’]
npzfile[‘x’]
array([0, 1, 2, 3, 4, 5, 6, 7, 8, 9])

savez_compressed: Save several arrays into a single file in compressed .npz format.

text file input and output

savetext: Save an array to a text file.
x = y = z = np.arange(0.0,5.0,1.0)
np.savetxt(‘test.out’, x, delimiter=’,’) # X is an array
np.savetxt(‘test.out’, (x,y,z)) # x,y,z equal sized 1D arrays
np.savetxt(‘test.out’, x, fmt=’%1.4e’) # use exponential notation

loadtxt:
from io import StringIO # StringIO behaves like a file object
c = StringIO(“0 1\n2 3”)
np.loadtxt©
array([[0., 1.],
[2., 3.]]

genfromtxt: Load data from a text file, with missing values handled as specified.
fromregex: Construct an array from a text file, using regular expression parsing.
fromstring:A new 1-D array initialized from text data in a string.
np.fromstring(‘1 2’, dtype=int, sep=’ ')

raw file

fromfile:
tofile: