Numpy学习笔记

多维数组基本操作

Numpy库是一个使用科学计算的基础库

创建ndarray

In [6]: import numpy as np
# 初始化一个列表
In [7]: list_data=[3,6,6.7,9,1,0]
# 使用numpy里的array方法获取数组
In [8]: array1 = np.array(list_data)

In [9]: array1
Out[9]: array([3. , 6. , 6.7, 9. , 1. , 0. ])

In [10]: list_data=[[1,2,3,4],[5,6,7,8]]

In [11]: array2 = np.array(list_data)

In [12]: array2
Out[12]:
array([[1, 2, 3, 4],
       [5, 6, 7, 8]])
       

查看数组的属性

# 维度
In [13]: array1.ndim
Out[13]: 1

In [14]: array2.ndim
Out[14]: 2
# 形状
In [15]: array2.shape
Out[15]: (2, 4)

In [16]: array1.shape
Out[16]: (6,)
# 数据类型
In [17]: array1.dtype
Out[17]: dtype('float64')

In [18]: array2.dtype
Out[18]: dtype('int32')

创建数组基础方法

指定特定数据类型

astype()函数可以直接转换数据类型

In [19]: arr1 = np.array([1,2,3], dtype=np.float64)

In [20]: arr1.dtype
Out[20]: dtype('float64')
# 或使用类型代码int_arr1 = arr1.astype('i4')
In [21]: int_arr1 = arr1.astype(np.int32)
In [22]: int_arr1.dtype
Out[22]: dtype('int32')

数组运算

与普通四则运算不同在于会延伸到数组里的每个元素

In [29]: arr = np.array([[1., 2., 3.], [4., 5., 6.]])

In [30]: arr*arr
Out[30]:
array([[ 1.,  4.,  9.],
       [16., 25., 36.]])

In [31]: arr-arr
Out[31]:
array([[0., 0., 0.],
       [0., 0., 0.]])

数组的索引和切片于列表类似

In [33]: arr3x3=np.array([[1, 2, 3], [4, 5, 6],[7, 8, 9]])

In [34]: arr3x3[:2,1:]
Out[34]:
array([[2, 3],
       [5, 6]])
# 选取第二行的前两列 [在第几行, 选取哪几列]
In [35]: arr3x3[1,:2]
Out[35]: array([4, 5])
# 选取第三列的前两行 [选取那几行, 在第几列]
In [36]: arr3x3[:2,2]
Out[36]: array([3, 6])
# 维度切片 [每一行,第一列]
In [40]: arr3x3[:,:1]
Out[40]:
array([[1],
       [4],
       [7]])

可对切片区域赋值

In [44]: arr3x3[:,2:] = 0

In [45]: arr3x3
Out[45]:
array([[1, 2, 0],
       [4, 5, 0],
       [7, 8, 0]])

利用布尔数组进行切片

In [49]: names = np.array(['Bob','Joe','Will','Bob','Will','Joe','Joe'])

In [50]: data = np.random.randn(7,4)

In [51]: names
Out[51]: array(['Bob', 'Joe', 'Will', 'Bob', 'Will', 'Joe', 'Joe'], dtype='<U4')

In [52]: data
Out[52]:
array([[-0.04231319, -0.52927321,  1.01094362,  0.28275029],
       [-0.5517945 ,  0.15585963, -0.95589379, -1.94704089],
       [ 0.271432  ,  0.14834617, -0.41995274,  0.69163998],
       [ 0.53585224,  0.45810894, -0.1749757 ,  0.82619417],
       [-0.03514554, -0.34414769,  1.14273879,  0.85434894],
       [ 1.57857093,  0.84154066, -0.18796459,  0.6071804 ],
       [ 0.95654895, -0.12625911, -2.09689247,  0.30145266]])

In [54]: names=='Bob'
Out[54]: array([ True, False, False,  True, False, False, False])

In [55]: data[names=='Bob']
Out[55]:
array([[-0.04231319, -0.52927321,  1.01094362,  0.28275029],
       [ 0.53585224,  0.45810894, -0.1749757 ,  0.82619417]])

# 选取 names=='Bob'的行，中的前三列
In [56]: data[names=='Bob',:3]
Out[56]:
array([[-0.04231319, -0.52927321,  1.01094362],
       [ 0.53585224,  0.45810894, -0.1749757 ]])

索引进阶

In [65]: arr = np.empty((8,4))

In [67]: for i in range(8):
    ...:     arr[i] = i
    
In [68]: arr
Out[68]:
array([[0., 0., 0., 0.],
       [1., 1., 1., 1.],
       [2., 2., 2., 2.],
       [3., 3., 3., 3.],
       [4., 4., 4., 4.],
       [5., 5., 5., 5.],
       [6., 6., 6., 6.],
       [7., 7., 7., 7.]])
# 选取指定行
In [71]: arr[[4,2,5,-7]]
Out[71]:
array([[4., 4., 4., 4.],
       [2., 2., 2., 2.],
       [5., 5., 5., 5.],
       [1., 1., 1., 1.]])

# reshape 指定矩阵形状
In [77]: arr = np.arange(32).reshape((8,4))

In [78]: arr
Out[78]:
array([[ 0,  1,  2,  3],
       [ 4,  5,  6,  7],
       [ 8,  9, 10, 11],
       [12, 13, 14, 15],
       [16, 17, 18, 19],
       [20, 21, 22, 23],
       [24, 25, 26, 27],
       [28, 29, 30, 31]])
# 选取指定坐标位置的值
In [79]: arr[[1,5,4,3],[3,2,1,0]]
Out[79]: array([ 7, 22, 17, 12])

# 选取指定行，指定列。 如：第2行的4，3，2，1列
In [80]: arr[[1,5,4,3]][:,[3,2,1,0]]
Out[80]:
array([[ 7,  6,  5,  4],
       [23, 22, 21, 20],
       [19, 18, 17, 16],
       [15, 14, 13, 12]])

数组的转置和轴对换

转置和运算

numpy.linalg 有许多关于矩阵运算的方法

# 转置
In [82]: arr.T
Out[82]:
array([[ 0,  4,  8, 12, 16, 20, 24, 28],
       [ 1,  5,  9, 13, 17, 21, 25, 29],
       [ 2,  6, 10, 14, 18, 22, 26, 30],
       [ 3,  7, 11, 15, 19, 23, 27, 31]])

# dot()计算矩阵相乘（外积），若参数都为一维矩阵则计算内积，即对应数相乘之和
In [83]: np.dot(arr.T,arr)
Out[83]:
array([[2240, 2352, 2464, 2576],
       [2352, 2472, 2592, 2712],
       [2464, 2592, 2720, 2848],
       [2576, 2712, 2848, 2984]])
       
In [60]: arr = np.random.randn(4,4)

In [61]: mat = arr.T.dot(arr)

In [62]: mat.dot(inv(mat))
Out[62]:
array([[ 1.00000000e+00, -1.77635684e-15, -4.44089210e-16,
        -2.22044605e-16],
       [ 0.00000000e+00,  1.00000000e+00, -1.33226763e-15,
        -2.22044605e-16],
       [ 2.66453526e-15,  0.00000000e+00,  1.00000000e+00,
        -2.22044605e-16],
       [ 6.66133815e-16,  4.44089210e-16,  0.00000000e+00,
         1.00000000e+00]])

numpy.linalg中的常用函数

轴对换

In [88]: arr = np.arange(24).reshape((2,3,4))

In [89]: arr
Out[89]:
array([[[ 0,  1,  2,  3],
        [ 4,  5,  6,  7],
        [ 8,  9, 10, 11]],

       [[12, 13, 14, 15],
        [16, 17, 18, 19],
        [20, 21, 22, 23]]])

X轴和Y轴对换

A2x3x4 转置 A3x2x4

方法一

将对应坐标进行转换，例如：若X轴和Y轴对换，则只需将坐标0,1,0 的值填入坐标 1,0,0中。（对其他轴对换也适用）

方法二

逐个面进行转置
底部

# 或者 arr.swapaxes(0,1)
In [90]: arr.transpose(1,0,2)
Out[90]:
array([[[ 0,  1,  2,  3],
        [12, 13, 14, 15]],

       [[ 4,  5,  6,  7],
        [16, 17, 18, 19]],

       [[ 8,  9, 10, 11],
        [20, 21, 22, 23]]])

X轴和Z轴对换

A2x3x4 转置 A4x3x2
正面

#或者 arr.swapaxes(0,2)
In [91]: arr.transpose(2,1,0)
Out[91]:
array([[[ 0, 12],
        [ 4, 16],
        [ 8, 20]],

       [[ 1, 13],
        [ 5, 17],
        [ 9, 21]],

       [[ 2, 14],
        [ 6, 18],
        [10, 22]],

       [[ 3, 15],
        [ 7, 19],
        [11, 23]]])

Y轴和Z轴对换

A2x3x4 转置 A2x4x3
侧面

# 或者 arr.swapaxes(1,2)
In [92]: arr.transpose(0,2,1)
Out[92]:
array([[[ 0,  4,  8],
        [ 1,  5,  9],
        [ 2,  6, 10],
        [ 3,  7, 11]],

       [[12, 16, 20],
        [13, 17, 21],
        [14, 18, 22],
        [15, 19, 23]]])

通用函数

In [2]: arr = np.arange(10)

In [3]: arr
Out[3]: array([0, 1, 2, 3, 4, 5, 6, 7, 8, 9])
# 计算平方根
In [4]: np.sqrt(arr)
Out[4]:
array([0.        , 1.        , 1.41421356, 1.73205081, 2.        ,
       2.23606798, 2.44948974, 2.64575131, 2.82842712, 3.        ])
#计算e的x指数
In [5]: np.exp(arr)
Out[5]:
array([1.00000000e+00, 2.71828183e+00, 7.38905610e+00, 2.00855369e+01,
       5.45981500e+01, 1.48413159e+02, 4.03428793e+02, 1.09663316e+03,
       2.98095799e+03, 8.10308393e+03])

In [6]: arr
Out[6]: array([0, 1, 2, 3, 4, 5, 6, 7, 8, 9])

In [8]: arr = arr.astype('f8')
# 具有相同的数据类型的数组可以直接对原数组进行操作
In [9]: np.sqrt(arr,arr)
Out[9]:
array([0.        , 1.        , 1.41421356, 1.73205081, 2.        ,
       2.23606798, 2.44948974, 2.64575131, 2.82842712, 3.        ])

In [10]: arr
Out[10]:
array([0.        , 1.        , 1.41421356, 1.73205081, 2.        ,
       2.23606798, 2.44948974, 2.64575131, 2.82842712, 3.        ])

更多通用函数

用数组处理数据

In [11]: points = np.arange(-5,5,0.01)

In [12]: xs,ys=np.meshgrid(points,points)

In [13]: ys
Out[13]:
array([[-5.  , -5.  , -5.  , ..., -5.  , -5.  , -5.  ],
       [-4.99, -4.99, -4.99, ..., -4.99, -4.99, -4.99],
       [-4.98, -4.98, -4.98, ..., -4.98, -4.98, -4.98],
       ...,
       [ 4.97,  4.97,  4.97, ...,  4.97,  4.97,  4.97],
       [ 4.98,  4.98,  4.98, ...,  4.98,  4.98,  4.98],
       [ 4.99,  4.99,  4.99, ...,  4.99,  4.99,  4.99]])

In [14]: xs
Out[14]:
array([[-5.  , -4.99, -4.98, ...,  4.97,  4.98,  4.99],
       [-5.  , -4.99, -4.98, ...,  4.97,  4.98,  4.99],
       [-5.  , -4.99, -4.98, ...,  4.97,  4.98,  4.99],
       ...,
       [-5.  , -4.99, -4.98, ...,  4.97,  4.98,  4.99],
       [-5.  , -4.99, -4.98, ...,  4.97,  4.98,  4.99],
       [-5.  , -4.99, -4.98, ...,  4.97,  4.98,  4.99]])

In [15]: z = np.sqrt(xs**2+ys**2)

In [16]: z
Out[16]:
array([[7.07106781, 7.06400028, 7.05693985, ..., 7.04988652, 7.05693985,
        7.06400028],
       [7.06400028, 7.05692568, 7.04985815, ..., 7.04279774, 7.04985815,
        7.05692568],
       [7.05693985, 7.04985815, 7.04278354, ..., 7.03571603, 7.04278354,
        7.04985815],
       ...,
       [7.04988652, 7.04279774, 7.03571603, ..., 7.0286414 , 7.03571603,
        7.04279774],
       [7.05693985, 7.04985815, 7.04278354, ..., 7.03571603, 7.04278354,
        7.04985815],
       [7.06400028, 7.05692568, 7.04985815, ..., 7.04279774, 7.04985815,
        7.05692568]])
In [18]: import matplotlib.pyplot as plt
In [27]: plt.imshow(z,cmap=plt.cm.Blues);plt.colorbar();plt.title("image $\sqrt{x^2+y^2}$")
Out[27]: Text(0.5, 1, 'image $\\sqrt{x^2+y^2}$')

where()函数：where(判断语句,True情况执行,False情况执行)，与if语句功能类似。

统计方法函数

# 生成正态分布的数组
In [28]: arr=np.random.randn(5,4)

In [29]: arr
Out[29]:
array([[ 1.05174756,  0.94627221, -1.47924905, -0.0400526 ],
       [ 0.37094514, -0.27207601,  1.31544288,  0.02487793],
       [ 1.13919491, -0.81736487,  0.21520435,  0.28251311],
       [-1.10817169,  0.22068492, -0.95416473,  0.06779754],
       [-0.02334601,  0.16765813, -1.30903895,  0.56600925]])
# 计算平均值 np.mean(arr) 或
In [30]: arr.mean()
Out[30]: 0.01824420114243558
# 计算总和
In [32]: arr.sum()
Out[32]: 0.3648840228487116
# 计算每一列的平均值，axis表示在该轴上的
# 0表示x轴，1表示y轴
In [34]: arr.mean(axis=0)
Out[34]: array([ 0.28607398,  0.04903488, -0.4423611 ,  0.18022904])
# 统计正数的个数
In [37]: (arr>0).sum()
Out[37]: 12

函数	说明
mean	算术平均数，零长度的数组为NaN
sum	对数组中quanbu
std、var	分别为标准差和方差、自由度可调（默认为n）
min、max	最大值和最小值
argmin、argmax	分别为最大和最小元素的索引
cumsum	所有元素的累计和
cumprod	所有元素的累计积

布尔型数组的方法

any() 用于测试数组中是否存在一个或多个True;
all() 则检查数组中所有值是否都是True

数组排序

# 对指定轴向的数组排序，0为x轴，1为y轴
# 注意：指定y轴会令原本数组改变
In [39]: arr.sort(1)

In [40]: arr
Out[40]:
array([[-1.47924905, -0.0400526 ,  0.94627221,  1.05174756],
       [-0.27207601,  0.02487793,  0.37094514,  1.31544288],
       [-0.81736487,  0.21520435,  0.28251311,  1.13919491],
       [-1.10817169, -0.95416473,  0.06779754,  0.22068492],
       [-1.30903895, -0.02334601,  0.16765813,  0.56600925]])

集合运算

In [47]: ints = np.array([3, 3, 3, 2, 2, 1, 1, 4, 4])

In [48]: np.unique(ints)
Out[48]: array([1, 2, 3, 4])
# 上下等价np.unique(ints) == sorted(set(ints))
In [49]: sorted(set(ints))
Out[49]: [1, 2, 3, 4]

数组文件输入和输出

In [4]: arr1 = np.arange(10)

In [5]: arr2 = np.random.randn(10)
# 数组的文件输出
In [6]: np.save('arr1_array',arr1)
# 数组的文件输入
In [7]: np.load('arr1_array.npy')
Out[7]: array([0, 1, 2, 3, 4, 5, 6, 7, 8, 9])
# 多个数组的文件输出，以关键字形式保存
In [8]: np.savez('arr1+2_array.npz',a=arr1,b=arr2)
# 多个数组的文件输入
In [9]: arr1ANDarr2 = np.load('arr1+2_array.npz')
In [10]: arr1ANDarr2['b']
Out[10]:
array([-2.4996369 , -1.88216122, -0.75614113, -1.3758831 ,  1.00322028,
        0.04897909, -0.69571353, -1.82350073, -0.54210798,  1.28929306])
# 以数据压缩的形式的文件输出
In [11]: np.savez_compressed('arrays_compressed.npz',a=arr1,b=arr2)

生成指定分布的随机数组

# 更改随机种子
In [74]:np.random.seed(1245)
# 独立的随机数生成器，与其他生成器隔离开
In [75]: rng = np.random.RandomState(1234)
In [76]: rng.randn(10)
Out[76]:
array([ 0.47143516, -1.19097569,  1.43270697, -0.3126519 , -0.72058873,
        0.88716294,  0.85958841, -0.6365235 ,  0.01569637, -2.24268495])

In [77]: rng.normal(size=(3,3))
Out[77]:
array([[ 1.15003572,  0.99194602,  0.95332413],
       [-2.02125482, -0.33407737,  0.00211836],
       [ 0.40545341,  0.28909194,  1.32115819]])

简单的例子1

可见模式

In [78]: import random

In [79]: position = 0

In [80]: walk = [position]

In [81]: steps =1000

In [82]: for i in range(steps):
    ...:     step = 1 if random.randint(0,1) else -1
    ...:     position += step
    ...:     walk.append(position)
    ...:

In [83]: import matplotlib.pyplot as plt

In [84]: plt.plot(walk[:100])
Out[84]: [<matplotlib.lines.Line2D at 0x14880408dd8>]

数组模式

In [111]: nsteps = 1000

In [112]: draws=np.random.randint(0,2,size=nsteps)

In [113]: steps=np.where(draws>0,1,-1)
# cumsum 计算累计和，可对轴进行
In [115]: walk=steps.cumsum()

In [116]: walk.max()
Out[116]: 62
## argmax() 返回的第一个TRUE下标
## argmin() 返回的第一个FALSE下标
In [119]: (np.abs(walk) >=10).argmax()
Out[119]: 47

二维数组模式

In [125]: nwalks=5000

In [126]: nsteps=1000

In [127]: draws = np.random.randint(0,2,size=(nwalks,nsteps))

In [128]: draws
Out[128]:
array([[0, 0, 0, ..., 0, 1, 0],
       [0, 1, 1, ..., 0, 0, 1],
       [0, 0, 0, ..., 1, 1, 0],
       ...,
       [1, 1, 1, ..., 1, 0, 0],
       [1, 1, 0, ..., 1, 1, 1],
       [0, 1, 1, ..., 1, 1, 1]])

In [130]: steps=np.where(draws>0,1,-1)

In [131]: walks=steps.cumsum(1)

In [132]: walks
Out[132]:
array([[ -1,  -2,  -3, ..., -16, -15, -16],
       [ -1,   0,   1, ...,   0,  -1,   0],
       [ -1,  -2,  -3, ...,   2,   3,   2],
       ...,
       [  1,   2,   3, ...,   2,   1,   0],
       [  1,   2,   1, ..., -38, -37, -36],
       [ -1,   0,   1, ...,  12,  13,  14]], dtype=int32)

In [133]: steps.cumsum(0)
Out[133]:
array([[ -1,  -1,  -1, ...,  -1,   1,  -1],
       [ -2,   0,   0, ...,  -2,   0,   0],
       [ -3,  -1,  -1, ...,  -1,   1,  -1],
       ...,
       [ 84,  52, -62, ..., -72,  84, -78],
       [ 85,  53, -63, ..., -71,  85, -77],
       [ 84,  54, -62, ..., -70,  86, -76]], dtype=int32)

In [134]: steps.cumsum()
Out[134]: array([  -1,   -2,   -3, ..., 1916, 1917, 1918], dtype=int32)

In [135]: walks.max()
Out[135]: 126
# any(1) Y轴向的有TRUE的数组
In [136]: hits30 = (np.abs(walks) >= 30).any(1)

In [137]: hits30
Out[137]: array([ True, False, False, ..., False,  True, False])

In [138]: crossing_times = (np.abs(walks[hits30]) 

In [140]: crossing_times.mean()
Out[140]: 499.10712183637435

多维数组进阶操作

数组重组和散开

重组

In [152]:arr = np.arange(15)
# -1表示自动分配适合的列数，F表示按列顺序重组
In [153]: arr.reshape((5,-1),order='F')
Out[153]:
array([[ 0,  5, 10],
       [ 1,  6, 11],
       [ 2,  7, 12],
       [ 3,  8, 13],
       [ 4,  9, 14]])
# 默认order=’C‘，表示按行顺序重组
In [155]: arr.reshape((5,3))
Out[155]:
array([[ 0,  1,  2],
       [ 3,  4,  5],
       [ 6,  7,  8],
       [ 9, 10, 11],
       [12, 13, 14]])

散开

# flatten()和ravel()相似
In [161]: arr.reshape((5,3),order='F').ravel()
Out[161]: array([ 0,  5, 10,  1,  6, 11,  2,  7, 12,  3,  8, 13,  4,  9, 14])

In [162]: arr
Out[162]: array([ 0,  1,  2,  3,  4,  5,  6,  7,  8,  9, 10, 11, 12, 13, 14])
# 可以指定行('C')或列('F')散开
In [169]: arr.reshape((5,3),order='F').ravel('F')
Out[169]: array([ 0,  1,  2,  3,  4,  5,  6,  7,  8,  9, 10, 11, 12, 13, 14])

数组合并和拆分

In [180]: arr1=np.arange(1,7).reshape(-1,3)

In [181]: arr2=np.arange(7,13).reshape(-1,3)
# 按行合并（X轴向合并）
# np.vstack((arr1,arr2))或
In [182]: np.concatenate([arr1,arr2],axis=0)
Out[182]:
array([[ 1,  2,  3],
       [ 4,  5,  6],
       [ 7,  8,  9],
       [10, 11, 12]])
# 按列合并（Y轴向合并）
# np.hstack((arr1,arr2)) 或
In [183]: np.concatenate([arr1,arr2],axis=1)
Out[183]:
array([[ 1,  2,  3,  7,  8,  9],
       [ 4,  5,  6, 10, 11, 12]])
# 拆分
# 指定索引处拆分 split(array, index)
In [186]: np.split(np.vstack((arr1,arr2)),[1,3])
Out[186]:
[array([[1, 2, 3]]), array([[4, 5, 6],
        [7, 8, 9]]), array([[10, 11, 12]])]

更多方法

元素的重复操作

repeat() 一维数组对元素重复操作，多维数组轴向对元素重复操作
tile(array,) 对整个多维数组进行重复操作，如铺瓷砖一般

一维数组

In [195]: arr = np.arange(4)
# 每个元素重复两次
In [196]: arr.repeat(2)
Out[196]: array([0, 0, 1, 1, 2, 2, 3, 3])
# 指定下标元素重复的次数
In [198]: arr.repeat([2,3,1,2])
Out[198]: array([0, 0, 1, 1, 1, 2, 3, 3])

多维数组

In [211]: arr = np.random.randn(2,3)

In [212]: arr
Out[212]:
array([[ 0.69330678, -1.56588234,  0.27861828],
       [-1.63229888,  0.40831778, -0.32360164]])

In [213]: arr.repeat([1,2],axis=0)
Out[213]:
array([[ 0.69330678, -1.56588234,  0.27861828],
       [-1.63229888,  0.40831778, -0.32360164],
       [-1.63229888,  0.40831778, -0.32360164]])

In [214]: arr.repeat([1,2,1],axis=1)
Out[214]:
array([[ 0.69330678, -1.56588234, -1.56588234,  0.27861828],
       [-1.63229888,  0.40831778,  0.40831778, -0.32360164]])
# 瓷砖平铺2次
In [215]: np.tile(arr,2)
Out[215]:
array([[ 0.69330678, -1.56588234,  0.27861828,  0.69330678, -1.56588234,
         0.27861828],
       [-1.63229888,  0.40831778, -0.32360164, -1.63229888,  0.40831778,
        -0.32360164]])
# 瓷砖垂直2次水平3次
In [216]: np.tile(arr,(2,3))
Out[216]:
array([[ 0.69330678, -1.56588234,  0.27861828,  0.69330678, -1.56588234,
         0.27861828,  0.69330678, -1.56588234,  0.27861828],
       [-1.63229888,  0.40831778, -0.32360164, -1.63229888,  0.40831778,
        -0.32360164, -1.63229888,  0.40831778, -0.32360164],
       [ 0.69330678, -1.56588234,  0.27861828,  0.69330678, -1.56588234,
         0.27861828,  0.69330678, -1.56588234,  0.27861828],
       [-1.63229888,  0.40831778, -0.32360164, -1.63229888,  0.40831778,
        -0.32360164, -1.63229888,  0.40831778, -0.32360164]])

广播

广播需注意数组的形状（最小行或列为1，不可为空），如：在0(1)轴向上广播，则行(列)数最小要为1，列(行)数需与目标数组相同。

沿轴向传播

In [20]: arr = np.arange(15).reshape(3,5)

In [21]: arr.mean(0)
Out[21]: array([5., 6., 7., 8., 9.])

In [22]: list_mean = arr.mean(0)

In [23]: list_mean.shape
Out[23]: (5,)
# 重建数组的形状
# list_mean.reshape(1,5) 或

In [26]: list_mean[np.newaxis,:]
Out[26]: array([[5., 6., 7., 8., 9.]])

In [27]: arr - list_mean[np.newaxis,:]
Out[27]:
array([[-5., -5., -5., -5., -5.],
       [ 0.,  0.,  0.,  0.,  0.],
       [ 5.,  5.,  5.,  5.,  5.]])

In [28]: (arr - list_mean[np.newaxis,:]).mean(0)
Out[28]: array([0., 0., 0., 0., 0.])

广播初始化数组

In [47]: arr = np.arange(16).reshape(4,4)

In [48]: col = np.array([2,3,5,8])

In [51]: arr[:] = col[np.newaxis,:]

In [52]: arr
Out[52]:
array([[2, 3, 5, 8],
       [2, 3, 5, 8],
       [2, 3, 5, 8],
       [2, 3, 5, 8]])
     
In [57]: arr[1:3] = [[8],[9]]

In [58]: arr
Out[58]:
array([[2, 3, 5, 8],
       [8, 8, 8, 8],
       [9, 9, 9, 9],
       [2, 3, 5, 8]])

结构化数组

结构化数组可以由不同的数据类型组成的数组。

# 定义结构化数组的数据类型
In [89]: UserDef_dtype = [('F_X',np.float64),('I_Y',np.int32)]
# sarr = np.array([(1.6,3),(np.pi,-2)],[('X',np.float64),('Y',np.int32)]) 等价
In [90]: sarr = np.array([(1.6,3),(np.pi,-2)],dtype=UserDef_dtype)

In [92]: sarr
Out[92]:
array([(1.6       ,  3), (3.14159265, -2)],
      dtype=[('F_X', '<f8'), ('I_Y', '<i4')])

In [93]: sarr['F_X']
Out[93]: array([1.6       , 3.14159265])

结构化多维数组

多维数组

In [97]: UserDef_dtype = [('x',np.int64,3),('y',np.int32)]

In [98]: arr = np.zeros(4,dtype=UserDef_dtype)

In [99]: arr
Out[99]:
array([([0, 0, 0], 0), ([0, 0, 0], 0), ([0, 0, 0], 0), ([0, 0, 0], 0)],
      dtype=[('x', '<i8', (3,)), ('y', '<i4')])

In [101]: arr[0][0][1]
Out[101]: 0

In [102]: arr[0]['x']
Out[102]: array([0, 0, 0], dtype=int64)

In [103]: arr['x']
Out[103]:
array([[0, 0, 0],
       [0, 0, 0],
       [0, 0, 0],
       [0, 0, 0]], dtype=int64)

嵌套dtype

In [104]: UserDef_dtype = [('x',[('a','f8'),('b','f4')]),('y',np.int32)]

In [105]: data = np.array([((1,2),5),((3,4),6)],dtype=UserDef_dtype)

In [106]: data['x']
Out[106]: array([(1., 2.), (3., 4.)], dtype=[('a', '<f8'), ('b', '<f4')])

In [107]: data['x']['a']
Out[107]: array([1., 3.])

间接排序

argsort() 返回排序后的索引（下标）数组

Out[109]: array([-0.38962844, -2.14157925,  1.7248753 ,  0.84273655, -0.30907011])

In [111]: indexer = value.argsort()

In [112]: indexer
Out[112]: array([1, 0, 4, 3, 2], dtype=int64)

In [113]: value[indexer]
Out[113]: array([-2.14157925, -0.38962844, -0.30907011,  0.84273655,  1.7248753 ])

lexsort() 可传入多个参数进行排序，即对多个键值组进行排序，如：lexsort(arr1,arr2,arr3) 则是先arr3再arr2再arr1进行排序。

In [116]: first_name = np.array(['Bob', 'Jane', 'Steve', 'Bill', 'Barbara'])

In [117]: last_name = np.array(['Jones', 'Arnold', 'Arnold', 'Jones', 'Walters'])

In [118]: indexer = np.lexsort((first_name,last_name))

In [119]: indexer
Out[119]: array([1, 2, 3, 0, 4], dtype=int64)

In [126]: for i in indexer:
     ...:     print(first_name[i] + " " + last_name[i])
     ...:
Jane Arnold
Steve Arnold
Bill Jones
Bob Jones
Barbara Walters

其他排序

mergesot（合并排序）、quicksort（快速排序）和heapsort（堆排序），使用方法更改argsort()中的kind参数argsort(kind=‘mergesort’)

partition(array,num) 返回的数组arr头num个为数组中最小的num个元素
argpartition(array,num) 返回partition()后的索引数组

searchsorted() 二分查找，在有序数组中返回查找到的索引

归类分组

In [136]: data = np.floor(np.random.uniform(0,10000,size=50))

In [137]: group = np.array([0,100,1000,5000,10000])

In [138]: data
Out[138]:
array([4343., 8770., 7665., 9782., 8358., 9207., 4158., 6187., 9370.,
       5887., 7608., 5661., 4172., 5920.,  474., 9362., 4400., 9747.,
        985., 2734., 2967., 1455., 7431., 1031., 2083., 7761., 3859.,
       3220., 3252., 6401.,  745., 9711., 7131.,  928., 3021., 3815.,
       1662., 5869., 4557., 1048., 7624., 6371., 6308., 1509., 4979.,
       6245., 4965., 1116., 8241., 2490.])

In [139]: group.searchsorted(data)
Out[139]:
array([3, 4, 4, 4, 4, 4, 3, 4, 4, 4, 4, 4, 3, 4, 2, 4, 3, 4, 2, 3, 3, 3,
       4, 3, 3, 4, 3, 3, 3, 4, 2, 4, 4, 2, 3, 3, 3, 4, 3, 3, 4, 4, 4, 3,
       3, 4, 3, 3, 4, 3], dtype=int64)
# pandas 可以使用该结果利用groupby()进行针对分组的数据分析
In [140]: import pandas as pd;

In [141]: pd.Series(data).groupby(group.searchsorted(data))
Out[141]: <pandas.core.groupby.generic.SeriesGroupBy object at 0x0000021C2C5F6B38>

In [142]: pd.Series(data).groupby(group.searchsorted(data)).mean()
Out[142]:
2     783.000000
3    3038.000000
4    7609.041667
dtype: float64

使用numba库编写快速函数

可以将python代码转换成机器代码

使用jit()函数

In [4]: import numpy as np,  numba as nb

# x-y的平均数
In [5]: def mean_distance(x, y):
   ...:     nx = len(x)
   ...:     result = 0.0
   ...:     count = 0
   ...:     for i in range(nx):
   ...:         result += x[i] - y[i]
   ...:         count += 1
   ...:     return result / count
   ...:

In [6]: x = np.random.randn(10000000)

In [7]: y = np.random.randn(10000000)

In [8]: %timeit mean_distance(x,y)
7.55 s ± 622 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)

In [9]: %timeit (x-y).mean()
59.1 ms ± 4.6 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)

In [10]: numba_mean_distance = nb.jit(mean_distance)

In [11]: %timeit numba_mean_distance(x,y)
22 ms ± 3.87 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)

使用装饰器

 In [12] @nb.jit
    ...: def mean_distance(x,y):
    ...:     nx = len(x)
    ...:     result = 0.0
    ...:     count =0
    ...:     for i in range(nx):
    ...:         result += x[i] -y[i]
    ...:         count += 1
    ...:     return result/count
In [13]: %timeit mean_distance(x,y)
22.8 ms ± 1.02 ms per loop (mean ± std. dev. of 7 runs, 100 loops each)

自定义numpy.ufunc对象

In [17]: from numba import vectorize

In [18]: @vectorize
    ...: def my_add(x,y):
    ...:     return x+y
    ...:

In [19]: x = np.arange(10)

In [21]: my_add(x,x)
Out[21]: array([ 0,  2,  4,  6,  8, 10, 12, 14, 16, 18], dtype=int64)

高级数组的输出输入

内存映像文件对磁盘读写

# memmap() 需传入参数文件路径、数据类型、形状以及文件模式
In [39]: mmap = np.memmap('mymmap', dtype='float64',mode='w+',shape=(10000,10000))

In [40]: mmap
Out[40]:
memmap([[0., 0., 0., ..., 0., 0., 0.],
        [0., 0., 0., ..., 0., 0., 0.],
        [0., 0., 0., ..., 0., 0., 0.],
        ...,
        [0., 0., 0., ..., 0., 0., 0.],
        [0., 0., 0., ..., 0., 0., 0.],
        [0., 0., 0., ..., 0., 0., 0.]])
# 返回磁盘上的数据的一个视图     
In [43]: section=mmap[:5]
# 将值赋给试图，此时数据被缓存在内存中
In [44]: section[:]=np.random.randn(5,10000)
# flush() 则将其写入磁盘
In [45]: mmap.flush()

In [46]: mmap
Out[46]:
memmap([[-0.69302268, -1.16286652,  1.01013265, ...,  0.73541862,
         -0.52401082,  0.79853806],
        [-0.1564463 ,  0.56630705,  1.01664617, ...,  0.52664146,
          0.61622246,  0.29153165],
        [-0.17849793, -0.66249437, -0.44119979, ...,  0.07165869,
         -0.09034793, -2.51524144],
        ...,
        [ 0.        ,  0.        ,  0.        , ...,  0.        ,
          0.        ,  0.        ],
        [ 0.        ,  0.        ,  0.        , ...,  0.        ,
          0.        ,  0.        ],
        [ 0.        ,  0.        ,  0.        , ...,  0.        ,
          0.        ,  0.        ]])

In [47]: del mmap
# 即使删除依旧存在于磁盘中
In [53]: mmap = np.memmap('mymmap', dtype='float64',mode='r',shape=(10000,10000))

In [54]: mmap
Out[54]:
memmap([[-0.69302268, -1.16286652,  1.01013265, ...,  0.73541862,
         -0.52401082,  0.79853806],
        [-0.1564463 ,  0.56630705,  1.01664617, ...,  0.52664146,
          0.61622246,  0.29153165],
        [-0.17849793, -0.66249437, -0.44119979, ...,  0.07165869,
         -0.09034793, -2.51524144],
        ...,
        [ 0.        ,  0.        ,  0.        , ...,  0.        ,
          0.        ,  0.        ],
        [ 0.        ,  0.        ,  0.        , ...,  0.        ,
          0.        ,  0.        ],
        [ 0.        ,  0.        ,  0.        , ...,  0.        ,
          0.        ,  0.        ]])

提高性能

访问内存中连续存储方式的数组一般是最快的，数组连续类型分为Fortran（列优先）和C（行优先），可通过flags属性查看信息

In [55]: arr_c = np.ones((1000,1000),order='C')

In [56]: arr_f = np.ones((1000,1000),order='F')

In [57]: arr_c.flags
Out[57]:
  C_CONTIGUOUS : True
  F_CONTIGUOUS : False
  OWNDATA : True
  WRITEABLE : True
  ALIGNED : True
  WRITEBACKIFCOPY : False
  UPDATEIFCOPY : False
# copy()可更改连续类型
In [58]: arr_c.copy('F').flags
Out[58]:
  C_CONTIGUOUS : False
  F_CONTIGUOUS : True
  OWNDATA : True
  WRITEABLE : True
  ALIGNED : True
  WRITEBACKIFCOPY : False
  UPDATEIFCOPY : False
#行优先只在行切片下连续，列优先只在列切片下连续
In [60]: arr_c[:50].flags
Out[60]:
  C_CONTIGUOUS : True
  F_CONTIGUOUS : False
  OWNDATA : False
  WRITEABLE : True
  ALIGNED : True
  WRITEBACKIFCOPY : False
  UPDATEIFCOPY : False
In [62]: arr_c[:,:5].flags
Out[62]:
  C_CONTIGUOUS : False
  F_CONTIGUOUS : False
  OWNDATA : False
  WRITEABLE : True
  ALIGNED : True
  WRITEBACKIFCOPY : False
  UPDATEIFCOPY : False

In [63]: arr_c.copy('F')[:,:5].flags
Out[63]:
  C_CONTIGUOUS : False
  F_CONTIGUOUS : True
  OWNDATA : False
  WRITEABLE : True
  ALIGNED : True
  WRITEBACKIFCOPY : False
  UPDATEIFCOPY : False