python numpy.sqrt_使用numpy在python中进行向量化空间距离

有eucl_dist软件包(免责声明：我是它的作者),它基本上包含两种方法来解决计算平方欧几里德距离的问题,该方法比SciPy的cdist更有效,尤其是对于大型数组(具有适当的列数).

我们将使用其source code中的一些代码来适应此处的问题,从而为我们提供两种方法.

方法1

def pdist_squareformed_numpy(a):

a_sumrows = np.einsum('ij,ij->i',a,a)

dist = a_sumrows[:,None] + a_sumrows -2*np.dot(a,a.T)

np.fill_diagonal(dist,0)

return dist

方法#2

另一种方法是创建输入数组的“扩展”版本,在github源代码链接中再次进行了详细讨论,以获取第二种方法,这种方法适用于较少的列,例如此处所示-

def ext_arrs(A,B, precision="float64"):

nA,dim = A.shape

A_ext = np.ones((nA,dim*3),dtype=precision)

A_ext[:,dim:2*dim] = A

A_ext[:,2*dim:] = A**2

nB = B.shape[0]

B_ext = np.ones((dim*3,nB),dtype=precision)

B_ext[:dim] = (B**2).T

B_ext[dim:2*dim] = -2.0*B.T

return A_ext, B_ext

def pdist_squareformed_numpy_v2(a):

A_ext, B_ext = ext_arrs(a,a)

dist = A_ext.dot(B_ext)

np.fill_diagonal(dist,0)

return dist

请注意,这些给出了平方欧几里德距离.因此,对于实际距离,如果需要最终输出,我们想使用np.sqrt().

样品运行-

In [380]: np.random.seed(0)

...: a = np.random.rand(5,3)

In [381]: from scipy.spatial.distance import cdist

In [382]: cdist(a,a)

Out[382]:

array([[0. , 0.29, 0.42, 0.2 , 0.57],

[0.29, 0. , 0.58, 0.42, 0.76],

[0.42, 0.58, 0. , 0.45, 0.9 ],

[0.2 , 0.42, 0.45, 0. , 0.51],

[0.57, 0.76, 0.9 , 0.51, 0. ]])

In [383]: np.sqrt(pdist_squareformed_numpy(a))

Out[383]:

array([[0. , 0.29, 0.42, 0.2 , 0.57],

[0.29, 0. , 0.58, 0.42, 0.76],

[0.42, 0.58, 0. , 0.45, 0.9 ],

[0.2 , 0.42, 0.45, 0. , 0.51],

[0.57, 0.76, 0.9 , 0.51, 0. ]])

In [384]: np.sqrt(pdist_squareformed_numpy_v2(a))

Out[384]:

array([[0. , 0.29, 0.42, 0.2 , 0.57],

[0.29, 0. , 0.58, 0.42, 0.76],

[0.42, 0.58, 0. , 0.45, 0.9 ],

[0.2 , 0.42, 0.45, 0. , 0.51],

[0.57, 0.76, 0.9 , 0.51, 0. ]])

1万点的计时-

In [385]: a = np.random.rand(10000,3)

In [386]: %timeit cdist(a,a)

1 loop, best of 3: 309 ms per loop

# Approach #1

In [388]: %timeit pdist_squareformed_numpy(a) # squared eucl distances

1 loop, best of 3: 668 ms per loop

In [389]: %timeit np.sqrt(pdist_squareformed_numpy(a)) # actual eucl distances

1 loop, best of 3: 812 ms per loop

# Approach #2

In [390]: %timeit pdist_squareformed_numpy_v2(a) # squared eucl distances

1 loop, best of 3: 237 ms per loop

In [391]: %timeit np.sqrt(pdist_squareformed_numpy_v2(a)) # actual eucl distances

1 loop, best of 3: 395 ms per loop

第二种方法似乎在性能上接近cdist！