使用^{}-# Perform lex sort and get sorted data
sorted_idx = np.lexsort(data.T)
sorted_data = data[sorted_idx,:]
# Get unique row mask
row_mask = np.append([True],np.any(np.diff(sorted_data,axis=0),1))
# Get unique rows
out = sorted_data[row_mask]
样本运行-In [199]: data
Out[199]:
array([[1, 8, 3, 3, 4],
[1, 8, 9, 9, 4],
[1, 8, 3, 3, 4],
[1, 8, 3, 3, 4],
[1, 8, 0, 3, 4],
[1, 8, 9, 9, 4]])
In [200]: sorted_idx = np.lexsort(data.T)
...: sorted_data = data[sorted_idx,:]
...: row_mask = np.append([True],np.any(np.diff(sorted_data,axis=0),1))
...: out = sorted_data[row_mask]
...:
In [201]: out
Out[201]:
array([[1, 8, 0, 3, 4],
[1, 8, 3, 3, 4],
[1, 8, 9, 9, 4]])
运行时测试-
本节乘以迄今为止提出的解决方案中提出的所有方法。In [34]: data = np.random.randint(0,10,(10000,10))
In [35]: def tuple_based(data):
...: new_array = [tuple(row) for row in data]
...: return np.unique(new_array)
...:
...: def lexsort_based(data):
...: sorted_data = data[np.lexsort(data.T),:]
...: row_mask = np.append([True],np.any(np.diff(sorted_data,axis=0),1))
...: return sorted_data[row_mask]
...:
...: def unique_based(a):
...: a = np.ascontiguousarray(a)
...: unique_a = np.unique(a.view([('', a.dtype)]*a.shape[1]))
...: return unique_a.view(a.dtype).reshape((unique_a.shape[0], a.shape[1]))
...:
In [36]: %timeit tuple_based(data)
10 loops, best of 3: 63.1 ms per loop
In [37]: %timeit lexsort_based(data)
100 loops, best of 3: 8.92 ms per loop
In [38]: %timeit unique_based(data)
10 loops, best of 3: 29.1 ms per loop
1699
被折叠的 条评论
为什么被折叠?
到【灌水乐园】发言
