7
With a small 10 line test file I tried 2 approaches - parse the whole thing and select the last N lines, versus load all lines, but only parse the last N:
使用一個小的10行測試文件,我嘗試了2種方法 - 解析整個事物並選擇最后N行,而不是加載所有行,但只解析最后N:
In [1025]: timeit np.genfromtxt('stack38704949.txt',delimiter=',')[-5:]
1000 loops, best of 3: 741 µs per loop
In [1026]: %%timeit
...: with open('stack38704949.txt','rb') as f:
...: lines = f.readlines()
...: np.genfromtxt(lines[-5:],delimiter=',')
1000 loops, best of 3: 378 µs per loop
This was tagged as a duplicate of Efficiently Read last 'n' rows of CSV into DataFrame. The accepted answer there used
這被標記為有效地將最后'n'行CSV讀入DataFrame的副本。那里接受的答案
from collections import deque
and collected the last N lines in that structure. It also u