目标:读取一个大的csv,8GB,16个列。对每一行验证name列是否在某个list中,获得对应id列的值。
坑:csv reader 巨慢
使用
with open(...) as f:
for line in f:
process(line) # <do something with line>
读10000行需要20s
使用
with open("test.csv",newline = '') as f:
reader = csv.reader(f)
for row in reader:
读10000行需要2min