原始语料:546M
1.readlines()
with codecs.open(combine_bigram_remove_freq_1_filename, encoding='utf-8') as f:
temp_list = [item for item in f.readlines()]
CPU usage:34.6% ==> 61.7%
占用内存:2.168G
耗时:5.66499996185s
2.
with codecs.open(combine_bigram_remove_freq_1_filename, encoding='utf-8') as f:
temp_list = [item for item in f]
CPU usage:35.3% ==> 62.0%
占用内存:2.136G
耗时:48.2940001488s
3.read()
with codecs.open(combine_bigram_remove_freq_1_filename, encoding='utf-8') as f:
temp_str =f.read()
CPU usage:30.4% ==> 36.3%
占用内存:0.48G
耗时:48.2940001488s
4.readline()
f = codecs.open(filename, encoding='utf-8')
start_time = time.time()
while 1:
line = f.readline()
if not line:
end_time = time.time()
print end_time-start_time
break
耗时:50.6809999943
readline,readlines,read占用内存分析
最新推荐文章于 2023-06-02 15:51:41 发布