python开发的程序内存越来越大_为什么我的python进程占用了这么多内存？

I'm working on a project that involves using python to read, process and write files that are sometimes as large as a few hundred megabytes. The program fails occasionally when I try to process some particularly large files. It does not say 'memory error', but I suspect that is the problem (in fact it gives no reason at all for failing').

I've been testing the code on smaller files and watching 'top' to see what memory usage is like, and it typically reaches 60%. top says that I have 4050352k total memory, so 3.8Gb.

Meanwhile I'm trying to track memory usage within python itself (see my question from yesterday) with the following little bit of code:

mem = 0

for variable in dir():

variable_ = vars()[variable]

try:

if str(type(variable_))[7:12] == 'numpy':

numpy_ = True

else:

numpy_ = False

except:

numpy_ = False

if numpy_:

mem_ = variable_.nbytes

else:

mem_ = sys.getsizeof(variable)

mem += mem_

print variable+ type: '+str(type(variable_))+' size: '+str(mem_)

print 'Tot