python 写内存测试程序,如何实现内存密集型python脚本进行测试

I've applied a cgroups rule to a specific user, and I'd like to test whether memory of the programs running from the above user has been limited as expected. I tried with the following script:

import string

import random

if __name__ == '__main__':

d = {}

i = 0;

for i in range(0, 100000000):

val = ''.join(random.choice(string.ascii_uppercase + string.digits) for _ in range(200)) # generate ramdom string of size 200

d[i] = val

if i % 10000 == 0:

print i

When I monitored the process via ps command, it turned out to be that the %MEM is increased to 4.8 and never changed when both cgroups service is on and off:

$ ps aux | grep mem_intensive.py

USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND

jason 11531 88.5 4.8 3312972 3191236 pts/0 R+ 22:16 0:07 python mem_intensive.py

In this scenario, total memory is 62GB, thus 4.8% of it about 3GB. I set the limit to be 4GB without any other processes running on this user.

So could anyone give me some idea about this problematic python script? Thanks in advance.

解决方案

I've played a bit with your script, and it keeps growing, albeit slowly. The bottleneck is using random.choice. If you want to fill memory fast, generating randomness works against you. So just using fixed strings does exhaust the memory rather quickly. If using the following, while wanting to watch how it grows you'd probably throw a time.sleep() after your print:

if __name__ == '__main__':

d = {}

i = 0;

for i in range(0, 100000000):

d[i] = 'A'*1024

if i % 10000 == 0:

print(i)

filling memory faster:

just an one-liner:

['A'*1024 for _ in xrange(0, 1024*1024*1024)]