http://my.huhoo.net/archives/2010/05/malloptmallocnew.html
同事在项目中使用new/delete的时候发现一个奇怪的现象:
int32_t i;
std::queue<char *> qTest;
for (i = 0; i < 100000; i ++) {
char *p = new char[100];
qTest.push(p);
char *p1 = qTest.front();
delete[] p1;
qTest.pop();
}
当在一个循环内,如果申请空间,在循环内并释放掉,内存不会引起增长,即使重复上面的单元也不会增长内存,但是当:
其实,这个与malloc的实现有关,一般来说,系统都会有默认的malloc行为,一般来说(针对FreeBSD和Linux系列),对于大于1M的数 据,malloc行为会直接调用系统的接口,直接向操作系统申请一块比数据块更大内存,然后划分成若干过chunk单元(这里会有一些算法设计),而这些 chunk分配给数据后,肯定还会剩下很多的trunk单元,留给以后的分配用,但这里耗费系统资源,代价较大;而对于小于等于1M的数据,则 malloc行为会利用那些trunk单元,占用系统资源少,速度快。同理free的时候,数据占用的内存被释放,如果大于1M,则系统会回收掉内存,节 省资源,而小于等于1M的数据,则内存释放时,不会还给操作系统,以空trunk形式存在,并由当前进程空间维护着。说到底,这种策略就是一个内存池的概 念。
内存池灵活度更加方便,一般来说(针对FreeBSD和Linux系列),每个trunk最小容纳的字节数是16bytes,而自行设计的内存池,可以更 加优化,节省大量产生的内存碎片,像上面的代码就是因为内存虽然释放,但是这些内存都没有还给操作系统,导致内存碎片越来越多,最好的设计方式就是使用一 个内存池,针对经常使用的分配大小,可以多分配这样的trunk,而其他大小的trunk可以少产生,从而达到优化目的。
另外,也可以使用mallopt来直接调整malloc的行为:
to be set, and VALUE the new value to be set. Possible choices
for PARAM, as defined in `malloc.h', are:
`M_TRIM_THRESHOLD'
This is the minimum size (in bytes) of the top-most,
releasable chunk that will cause `sbrk' to be called with a
negative argument in order to return memory to the system.
`M_TOP_PAD'
This parameter determines the amount of extra memory to
obtain from the system when a call to `sbrk' is required. It
also specifies the number of bytes to retain when shrinking
the heap by calling `sbrk' with a negative argument. This
provides the necessary hysteresis in heap size such that
excessive amounts of system calls can be avoided.
`M_MMAP_THRESHOLD'
All chunks larger than this value are allocated outside the
normal heap, using the `mmap' system call. This way it is
guaranteed that the memory for these chunks can be returned
to the system on `free'. Note that requests smaller than
this threshold might still be allocated via `mmap'.
`M_MMAP_MAX'
The maximum number of chunks to allocate with `mmap'.
Setting this to zero disables all use of `mmap'.
值得注意的是mallopt是malloc底层的函数,需要使用info mallopt来查看相关帮助信息。
int32_t i;而发现通过delete[]后,内存并没有减少,即使重复上面的单元,内存只会增加,不会减少,用valgrind的memcheck工具查看,也无内存泄漏。
std::queue<char *> qTest;
for (i = 0; i < 100000; i ++) {
char *p = new char[100];
qTest.push(p);
//char *p1 = qTest.front();
//delete[] p1;
//qTest.pop();
}
while( !qTest.empty() ) {
char *p1 = qTest.front();
delete[] p1;
qTest.pop();
}
其实,这个与malloc的实现有关,一般来说,系统都会有默认的malloc行为,一般来说(针对FreeBSD和Linux系列),对于大于1M的数 据,malloc行为会直接调用系统的接口,直接向操作系统申请一块比数据块更大内存,然后划分成若干过chunk单元(这里会有一些算法设计),而这些 chunk分配给数据后,肯定还会剩下很多的trunk单元,留给以后的分配用,但这里耗费系统资源,代价较大;而对于小于等于1M的数据,则 malloc行为会利用那些trunk单元,占用系统资源少,速度快。同理free的时候,数据占用的内存被释放,如果大于1M,则系统会回收掉内存,节 省资源,而小于等于1M的数据,则内存释放时,不会还给操作系统,以空trunk形式存在,并由当前进程空间维护着。说到底,这种策略就是一个内存池的概 念。
内存池灵活度更加方便,一般来说(针对FreeBSD和Linux系列),每个trunk最小容纳的字节数是16bytes,而自行设计的内存池,可以更 加优化,节省大量产生的内存碎片,像上面的代码就是因为内存虽然释放,但是这些内存都没有还给操作系统,导致内存碎片越来越多,最好的设计方式就是使用一 个内存池,针对经常使用的分配大小,可以多分配这样的trunk,而其他大小的trunk可以少产生,从而达到优化目的。
另外,也可以使用mallopt来直接调整malloc的行为:
int mallopt (int PARAM, int VALUE)When calling `mallopt', the PARAM argument specifies the parameter
to be set, and VALUE the new value to be set. Possible choices
for PARAM, as defined in `malloc.h', are:
`M_TRIM_THRESHOLD'
This is the minimum size (in bytes) of the top-most,
releasable chunk that will cause `sbrk' to be called with a
negative argument in order to return memory to the system.
`M_TOP_PAD'
This parameter determines the amount of extra memory to
obtain from the system when a call to `sbrk' is required. It
also specifies the number of bytes to retain when shrinking
the heap by calling `sbrk' with a negative argument. This
provides the necessary hysteresis in heap size such that
excessive amounts of system calls can be avoided.
`M_MMAP_THRESHOLD'
All chunks larger than this value are allocated outside the
normal heap, using the `mmap' system call. This way it is
guaranteed that the memory for these chunks can be returned
to the system on `free'. Note that requests smaller than
this threshold might still be allocated via `mmap'.
`M_MMAP_MAX'
The maximum number of chunks to allocate with `mmap'.
Setting this to zero disables all use of `mmap'.
值得注意的是mallopt是malloc底层的函数,需要使用info mallopt来查看相关帮助信息。