多线程下内存分配方式的比较(TCMalloc vs mt_alloc)

当软件性能优化到一定程度之后,用vturn查看hotspots,将会发现malloc/delete会花费很高比例的时间,此时如果是多线程程序,频繁的lock将会是一个瓶颈,这里有一篇oracle的文章,很好的介绍了这样的情况http://www.oracle.com/technetwork/articles/servers-storage-dev/mem-alloc-1557798.html

在linux平台,此时我们主要考虑到两种优化方案,各自都有很细节的文章可以参考
1. google的MTMalloc (http://goog-perftools.sourceforge.net/doc/tcmalloc.html)

2. gnu的__mt_alloc (http://gcc.gnu.org/onlinedocs/libstdc++/manual/bk01pt12ch32.html)


下面我做了一个简单的测试程序
// test app
void* vectormemorycost(void*){
    cout << "malloc test"<< endl;
    int blocksize = 512;
    for(int j=0;j<20; ++j){
        int sum=25000;
        char* p1[sum];
        for(int i=0; i<sum;++i){
            p1[i]=(char*)malloc(blocksize);
        }

        for(int i=0; i<sum;++i){
            free(p1[i]);
        }
    }

    return 0;
}

void* mtallocateMemorycost(void*){
    cout << "mt alloc test"<< endl;
    typedef pod value_type;
    typedef __gnu_cxx::__mt_alloc<value_type> allocator_type;
    typedef __gnu_cxx::__pool_base::_Tune tune_type;

///_M_align(__align)
///_M_max_bytes(__maxb)
///_M_min_bin(__minbin),
///_M_chunk_size(__chunk)
///_M_max_threads(__maxthreads),
///_M_freelist_headroom(__headroom)
///_M_force_new(__force)

    tune_type t_our(16, 510, 32, 5120, 20, 10, false);
    allocator_type a;);
    a._M_set_options(t_our);
    int blocksize = 512;
for(int j=0;j<20; ++j){
    int sum=25000;
    allocator_type::pointer p1[sum];
    for(int i=0; i<sum;++i){
    p1[i]=a.allocate(blocksize);
    }

    for(int i=0; i<sum;++i){
    a.deallocate(p1[i], blocksize);
    }
}
    return 0;
}




typedef void*(*pFoo)(void*);

void mtallocatePrivate(pFoo func){
    int i;
    int cores = 4;
    pthread_t threads[cores];
    /* Create threads to do the work. */
    for (i = 0; i < cores; ++i)
        // pthread_create (&(threads[i]), NULL, vectormemorycost, NULL);
        pthread_create (&(threads[i]), NULL, func, NULL);

    /* Wait for all threads to finish. */
    for (i = 0; i < cores; ++i)
        pthread_join (threads[i], NULL);
}

void mtallocate(){
    for(int i=0;i<5;++i){
        mtallocatePrivate(mtallocateMemorycost);
        mtallocatePrivate(vectormemorycost);
    }
}




改变的tune_type参数,发现gnu的最优的情况比malloc快一两倍,最糟糕的情况是反而比malloc慢十倍,很依赖具体的实现情况,另外tcmalloc使用很简单,我们只要改一下link的库,测试的结果是比不用tcmalloc有七八倍的性能提高,从结构上来讲gnu的在更高层次的代码上优化,想通过memory pool和thread id的比较来减少申请内存的次数和不同线程之间的竞争,它最终调用的还是malloc尤其是当申请的内存大于_M_max_bytes的时候就完全是调用malloc了,而TCMalloc是替换掉了系统的malloc,是更加底层的优化。

总结一下,gnu的使用麻烦,接口繁琐,还可能有负面效应,google的比gnu的速度快,使用也方便很多。
已标记关键词 清除标记
相关推荐
<p> <strong><span style="font-size:16px;color:#003399;">会用Python分析金融数据 or 金融行业会用Python</span></strong> </p> <p> <strong><span style="font-size:16px;color:#003399;">职场竞争力更高</span></strong> </p> <p> <br /> </p> <p> <img src="https://img-bss.csdnimg.cn/202012231042221925.png" alt="" /> </p> <p> <br /> </p> <p> <br /> </p> <p> <strong><span style="font-size:16px;color:#003399;">Python金融数据分析入门到实战</span></strong> </p> <p> <strong><span style="font-size:16px;color:#003399;">Get√金融行业数据分析必备技能</span></strong> </p> <p> <img src="https://img-bss.csdnimg.cn/202012231042438069.png" alt="" /> </p> <p> <br /> </p> <p> <br /> </p> <p> <strong><span style="font-size:16px;color:#003399;">以股票量化交易为应用场景</span></strong> </p> <p> <strong><span style="font-size:16px;color:#003399;">完成技术指标实现的全过程</span></strong> </p> <p> <br /> </p> <p> <span style="font-size:14px;">课程选取股票量化交易为应用场景,由股票数据的获取、技术指标的实现,逐步进阶到策略的设计</span><span style="font-size:14px;">和回测,由浅入深、由技术到思维地为同学们讲解Python金融数据分析在股票量化交易中的应用</span><span style="font-size:14px;">。</span> </p> <p> <br /> </p> <p> <span style="font-size:14px;"><br /> </span> </p> <p> <img src="https://img-bss.csdnimg.cn/202012231043183686.png" alt="" /> </p> <p> <br /> </p> <p> <br /> </p> <p> <strong><span style="font-size:16px;color:#003399;">以Python为编程语言</span></strong> </p> <p> <strong><span style="font-size:16px;color:#003399;">解锁3大主流数据分析工具</span></strong> </p> <p> <br /> </p> <p> <span style="font-size:14px;">Python做金融具有先天优势,课程提取了Python数据分析工具NumPy、Pandas及可视化工具</span><span style="font-size:14px;">Matplotlib的关键点详细讲解,帮助同学掌握数据分析的关键技能。</span> </p> <p> <img src="https://img-bss.csdnimg.cn/202012231043472858.png" alt="" /> </p> <p> <strong><span style="font-size:16px;color:#003399;"><br /> </span></strong> </p> <p> <strong><span style="font-size:16px;color:#003399;">2大购课福利</span></strong> </p> <p> <strong><span style="font-size:16px;color:#003399;"><br /> </span></strong> </p> <p> <img src="https://img-bss.csdnimg.cn/202012300628195864.png" alt="" /> </p>
<p class="MsoNormal"> <span style="font-family:宋体;">(</span><span>1</span><span style="font-family:宋体;">)</span><span>HTML5 WebSocket</span><span style="font-family:宋体;">、异常重连、心跳检测;</span> </p> <p class="MsoNormal"> <span style="font-family:宋体;">(</span><span>2</span><span style="font-family:宋体;">)</span><span>SockJS</span><span style="font-family:宋体;">、</span><span>Stomp</span><span style="font-family:宋体;">、</span><span>RabbitMQ Stomp</span><span style="font-family:宋体;">消息代理;</span> </p> <p class="MsoNormal"> <span style="font-family:宋体;">(</span><span>3</span><span style="font-family:宋体;">)分别用</span><span>Nginx</span><span style="font-family:宋体;">和</span><span>Spring Cloud Gateway</span><span style="font-family:宋体;">实现多实例负载均衡;</span> </p> <p class="MsoNormal"> <span style="font-family:宋体;">(</span><span>4</span><span style="font-family:宋体;">)可靠消息推送(</span><span>Stomp</span><span style="font-family:宋体;">持久化队列、客户端</span><span>ACK</span><span style="font-family:宋体;">确认机制);</span> </p> <p class="MsoNormal"> <span style="font-family:宋体;">(</span><span>5</span><span style="font-family:宋体;">)</span><span>Java</span><span style="font-family:宋体;">原生、</span><span>Stomp</span><span style="font-family:宋体;">客户端实现(非浏览器客户端);</span> </p> <p class="MsoNormal"> <span style="font-family:宋体;">(</span><span>6</span><span style="font-family:宋体;">)</span><span>Websocket</span><span style="font-family:宋体;">拦截器结合</span><span> Spring security</span><span style="font-family:宋体;">、</span><span>jwt token</span><span style="font-family:宋体;">认证授权。</span> </p> <p class="MsoNormal"> <span style="font-family:宋体;">(</span><span>7</span><span style="font-family:宋体;">)</span><span>VUE+elementUI</span><span style="font-family:宋体;">前后分离实现。</span> </p>
©️2020 CSDN 皮肤主题: 大白 设计师:CSDN官方博客 返回首页