Tcmalloc SetMemoryReleaseRate(double) 和 ReleaseFreeMemory()浅析

问题

Tcmalloc 由于使用了精心设计的 cache，进而大大提高了malloc 和 free 的效率，但由之而来的是 cache 大小的难以把控，容易出现 cache 占用过大，进而 OOM 的问题，tcmalloc 提供了两个方案解决这个问题

• 展现 tcmalloc 维护的内存状态解决

  //virtual void GetStats(char* buffer, int buffer_length);
  MallocExtension::instance()->GetStats(buf, length);
  

解决

1.ReleaseFreeMemory

作用

• 一次性，全部释放 tcmalloc 的 cache 内存

函数实现

// Same as ReleaseToSystem() but release as much memory as possible.
virtual void ReleaseFreeMemory();
// 实现
void MallocExtension::ReleaseFreeMemory() {
  ReleaseToSystem(static_cast<size_t>(-1));   // SIZE_T_MAX
}
//加锁释放，尽可能多的cache
virtual void ReleaseToSystem(size_t num_bytes) {
  SpinLockHolder h(Static::pageheap_lock());
  if (num_bytes <= extra_bytes_released_) {
    // We released too much on a prior call, so don't release any
    // more this time.
    extra_bytes_released_ = extra_bytes_released_ - num_bytes;
    return;
  }
  num_bytes = num_bytes - extra_bytes_released_;
  // num_bytes might be less than one page.  If we pass zero to
  // ReleaseAtLeastNPages, it won't do anything, so we release a whole
  // page now and let extra_bytes_released_ smooth it out over time.
  Length num_pages = max<Length>(num_bytes >> kPageShift, 1);
  size_t bytes_released = Static::pageheap()->ReleaseAtLeastNPages(
    num_pages) << kPageShift;
  if (bytes_released > num_bytes) {
    extra_bytes_released_ = bytes_released - num_bytes;
  } else {
    // The PageHeap wasn't able to release num_bytes.  Don't try to
    // compensate with a big release next time.  Specifically,
    // ReleaseFreeMemory() calls ReleaseToSystem(LONG_MAX).
    extra_bytes_released_ = 0;
  }
}

劣势

• 对 page heap 加锁，cache 较大时，占用锁时间较长，可能阻塞较长时间
• 对 tcmalloc 源码的阅读，会在之后的博客中分享

2.SetMemoryReleaseRate

作用

• 控制释放 cache 给 OS的速度

函数实现

// Sets the rate at which we release unused memory to the system.
// Zero means we never release memory back to the system.  Increase
// this flag to return memory faster; decrease it to return memory
// slower.  Reasonable rates are in the range [0,10].  (Currently
// only implemented in tcmalloc).
virtual void SetMemoryReleaseRate(double rate);
// 实现
virtual void SetMemoryReleaseRate(double rate) {
  FLAGS_tcmalloc_release_rate = rate;
}

rate的含义

• 官方建议 [0 , 10]
• 0 代表不归还，数字越大归还越快，但最大是多少呢？以及是否是线性的呢？

// Incrementally release some memory to the system.
// IncrementalScavenge(n) is called whenever n pages are freed.
// 大意是，释放 n 个 pages 时， 通过 IncrementalScavenge 归还一定大小 cache 给 OS
void IncrementalScavenge(Length n);
// 实现
void PageHeap::IncrementalScavenge(Length n) {
  // Fast path; not yet time to release memory
  // 需要等待的 pages 数，每释放 scavenge_counter_ 个 pages 归还一次给OS
  scavenge_counter_ -= n;
  if (scavenge_counter_ >= 0) return;  // Not yet time to scavenge
  const double rate = FLAGS_tcmalloc_release_rate;
  if (rate <= 1e-6) {
    // 0 不释放
    scavenge_counter_ = kDefaultReleaseDelay;
    return;
  }
  ++stats_.scavenge_count;
  Length released_pages = ReleaseAtLeastNPages(1);
  if (released_pages == 0) {
    scavenge_counter_ = kDefaultReleaseDelay;
  } else {
    // Compute how long to wait until we return memory.
    // FLAGS_tcmalloc_release_rate==1 means wait for 1000 pages
    // after releasing one page.
    // 反比关系，1000 / rate 为要等待的 pages，所以 rate 可以理解为 [1, 1000]
    // 1 时等待1000个pages， 1000时等待1个page
    const double mult = 1000.0 / rate;
    double wait = mult * static_cast<double>(released_pages);
    if (wait > kMaxReleaseDelay) {
      // Avoid overflow and bound to reasonable range.
      wait = kMaxReleaseDelay;
    }
    scavenge_counter_ = static_cast<int64_t>(wait);
  }
}

• 由此可见，建议在 [0, 1000] 配置
• 1 时归还率为 1 / 1000， 1000时归还率为 50%
• 数字越大越好，非线性反比关系