golang 内存优化_Go系统的内存优化

golang 内存优化

介绍(Introduction)

Despite its growing popularity as a systems language, Go programs are susceptible to severe performance regressions at large scale. In systems with high memory usage, garbage collection (GC) can cause performance regressions by cannibalizing resources from the main program. Heavy GC cycles can add hundreds of milliseconds of latency to a request, resulting in degraded user experience.

尽管Go程序作为一种系统语言越来越受欢迎，但它很容易受到大规模严重性能下降的影响。 在内存使用率很高的系统中，垃圾回收(GC)可能会吞噬主程序中的资源，从而导致性能下降。 繁重的GC周期会为请求增加数百毫秒的延迟，从而降低用户体验。

The goal of this post is to help you understand:

这篇文章的目的是帮助您了解：

• How Go GC works at a high level? Why would it impact your system’s performance?
  Go GC如何在高水平上工作？ 为什么会影响系统性能？
• What causes GC pressure (more resources spent on GC)?
  是什么导致GC压力(在GC上花费了更多资源)？
• How to determine if GC pressure is the cause of your performance problems?
  如何确定GC压力是否是性能问题的原因？
• How to measure and profile your program’s heap usage?
  如何测量和分析程序的堆使用情况？
• How to identify which part of the code is the culprit?
  如何识别罪魁祸首？
• What are some steps you can take to lower heap usage and GC pressure?
  您可以采取哪些步骤来降低堆使用率和GC压力？

Go中的垃圾收集如何工作？ (How does garbage collection work in Go?)

Go does not require manual memory management i.e., you do not need to manually allocate memory and clear it once you’re done using it. Such functionalities are abstracted away from the user, to minimize errors that could lead to memory leaks. Go has a built-in garbage collector, which reclaims memory once it is no longer in use by the program.

Go不需要手动内存管理，即，您不需要手动分配内存并在使用完后清除它。 此类功能是从用户那里抽象出来的，以最大程度地减少可能导致内存泄漏的错误。 Go有一个内置的垃圾收集器，一旦程序不再使用它，它将回收内存。

Go uses a concurrent, tricolor mark-and-sweep algorithm for garbage collection. This algorithm allows GC to run concurrently with the mutator (main program), without a stop-the-world pause — when all goroutines are paused while GC runs. Go GC also aims to utilize no more than 25% of available CPU. Both these features are highly beneficial since it leaves plenty of resources for the mutator to continue running without significant impact to performance (throughout, latency, etc.)

Go使用并发的三色标记和清除算法进行垃圾收集。 该算法允许GC与mutator(主程序)并发运行，而没有世界停顿的停顿-当GC运行时所有goroutine都停顿了。 Go GC还旨在利用不超过25％的可用CPU。 这两个功能都是非常有益的，因为它为mutator留下了足够的资源来继续运行，而不会对性能造成重大影响(贯穿，延迟等)。

The algorithm works by dividing objects on the heap into three sets (colors) during the mark phase:

该算法通过在标记阶段将堆上的对象分为三组(颜色)来工作：

• White = collectible since it’s not in use in memory

  白色=可收藏，因为未在内存中使用

• Black = not collectible since it’s definitely in use in memory

  黑色=无法收集，因为它肯定在内存中使用

• Grey = might be collectible, not determined yet

  灰色=可能是可收藏的，尚未确定

As the number of objects on the heap increases, it increases the time spent in the marking phase. Later, this collectible memory is reclaimed during the sweep phase. Sweeping occurs when a goroutine attempts to allocate new objects in memory.

随着堆上对象数量的增加，这会增加标记阶段所花费的时间。 以后，在扫描阶段将回收此可收集的内存。 当goroutine尝试在内存中分配新对象时，就会发生清除。

为什么会影响系统性能？ (Why would it impact system performance?)

If the rate of memory allocation in the mutator (main program) is very high, then Go GC will start to “steal” more goroutines from the mutator to assist with the marking phase. This has two effects — firstly, it speeds up the GC process by providing more resources, and secondly, it takes away resources from the mutator, which slows down the rate of memory allocation. This is important to ensure that the rate of memory allocation does exceed the rate of memory cleanup, which could cause the heap to grow out of control, potentially resulting in out-of-memory crashes.

如果更改器(主程序)中的内存分配率很高，则Go GC将开始从更改器中“窃取”更多goroutine，以协助标记阶段。 这有两个作用-首先，它通过提供更多资源来加速GC进程，其次，它从mutator中夺走了资源，这减慢了内存分配的速度。 这对于确保内存分配速率确实超过内存清除速率非常重要，因为这可能导致堆增长失控，并可能导致内存不足崩溃。

When the garbage collector starts to steal resources from the main program, it can start to have a significant impact on the performance of the main program, since CPU resources are limited. This typically manifests in the form of “tail latency”, i.e, the higher percentiles of latency (p99, p999, etc.) compared to the average latency, and can have an adverse effect on user experience. A user will remember the worst or slowest experiences more than the average request, and this can cause user dissatisfaction.

当垃圾收集器开始从主程序中窃取资源时，由于CPU资源有限，它可能开始对主程序的性能产生重大影响。 这通常以“尾部等待时间”的形式表现出来&#