Wireshark----wmem 内存池、内存管理的学习--README.wmem 翻译

1. 什么是内存池？

当创建大量消耗小内存的对象时，频繁调用new/malloc会导致大量的内存碎片，致使效率降低。内存池的概念就是预先在内存中申请一定数量的，大小相等 的内存块留作备用，当有新的内存需求时，就先从内存池中分配内存给这个需求，不够了之后再申请新的内存。这样做最显著的优势就是能够减少内存碎片，提升效率。
内存池的实现方式有很多，性能和适用范围也不一样。
————————————————
原文链接：https://blog.csdn.net/None_yel/article/details/89676966

2. 查看wireshark的官方资料

原文：doc/README.wmem · master · Wireshark Foundation / wireshark · GitLab

---

翻译：

一、简介

'wmem' 内存管理器是 Wireshark 的内存管理框架，取代在 Wireshark 2.0 中删除的旧 'emem' 框架。

为了使内存管理更容易并减少内存泄漏，Wireshark 提供了自己的内存管理 API。这个 API 是在 wsutil/wmem/ 内部实现并提供内存池和函数即使面对异常（许多剖析功能可以出现）。解剖的内存范围在 epan/wmem_scopes.h 中定义。

正确使用这些函数会让你的代码更快，并大大减少在特殊情况下它会泄漏内存的可能性。

Wmem 最初是在这封发给 wireshark-dev 邮件列表的电子邮件中构思的：
https://www.wireshark.org/lists/wireshark-dev/201210/msg00178.html（翻译见文末）

2. 消费者使用

如果您正在编写解析器或其他“用户空间”代码，则使用 wmem应该与使用 malloc 或 g_malloc 或您使用的其他任何东西非常相似至。您需要做的就是包含标题 (epan/wmem_scopes.h) 和可选的获取内存池的句柄（如果你想*创建*一个内存池，请参阅下面的“3. 生产者的用法”部分）。

内存池是指向 wmem_allocator_t 类型对象的不透明指针，并且它是传递给几乎每次调用 wmem 的第一个参数。除了那个参数（以及函数以 wmem_ 为前缀的事实）用法与 glib 和其他实用程序库非常相似。例如：

    wmem_alloc(myPool, 20);

在 myPool 指向的池中分配 20 个字节。

2.1 内存池生命周期

每个内存池都应该有一个定义的生命周期或作用域，之后所有的该池中的内存被无条件释放。当你选择分配内存时在池中，您*必须*注意它的生命周期：如果生命周期短于您需要，您的代码将包含 use-after-free 错误；如果寿命更长超出您的需要，您的代码可能包含无法检测到的内存泄漏。在任一情况下，风险大于收益。

如果不存在其生命周期与您的内存生命周期相匹配的池，则您有两个选项：创建一个新池（请参阅本文档的第 3 节）或使用 NULL 水池。任何带有指向 wmem_allocator_t 的指针的函数也可以被传递 NULL 代替，在这种情况下，内存是手动管理的（就像 malloc 或 g_malloc)。像这样分配的内存*必须*手动传递给 wmem_free()为了防止内存泄漏（但是这些内存泄漏至少会显示在 valgrind 中）。请注意，将 wmem_allocated 内存直接传递给 free()或 g_free() 不安全；手动管理内存的后备类型可能是毫无预兆地改变了。

2.2 Wireshark 全局池

包含 wmem_scopes.h 头文件的解析器将具有三个可用的池自动发送给他们：pinfo->pool、wmem_file_scope() 和wmem_epan_scope();还有一个 wmem_packet_scope() 用于 `pinfo` 参数不可访问，但应首选 pinfo->pool。

pinfo 池的范围仅限于每个数据包的剖析，这意味着任何在其中分配的内存将在当前结束时自动释放包。文件池的范围与每个文件的剖析类似，这意味着在其中分配的任何内存都将在当前捕获文件已关闭。

注意：在适当范围之外使用这些池（例如，使用文件没有打开文件时的池）将抛出一个断言。有关详细信息，请参阅 epan/wmem_scopes.c 中的注释。

epan 池的范围为库的生命周期 - 在其中分配的内存是在调用 epan_cleanup() 之前不会被释放，这通常但不一定在程序的最后。

2.3 Pinfo 池

某些分配（例如 AT_STRINGZ 地址分配和任何可能最终被传递给 add_new_data_source) 需要他们的内存持续大约比通常的数据包范围长一点 - 基本上直到下一个数据包被剖析。这其实就是 Wireshark 的 pinfo 的范围结构，所以 pinfo 结构有一个 'pool' 成员，它是一个 wmem 池范围到 pinfo 结构的生命周期。

2.4 接口

每个函数的完整文档（参数、返回值、行为）在这些函数的头文件中以 Doxygen 格式存在（或将存在）。这只是您应该查看哪些头文件的概述。

2.4.1 核心API

wmem_core.h
 - 基本内存管理功能（wmem_alloc、wmem_realloc、wmem_free）。

2.4.2 字符串

wmem_strutl.h
 - 用于操作以 null 结尾的 C 样式字符串的实用程序函数。
   strdup 和 strdup_printf 等函数。

wmem_strbuf.h
 - 托管字符串对象实现，类似于 C++ 中的 std::string 或
   来自 Glib 的 GString。

2.4.3 容器数据结构

wmem_array.h
 - 一个可增长的数组（AKA 向量）实现。

wmem_list.h
 - 双向链表实现。

wmem_map.h
 - 哈希映射（AKA 哈希表）实现。

wmem_multimap.h
 - 哈希多重映射（可以使用相同键存储多个值的映射）
   执行。

wmem_queue.h
 - 队列实现（先进先出）。

wmem_stack.h
 - 堆栈实现（后进先出）。

wmem_tree.h
 - 平衡二叉树（红黑树）实现。

2.4.4 其他实用程序

wmem_miscutl.h
 - 杂项。实用功能，如 memdup。

2.5 回调

警告：您可能实际上并不需要这些；仅当您使用它们时
         确保您了解其中的危险。

有时（尽管希望很少）可能需要将数据存储在 wmem 中在释放之前需要额外清理的池。例如，也许您有一个指向需要关闭的文件句柄的指针。在这种情况下，您可以使用wmem_register_callback 函数注册回调在 wmem_user_cb.h 中声明。每次释放池中的内存时，所有首先调用已注册的清理函数。

请注意，未定义回调调用顺序，您不能依赖在另一个之前或之后调用某些回调。

警告：手动释放或移动内存（使用 wmem_free 或 wmem_realloc）
         不会触发任何回调。调用任何一个都是错误的
         如果您注册了回调来处理内存中的那些函数
         与那段记忆的内容。

3. 生产者使用

注意：如果你只是写一个解剖器，你可能不需要阅读本节。

旧的 emem 框架的问题之一是基本上有两个分配器后端（glib 和 mmap）混在一起 if 语句、环境变量和#ifdefs。在 wmem 中不同分配器后端被干净地分离出来，由池的所有者来选择一个。

3.1 可用的分配器后端

每个可用的分配器类型在 wmem_allocator_type_t 枚举定义在 wmem_core.h 中。见 doxygen 该头文件中的注释以了解每种类型的详细信息。

3.2 创建池

要创建池，请包含常规 wmem 标头并调用具有适当类型值的 wmem_allocator_new() 函数。
例如：

    #include <wsutil/wmem/wmem.h>

    wmem_allocator_t *myPool;
    myPool = wmem_allocator_new(WMEM_ALLOCATOR_SIMPLE);

从这里开始，你不需要记住你使用了哪种类型的分配器（尽管欢迎分配器作者公开其他特定于分配器的头文件中的辅助函数）。 “myPool”变量可以传递并在分配请求中正常使用，如本文第 2 节所述文档。

3.3 销毁池

无论您使用哪个分配器创建池，它都可以被销毁调用函数 wmem_destroy_allocator()。例如：

    #include <wsutil/wmem/wmem.h>

    wmem_allocator_t *myPool;

    myPool = wmem_allocator_new(WMEM_ALLOCATOR_SIMPLE);

    /* 在 myPool 中分配一些内存 ... */

    wmem_destroy_allocator(myPool);

销毁池将释放其中分配的所有内存。

3.4 复用池

可以释放池中的所有内存而不破坏它，允许以后重复使用。根据分配器的类型，这样做（通过调用 wmem_free_all()）可能比完全销毁和重新创建池便宜得多。因此，建议使用此方法，尤其是当池的范围为循环的单个迭代时。例如：

    #include <wsutil/wmem/wmem.h>

    wmem_allocator_t *myPool;

    myPool = wmem_allocator_new(WMEM_ALLOCATOR_SIMPLE);
    for （...） {

        /* 在 myPool 中分配一些内存 ... */

        /* 释放内存，比销毁和重建更快
           每次通过循环池。 */
        wmem_free_all(myPool);
    }
    wmem_destroy_allocator(myPool);

4. 内部设计

尽管使用 Wireshark 的标准 C90 编写，wmem 遵循相当面向对象的设计模式。尽管效率始终是一个问题，但编写 wmem 的主要目标是可维护性和防止内存泄漏。

4.1 结构体_wmem_allocator_t

wmem 的核心是定义在 wmem_allocator.h 头文件。此结构使用 C 函数指针实现一种常见的面向对象设计模式，称为接口（也对于那些更熟悉 C++ 的人来说，它被称为抽象类）。

不同的分配器实现可以通过以下方式提供完全相同的接口将它们自己的功能分配给结构实例的成员。该结构由三组八名成员组成。

4.1.1 实施细节

 - 私人数据
 - 类型

private_data 指针是一个 void 指针，分配器实现可以使用它来存储它需要的任何内部结构。指向 private_data 的指针被传递给分配器实现必须定义的几乎所有其他函数。

type 字段是 wmem_allocator_type_t 类型的枚举（参见
第 3.1 节）。它的值由 wmem_allocator_new() 函数设置，而不是由特定于实现的构造函数设置。分配器实现应将此字段视为只读。

4.1.2 消费者函数

 - walloc()
 - wfree()
 - wrealloc()

这些函数指针应该设置为语义明显类似于它们的标准库同名的函数。每个都有一个额外的参数，它是分配器的 private_data 指针的副本。

请注意，在大多数情况下，用户代码不会直接调用 wrealloc() 和 wfree() - 它们主要是 wmem 可能想要实现的数据结构使用的优化（例如，实现动态大小的没有某种形式的 realloc 的数组）。

另请注意，分配器不必以任何方式处理 NULL 指针或 0 长度请求 - 这些检查是在 wmem 中以与分配器无关的方式完成的。分配器作者可以假设所有传入的指针（到 wrealloc 和 wfree）都是非 NULL，并且所有传入的长度（到 walloc 和 wrealloc）都是非 0。

4.1.3 生产者/管理者功能

 - free_all()
 - gc()
 - cleanup()

所有这些函数都只接受一个参数，即分配器的
private_data 指针。

free_all() 函数应该释放池中当前分配的所有内存。请注意，这不一定与调用 free() 完全相同
在所有分配的块上 - 允许 free_all() 进行额外的清理或使用一次释放一个块时不可用的优化。

gc() 函数应该尽其所能通过将未使用的块返回给操作系统、优化内部数据结构等来减少解析器中过多的内存使用。

cleanup() 函数应该进行任何最终清理并释放所有内存。
它基本上相当于一个析构函数。为简单起见，wmem 保证在调用此函数之前立即调用 free_all()。没有这样的保证 gc() 已经（曾经）被调用过。

4.2 与池无关的 API

emem 的问题之一是 API（包括公共数据结构）需要为每个实现的范围提供包装函数。即使 emem 中有一个堆栈实现，它也不一定可用于文件范围内存，除非有人花时间为接口编写 se_stack_wrapper 函数。

在 wmem 中，所有公共 API 都将池作为第一个参数，因此它们可以被编写一次并与任何可用的内存池一起使用。像 wmem 的堆栈实现这样的数据结构只在创建时使用池 - 提供的指针与数据结构一起存储在内部，随后的调用（如 push 和 pop）将使用堆栈本身而不是池。

4.3 调试

wmem 的主要调试控件是 WIRESHARK_DEBUG_WMEM_OVERRIDE 环境变量。如果设置，此值将强制所有对 wmem_allocator_new() 的调用返回相同类型的分配器，而不管代码通常请求哪种类型。它目前具有三个有效值：

 - 值“simple”强制使用 WMEM_ALLOCATOR_SIMPLE。 valgrind 脚本当前设置了这个值，因为简单分配器是唯一一个内存分配可以被 valgrind 正确跟踪的分配器。

 - 值“strict”强制使用 WMEM_ALLOCATOR_STRICT。 fuzz-test 脚本当前设置了这个值，因为 fuzz-testing 的目标是找到尽可能多的错误。

 - 值“block”强制使用 WMEM_ALLOCATOR_BLOCK。这目前没有被任何脚本使用，但对
对块分配器进行压力测试。

 - 值“block_fast”强制使用 WMEM_ALLOCATOR_BLOCK_FAST。这目前没有被任何脚本使用，但对于对快速块分配器进行压力测试很有用。

请注意，无论此变量的值如何，调用特定于分配器的辅助函数始终是安全的。如果分配器参数的类型错误，则它们必须是安全的空操作。

4.4 测试

有一个简单的 wmem 测试套件，位于文件 wmem_test.c 和构建时应该自动构建到二进制“wmem_test”中 wireshark。它至少包含所有现有功能的基本测试。
该套件由构建机器人通过 shell 脚本自动运行test/test.py 调用 test/suite_unittests.py。

wmem 中添加的新功能（分配器、数据结构、实用程序功能等）还必须在此套件中添加测试。

测试套件可能会使用更熟悉 Glib 测试框架的人进行的清理工作，但它确实完成了这项工作。

5. 性能说明

由于我自己的错误判断，有一个持久的想法是 wmem 在一般情况下比其他分配器神奇地快。这是错误的。

首先，wmem 支持多种不同的分配器后端（参见本文档的第 3 节和第 4 节），因此无论如何尝试将“wmem”的性能与另一个系统进行比较是令人困惑和误导的。

其次，任何现代系统提供的 malloc 已经有一个非常聪明和高效的分配器算法，它利用了块、竞技场和各种其他花哨的技巧。尝试比 libc 的分配器更快通常是浪费时间，除非您有特定的分配模式需要优化。

第三，虽然历史上存在过在内核前面放置一些东西以减少上下文切换的数量的论据，但现代 libc 实现应该已经这样做了。进行动态库调用仍然比调用本地定义的链接器优化函数稍微贵一些，但差异太小了，无法在意。

话虽如此，确实 *some* wmem 的分配器可以是在*某些*用例中，比您的标准 libc malloc 快得多：
 - BLOCK 和 BLOCK_FAST 分配器都提供了非常有效的 free_all 操作，这比在每个单独的分配上调用 free() 快很多数量级。
 - BLOCK_FAST 分配器特别针对 Wireshark 的数据包范围池进行了优化。它具有极短的、定义明确的生命周期和非常规则的分配模式；我能够使用这些知识轻松击败 libc，*在那个特定的用例中*。

原文：

1. Introduction

The 'wmem' memory manager is Wireshark's memory management framework, replacing
the old 'emem' framework which was removed in Wireshark 2.0.

In order to make memory management easier and to reduce the probability of
memory leaks, Wireshark provides its own memory management API. This API is
implemented inside wsutil/wmem/ and provides memory pools and functions that make
it easy to manage memory even in the face of exceptions (which many dissector
functions can raise). Memory scopes for dissection are defined in epan/wmem_scopes.h.

Correct use of these functions will make your code faster, and greatly reduce
the chances that it will leak memory in exceptional cases.

Wmem was originally conceived in this email to the wireshark-dev mailing list:
https://www.wireshark.org/lists/wireshark-dev/201210/msg00178.html

2. Usage for Consumers

If you're writing a dissector, or other "userspace" code, then using wmem
should be very similar to using malloc or g_malloc or whatever else you're used
to. All you need to do is include the header (epan/wmem_scopes.h) and optionally
get a handle to a memory pool (if you want to *create* a memory pool, see the
section "3. Usage for Producers" below).

A memory pool is an opaque pointer to an object of type wmem_allocator_t, and
it is the very first parameter passed to almost every call you make to wmem.
Other than that parameter (and the fact that functions are prefixed wmem_)
usage is very similar to glib and other utility libraries. For example:

    wmem_alloc(myPool, 20);

allocates 20 bytes in the pool pointed to by myPool.

2.1 Memory Pool Lifetimes

Every memory pool should have a defined lifetime, or scope, after which all the
memory in that pool is unconditionally freed. When you choose to allocate memory
in a pool, you *must* be aware of its lifetime: if the lifetime is shorter than
you need, your code will contain use-after-free bugs; if the lifetime is longer
than you need, your code may contain undetectable memory leaks. In either case,
the risks outweigh the benefits.

If no pool exists whose lifetime matches the lifetime of your memory, you have
two options: create a new pool (see section 3 of this document) or use the NULL
pool. Any function that takes a pointer to a wmem_allocator_t can also be passed
NULL instead, in which case the memory is managed manually (just like malloc or
g_malloc). Memory allocated like this *must* be manually passed to wmem_free()
in order to prevent memory leaks (however these memory leaks will at least show
up in valgrind). Note that passing wmem_allocated memory directly to free()
or g_free() is not safe; the backing type of manually managed memory may be
changed without warning.

2.2 Wireshark Global Pools

Dissectors that include the wmem_scopes.h header file will have three pools available
to them automatically: pinfo->pool, wmem_file_scope() and
wmem_epan_scope(); there is also a wmem_packet_scope() for cases when the
`pinfo` argument is not accessible, but pinfo->pool should be preferred.

The pinfo pool is scoped to the dissection of each packet, meaning that any
memory allocated in it will be automatically freed at the end of the current
packet. The file pool is similarly scoped to the dissection of each file,
meaning that any memory allocated in it will be automatically freed when the
current capture file is closed.

NB: Using these pools outside of the appropriate scope (e.g. using the file
    pool when there isn't a file open) will throw an assertion.
    See the comment in epan/wmem_scopes.c for details.

The epan pool is scoped to the library's lifetime - memory allocated in it is
not freed until epan_cleanup() is called, which is typically but not necessarily
at the very end of the program.

2.3 The Pinfo Pool

Certain allocations (such as AT_STRINGZ address allocations and anything that
might end up being passed to add_new_data_source) need their memory to stick
around a little longer than the usual packet scope - basically until the
next packet is dissected. This is, in fact, the scope of Wireshark's pinfo
structure, so the pinfo struct has a 'pool' member which is a wmem pool scoped
to the lifetime of the pinfo struct.

2.4 API

Full documentation for each function (parameters, return values, behaviours)
lives (or will live) in Doxygen-format in the header files for those functions.
This is just an overview of which header files you should be looking at.

2.4.1 Core API

wmem_core.h
 - Basic memory management functions (wmem_alloc, wmem_realloc, wmem_free).

2.4.2 Strings

wmem_strutl.h
 - Utility functions for manipulating null-terminated C-style strings.
   Functions like strdup and strdup_printf.

wmem_strbuf.h
 - A managed string object implementation, similar to std::string in C++ or
   GString from Glib.

2.4.3 Container Data Structures

wmem_array.h
 - A growable array (AKA vector) implementation.

wmem_list.h
 - A doubly-linked list implementation.

wmem_map.h
 - A hash map (AKA hash table) implementation.

wmem_multimap.h
 - A hash multimap (map that can store multiple values with the same key)
   implementation.

wmem_queue.h
 - A queue implementation (first-in, first-out).

wmem_stack.h
 - A stack implementation (last-in, first-out).

wmem_tree.h
 - A balanced binary tree (red-black tree) implementation.

2.4.4 Miscellaneous Utilities

wmem_miscutl.h
 - Misc. utility functions like memdup.

2.5 Callbacks

WARNING: You probably don't actually need these; use them only when you're
         sure you understand the dangers.

Sometimes (though hopefully rarely) it may be necessary to store data in a wmem
pool that requires additional cleanup before it is freed. For example, perhaps
you have a pointer to a file-handle that needs to be closed. In this case, you
can register a callback with the wmem_register_callback function
declared in wmem_user_cb.h. Every time the memory in a pool is freed, all
registered cleanup functions are called first.

Note that callback calling order is not defined, you cannot rely on a
certain callback being called before or after another.

WARNING: Manually freeing or moving memory (with wmem_free or wmem_realloc)
         will NOT trigger any callbacks. It is an error to call either of
         those functions on memory if you have a callback registered to deal
         with the contents of that memory.

3. Usage for Producers

NB: If you're just writing a dissector, you probably don't need to read
    this section.

One of the problems with the old emem framework was that there were basically
two allocator backends (glib and mmap) that were all mixed together in a mess
of if statements, environment variables and #ifdefs. In wmem the different
allocator backends are cleanly separated out, and it's up to the owner of the
pool to pick one.

3.1 Available Allocator Back-Ends

Each available allocator type has a corresponding entry in the
wmem_allocator_type_t enumeration defined in wmem_core.h. See the doxygen
comments in that header file for details on each type.

3.2 Creating a Pool

To create a pool, include the regular wmem header and call the
wmem_allocator_new() function with the appropriate type value.
For example:

    #include <wsutil/wmem/wmem.h>

    wmem_allocator_t *myPool;
    myPool = wmem_allocator_new(WMEM_ALLOCATOR_SIMPLE);

From here on in, you don't need to remember which type of allocator you used
(although allocator authors are welcome to expose additional allocator-specific
helper functions in their headers). The "myPool" variable can be passed around
and used as normal in allocation requests as described in section 2 of this
document.

3.3 Destroying a Pool

Regardless of which allocator you used to create a pool, it can be destroyed
with a call to the function wmem_destroy_allocator(). For example:

    #include <wsutil/wmem/wmem.h>

    wmem_allocator_t *myPool;

    myPool = wmem_allocator_new(WMEM_ALLOCATOR_SIMPLE);

    /* Allocate some memory in myPool ... */

    wmem_destroy_allocator(myPool);

Destroying a pool will free all the memory allocated in it.

3.4 Reusing a Pool

It is possible to free all the memory in a pool without destroying it,
allowing it to be reused later. Depending on the type of allocator, doing this
(by calling wmem_free_all()) can be significantly cheaper than fully destroying
and recreating the pool. This method is therefore recommended, especially when
the pool would otherwise be scoped to a single iteration of a loop. For example:

    #include <wsutil/wmem/wmem.h>

    wmem_allocator_t *myPool;

    myPool = wmem_allocator_new(WMEM_ALLOCATOR_SIMPLE);
    for (...) {

        /* Allocate some memory in myPool ... */

        /* Free the memory, faster than destroying and recreating
           the pool each time through the loop. */
        wmem_free_all(myPool);
    }
    wmem_destroy_allocator(myPool);

4. Internal Design

Despite being written in Wireshark's standard C90, wmem follows a fairly
object-oriented design pattern. Although efficiency is always a concern, the
primary goals in writing wmem were maintainability and preventing memory
leaks.

4.1 struct _wmem_allocator_t

The heart of wmem is the _wmem_allocator_t structure defined in the
wmem_allocator.h header file. This structure uses C function pointers to
implement a common object-oriented design pattern known as an interface (also
known as an abstract class to those who are more familiar with C++).

Different allocator implementations can provide exactly the same interface by
assigning their own functions to the members of an instance of the structure.
The structure has eight members in three groups.

4.1.1 Implementation Details

 - private_data
 - type

The private_data pointer is a void pointer that the allocator implementation can
use to store whatever internal structures it needs. A pointer to private_data is
passed to almost all of the other functions that the allocator implementation
must define.

The type field is an enumeration of type wmem_allocator_type_t (see
section 3.1). Its value is set by the wmem_allocator_new() function, not
by the implementation-specific constructor. This field should be considered
read-only by the allocator implementation.

4.1.2 Consumer Functions

 - walloc()
 - wfree()
 - wrealloc()

These function pointers should be set to functions with semantics obviously
similar to their standard-library namesakes. Each one takes an extra parameter
that is a copy of the allocator's private_data pointer.

Note that wrealloc() and wfree() are not expected to be called directly by user
code in most cases - they are primarily optimizations for use by data
structures that wmem might want to implement (it's inefficient, for example, to
implement a dynamically sized array without some form of realloc).

Also note that allocators do not have to handle NULL pointers or 0-length
requests in any way - those checks are done in an allocator-agnostic way
higher up in wmem. Allocator authors can assume that all incoming pointers
(to wrealloc and wfree) are non-NULL, and that all incoming lengths (to walloc
and wrealloc) are non-0.

4.1.3 Producer/Manager Functions

 - free_all()
 - gc()
 - cleanup()

All of these functions take only one parameter, which is the allocator's
private_data pointer.

The free_all() function should free all the memory currently allocated in the
pool. Note that this is not necessarily exactly the same as calling free()
on all the allocated blocks - free_all() is allowed to do additional cleanup
or to make use of optimizations not available when freeing one block at a time.

The gc() function should do whatever it can to reduce excess memory usage in
the dissector by returning unused blocks to the OS, optimizing internal data
structures, etc.

The cleanup() function should do any final cleanup and free any and all memory.
It is basically the equivalent of a destructor function. For simplicity, wmem
is guaranteed to call free_all() immediately before calling this function. There
is no such guarantee that gc() has (ever) been called.

4.2 Pool-Agnostic API

One of the issues with emem was that the API (including the public data
structures) required wrapper functions for each scope implemented. Even
if there was a stack implementation in emem, it wasn't necessarily available
for use with file-scope memory unless someone took the time to write se_stack_
wrapper functions for the interface.

In wmem, all public APIs take the pool as the first argument, so that they can
be written once and used with any available memory pool. Data structures like
wmem's stack implementation only take the pool when created - the provided
pointer is stored internally with the data structure, and subsequent calls
(like push and pop) will take the stack itself instead of the pool.

4.3 Debugging

The primary debugging control for wmem is the WIRESHARK_DEBUG_WMEM_OVERRIDE
environment variable. If set, this value forces all calls to
wmem_allocator_new() to return the same type of allocator, regardless of which
type is requested normally by the code. It currently has three valid values:

 - The value "simple" forces the use of WMEM_ALLOCATOR_SIMPLE. The valgrind
   script currently sets this value, since the simple allocator is the only
   one whose memory allocations are trackable properly by valgrind.

 - The value "strict" forces the use of WMEM_ALLOCATOR_STRICT. The fuzz-test
   script currently sets this value, since the goal when fuzz-testing is to find
   as many errors as possible.

 - The value "block" forces the use of WMEM_ALLOCATOR_BLOCK. This is not
   currently used by any scripts, but is useful for stress-testing the block
   allocator.

 - The value "block_fast" forces the use of WMEM_ALLOCATOR_BLOCK_FAST. This is
   not currently used by any scripts, but is useful for stress-testing the fast
   block allocator.

Note that regardless of the value of this variable, it will always be safe to
call allocator-specific helpers functions. They are required to be safe no-ops
if the allocator argument is of the wrong type.

4.4 Testing

There is a simple test suite for wmem that lives in the file wmem_test.c and
should get automatically built into the binary 'wmem_test' when building
Wireshark. It contains at least basic tests for all existing functionality.
The suite is run automatically by the build-bots via the shell script
test/test.py which calls out to test/suite_unittests.py.

New features added to wmem (allocators, data structures, utility
functions, etc.) MUST also have tests added to this suite.

The test suite could potentially use a clean-up by someone more
intimately familiar with Glib's testing framework, but it does the job.

5. A Note on Performance

Because of my own bad judgment, there is the persistent idea floating around
that wmem is somehow magically faster than other allocators in the general case.
This is false.

First, wmem supports multiple different allocator backends (see sections 3 and 4
of this document), so it is confusing and misleading to try and compare the
performance of "wmem" in general with another system anyways.

Second, any modern system-provided malloc already has a very clever and
efficient allocator algorithm that makes use of blocks, arenas and all sorts of
other fancy tricks. Trying to be faster than libc's allocator is generally a
waste of time unless you have a specific allocation pattern to optimize for.

Third, while there were historically arguments to be made for putting something
in front of the kernel to reduce the number of context-switches, modern libc
implementations should already do that. Making a dynamic library call is still
marginally more expensive than calling a locally-defined linker-optimized
function, but it's a difference too small to care about.

With all that said, it is true that *some* of wmem's allocators can be
substantially faster than your standard libc malloc, in *some* use cases:
 - The BLOCK and BLOCK_FAST allocators both provide very efficient free_all
   operations, which can be many orders of magnitude faster than calling free()
   on each individual allocation.
 - The BLOCK_FAST allocator in particular is optimized for Wireshark's packet
   scope pool. It has an extremely short, well-defined lifetime, and a very
   regular pattern of allocations; I was able to use that knowledge to beat libc
   rather handily, *in that specific use case*.

/*
 * Editor modelines  -  https://www.wireshark.org/tools/modelines.html
 *
 * Local variables:
 * c-basic-offset: 4
 * tab-width: 8
 * indent-tabs-mode: nil
 * End:
 *
 * vi: set shiftwidth=4 tabstop=8 expandtab:
 * :indentSize=4:tabSize=8:noTabs=true:
 */

wmem 内存管理的构思邮件翻译：

TL;DR - Jakub 最近提议对 emem [1] 进行一些更改。当我认为它们是一个非常好的主意，我相信从长远来看当前的 emem 设计有太多的基本限制，使其值得适应我们未来的需求。我建议应该是
逐渐弃用以支持别的东西，我写了一个其他东西可能是什么的简单例子。

--- 长版 ---

在我最近与 emem 的冒险之后，我花了一些时间来研究 Wireshark 现在和将来在其内存管理器中可能需要的功能。预测未来总是一件棘手的事情，但我尽量不偏离明显的太远：
- 当前季节性和短暂的任意实例分配器。如果我们想要支持一次打开多个文件，这些将是必要的。
- 线程安全的分配器。如果我们想要对单个文件进行多线程剖析或重新剖析，这是必要的。
- 具有不同范围的分配器。我可以想到几个不同的地方，新的分配器范围可能会简化现有代码，我相信还有其他地方。

考虑到这些想法，然后我仔细研究了当前emem 设计并试图估计所涉及的工作量
适应（和维护）它向前发展。我得出结论，在从长远来看，emem 有太多的基本限制，使其值得适应：
- 支持任意季节性和临时池将需要非平凡的 API 更改，导致依赖代码的大量痛苦。
- 当前的 glib 和 mmap 分配器经常挤在一起并排生活在相同的功能中。正如我最近发现的，试图调整其中任何一个都会产生很多意想不到的副作用。
- 添加单个新范围需要为每个API 函数（*_alloc、*_alloc0、*_strdup、*_strndup 等）。最好这只是额外的工作，但在最坏的情况下会导致 API 不一致，例如，有一个 ep_strbuf 实现但没有等效的 se_ function（顺便说一下，这是一个真实的例子）。

我想建议逐渐弃用 emem 以支持使用这些设计的新内存管理框架牢记要求。一个单独的新界面将减轻痛苦通过允许我们将 emem 维护为已弃用的接口来迁移只要有必要。

本着用工作代码支持想法的精神，我写了一个我认为这样一个新框架可能看起来像的简单版本。
它干净地分离出分配器后端（glib、mmap 等）和核心接口中的实用功能（strdup 等）。它也是
要求所有 API 函数都显式传递给作用域他们想要使用的分配器实例。这使得添加新的变得微不足道范围或支持当前范围的多个实例。因为分配器是明确区分的，所以使它们中的任何一个成为线程安全的都是一个相当简单的操作。

我已经链接了一个包含以下文件的 tarball [2]：
- wmem_allocator.h - 分配器接口的定义
- wmem_allocator_glib.* - 分配器的简单实现由 g_malloc 和单链表支持的接口。
- wmem_core.* - wmem_alloc() 和 wmem_alloc0() 的实现
- wmem_strutl.* - wmem_strcpy() 和 wmem_strncpy() 的实现
- wmem.h - wmem 消费者包含的通用头文件，它仅包含 wmem_core.h、wmem_strutl.h 和任何其他可能创建的内容

用法可能如下所示：

wmem_allocator_t *ep_scope = wmem_create_glib_allocator();
doWork(ep_scope);
wmem_destroy_glib_allocator(ep_scope);

然后在 doWork，而不是 ep_alloc(numBytes) 你会调用wmem_alloc（ep_scope，numBytes）。

或者，如果外部块处于循环中，则不必每次创建/销毁，并且可以简单地调用 wmem_free_all(ep_scope)在调用 doWork() 之间。

想法、评论和建设性的批评总是受欢迎的。
你怎么看？
埃文

-扩展：

一个内存池的设计与实现的过程*（强烈推荐）：性能优化-高效内存池的设计与实现 - 知乎