linux 命令记忆_感谢您的记忆，Linux

学到更多。 开发更多。 连接更多。

新的developerWorks Premium会员计划可通过Safari图书在线获得对强大的开发工具和资源的无障碍访问权，其中包括500个顶级技术标题（数十个专门针对Java开发人员），主要开发人员活动的超低折扣，最近O'Reilly的视频重播会议等。 立即注册 。

分配每个Java对象的Java堆是编写Java应用程序时与您最紧密联系的内存区域。 JVM旨在将我们与主机的特性隔离开来，因此在考虑内存时自然会想到堆。 毫无疑问，您遇到过Java堆OutOfMemoryError —由对象泄漏或由于堆的大小不足以存储所有数据而引起—并且可能已经学会了一些调试这些情况的技巧。 但是随着Java应用程序处理更多的数据和更多的并发负载，您可能会开始遇到OutOfMemoryError ，这些OutOfMemoryError无法使用常规的技巧来解决-即使Java堆未满，也会引发错误。 发生这种情况时，您需要了解Java Runtime Environment（JRE）内部的情况。

Java应用程序在Java运行时的虚拟化环境中运行，但是运行时本身是用某种语言（例如C）编写的本机程序，该语言消耗本机资源，包括本机内存 。 本机内存是可用于运行时进程的内存，与Java应用程序使用的Java堆内存区分开。 每个虚拟化资源（包括Java堆和Java线程）都必须与虚拟机运行时使用的数据一起存储在本机内存中。 这意味着主机硬件和操作系统（OS）对本地内存的限制会影响您可以使用Java应用程序执行的操作。

本文是在不同平台上涉及同一主题的两篇文章之一。 在这两者中，您将了解什么是本机内存，Java运行时如何使用它，用尽了什么以及如何调试本机OutOfMemoryError 。 本文介绍Windows和Linux，并且不关注任何特定的运行时实现。 随附的文章介绍了AIX，并重点介绍了Java®IBM®Developer Kit。 （该文章中有关IBM实现的信息对于AIX以外的平台也适用，因此，如果您使用Linux上的Java Java开发人员工具包或Windows的IBM 32位运行时环境，您可能会发现该文章也很有用。 ）

回顾本机内存

我将从解释操作系统和底层硬件对本机内存的限制开始。 如果您熟悉使用C之类的语言管理动态内存，那么您可能需要跳到下一部分 。

硬件限制

本机进程遇到的许多限制是由硬件而不是操作系统施加的。 每台计算机都有一个处理器和一些随机存取内存（RAM），也称为物理内存。 处理器将数据流解释为要执行的指令； 它具有一个或多个执行整数和浮点运算以及更高级计算的处理单元。 处理器具有许多寄存器 -非常快的存储元件，用作执行的计算的工作存储； 寄存器大小决定了单个计算可以使用的最大数量。

处理器通过内存总线连接到物理内存。 物理地址的大小（处理器用于索引物理RAM的地址）限制了可以寻址的内存量。 例如，一个16位物理地址可以从0x0000到0xFFFF寻址，这提供2 ^ 16 = 65536个唯一的存储位置。 如果每个地址都引用一个存储字节，则16位物理地址将允许处理器寻址64KB的存储器。

推荐资源

• “ 感谢您的记忆（对于AIX） ”
• “ IBM Java监视和诊断工具-内存分析器 ”
• “ Java的指导性调试 ”

处理器被描述为一定数量的位。 尽管存在一些例外情况（例如390 31位），但它通常指的是寄存器的大小，其中指的是物理地址的大小。 对于台式机和服务器平台，此数字为31、32或64；对于台式机和服务器平台，此数字为31。 对于嵌入式设备和微处理器，它可以低至4。物理地址大小可以与寄存器宽度相同，但可以更大或更小。 当运行合适的操作系统时，大多数64位处理器都可以运行32位程序。

表1列出了一些流行的Linux和Windows体系结构及其寄存器和物理地址大小：

表1.一些流行处理器架构的寄存器和物理地址大小

建筑	寄存器宽度（位）	物理地址大小（位）
（现代）英特尔®x86	32	32 具有物理地址扩展功能的36（奔腾Pro及更高版本）
x86 64	64	目前为48位（范围会在以后增加）
PPC64	64	电源5时为50位
390 31位	32	31
390 64位	64	64

操作系统和虚拟内存

如果编写的应用程序要在没有操作系统的情况下直接在处理器上运行，则可以使用处理器可以寻址的所有内存（假设已连接足够的物理RAM）。 但是要享受多任务处理和硬件抽象等功能，几乎每个人都使用某种操作系统来运行其程序。

在Windows和Linux等多任务操作系统中，有多个程序使用系统资源，包括内存。 每个程序都需要分配物理内存的区域才能使用。可以设计一个操作系统，使每个程序都可以直接与物理内存一起使用，并且可以信任仅使用已分配的内存。 某些嵌入式OS的工作方式是这样的，但是在由许多未经测试的程序组成的环境中不可行，因为任何程序都可能破坏其他程序或OS本身的内存。

虚拟内存允许多个进程共享物理内存，而不会破坏彼此的数据。 在具有虚拟内存的操作系统（例如Windows，Linux等）中，每个程序都有其自己的虚拟地址空间-地址的逻辑区域，其大小由该系统上的地址大小决定（因此31、32或台式机和服务器平台为64位）。 进程的虚拟地址空间中的区域可以映射到物理内存，文件或任何其他可寻址存储。 操作系统可以在不使用物理内存时将数据移入或移出交换区域（Windows上的页面文件或Linux上的交换分区 ），以充分利用物理内存。 当程序尝试使用虚拟地址访问内存时，操作系统结合片上硬件会将该虚拟地址映射到物理位置。 该位置可以是物理RAM，文件或页面文件/交换分区。 如果内存区域已移至交换空间，则在使用之前将其加载回物理内存。 图1显示了虚拟内存如何通过将进程地址空间的区域映射到共享资源来工作的：

图1.虚拟内存将进程地址空间映射到物理资源

程序的每个实例都作为一个进程运行。 Linux和Windows上的进程是有关操作系统控制的资源的信息的集合（例如文件和套接字信息），通常是一个虚拟地址空间（在某些体系结构上超过一个）和至少一个执行线程。

虚拟地址空间大小可以小于处理器的物理地址大小。 Intel x86 32位最初具有32位物理地址，该地址允许处理器寻址4GB的存储。 后来，添加了一项称为“物理地址扩展（PAE）”的功能，该功能将物理地址的大小扩展到了36位-允许安装和寻址多达64GB的RAM。 PAE允许操作系统将32位4GB虚拟地址空间映射到较大的物理地址范围，但不允许每个进程拥有64GB虚拟地址空间。 这意味着，如果您将超过4GB的内存放入32位Intel服务器中，则无法将所有内存直接映射到单个进程中。

地址窗口扩展功能允许Windows进程将其32位地址空间的一部分作为滑动窗口映射到更大的内存区域中。 Linux基于将区域映射到虚拟地址空间而使用类似的技术。 这意味着尽管您不能直接引用超过4GB的内存，但是可以使用更大的内存区域。

内核空间和用户空间

尽管每个进程都有其自己的地址空间，但程序通常无法使用所有地址空间。 地址空间分为用户空间和内核空间 。 内核是主要的OS程序，包含用于与计算机硬件接口，调度程序以及提供诸如网络和虚拟内存之类的服务的逻辑。

作为计算机启动顺序的一部分，操作系统内核将运行并初始化硬件。 内核配置了硬件及其内部状态后，便会开始第一个用户空间进程。 如果用户程序需要来自OS的服务，则它可以执行称为系统调用的操作，该操作会跳入内核程序，然后执行请求。 通常需要系统调用来进行诸如读写文件，联网和启动新进程之类的操作。

执行系统调用时，内核需要访问其自身的内存和调用进程的内存。 因为正在执行当前线程的处理器被配置为使用当前进程的地址空间映射来映射虚拟地址，所以大多数OS将每个进程地址空间的一部分映射到公共内核内存区域。 地址空间中映射供内核使用的部分称为内核空间； 可由用户应用程序使用的其余部分称为用户空间。

内核与用户空间之间的平衡因操作系统而异，甚至在运行于不同硬件体系结构上的同一操作系统实例之间也有所不同。 平衡通常是可配置的，可以进行调整以为用户应用程序或内核提供更多空间。 挤压内核区域可能会导致一些问题，例如限制可以同时登录的用户数或可以运行的进程数。 较小的用户空间意味着应用程序程序员的工作空间较小。

默认情况下，32位Windows具有2GB用户空间和2GB内核空间。 通过在启动配置中添加/3GB开关并使用/LARGEADDRESSAWARE开关重新链接应用程序，可以在某些Windows版本上将余额转移到3GB用户空间和1GB内核空间。 在32位Linux上，默认值为3GB用户空间和1GB内核空间。 一些Linux发行版提供了支持4GB用户空间的hugemem内核。 为实现此目的，内核被赋予了自己的地址空间，该地址空间在进行系统调用时会使用。 用户空间的增加被较慢的系统调用所抵消，因为操作系统必须在每次进行系统调用时在地址空间之间复制数据并重置进程地址空间映射。 图2显示了32位Windows的地址空间布局：

图2. 32位Windows的地址空间布局

图3显示了32位Linux的地址空间安排：

图3. 32位Linux的地址空间布局

在Linux 390 31位上也使用了单独的内核地址空间，其中2GB的地址空间较小，因此不希望分割单个地址空间。 但是，390体系结构可以同时使用多个地址空间，而不会影响性能。

进程地址空间必须包含程序所需的所有内容，包括程序本身及其使用的共享库（Windows上的DLL，Linux上的.so文件）。 共享库不仅会占用程序无法用来存储数据的空间，而且还会使地址空间碎片化，并减少可作为连续块分配的内存量。 这在具有3GB用户空间的Windows x86上运行的程序中很明显。 DLL是使用首选的加载地址构建的：加载DLL时，它将被映射到特定位置的地址空间中，除非该位置已被占用，在这种情况下，它将被重新定位并加载到其他位置。 最初设计Windows NT时具有2GB的用户空间，因此可以将系统库构建为在2GB的边界附近加载，从而使大部分用户区域可供应用程序使用是有意义的。 当用户区域扩展到3GB时，系统共享库仍加载接近2GB的空间-现在位于用户空间的中间。 尽管总用户空间为3GB，但由于共享库妨碍了分配3GB的内存块，因此是不可能的。

在Windows上使用/3GB开关会将内核空间减少到最初设计的一半。 在某些情况下，可能会耗尽1GB的内核空间并遇到缓慢的I / O或创建新用户会话的问题。 尽管/3GB开关对于某些应用程序来说可能非常有价值，但是在使用它的任何环境都应该在部署之前进行全面的负载测试。

本机内存泄漏或过多的本机内存使用将导致不同的问题，具体取决于您耗尽地址空间还是用尽物理内存。 通常，只有32位进程才会用尽地址空间-因为最大4GB的分配很容易。 64位进程的用户空间为数百或数千GB，即使尝试也很难填满。 如果确实用尽了Java进程的地址空间，那么Java运行时可能会开始显示奇怪的症状，我将在本文稍后介绍。 在具有比物理内存更多的进程地址空间的系统上运行时，内存泄漏或过多使用本机内存将迫使OS换出某些本机进程的虚拟地址空间的后备存储。 访问已交换的内存地址比读取常驻（位于物理内存中）地址要慢得多，因为操作系统必须从硬盘驱动器中提取数据。 可以分配足够的内存以耗尽所有物理内存和所有交换内存（分页空间）； 在Linux上，这会触发内核内存不足（OOM）杀手，它会强行杀死最耗内存的进程。 在Windows上，分配开始失败的方式与地址空间已满时的方式相同。

如果您同时尝试使用比物理内存更多的虚拟内存，那么很明显，问题就在此过程被淘汰之前，因为它是内存消耗。 系统将崩溃—也就是说，将大部分时间都花在从交换空间来回复制内存上。 发生这种情况时，计算机和单个应用程序的性能将变得很差，以至于用户不能不注意存在问题。 换出JVM的Java堆时，垃圾收集器的性能会变得非常差，以致应用程序可能会挂起。 如果在一台计算机上同时使用多个Java运行时，则物理内存必须足以容纳所有Java堆。

Java运行时如何使用本机内存

Java运行时是一个OS进程，受我在上一节中概述的硬件和OS约束的约束。 运行时环境提供的功能由一些未知的用户代码驱动； 这使得无法预测运行时环境在每种情况下将需要哪些资源。 Java应用程序在托管Java环境中采取的每项操作都可能会影响提供该环境的运行时的资源要求。 本部分描述Java应用程序如何以及为何使用本机内存。

Java堆和垃圾回收

Java堆是分配对象的内存区域。 大多数Java SE实现都有一个逻辑堆，尽管某些专业的Java运行时（例如实现Java实时规范（RTSJ）的运行时有多个堆）。 可以根据用于管理堆内存的垃圾回收（GC）算法将单个物理堆分为多个逻辑部分。 这些部分通常实现为在Java内存管理器（包括垃圾收集器）控制下的本地内存的连续平板。

使用-Xmx和-Xms选项从Java命令行控制堆的大小（ mx是堆的最大大小， ms是初始大小）。 尽管逻辑堆（活动使用的内存区域）可以根据堆上的对象数和在GC中花费的时间而增长和缩小，但使用的本机内存量保持不变，并由-Xmx决定。 -Xmx值：最大堆大小。 大多数GC算法都将堆分配为连续的内存，因此当需要扩展堆时就不可能分配更多的本机内存。 所有堆内存必须预先预留。

保留本机内存与分配本机内存不同。 当保留本机内存时，它不支持物理内存或其他存储。 尽管保留地址空间的大块不会耗尽物理资源，但确实会阻止该内存用于其他目的。 由从未使用的保留内存引起的泄漏与泄漏分配的内存一样严重。

一些垃圾收集器通过在堆的使用区域缩小时取消使用（释放其后备存储）部分堆，从而最大程度地减少物理内存的使用。

需要更多的本机内存来维护维护Java堆的内存管理系统的状态。 必须分配数据结构以跟踪空闲存储并记录收集垃圾时的进度。 这些数据结构的确切大小和性质随实现的不同而变化，但是许多与堆的大小成正比。

即时（JIT）编译器

JIT编译器在运行时将Java字节码编译为优化的本机可执行代码。 这极大地提高了Java运行时的运行速度，并允许Java应用程序以与本机代码相当的速度运行。

字节码编译使用本机内存（与gcc类的静态编译器需要运行内存的方式相同），但是JIT的输入（字节码）和输出（可执行代码）也必须存储在本机内存中。 包含许多JIT编译方法的Java应用程序比较小的应用程序使用更多的本机内存。

类和类加载器

Java应用程序由定义对象结构和方法逻辑的类组成。 他们还使用Java运行时类库中的类（例如java.lang.String ），并且可能使用第三方库。 只要使用这些类，它们就需要存储在内存中。

类的存储方式因实现而异。 Sun JDK使用永久生成（PermGen）堆区域。 从Java 5开始的IBM实现为每个类加载器分配本机内存，并在其中存储类数据。 现代Java运行时具有诸如类共享之类的技术，这些技术可能需要将共享内存的区域映射到地址空间中。 要了解这些分配机制如何影响Java运行时的本机资源，您需要阅读该实现的技术文档。 但是，一些普遍真理会影响所有实现。

在最基本的级别上，使用更多的类会占用更多的内存。 （这可能意味着您的本机内存使用量刚刚增加，或者您必须显式调整区域的大小（例如PermGen或共享类缓存），以允许所有类都适合。）请记住，不仅您的应用程序需要适合; 框架，应用程序服务器，第三方库和Java运行时包含按需加载并占用空间的类。

Java运行时可以卸载类以回收空间，但只能在严格的条件下进行。 卸载单个类是不可能的。 取而代之的是，卸载类加载器，并使用它们加载的所有类。 仅在以下情况下才能卸载类加载器：

• Java堆不包含对表示该类加载器的java.lang.ClassLoader对象的引用。
• Java堆不包含对任何表示该类加载器加载的类的java.lang.Class对象的引用。
• 由该类加载器加载的任何类的对象都不会在Java堆上处于活动状态（引用）。

值得注意的是，Java运行时为所有Java应用程序创建的三个默认类加载器-bootstrap ， extension和application-永远无法满足这些条件。 因此，不能在运行时释放任何系统类（例如java.lang.String ）或通过应用程序类加载器加载的任何应用程序类。

即使类加载器符合收集条件，运行时也仅在GC周期中收集类加载器。 一些实现仅在某些GC周期上卸载类加载器。

也有可能在没有意识到的情况下在运行时生成类。 许多JEE应用程序使用JavaServer Pages（JSP）技术来生成Web页面。 使用JSP为执行的每个.jsp页面生成一个类，该类将持续加载它们的类加载器的生存期-通常是Web应用程序的生存期。

生成类的另一种常见方法是使用Java反射。 反射的工作方式在Java实现中有所不同，但是Sun和IBM实现都使用我现在将描述的方法。

使用java.lang.reflect API时，Java运行时必须将反射对象的方法（例如java.lang.reflect.Field ）连接到要反射的对象或类。 这可以通过使用Java本机接口（JNI）访问器来完成，该访问器只需要很少的设置但是使用起来很慢，或者通过在运行时为要反映的每种对象类型动态地构建类。 后一种方法设置较慢，但运行速度较快，因此非常适合经常反映特定类的应用程序。

Java运行时在类被反映的前几次使用JNI方法，但是在多次使用后，访问器膨胀为字节码访问器，这涉及构建一个类并通过新的类加载器加载它。 进行大量反射会导致创建许多访问器类和类加载器。 保留对反射对象的引用会使这些类保持活动并继续占用空间。 由于创建字节码访问器的速度很慢，因此Java运行时可以缓存这些访问器以供以后使用。 一些应用程序和框架还缓存反射对象，从而增加其本机覆盖区。

杰尼

JNI允许本机代码（使用本机编译语言（例如C和C ++）编写的应用程序）调用Java方法，反之亦然。 Java运行时本身严重依赖JNI代码来实现类库功能，例如文件和网络I / O。 JNI应用程序可以通过三种方式增加Java运行时的本机资源：

• JNI应用程序的本机代码被编译到共享库或可执行文件中，该共享库或可执行文件已加载到进程地址空间中。 大型本地应用程序仅通过加载就可以占据进程地址空间的很大一部分。
• 本机代码必须与Java运行时共享地址空间。 由本机代码执行的任何本机内存分配或内存映射都会占用Java运行时内存。
• 某些JNI函数可以将本机内存用作其正常操作的一部分。 Get Type ArrayElements和Get Type ArrayRegion函数可以将Java堆数据复制到本机内存缓冲区中，以供本机代码使用。 是否进行复制取决于运行时的实现。 （用于Java 5.0及更高版本的IBM Developer Kit创建了本机副本。）以这种方式访问​​大量Java堆数据可以使用相应数量的本机堆。

蔚来

Java 1.4中添加的新I / O（NIO）类引入了一种基于通道和缓冲区执行I / O的新方法。 除了由Java堆上的内存支持的I / O缓冲区外，NIO还增加了对直接 ByteBuffer的支持（使用java.nio.ByteBuffer.allocateDirect()方法分配），该ByteBuffer由本机内存而不是Java堆支持。 Direct ByteBuffer可以直接传递给本机OS库函数以执行I / O-在某些情况下，它们可以显着加快速度，因为它们可以避免在Java堆和本机堆之间复制数据。

对于直接ByteBuffer数据的存储位置很容易感到困惑。 应用程序仍然使用Java堆上的对象来编排I / O操作，但是保存数据的缓冲区保留在本机内存中-Java堆对象仅包含对本机堆缓冲区的引用。 非直接ByteBuffer将其数据保存在Java堆上的byte[]数组中。 图4显示了直接和非直接ByteBuffer对象之间的区别：

图4.直接和非直接java.nio.ByteBuffer的内存拓扑

直接ByteBuffer对象会自动清除其本机缓冲区，但只能作为Java堆GC的一部分进行清理，因此它们不会自动响应本机堆上的压力。 GC仅在Java堆变得非常满而无法为堆分配请求服务或Java应用程序显式请求该请求时才发生（不建议这样做，因为这会导致性能问题）。

病理情况是本机堆已满，并且一个或多个直接ByteBuffers可以用于GC（并且可以释放以在本机堆上腾出一些空间），但是Java堆大部分为空，因此不会发生GC。

线程数

应用程序中的每个线程都需要内存来存储其堆栈 （调用函数时，用于保存局部变量和维护状态的内存区域）。 每个Java线程都需要堆栈空间才能运行。 根据实现的不同，Java线程可以具有单独的本机堆栈和Java堆栈。 除了堆栈空间外，每个线程还需要一些本机内存以用于线程本地存储和内部数据结构。

堆栈大小因Java实现和体系结构而异。 一些实现允许您指定Java线程的堆栈大小。 典型值为256KB至756KB。

尽管每个线程使用的内存量很小，但是对于具有数百个线程的应用程序，线程堆栈的总内存使用量可能很大。 运行具有比可用处理器更多的线程的线程的应用程序通常效率低下，并且可能导致性能下降以及内存使用增加。

我怎么知道我是否耗尽了本机内存？

Java运行时在处理Java堆用尽方面与在本机堆中用尽方面有很大不同，尽管两种情况都可能出现类似的症状。 当Java堆耗尽时，Java应用程序很难运行，因为Java应用程序不分配对象就很难做任何事情。 一旦Java堆填满，就会产生较差的GC性能和表示完整Java堆的OutOfMemoryError 。

相反，一旦Java运行时启动并且应用程序处于稳定状态，它就可以继续运行，并具有完全的本机堆耗尽功能。 它不一定显示任何奇怪的行为，因为需要本机内存分配的操作比需要Java堆分配的操作少得多。 尽管需要本机内存的操作因JVM的实现而异，但一些流行的示例是：启动线程，加载类以及执行某些类型的网络和文件I / O。

本地内存不足行为也比Java堆栈内存不足行为一致，因为本地内存分配没有单一的控制点。 尽管所有Java堆分配都在Java内存管理系统的控制下，但是任何本机代码（无论是在JVM，Java类库还是应用程序代码内部）都可以执行本机内存分配，并使其失败。 然后，尝试分配的代码可以按设计人员的要求进行处理：它可以通过JNI接口抛出OutOfMemoryError ，在屏幕上打印一条消息，静默失败，然后再试一次，或者执行其他操作。

缺乏可预测的行为意味着没有一种简单的方法可以识别本机内存耗尽。 相反，您需要使用来自OS和Java运行时的数据来确认诊断。

用完本机内存的示例

为了帮助您了解本地内存耗尽如何影响您所使用的Java实现，本文的示例代码（请参见下载 ）包含一些Java程序，这些Java程序以不同的方式触发本地堆耗尽。 这些示例使用用C编写的本机库来消耗所有本机地址空间，然后尝试执行一些使用本机内存的操作。 提供的示例已经构建，尽管示例包的顶级目录中的README.html文件中提供了有关编译示例的说明。

com.ibm.jtc.demos.NativeMemoryGlutton类提供了gobbleMemory()方法，该方法在循环中调用malloc ，直到几乎所有本机内存都用完为止。 完成任务后，将打印分配给标准错误的字节数，如下所示：

Allocated 1953546736 bytes of native memory before running out

已为在32位Windows上运行的Sun和IBM Java运行时捕获了每个演示的输出。 提供的二进制文件已在以下位置进行了测试：

• Linux x86
• Linux PPC 32
• Linux 390 31
• Windows x86

以下版本的Sun Java运行时用于捕获输出：

java version "1.5.0_11"
Java(TM) 2 Runtime Environment, Standard Edition (build 1.5.0_11-b03)
Java HotSpot(TM) Client VM (build 1.5.0_11-b03, mixed mode)

使用的IBM Java运行时版本为：

java version "1.5.0"
Java(TM) 2 Runtime Environment, Standard Edition (build pwi32devifx-20071025 (SR
6b))
IBM J9 VM (build 2.3, J2RE 1.5.0 IBM J9 2.3 Windows XP x86-32 j9vmwi3223-2007100
7 (JIT enabled)
J9VM - 20071004_14218_lHdSMR
JIT  - 20070820_1846ifx1_r8
GC   - 200708_10)
JCL  - 20071025

内存不足时尝试启动线程

当进程地址空间用完时， com.ibm.jtc.demos.StartingAThreadUnderNativeStarvation类尝试启动线程。 这是一种发现Java进程内存不足的常见方法，因为许多应用程序在其整个生命周期中都会启动线程。

在IBM Java运行时上运行时， StartingAThreadUnderNativeStarvation演示的输出为：

Allocated 1019394912 bytes of native memory before running out
JVMDUMP006I Processing Dump Event "systhrow", detail 
"java/lang/OutOfMemoryError" - Please Wait.
JVMDUMP007I JVM Requesting Snap Dump using 'C:\Snap0001.20080323.182114.5172.trc'
JVMDUMP010I Snap Dump written to C:\Snap0001.20080323.182114.5172.trc
JVMDUMP007I JVM Requesting Heap Dump using 'C:\heapdump.20080323.182114.5172.phd'
JVMDUMP010I Heap Dump written to C:\heapdump.20080323.182114.5172.phd
JVMDUMP007I JVM Requesting Java Dump using 'C:\javacore.20080323.182114.5172.txt'
JVMDUMP010I Java Dump written to C:\javacore.20080323.182114.5172.txt
JVMDUMP013I Processed Dump Event "systhrow", detail "java/lang/OutOfMemoryError".
java.lang.OutOfMemoryError: ZIP006:OutOfMemoryError, ENOMEM error in ZipFile.open
   at java.util.zip.ZipFile.open(Native Method)
   at java.util.zip.ZipFile.<init>(ZipFile.java:238)
   at java.util.jar.JarFile.<init>(JarFile.java:169)
   at java.util.jar.JarFile.<init>(JarFile.java:107)
   at com.ibm.oti.vm.AbstractClassLoader.fillCache(AbstractClassLoader.java:69)
   at com.ibm.oti.vm.AbstractClassLoader.getResourceAsStream(AbstractClassLoader.java:113)
   at java.util.ResourceBundle$1.run(ResourceBundle.java:1101)
   at java.security.AccessController.doPrivileged(AccessController.java:197)
   at java.util.ResourceBundle.loadBundle(ResourceBundle.java:1097)
   at java.util.ResourceBundle.findBundle(ResourceBundle.java:942)
   at java.util.ResourceBundle.getBundleImpl(ResourceBundle.java:779)
   at java.util.ResourceBundle.getBundle(ResourceBundle.java:716)
   at com.ibm.oti.vm.MsgHelp.setLocale(MsgHelp.java:103)
   at com.ibm.oti.util.Msg$1.run(Msg.java:44)
   at java.security.AccessController.doPrivileged(AccessController.java:197)
   at com.ibm.oti.util.Msg.<clinit>(Msg.java:41)
   at java.lang.J9VMInternals.initializeImpl(Native Method)
   at java.lang.J9VMInternals.initialize(J9VMInternals.java:194)
   at java.lang.ThreadGroup.uncaughtException(ThreadGroup.java:764)
   at java.lang.ThreadGroup.uncaughtException(ThreadGroup.java:758)
   at java.lang.Thread.uncaughtException(Thread.java:1315)
K0319java.lang.OutOfMemoryError: Failed to fork OS thread
   at java.lang.Thread.startImpl(Native Method)
   at java.lang.Thread.start(Thread.java:979)
   at com.ibm.jtc.demos.StartingAThreadUnderNativeStarvation.main(
StartingAThreadUnderNativeStarvation.java:22)

调用java.lang.Thread.start()尝试为新的OS线程分配内存。 该尝试失败，并引发OutOfMemoryError 。 JVMDUMP行通知用户Java运行时已产生其标准OutOfMemoryError调试数据。

尝试处理第一个OutOfMemoryError导致了第二个— :OutOfMemoryError, ENOMEM error in ZipFile.open的:OutOfMemoryError, ENOMEM error in ZipFile.open 。 当本机进程内存耗尽时，通常会出现多个OutOfMemoryError 。 Failed to fork OS thread消息可能是本机内存不足最常见的信号。

本文提供的示例触发了OutOfMemoryError簇，这些簇比您在自己的应用程序中可能看到的任何簇都严重。 部分原因是因为几乎所有本机内存都已用完，并且与实际应用程序不同，该内存稍后不会释放。 In a real application, as OutOfMemoryError s are thrown, threads shut down and the native-memory pressure is likely to be relieved for a bit, giving the runtime a chance to handle the error. The trivial nature of the test cases also means that whole sections of the class library (such as the security system) have not been initialised yet — and their initialisation is driven by the runtime trying to handle the out-of-memory condition. In a real application, you may see some of the errors shown here, but it's unlikely you will see them all together.

When the same test case is executed on the Sun Java runtime, the following console output is produced:

Allocated 1953546736 bytes of native memory before running out
Exception in thread "main" java.lang.OutOfMemoryError: unable to create new native thread
   at java.lang.Thread.start0(Native Method)
   at java.lang.Thread.start(Thread.java:574)
   at com.ibm.jtc.demos.StartingAThreadUnderNativeStarvation.main(
StartingAThreadUnderNativeStarvation.java:22)

Although the stack trace and the error message are slightly different, the behaviour is essentially the same: the native allocation fails and a java.lang.OutOfMemoryError is thrown. The only thing that distinguishes the OutOfMemoryError s thrown in this scenario from those thrown because of Java-heap exhaustion is the message.

Trying to allocate a direct ByteBuffer when out of native memory

The com.ibm.jtc.demos.DirectByteBufferUnderNativeStarvation class tries to allocate a direct (that is, natively backed) java.nio.ByteBuffer object when the address space is exhausted. When run on the IBM Java runtime, it produces the following output:

Allocated 1019481472 bytes of native memory before running out
JVMDUMP006I Processing Dump Event "uncaught", detail
"java/lang/OutOfMemoryError" - Please Wait.
JVMDUMP007I JVM Requesting Snap Dump using 'C:\Snap0001.20080324.100721.4232.trc'
JVMDUMP010I Snap Dump written to C:\Snap0001.20080324.100721.4232.trc
JVMDUMP007I JVM Requesting Heap Dump using 'C:\heapdump.20080324.100721.4232.phd'
JVMDUMP010I Heap Dump written to C:\heapdump.20080324.100721.4232.phd
JVMDUMP007I JVM Requesting Java Dump using 'C:\javacore.20080324.100721.4232.txt'
JVMDUMP010I Java Dump written to C:\javacore.20080324.100721.4232.txt
JVMDUMP013I Processed Dump Event "uncaught", detail "java/lang/OutOfMemoryError".
Exception in thread "main" java.lang.OutOfMemoryError: 
Unable to allocate 1048576 bytes of direct memory after 5 retries
   at java.nio.DirectByteBuffer.<init>(DirectByteBuffer.java:167)
   at java.nio.ByteBuffer.allocateDirect(ByteBuffer.java:303)
   at com.ibm.jtc.demos.DirectByteBufferUnderNativeStarvation.main(
   DirectByteBufferUnderNativeStarvation.java:29)
Caused by: java.lang.OutOfMemoryError
   at sun.misc.Unsafe.allocateMemory(Native Method)
   at java.nio.DirectByteBuffer.<init>(DirectByteBuffer.java:154)
   ... 2 more

In this scenario, an OutOfMemoryError is thrown that triggers the default error documentation. The OutOfMemoryError reaches the top of the main thread's stack and is printed on stderr .

When run on the Sun Java runtime, this test case produces the following console output:

Allocated 1953546760 bytes of native memory before running out
Exception in thread "main" java.lang.OutOfMemoryError
   at sun.misc.Unsafe.allocateMemory(Native Method)
   at java.nio.DirectByteBuffer.<init>(DirectByteBuffer.java:99)
   at java.nio.ByteBuffer.allocateDirect(ByteBuffer.java:288)
   at com.ibm.jtc.demos.DirectByteBufferUnderNativeStarvation.main(
DirectByteBufferUnderNativeStarvation.java:29)

Debugging approaches and techniques

The first thing to do when faced with a java.lang.OutOfMemoryError or an error message about lack of memory is to determine which kind of memory has been exhausted. The easiest way to do this is to first check if the Java heap is full. If the Java heap did not cause the OutOfMemory condition, then you should analyse the native-heap usage.

Checking your vendor's documentation

The guidelines in this article are general debugging principles applied to understanding native out-of-memory scenarios. Your runtime vendor may supply its own debugging instructions that it expects you to follow when working with its support teams. If you are raising a problem ticket with your runtime vendor (including IBM), always check its debugging and diagnostic documentation to see what steps you are expected to take when submitting a problem report.

Checking the Java heap

The method for checking the heap utilisation varies among Java implementations. On IBM implementations of Java 5 and 6, the javacore file produced when the OutOfMemoryError is thrown will tell you. The javacore file is usually produced in the working directory of the Java process and has a name of the form javacore. date . time . pid .txt. If you open the file in a text editor, you can find a section that looks like this:

0SECTION       MEMINFO subcomponent dump routine
NULL           =================================
1STHEAPFREE    Bytes of Heap Space Free: 416760 
1STHEAPALLOC   Bytes of Heap Space Allocated: 1344800

This section shows how much Java heap was free when the javacore was produced. Note that the values are in hexadecimal format. If the OutOfMemoryError was thrown because an allocation could not be satisfied, then the GC trace section will show this:

1STGCHTYPE     GC History  
3STHSTTYPE     09:59:01:632262775 GMT j9mm.80 -   J9AllocateObject() returning NULL!
32 bytes requested for object of class 00147F80

J9AllocateObject() returning NULL! means that the Java heap allocation routine completed unsuccessfully and an OutOfMemoryError will be thrown.

It is also possible for an OutOfMemoryError to be thrown because the garbage collector is running too frequently (a sign that the heap is full and the Java application will be making little or no progress). In this case, you would expect the Heap Space Free value to be very small, and the GC trace will show one of these messages:

1STGCHTYPE     GC History  
3STHSTTYPE     09:59:01:632262775 GMT j9mm.83 -     Forcing J9AllocateObject()
to fail due to excessive GC

1STGCHTYPE     GC History  
3STHSTTYPE     09:59:01:632262775 GMT j9mm.84 -     Forcing 
J9AllocateIndexableObject() to fail due to excessive GC

When the Sun implementation runs out of Java heap memory, it uses the exception message to show that it is Java heap that is exhausted:

Exception in thread "main" java.lang.OutOfMemoryError: Java heap space

IBM and Sun implementations both have a verbose GC option that produces trace data showing how full the heap is at every GC cycle. This information can be plotted with a tool such as the IBM Monitoring and Diagnostic Tools for Java - Garbage Collection and Memory Visualizer (GCMV) to show if the Java heap is growing.

Measuring native heap usage

If you have determined that your out-of-memory condition was not caused by Java-heap exhaustion, the next stage is to profile your native-memory usage.

The PerfMon tool supplied with Windows allows you to monitor and record many OS and process metrics, including native-memory use. It allows counters to be tracked in real time or stored in a log file to be reviewed offline. Use the Private Bytes counter to show the total address-space usage. If this approaches the limit of the user space (between 2 and 3GB as previously discussed), you should expect to see native out-of-memory conditions.

Linux has no PerfMon equivalent, but you have several alternatives. Command-line tools such as ps , top , and pmap can show an application's native-memory footprint. Although getting an instant snapshot of a process's memory usage is useful, you get a much greater understanding of how native memory is being used by plotting memory use over time. One way to do this is to use GCMV.

GCMV was originally written to plot verbose GC logs, allowing users to view changes in Java heap usage and GC performance when tuning the garbage collector. GCMV was later extended to allow it to plot other data sources, including Linux and AIX native-memory data. GCMV is shipped as a plug-in for the IBM Support Assistant (ISA).

To plot a Linux native-memory profile with GCMV, you must first collect native-memory data using a script. GCMV's Linux native-memory parser reads output from the Linux ps command interleaved with time stamps. A script is provided in the GCMV help documentation that collects data in the correct form. To find the script:

1. Download and install ISA Version 4 (or above) and install the GCMV tool plug-in.
2. Start ISA.
3. Click Help >> Help Contents from the menu bar to open the ISA help menu.
4. Find the Linux native-memory instructions in the left-hand pane under Tool:IBM Monitoring and Diagnostic Tools for Java - Garbage Collection and Memory Visualizer >> Using the Garbage Collection and Memory Visualizer >> Supported Data Types >> Native memory >> Linux native memory .

Figure 5 shows the location of the script in the ISA help file. If you do not have the GCMV Tool entry in your help file, it is most likely you do not have the GCMV plug-in installed.

Figure 5. Location of Linux native memory data capture script in ISA help dialog

The script provided in the GCMV help uses a ps command that only works with recent versions of ps . On some older Linux distributions, the command in the help file will not produce the correct information. To check the behaviour on your Linux distribution, try running ps -o pid,vsz=VSZ,rss=RSS . If your version of ps supports the new command-line argument syntax, the output will look like this:

PID    VSZ   RSS
 5826   3772  1960
 5675   2492   760

If your version of ps does not support the new syntax, the output will look like this:

PID VSZ,rss=RSS
 5826        3772
 5674        2488

If you are running an older version of ps , modify the native memory script replacing the line

ps -p $PID -o pid,vsz=VSZ,rss=RSS

with

ps -p $PID -o pid,vsz,rss

Copy the script from the help panel into a file (in this example called memscript.sh), find the process id (PID) of the Java process you want to monitor (in this example, 1234), and run:

./memscript.sh 1234 > ps.out

This will write the native-memory log into ps.out. To plot memory usage:

1. In ISA, select Analyze Problem from the Launch Activity drop-down menu.
2. Select the Tools tab near the top of the Analyze Problem panel.
3. Select IBM Monitoring and Diagnostic Tools for Java - Garbage Collection and Memory Visualizer .
4. Click the Launch button near the bottom of the tools panel.
5. Click the Browse button and locate the log file. Click OK to launch GCMV.

Once you have the profile of the native-memory use over time, you need to decide whether you are seeing a native-memory leak or are just trying to do too much in the available space. The native-memory footprint for even a well-behaved Java application is not constant from start-up. Several of the Java runtime systems — particularly the JIT compiler and classloaders — initialise over time, which can consume native memory. The memory growth from initialisation will plateau, but if your scenario has an initial native-memory footprint close to the limit of the address space, then this warm-up phase can be enough to cause native out-of-memory. Figure 6 shows an example GCMV native-memory plot from a Java stress test with the warm-up phase highlighted.

Figure 6. Example Linux native-memory plot from GCMV showing warm-up phase

It's also possible to have a native footprint that varies with workload. If your application creates more threads to handle incoming workload or allocates native-backed storage such as direct ByteBuffer s proportionally to how much load is being applied to your system, it's possible that you will run out of native memory under high load.

Running out of native memory because of JVM warm-up-phase native-memory growth, and growth proportional to load, are examples of trying to do too much in the available space. In these scenarios your options are:

• Reduce your native-memory use. Reducing your Java heap size is a good place to start.
• Restrict your native-memory use. If you have native-memory growth that changes with load, find a way to cap the load or the resources that are allocated because of it.
• Increase the amount of address space available to you. You can do this by tuning your OS (increasing your user space with the /3GB switch on Windows or a hugemem kernel on Linux, for example), changing platform (Linux typically has more user space than Windows), or moving to a 64-bit OS .

A genuine native-memory leak manifests as a continual growth in native heap that doesn't drop when load is removed or when the garbage collector runs. The rate of memory leak can vary with load, but the total leaked memory will not drop. Memory that is leaked is unlikely to be referenced, so it can be swapped out and stay swapped out.

When faced with a leak, your options are limited. You can increase the amount of user space (so there is more room to leak into) but that will only buy you time before you eventually run out of memory. If you have enough physical memory and address space, you can allow the leak to continue on the basis that you will restart your application before the process address space is exhausted.

What's using my native memory?

Once you have determined you are running out of native memory, the next logical question is: What's using that memory? Answering this question is hard because, by default, Windows and Linux do not store information about which code path is allocated a particular chunk of memory.

Your first step when trying to understand where your native memory has gone is to work out roughly how much native memory will be used based on your Java settings. An accurate value is difficult to work out without in-depth knowledge of the JVM's workings, but you can make a rough estimate based on the following guidelines:

• The Java heap occupies at least the -Xmx value.
• Each Java thread requires stack space. Stack size varies among implementations, but with default settings, each thread could occupy up to 756KB of native memory.
• Direct ByteBuffer s occupy at least the values supplied to the allocate() routine.

If your total is much less than your maximum user space, you are not necessarily safe. Many other components in a Java runtime could allocate enough memory to cause problems; however, if your initial calculations suggest you are close to your maximum user space, it is likely that you will have native-memory issues. If you suspect you have a native-memory leak or you want to understand exactly where your memory is going, several tools can help.

Microsoft provides the UMDH (user-mode dump heap) and LeakDiag tools for debugging native-memory growth on Windows. They both work in a similar way: recording which code path allocated a particular area of memory and providing a way to locate sections of code that allocate memory that isn't freed later. I refer you to the article "Umdhtools.exe: How to use Umdh.exe to find memory leaks on Windows" for instructions on using UMDH. In this article, I'll focus on what the output of UMDH looks like when run against a leaky JNI application.

The samples pack for this article contains a Java application called LeakyJNIApp ; it runs in a loop calling a JNI method that leaks native memory. The UMDH command takes a snapshot of the current native heap together with native stack traces of the code paths that allocated each region of memory. By taking two snapshots and using the UMDH tool to analyse the difference, you get a report of the heap growth between the two snapshots.

For LeakyJNIApp , the difference file contains this information:

// _NT_SYMBOL_PATH set by default to C:\WINDOWS\symbols
//
// Each log entry has the following syntax:
//
// + BYTES_DELTA (NEW_BYTES - OLD_BYTES) NEW_COUNT allocs BackTrace TRACEID
// + COUNT_DELTA (NEW_COUNT - OLD_COUNT) BackTrace TRACEID allocations
//     ... stack trace ...
//
// where:
//
//     BYTES_DELTA - increase in bytes between before and after log
//     NEW_BYTES - bytes in after log
//     OLD_BYTES - bytes in before log
//     COUNT_DELTA - increase in allocations between before and after log
//     NEW_COUNT - number of allocations in after log
//     OLD_COUNT - number of allocations in before log
//     TRACEID - decimal index of the stack trace in the trace database
//         (can be used to search for allocation instances in the original
//         UMDH logs).
//

+  412192 ( 1031943 - 619751)    963 allocs     BackTrace00468

Total increase == 412192

The important line is + 412192 ( 1031943 - 619751) 963 allocs BackTrace00468 . It shows that one backtrace has made 963 allocations that have not been freed — that have consumed 412192 bytes of memory. By looking in one of the snapshot files, you can associate BackTrace00468 with the meaningful code path. Searching for BackTrace00468 in the first snapshot shows:

000000AD bytes in 0x1 allocations (@ 0x00000031 + 0x0000001F) by: BackTrace00468
        ntdll!RtlpNtMakeTemporaryKey+000074D0
        ntdll!RtlInitializeSListHead+00010D08
        ntdll!wcsncat+00000224
        leakyjniapp!Java_com_ibm_jtc_demos_LeakyJNIApp_nativeMethod+000000D6

This shows the leak coming from the leakyjniapp.dll module in the Java_com_ibm_jtc_demos_LeakyJNIApp_nativeMethod function.

At the time of writing, Linux does not have a UMDH or LeakDiag equivalent. But there are still ways to debug native-memory leaks on Linux. The many memory debuggers available on Linux typically fall into one of the following categories:

• Preprocessor level. These require a header to be compiled in with the source under test. It's possible to recompile your own JNI libraries with one of these tools to track a native memory leak in your code. Unless you have the source code for the Java runtime itself, this cannot find a leak in the JVM (and even then compiling this kind of tool into a large project like a JVM would almost certainly be difficult and time-consuming). Dmalloc is an example of this kind of tool.
• Linker level. These require the binaries under test to be relinked with a debugging library. Again, this is feasible for individual JNI libraries but not recommended for entire Java runtimes because it is unlikely that the runtime vendor would support you running with modified binaries. Ccmalloc is an example of this kind of tool.
• Runtime-linker level. These use the LD_PRELOAD environment variable to preload a library that replaces the standard memory routines with instrumented versions. They do not require recompilation or relinking of source code, but many of them do not work well with Java runtimes. A Java runtime is a complicated system that can use memory and threads in unusual ways that can confuse or break this kind of tool. It is worth experimenting with a few to see if they work in your scenario. NJAMD is an example of this kind of tool.
• Emulator-based. The Valgrind memcheck tool is the only example of this type of memory debugger. It emulates the underlying processor in a similar way to how a Java runtime emulates the JVM. It is possible to run Java under Valgrind, but the heavy performance impact (10 to 30 times slower) means that it would be very hard to run large, complicated Java applications in this manner. Valgrind is currently available on Linux x86, AMD64, PPC 32, and PPC 64. If you use Valgrind, try to narrow down the problem to the smallest test case you can (preferably cutting out the entire Java runtime if possible) before using it.

For simple scenarios that can tolerate the performance overhead, Valgrind memcheck is the most simple and user-friendly of the available free tools. It can provide a full stack trace for code paths that are leaking memory in the same way that UMDH can on Windows.

The LeakyJNIApp is simple enough to run under Valgrind. The Valgrind memcheck tool can print out a summary of leaked memory when the emulated program ends. By default, the LeakyJNIApp program runs indefinitely; to make it shut down after a fixed time period, pass the run time in seconds as the only command-line argument.

Some Java runtimes use thread stacks and processor registers in unusual ways; this can confuse some debugging tools, which expect native programs to abide by standard conventions of register use and stack structure. When using Valgrind to debug leaking JNI applications, you may find that many warnings are thrown up about the use of memory, and some thread stacks will look odd; these are due to the way the Java runtime structures its data internally and are nothing to worry about.

To trace the LeakyJNIApp with the Valgrind memcheck tool, use this command (on a single line):

valgrind --trace-children=yes --leak-check=full 
java -Djava.library.path=. com.ibm.jtc.demos.LeakyJNIApp 10

The --trace-children=yes option to Valgrind makes it trace any processes that are started by Java launcher. Some versions of the Java launcher reexecute themselves (they restart themselves from the beginning, again having set environment variables to change behaviour). If you don't specify --trace-children , you might not trace the actual Java runtime.

The --leak-check=full option requests that full stack traces of leaking areas of code are printed at the end of the run, rather than just summarising the state of the memory.

Valgrind prints many warnings and errors while the command runs (most of which are uninteresting in this context), and finally prints a list of leaking call stacks in ascending order of amount of memory leaked. The end of the summary section of the Valgrind output for LeakyJNIApp on Linux x86 is:

==20494== 8,192 bytes in 8 blocks are possibly lost in loss record 36 of 45
==20494==    at 0x4024AB8: malloc (vg_replace_malloc.c:207)
==20494==    by 0x460E49D: Java_com_ibm_jtc_demos_LeakyJNIApp_nativeMethod
(in /home/andhall/LeakyJNIApp/libleakyjniapp.so)
==20494==    by 0x535CF56: ???
==20494==    by 0x46423CB: gpProtectedRunCallInMethod 
(in /usr/local/ibm-java2-i386-50/jre/bin/libj9vm23.so)
==20494==    by 0x46441CF: signalProtectAndRunGlue 
(in /usr/local/ibm-java2-i386-50/jre/bin/libj9vm23.so)
==20494==    by 0x467E0D1: j9sig_protect 
(in /usr/local/ibm-java2-i386-50/jre/bin/libj9prt23.so)
==20494==    by 0x46425FD: gpProtectAndRun 
(in /usr/local/ibm-java2-i386-50/jre/bin/libj9vm23.so)
==20494==    by 0x4642A33: gpCheckCallin 
(in /usr/local/ibm-java2-i386-50/jre/bin/libj9vm23.so)
==20494==    by 0x464184C: callStaticVoidMethod
(in /usr/local/ibm-java2-i386-50/jre/bin/libj9vm23.so)
==20494==    by 0x80499D3: main 
(in /usr/local/ibm-java2-i386-50/jre/bin/java)
==20494== 
==20494== 
==20494== 65,536 (63,488 direct, 2,048 indirect) bytes in 62 blocks are definitely 
lost in loss record 42 of 45
==20494==    at 0x4024AB8: malloc (vg_replace_malloc.c:207)
==20494==    by 0x460E49D: Java_com_ibm_jtc_demos_LeakyJNIApp_nativeMethod 
(in /home/andhall/LeakyJNIApp/libleakyjniapp.so)
==20494==    by 0x535CF56: ???
==20494==    by 0x46423CB: gpProtectedRunCallInMethod 
(in /usr/local/ibm-java2-i386-50/jre/bin/libj9vm23.so)
==20494==    by 0x46441CF: signalProtectAndRunGlue 
(in /usr/local/ibm-java2-i386-50/jre/bin/libj9vm23.so)
==20494==    by 0x467E0D1: j9sig_protect 
(in /usr/local/ibm-java2-i386-50/jre/bin/libj9prt23.so)
==20494==    by 0x46425FD: gpProtectAndRun 
(in /usr/local/ibm-java2-i386-50/jre/bin/libj9vm23.so)
==20494==    by 0x4642A33: gpCheckCallin 
(in /usr/local/ibm-java2-i386-50/jre/bin/libj9vm23.so)
==20494==    by 0x464184C: callStaticVoidMethod
(in /usr/local/ibm-java2-i386-50/jre/bin/libj9vm23.so)
==20494==    by 0x80499D3: main 
(in /usr/local/ibm-java2-i386-50/jre/bin/java)
==20494== 
==20494== LEAK SUMMARY:
==20494==    definitely lost: 63,957 bytes in 69 blocks.
==20494==    indirectly lost: 2,168 bytes in 12 blocks.
==20494==      possibly lost: 8,600 bytes in 11 blocks.
==20494==    still reachable: 5,156,340 bytes in 980 blocks.
==20494==         suppressed: 0 bytes in 0 blocks.
==20494== Reachable blocks (those to which a pointer was found) are not shown.
==20494== To see them, rerun with: --leak-check=full --show-reachable=yes

The second line of the stacks shows that the memory was leaked by the com.ibm.jtc.demos.LeakyJNIApp.nativeMethod() method.

Several proprietary debugging applications that can debug native-memory leaks are also available. More tools (both open-source and proprietary) are being developed all the time, and it's worth researching the current state of the art.

Currently debugging native-memory leaks on Linux with the freely available tools is more challenging than doing the same on Windows. Whereas UMDH allows native leaks on Windows to be debugged in situ , on Linux you will probably need to do some traditional debugging rather than rely on a tool to solve the problem for you. Here are some suggested debugging steps:

• Extract a test case. Produce a stand-alone environment that you can reproduce the native leak with. It will make debugging much simpler.
• Narrow the test case as far as possible. Try stubbing out functions to identify which code paths are causing the native leak. If you have your own JNI libraries, try stubbing them out entirely one at a time to determine if they are causing the leak.
• Reduce the Java heap size. The Java heap is likely to be the largest consumer of virtual address space in the process. By reducing the Java heap, you make more space available for other users of native memory.
• Correlate the native process size. Once you have a plot of native-memory use over time, you can compare it to application workload and GC data. If the leak rate is proportional to the level of load, it suggests that the leak is caused by something on the path of each transaction or operation. If the native process size drops significantly when a GC happens, it suggests that you are not seeing a leak — you are seeing a buildup of objects with a native backing (such as direct ByteBuffer s). You can reduce the amount of memory held by native-backed objects by reducing the Java heap size (thereby forcing collections to occur more frequently) or by managing them yourself in an object cache rather than relying on the garbage collector to clean up for you.

If you identify a leak or memory growth that you think is coming out of the Java runtime itself, you may want to engage your runtime vendor to debug further.

Removing the limit: Making the change to 64-bit

It is easy to hit native out-of-memory conditions with 32-bit Java runtimes because the address space is relatively small. The 2 to 4GB of user space that 32-bit OSs provide is often less than the amount of physical memory attached to the system, and modern data-intensive applications can easily scale to fill the available space.

If your application cannot be made to fit in a 32-bit address space, you can gain a lot more user space by moving to a 64-bit Java runtime. If you are running a 64-bit OS, then a 64-bit Java runtime will open the door to huge Java heaps and fewer address-space-related headaches. Table 2 lists the user spaces currently available with 64-bit OSs:

Table 2. User space sizes on 64-bit OSs

操作系统	Default user space size
Windows x86-64	8192GB
Windows Itanium	7152GB
Linux x86-64	500GB
Linux PPC64	1648GB
Linux 390 64	4EB

Moving to 64-bit is not a universal solution to all native-memory problems, however; you still need sufficient physical memory to hold all of your data. If your Java runtime won't fit in physical memory then performance will be intolerably poor because the OS is forced to thrash Java runtime data back and forth from swap space. For the same reason, moving to 64-bit is no permanent solution to a memory leak — you are just providing more space to leak into, which will only buy time between forced restarts.

It's not possible to use 32-bit native code with a 64-bit runtime; any native code (JNI libraries, JVM Tool Interface [JVMTI], JVM Profiling Interface [JVMPI], and JVM Debug Interface [JVMDI] agents) must be recompiled for 64-bit. A 64-bit runtime's performance can also be slower than the corresponding 32-bit runtime on the same hardware. A 64-bit runtime uses 64-bit pointers (native address references), so the same Java object on 64-bit takes up more space than an object containing the same data on 32-bit. Larger objects mean a bigger heap to hold the same amount of data while maintaining similar GC performance, which makes the OS and hardware caches less efficient. Surprisingly, a larger Java heap does not necessarily mean longer GC pause times, because the amount of live data on the heap might not have increased, and some GC algorithms are more effective with larger heaps.

Some modern Java runtimes contain technology to mitigate 64-bit "object bloat" and improve performance. These features work by using shorter references on 64-bit runtimes. This is called compressed references on IBM implementations and compressed oops on Sun implementations.

A comparative study of Java runtime performance is beyond this article's scope, but if you are considering a move to 64-bit it is worth testing your application early to understand how it performs. Because changing the address size affects the Java heap, you will need to retune your GC settings on the new architecture rather than just port your existing settings.

结论

An understanding of native memory is essential when you design and run large Java applications, but it's often neglected because it is associated with the grubby hardware and OS details that the Java runtime was designed to save us from. The JRE is a native process that must work in the environment defined by these grubby details. To get the best performance from your Java application, you must understand how the application affects the Java runtime's native-memory use.

Running out of native memory can look similar to running out of Java heap, but it requires a different set of tools to debug and solve. The key to fixing native-memory issues is to understand the limits imposed by the hardware and OS that your Java application is running on, and to combine this with knowledge of the OS tools for monitoring native-memory use. By following this approach, you'll be equipped to solve some of the toughest problems your Java application can throw at you.

---

翻译自: https://www.ibm.com/developerworks/java/library/j-nativememory-linux/index.html