c++重新定义数据结构_结构类别的重新定义

c++重新定义数据结构

介绍 (Introduction)

The Android Runtime (ART) in Android 11 introduces an extension called Structural Class Redefinition to the JVMTI API. This post will cover the capabilities of structural redefinition, as well as some of the considerations and trade-offs we encountered while implementing the feature and how we solved them. Structural class redefinition is a runtime feature that extends the class-redefinition functionality added in Android 8. Structural class redefinition allows tools, such as the Android Studio Apply Changes, to modify the class structure itself, allowing one to add new fields and methods to classes (the older ‘normal’ class redefinition limits the changes to the implementations of methods already present in the class).

Android 11中的Android运行时(ART)向JVMTI API引入了一个称为“结构类重新定义”的扩展。 这篇文章将介绍结构性重新定义的功能，以及在实现此功能时遇到的一些注意事项和权衡取舍以及我们如何解决它们。 结构化类重定义是一种运行时功能，它扩展了Android 8中添加的类重新定义功能。结构化类重定义允许诸如Android Studio Apply Changes之类的工具修改类结构本身，从而允许向类中添加新的字段和方法。 (较旧的“普通”类重新定义将更改限制在该类中已经存在的方法的实现上)。

This can be leveraged into great features such as extending apply changes to support adding new resources to apps. You can read about how the Android Studio ‘Apply Changes’ functionality works here and in a future blog post about how it was extended using structural class redefinition coming soon. More complex and powerful tools are being built into Android Studio to take advantage of these new features.

这可以被利用为强大的功能，例如扩展应用程序更改以支持向应用程序添加新资源。 您可以在此处阅读有关Android Studio“应用更改”功能的工作原理，并在以后的博客文章中了解有关如何使用结构化类重新定义对其进行扩展的信息。 为了利用这些新功能，Android Studio内置了更复杂，功能更强大的工具。

JVMTI is a standard API developer tools can use to interact and control the runtime at a very deep level. It is used to implement many developer tools you are likely familiar with, from the Android Studio Network and Memory profilers to the debugger to mocking frameworks like dexmaker-mockito-inline and MockK to the new Layout and Database inspectors. You can find out more about the Android JVMTI implementation and how to use it for your own tools from the android documentation.

JVMTI是标准的API开发人员工具，可用于在很深的层次上进行交互和控制运行时。 它用于实现许多您可能熟悉的开发人员工具，从Android Studio 网络和内存分析器到调试器，再到诸如dexmaker-mockito-inline和MockK之类的模拟框架， 再到新的Layout和Database检查器。 您可以从android文档中找到有关Android JVMTI实现以及如何将其用于自己的工具的更多信息 。

结构的重新定义 (Structural Redefinition)

Structural class redefinition improves the support for class redefinition added in Android Oreo (8.0). In Oreo, only the definitions of existing methods could be changed. The object layout and the sets of fields and methods a class defines could not be changed in any way.

结构化类重定义改进了对Android Oreo(8.0)中添加的类重定义的支持。 在Oreo中，只能更改现有方法的定义。 类定义的对象布局以及字段和方法集不能以任何方式更改。

Structural class redefinition allows much more freedom in modifying classes, enabling one to add entirely new methods and fields to existing classes. There are no restrictions on what types of methods and fields may be added. Newly added fields are initialized to 0 or null, but agents can use other JVMTI functionality to initialize them, if desired. As with standard class redefinition, methods currently executing will continue to use their old definition, though subsequent calls to them will use the new definition. In order to ensure structural redefinition can have clear and consistent semantics the following changes are not allowed:

结构化类的重新定义为修改类提供了更大的自由度，使人们可以向现有类中添加全新的方法和字段。 对于可以添加哪种类型的方法和字段没有任何限制。 新添加的字段被初始化为0或null，但是如果需要，代理可以使用其他JVMTI功能对其进行初始化。 与标准类重新定义一样，当前执行的方法将继续使用其旧定义，尽管对它们的后续调用将使用新定义。 为了确保结构重定义可以具有清晰一致的语义，不允许进行以下更改：

• Fields and methods may not be removed or have their attributes changed
  字段和方法可能不会被删除或属性已更改
• Classes may not change their name
  班级可能不会更改其名称
• Class hierarchy (superclass and implemented interfaces) may not be changed
  类层次结构(超类和已实现的接口)可能不会更改

Combined with support from Android Studio structural class redefinition can be used to implement Apply Changes for most common edits. The rest of this post will cover part of how we went about implementing this feature and some of the considerations and tradeoffs we make when implementing new runtime features.

结合Android Studio结构类重定义的支持，可用于对大多数常见编辑实施“应用更改”。 这篇文章的其余部分将介绍我们如何实现此功能的部分内容，以及实现新的运行时功能时需要进行的一些考虑和权衡。

一，无害 (First, do no harm)

The primary challenge of structural redefinition is to implement it without affecting apps running in release mode. For every developer running their code in debuggable mode and using tools such as Apply Changes or the debugger there are millions of users running these apps normally on their phones. Therefore, one of the paramount requirements for any new developer feature in ART is to not affect runtime performance when the app is not debuggable. This means that we are not able to make major changes to the core internals of the runtime. We cannot change things such as the basic layout, allocation, or garbage collection of objects, how classes are linked and loaded, or how dex-code is executed.

结构性重新定义的主要挑战是在不影响以发布模式运行的应用程序的情况下实现它。 对于每个在可调试模式下运行其代码并使用诸如“应用更改”或调试器之类的开发人员来说，数百万的用户通常会在其手机上运行这些应用程序。 因此，ART中任何新开发人员功能的首要要求之一是在应用不可调试时不影响运行时性能。 这意味着我们无法对运行时的核心内部进行重大更改。 我们不能更改诸如对象的基本布局，分配或垃圾收集，如何链接和加载类或如何执行dex代码之类的事情。

Objects, including java.lang.Class objects (which contain their class’s static-fields in ART), have their size and layout determined at the time they are loaded. This allows for very efficient execution since, for example in the ‘Parrot’ class shown in the image below, we always know that the `piningFor` field is contained at offset `0x8` of any Parrot. This means that ART is able to generate efficient code but makes it impossible to alter the layout of objects after they are created, since by adding new fields we change the layout of not just objects of that class but also all of its subtypes. To implement this feature what we do instead is transparently and atomically replace all the old classes and instances with their redefined counterparts.

对象(包括java.lang.Class对象(在ART中包含其类的静态字段))的大小和布局在加载时确定。 因为例如在下图所示的'Parrot'类中，我们始终知道'piningFor'字段包含在任何Parrot的偏移量'0x8'处，所以这允许非常高效的执行。 这意味着ART能够生成高效的代码，但无法在创建对象后更改对象的布局，因为通过添加新字段，我们不仅更改了该类对象的布局，还更改了其所有子类型的布局。 为了实现此功能，我们要做的是透明地和原子地将所有旧类和实例替换为其重新定义的对应类。

In order to provide structural class redefinition in a way that does not degrade performance we need to reach deep into the internals of the runtime. Fundamentally, doing a structural redefinition of a class has 4 main steps.

为了以不降低性能的方式提供结构类的重新定义，我们需要深入了解运行时的内部。 从根本上讲，对类进行结构性重新定义有四个主要步骤。

1. Create new java.lang.Class objects for all types being changed, with the new class definition.
   使用新的类定义为要更改的所有类型创建新的java.lang.Class对象。
2. Recreate all objects of the old types with the newly redefined type.
   用新定义的类型重新创建所有旧类型的对象。
3. Replace/update all the old objects with their corresponding new objects.
   用其对应的新对象替换/更新所有旧对象。
4. Ensure all compiled code and runtime state is correct with respect to the new type layout.
   确保所有编译代码和运行时状态相对于新类型布局而言都是正确的。

追逐表现 (Chasing Performance)

Like many programs ART is inherently multithreaded, both due to the (potentially) multi-threaded nature of the dex-code it runs and in order to avoid runtime pauses. At any point the runtime might be doing any number of things at once: running java language code, performing garbage-collection, loading classes, allocating objects, running finalizers or many other things.

像许多程序一样，ART本身就是多线程的，这既是由于它运行的dex代码的(潜在)多线程特性，也是为了避免运行时暂停。 在任何时候，运行时都可能一次执行许多操作：运行Java语言代码，执行垃圾回收，加载类，分配对象，运行终结器或许多其他事情。

This means that doing redefinition in a naive way has obvious races. For example what if, after we had already recreated all the old objects a new instance was created? We therefore need to be extremely careful about how we go about performing each of the steps, to ensure that nothing is ever able to see or create an inconsistent state. We need to ensure that every thread sees the transformation shown in the diagram above happens atomically, all at once.

这意味着以幼稚的方式进行重新定义会产生明显的竞争。 例如，如果在我们重新创建了所有旧对象之后创建了新实例，该怎么办？ 因此，我们必须非常谨慎地执行每个步骤，以确保没有人能够看到或创建不一致的状态。 我们需要确保每个线程都能看到上图所示的转换原子地，一次性地发生。

The naive solution to this would be to stop everything dead as soon as we start redefining something. We then perform the redefinition in the way described above (create new classes and objects then replace the old ones). This has the benefit of giving us the atomicity we need without any real effort. All code is paused during any time when inconsistencies could be observed so nothing can be seen. Unfortunately, there are several problems with this approach.

天真的解决方案是在我们开始重新定义某些东西时立即停止一切死亡。 然后，我们以上述方式执行重新定义(创建新的类和对象，然后替换旧的类和对象)。 这样做的好处是无需任何实际努力就可以为我们提供所需的原子性。 在发现不一致的任何时候，所有代码都会暂停，因此什么也看不到。 不幸的是，这种方法存在一些问题。

For one it would be horrendously slow. There might be large numbers of objects to recreate and classes to reload (for example one might want to edit the java.util.ArrayList class which can typically have thousands of instances at any one time). A more pressing issue is that allocation is not possible with all threads stopped. This is to prevent deadlocks where, for example we wait for a paused GC thread to finish work before allocating something. This restriction is embedded deep into the design of ART and its GC. Simply changing it to remove this restriction is not feasible, especially for a feature that is only used in debugging contexts. Since a major part of structural redefinition is re-allocating all the redefined objects this is obviously not acceptable.

首先，它会非常缓慢。 可能有大量要重新创建的对象和要重新加载的类(例如，一个人可能想要编辑java.util.ArrayList类，该类通常可以同时具有数千个实例)。 更为紧迫的问题是不可能在所有线程停止的情况下进行分配。 这是为了防止出现死锁，例如，在分配某些内容之前，我们等待已暂停的GC线程完成工作。 此限制已深深植入ART及其GC的设计中。 简单地更改它以消除此限制是不可行的，特别是对于仅在调试上下文中使用的功能。 由于结构重定义的主要部分是重新分配所有重新定义的对象，因此这显然是不可接受的。

So what do we do now? We still need to ensure that, as far as all Java Language code is concerned the change happens instantly but we cannot stop the world as we do so. Here we can take advantage of a restriction of the Java Language, the heap and significant class-loading state is not directly observable by threads and that important GC management threads will never allocate or load classes.

那么，我们现在该怎么办？ 就所有Java语言代码而言，我们仍然需要确保更改会立即发生，但我们不能因此而停步。 在这里，我们可以利用Java语言的限制，线程无法直接观察到堆和重要的类加载状态，并且重要的GC管理线程将永远不会分配或加载类。

This means that the only step we need to actually pause the rest of the runtime for is the replacement process. We can allocate all of the classes and new objects while other code is still running. Since none of those threads have any references to the new objects and the code being run is still the original no inconsistencies will be visible.

这意味着我们实际上需要暂停其余运行时间的唯一步骤是替换过程。 我们可以在其他代码仍在运行时分配所有类和新对象。 由于这些线程都没有对新对象的任何引用，并且所运行的代码仍然是原始代码，因此不会看到任何不一致之处。

For anyone interested in seeing how this is all actually implemented we have linked to some of the relevant pieces of the implementation. These link to the Android Code Search interface where you can explore how Android and AOSP are created.

对于有兴趣了解这是如何真正实现的任何人，我们已将其链接到实现的一些相关部分。 这些链接到Android代码搜索界面，您可以在其中探索如何创建Android和AOSP。

Since we are allowing app code to continue running, we need to be careful that the state of the world doesn’t change out from under us. To do this we have to carefully shut down various pieces of the runtime in sequence in order to ensure we can collect all the information we need and it is not invalidated while running. To accomplish this we need, at the instant of redefinition, to have a complete list of the java.lang.Class objects of the class being redefined¹ and all of its subclasses, a corresponding list of all the redefined Class objects, a complete list of all the instances of that class and a corresponding list of all the redefined objects.

由于我们允许应用程序代码继续运行，因此我们需要注意，世界的状况不会因我们而改变。 为此，我们必须按顺序仔细关闭运行时的各个部分，以确保我们可以收集所需的所有信息，并且在运行时该信息不会失效。 为此，在重新定义时，我们需要具有要重新定义的类的java.lang.Class对象及其所有子类的完整列表，以及所有已重新定义的Class对象的对应列表，以及重新定义的Class对象的完整列表。该类的所有实例以及所有重新定义的对象的对应列表。

Since loading new classes is rare (and we need the new Class objects in order to allocate the redefined instances) we can start by figuring out the list of classes being redefined and creating new Class objects for the redefined types. To ensure that this list remains valid and complete we need to completely stop class loading before we create this list². To do this we need to both stop new class loads from starting and wait for in-progress class definitions to finish. Once this is done we can safely collect and recreate all the redefined class objects.

由于很少加载新类(并且需要新的Class对象以分配重新定义的实例)，因此我们可以从确定要重新定义的类的列表并为重新定义的类型创建新的Class对象开始。 为确保此列表有效并完整，我们需要在创建此列表之前完全停止类加载 ²。 为此，我们既需要停止启动新的类，又要等待正在进行的类定义完成 。 完成此操作后，我们就可以安全地收集并重新创建所有重新定义的类对象。

Now that we have all the classes collected we need to find and recreate the instances that need to be replaced. Similar to what we did with classes, we need to pause allocations and wait for all threads to acknowledge it in order to ensure our list of objects doesn’t get stale³. Again like with classes we then simply collect all the old instances and create new versions of each of them.

现在，我们已经收集了所有类，我们需要查找并重新创建需要替换的实例。 与我们对类所做的类似，我们需要暂停分配并等待所有线程确认它 ，以确保我们的对象列表不会过时。 再次与类类似，然后我们仅收集所有旧实例并为每个旧实例 创建新版本 。

Now that we have all the new objects all that is left to do is copy the field values from the old objects and actually perform the replacement. Since once we start giving threads or objects references to the replacements they will no longer be unobservable and threads while running can arbitrarily change any fields we need to finally stop all threads before we perform these last few steps. Now that all other threads have been stopped we can copy the field values from the old to the new objects.

现在我们有了所有新对象，剩下要做的就是复制旧对象中的字段值并实际执行替换。 因为一旦我们开始为线程或对象提供对替换的引用，它们将不再是不可观察的，并且在运行时线程可以任意更改任何字段，因此我们需要在执行最后几步之前最终停止所有线程 。 现在所有其他线程都已停止，我们可以将字段值从旧对象复制到新对象 。

Once this is done we can actually walk through the heap and replace all the old instances with the new redefined instances. All that is left now is doing some miscellaneous work to ensure that things such as reflection objects and all the various runtime resolution caches are updated or cleared as needed. We also make sure to keep track of enough data to allow all the code running when redefinition started to continue running.

完成此操作后，我们实际上可以遍历堆并将所有旧实例替换为新的重新定义的实例 。 现在剩下的就是做一些杂项工作，以确保诸如反射对象和所有各种运行时分辨率缓存之类的东西根据需要进行更新或清除。 我们还确保跟踪足够的数据，以便在重新定义开始时继续运行所有代码。

结论 (Conclusion)

With the capabilities of structural redefinition many brand new, more powerful debugging and development tools are possible. We have already talked about the improvements to apply-changes and many other teams in Android are looking to use this to make other great tools. This is just one part of the many improvements and new features we add in every Android release. You might also want to read our recent blog-post about how we improved launch time for apps in Android 11 using IO prefetching.

通过结构重新定义的功能，可以使用许多全新的，功能更强大的调试和开发工具。 我们已经讨论过应用变更的改进，Android中的许多其他团队正在寻求使用此功能来开发其他出色的工具。 这只是我们在每个Android版本中添加的许多改进和新功能的一部分。 您可能还想阅读我们最近的博客文章，其中介绍了我们如何使用IO预取缩短了Android 11中应用程序的启动时间 。

[1] Before doing this we perform some checks to make sure that all classes are even eligible for redefinition and the new definitions are valid but these validations are not terribly interesting.

[1]在执行此操作之前，我们执行一些检查以确保所有类甚至都可以重新定义并且新定义有效，但是这些验证并不是十分有趣。

[2] Technically, it is safe for unrelated classes to continue loading but due to the way class loading works there is no way to distinguish these cases early enough to be useful.

[2]从技术上讲，对于不相关的类，继续加载是安全的，但是由于类加载的工作方式，因此无法尽早区分出这些情况以至有用。

[3] Again details of how allocation interacts with cross-thread synchronization in ART prevents us from only pausing allocations that are instances of our redefined classes.

[3]再次详细介绍了分配如何与ART中的跨线程同步进行交互，这阻止了我们仅暂停作为重新定义类实例的分配。

翻译自: https://medium.com/androiddevelopers/structural-class-redefinition-6fc0cbab9161

c++重新定义数据结构