新的QtQuick编译技术

The new QtQuick Compiler technology

新的QtQuick编译技术

Monday January 17, 2022 by Ulf Hermann | Comments

2022年1月17日星期一，Ulf Hermann | 评论

The new Qt Quick Compiler technology

新的Qt Quick编译技术

It's been a while since we've heard about what goes on inside and around Qt QML, our engine to interpret the QML language (not counting the recent announcement, that is). The last post strictly about this topic was what Lars wrote in 2018.

​我们已经有一段时间没有听说Qt-QML内部和周围的消息了，Qt-QML是我们解释QML语言的引擎（不包括最近的发布的版本）。关于这个话题的最后一篇文章是Lars在2018年写的。

We've been so silent because we've been prototyping new ways to make your QML run faster, and some of them turned out to be dead ends. There is no tracing JIT after all. This isn't cool, so we were somewhat silent. But now there is something to say. And, mind you, it's not cool either. It's hot. But let me take a step back first.

我们之所以一直保持沉默，是因为我们一直在设计新方法的原型，以使您的QML运行得更快，而其中一些方法最终被证明是死路一条。毕竟没有跟踪JIT。这不酷，所以我们有点沉默。但现在有话要说。而且，请注意，这也不酷。天气很热。但让我先退一步。

The Cold: Old Qt Quick Compiler

旧的一面：旧的Qt快速编译器

If you've been around for a while, you may have heard the term "Qt Quick Compiler" before. There used to be a qmlcompiler tool that compiled QML script code to C++. It first shipped with Qt 5.3. In our archives you can still find some of its documentation. What did it do? Quite simply, it compiled QML script code into a C++ representation that "remote controlled" the QML engine through private API. This way, the QML engine didn't have to touch the QML source code anymore, which lead to greatly improved startup performance.

​如果您已经使用QML有一段时间了，您可能以前听过“Qt Quick编译器”这个术语。曾经有一个QML编译器工具，它将QML脚本代码编译成C++。它首先随Qt5.3一起提供。在我们的历史档案中，你仍然可以找到它的一些文档。它做了什么？很简单，它将QML脚本代码编译成C++表示，通过私有API远程控制QML引擎。这样，QML引擎就不必再接触QML源代码，从而大大提高了启动性能。

This tool was later replaced by the byte code caching technology introduced in Qt 5.8 and 5.9, as that eventually did a better job at improving QML performance. How does this caching work? When interpreting a QML file from scratch, without any caching or other preprocessing, the QML engine will parse the source code, and produce a compilation unit from it. The compilation unit contains a compact byte code representation for each function and expression. The interpreter then uses this byte code as needed to evaluate bindings and execute signal handlers or JavaScript functions. Furthermore, the JIT can compile the byte code to native machine code for improved performance of hot functions. The cache retains the byte code across invocations of your program, so that the source code doesn't have to be parsed and analyzed over and over. The qmlcachegen tool does this at compile time and causes the byte code to be bundled into your binary.

这个工具后来被Qt5.8和5.9中引入的字节码缓存技术所取代，因为它最终在提高QML性能方面做得更好。这种缓存是如何工作的？当从头开始解释QML文件时，没有任何缓存或其他预处理，QML引擎将解析源代码，并从中生成编译单元。编译单元包含每个函数和表达式的紧凑字节码表示。然后，解释器根据需要使用该字节码来评估绑定并执行信号处理器或JavaScript函数。此外，JIT可以将字节码编译为本机代码，以提高热函数的性能。缓存在程序调用过程中保留字节码，因此不必反复解析和分析源代码。qmlcachegen工具在编译时执行此操作，并将字节码绑定到二进制文件中。

If you look at those two diagrams, you may wonder. In the first one we don't have to run any interpreter or JIT if we just compile all our QML files ahead of time. This should be simpler and faster, shouldn't it? So, why did we drop the direct compilation to C++?

如果你看这两张图，你可能会想。在第一个例子中，如果我们只是提前编译所有QML文件，就不必运行任何解释器或JIT。这应该更简单更快，不是吗？那么，为什么我们不直接编译成C++？

There are two dimensions to "fast" here: You not only want your QML components to load quickly, but you also want each function and expression to take the least possible time to execute. The old Qt Quick Compiler was supposed to address both aspects. However, it turned out that the new interpreter and JIT introduced in Qt 5.11 did a better job at optimizing the run time aspect. Now you may ask how the interpreter or the JIT can be faster than native machine code compiled from C++ as generated by qmlcompiler. We did ask ourselves the same question. It's important to remember that JavaScript is a heavily dynamic language. A function in JavaScript makes no promises whatsoever on the types of arguments that may be passed in or returned. At run time, we do have some indication about the types because we can introscpect the QObjects at play and check what kinds of properties and methods they offer. However, in early Qt 5 none of this information was available at compile time. Therefore, the old Qt Quick Compiler had to generate extremely generic code that worked with the equivalent of a QVariant for each and every value it manipulated. In order to do anything meaningful, the resulting code had to check the current type of these values at every step. This was slow. When interpreting, on the other hand, we could use our run time knowledge of the Qt metatype system to avoid some of the type checking overhead. This made the new interpreter and JIT eventually surpass the C++ code generated by the old Qt Quick Compiler.

这里的“快速”有两个维度：您不仅希望快速加载QML组件，还希望每个函数和表达式的执行时间尽可能短。旧的Qt快速编译器支持解决这两个方面的问题。然而，事实证明，Qt5.11中引入的新解释器和JIT在优化运行时方面做得更好。现在，您可能会问，编译器或JIT如何比由QML编译器生成的来自C++的本机代码更快。我们也问过自己同样的问题。记住JavaScript是一种重度动态的语言，这一点很重要。JavaScript中的函数不会对传入或返回的参数类型做出任何承诺。在运行时，我们确实有一些关于类型的指示，因为我们可以介绍正在使用的QObject，并检查它们提供了什么类型的属性和方法。然而，在Qt5早期，这些信息在编译时都不可用。因此，旧的Qt Quick编译器必须生成极其通用的代码，对于它所操作的每个值，该代码都与QVariant等效。为了做任何有意义的事情，生成的代码必须在每一步检查这些值的当前类型。这很慢。另一方面，在解释时，我们可以使用Qt元类型系统的运行时知识来避免一些类型检查开销。这使得新的解释器和JIT最终超过了由旧Qt Quick编译器生成的C++代码。

So, that was that, the old Qt Quick Compiler had hit a dead end. What could we do to further improve on QML performance?

就是说，旧的Qt Quick编译器已经走到了死胡同。我们可以做些什么来进一步改进QML性能？

The Cool: Declaring type information ahead of time

新的一面：提前声明类型信息

In order to generate better C++ code ahead of time, any QML-to-C++ compiler needs to know what kind of objects it's dealing with. If we know that a and b are integers, then the expression a + b becomes a much simpler affair than if we have to assume they are strings, objects, arrays, or anything else. There is one facility commonly used to provide type information to QML at run time:

为了提前生成更好的C++代码，任何QML到C++编译器都需要知道它处理的对象是什么类型。如果我们知道a和b是整数，那么表达式a+b就变得比假设它们是字符串、对象、数组或其他任何东西都简单得多。有一个工具通常用于在运行时向QML提供类型信息：

qmlRegisterType<MyType>("Some.Module", 3, 12, "T");

Here, you declare that your type MyType shall be available to QML with all its properties and methods. If MyType then has some properties a and b, we can determine their types by looking at its QMetaObject. At run time, we just process those registration calls as they come in. In order to do it at compile time we have to know in advance what types will have been registered at the time a QML file is interpreted. As the registration calls happen anywhere in generic C++ code, this problem is equivalent to the halting problem. Bad news.

​在这里，您声明您的类型MyType及其所有属性和方法应可供QML使用。如果MyType有一些属性a和b，我们可以通过查看它的QMetaObject来确定它们的类型。在运行时，我们只在注册调用传入时处理它们。为了在编译时执行此操作，我们必须提前知道在解释QML文件时将注册哪些类型。当注册调用发生在通用C++代码中的任何地方时，这个问题就等同于暂停问题。坏消息。

Yet, a large number of types is in fact well known. We know what properties a Rectangle has, and what a Timer is. Why do we know? Because we always register those types with the same names at the same places. So, we formalized this approach and provided a way to declare your QML types together with the C++ types backing them. The relevant facilities were provided in Qt 5.15 and you can read up on them at the blog post written back then.

​然而，事实上，许多类型是已知的。我们知道矩形Rectangle有什么属性，计时器Timer是什么。为什么我们知道？因为我们总是在相同的地方用相同的名字注册这些类型。因此，我们对这种方法进行了形式化，并提供了一种方法来声明您的QML类型和支持它们的C++类型。Qt5.15中提供了相关的工具，您可以在当时写的博客文章中阅读它们。

In addition, Qt 5.14 introduced the possibility to provide type information in function signatures, similar to how it is done in TypeScript:

此外，Qt 5.14引入了在函数签名中提供类型信息的可能性，类似于在TypeScript中的实现方式：

function add(a: int, b: int) : int { return a + b }

With these tools at hand, we could try the QML-to-C++ route again. Or so we thought. Something was missing, unfortunately.

有了这些工具，我们可以再次尝试QML到C++的路线。至少我们是这么想的。不幸的是，有些东西不见了。

The Warm: Organizing QML components into modules

警告：将QML组件组织到模块中

In order to know what QML files relate to which other QML files and to which C++ code, we also need to know the file system locations and import paths of all the bits and pieces at compile time. In Qt5, however, the only requirement for using qmlcachegen was that the QML files be added added to the resource file system. In the resource file system, you could map them to any paths you liked. Furthermore, you didn't need to tell anyone where those nice C++-declared QML types would end up and whether they would be accessible by your QML code. You didn't even need to declare what QML module a QML file would belong to, and therefore what types it would import implicitly. For all of this, interesting and brittle heuristics can be written, but ultimately a proper solution had to be found.

为了知道QML文件与哪些其他QML文件以及C++代码有关，我们还需要知道编译时所有文件和文件的位置和导入路径。但是，在Qt5中，使用qmlcachegen的唯一要求是将QML文件添加到资源文件系统中。在资源文件系统中，您可以将它们映射到您喜欢的任何路径。此外，您不需要告诉任何人这些C++明确声明的QML类型，将在哪里结束，以及它们是否可以被您的QML代码访问。您甚至不需要声明QML文件将属于哪个QML模块，也不需要声明它将隐式导入的类型。对于所有这些，可以编写有趣而脆弱的启发式算法，但最终必须找到合适的解决方案。

In Qt 6.2, there is a CMake API for building QML modules. You can read up on this in our recent blog posts. Yes, the new way of writing QML modules is somewhat restrictive. You cannot just move your QML files all over the place anymore, and you have to declare all the contents of your modules. But, with all this information, we can be confident that we know enough about your QML modules to generate good C++ code.

​在Qt6.2中，有一个用于构建QML模块的CMake API。你可以在我们最近的博客文章中读到这一点。是的，编写QML模块的新方法有些限制。您不能再到处移动QML文件，您必须声明模块的所有内容。但是，通过这些信息，我们可以确信我们对QML模块的了解足够好，生成了良好的C++代码。

The Hot: New Qt Quick Compiler

热点：新的Qt Quick编译器

In Qt 6.2 we've released a technology preview of our new QML Script Compiler, short qmlsc, that uses all the information explained above to generate C++ code from your QML functions and expressions. It replaces qmlcachegen, and simply generates C++ code in addition to byte code for functions it can exhaustively analyze.

在Qt 6.2中，我们发布了一个新的QML脚本编译器的技术预览，即短qmlsc，它使用上面解释的所有信息从QML函数和表达式中生成C++代码。它代替qmlcachegen，除了可以对其功能进行详尽分析的功能之外，还可以生成C++代码。

qmlsc is available with Qt for Device Creation. In Qt 6.3, some of its features are merged into qmlcachegen, which continues to be available with all versions of Qt.

qmlsc可与Qt一起用于设备创建。在Qt6.3中，它的一些特性被合并到qmlcachegen中，qmlcachegen在Qt的所有版本中仍然可用。

Mind that there are many JavaScript constructs that qmlsc cannot be represented in C++ in an efficient way. In contrast to the old qmlcompiler, qmlsc will skip the C++ code generation for functions that contain such constructs and only generate byte code to be interpreted. Yet, most common QML expressions are of a rather simple nature: Value lookups on QObjects, arithmetics, simple if/else or loop constructs. Those can easily be expressed in C++, and doing so provides a nice speedup.

注意，有许多JavaScript，qmlsc无法以有效的C++方式表示。与旧的QML编译器相反，qmlsc会跳过C++代码生成，以生成包含这些构造的函数，并且只生成要解释的字节代码。然而，最常见的QML表达式具有相当简单的性质：对QObject、算术、简单的if/else或循环构造的值查找。这些可以很容易地用C++来表达，这样做可以提供很好的加速。

Additionally, in Qt 6.3, there will be another compiler: The QML type compiler, or qmltc. With the QML type compiler, you will be able to compile your object structure to C++. Each QML component becomes a C++ class this way. Instantiating those classes from C++ will be much faster than the usual detour through QQmlComponent. qmltc, like qmlsc, is built on the availability of complete type information at compile time. In contrast to qmlsc, though, qmltc has no graceful fallback in case the type information is lacking. In order to use qmltc, you will have to provide all the information it requires. qmltc will be available with all versions of Qt.

此外，在Qt6.3中，还将有另一个编译器：QML类型编译器或qmltc。使用QML类型编译器，您将能够编译您的对象结构到C++。每个QML组件以这种方式变成C++类。从C++中实例化那些类将比QQML组件通常绕过的速度快得多。qmltc和qmlsc一样，是建立在编译时完整类型信息的可用性之上的。然而，与qmlsc相比，qmltc在缺少类型信息的情况下没有优雅的回退。为了使用qmltc，您必须提供它所需的所有信息。qmltc将与所有版本的Qt一起提供。

In a further post, the performance benefits of using qmlsc, and the differences between qmlsc and qmlcachegen will be explored.

在下一篇文章中，将探讨使用qmlsc的性能优势，以及qmlsc和qmlcachegen之间的差异。