Qt 6.5中的后期处理效果

Post-Processing Effects in Qt 6.5

Qt 6.5中的后期处理效果

February 20, 2023 by Laszlo Agocs | Comments

2023年2月20日，Laszlo Agics |评论

In Qt 6.5 Qt Quick 3D gains a new QML type: ExtendedSceneEnvironment. See the pre-release documentation at doc-snapshots here. What's more, the existing contents of the QtQuick3D.Effects module is now mostly deprecated, including all the pre-made post-processing effects in there, while recommending alternatives, such as ExtendedSceneEnvironment for true 3D post-processing, MultiEffect / Qt Quick Effect Maker for 2D oriented effects, or a custom 3D post-processing effect via the Effect type (which, to be clear, continues to available and supported and is part of the main QtQuick3D module).

在Qt 6.5中，Qt Quick 3D获得了一种新的QML类型：ExtendedSceneEnvironment。请参阅此处文档快照中的预发布文档。此外，QtQuick3D.Effects模块的现有内容现在大多被弃用，包括其中所有预先制作的后期处理效果，同时推荐其他选项，如用于真实3D后期处理的ExtendedSceneEnvironment、用于2D效果的MultiEffect/Qt Quick Effect Maker、，或通过“效果”类型的自定义3D后处理效果（很明显，它仍然可用并受支持，是主要QtQuick3D模块的一部分）。

What is going on here?

这是怎么回事？

In short, these new types together improve both the 2D and 3D post-processing effect story in Qt Quick and Qt Quick 3D. In this post we are going to look at some of the underlying details and some simple examples, focusing mostly on 3D post-processing, but also touching the world of 2D Qt Quick scenes to a degree, in order to show why these different APIs together make sense and how they provide a more consistent, more sensible story for 2D and 3D effect creation.

简而言之，这些新类型一起改进了Qt Quick和Qt Quick 3D中的2D和3D后处理效果。在这篇文章中，我们将研究一些基本细节和一些简单的示例，主要关注3D后处理，但也在一定程度上触及了2D Qt Quick场景的世界，以展示为什么这些不同的API一起有意义，以及它们如何为2D和3D效果创建提供更一致、更合理的故事。

Simple porting overview

简单的移植概述

First, let's try to visualize how the various components map onto each other, i.e. what are the recommended APIs and tools for new projects built with Qt 6.5 and newer. At the same time, do note that all the components shown are present and fully functional in Qt 6.5, nothing is removed at this point.

首先，让我们尝试可视化各种组件如何相互映射，即使用Qt 6.5和更高版本构建的新项目的推荐API和工具是什么。同时，请注意，Qt 6.5中显示的所有组件均已存在且功能齐全，此时未删除任何组件。

Strictly speaking this is a bit simplified, because as we will soon see, any of the 2D effect approaches are usable with a typical View3D item. This means that the QtQuick3D.Effects box could also link to the Effect Maker application, or even to ShaderEffect as well. Whereas if something that was previously only possible to implement via a custom Effect is now in Qt 6.5 provided by ExtendedSceneEnvironment, it may make sense to switch over to using the built-in feature instead, thus Effect on the left could link to ExtendedSceneEnvironment on the right as well.

严格来说，这有点简化，因为我们很快就会看到，任何2D效果方法都可以用于典型的View3D项目。这意味着QtQuick3D.Effects框也可以链接到EffectMaker应用程序，甚至可以链接到ShaderEffect。然而，如果以前只能通过自定义效果实现的东西现在在ExtendedSceneEnvironment提供的Qt 6.5中，那么切换到使用内置功能可能是有意义的，因此左侧的效果也可以链接到右侧的ExtendedScene环境。

Let's talk about 2D for a moment

让我们来谈谈2D

MultiEffect is another new type in Qt 6.5. It provides the most commonly used shader-based 2D effects for Qt Quick, effectively providing a replacement for the most used subset of Qt Graphical Effects. Qt Graphical Effects continues to be available in the Qt5Compat module, but should be treated as a porting aid for existing applications. New projects are encouraged to use MultiEffect for 2D effects, and for effects that are not covered by it, the new Qt Quick Effect Maker tool. On top, ShaderEffect continues to be available for those who prefer working directly with vertex and fragment shaders.

MultiEffect是Qt 6.5中的另一种新类型。它为Qt Quick提供了最常用的基于着色器的2D效果，有效地替代了Qt图形效果的最常用子集。Qt图形效果继续在Qt5Compat模块中提供，但应作为现有应用程序的移植辅助。鼓励新项目将MultiEffect用于2D效果，对于未包含的效果，则使用新的Qt Quick Effect Maker工具。在顶部，ShaderEffect继续适用于那些喜欢直接使用顶点和片段着色器的用户。

Unlike the old Graphical Effects, MultiEffect can efficiently combine multiple effects without the performance penalty of having to render each effect in the chain one by one, each working on the output of the previous one, involving a full render pass of its own. In addition, it provides an API that is better suited for visual design tools such as Qt Design Studio. (because it allows to easily state what effects are wanted, which can be as simple as ticking some checkboxes in the designer tool, instead of having to create and manage chains of standalone effect items that each live as individual objects in the Qt Quick scene)

与旧的“图形效果”不同，“多重效果”可以有效地组合多个效果，而无需逐个渲染链中的每个效果，每个效果都使用前一个效果的输出，包括自己的完整渲染过程。此外，它还提供了一个更适合QtDesignStudio等可视化设计工具的API。（因为它可以很容易地说明所需的效果，只需在设计器工具中勾选一些复选框即可，而无需创建和管理独立的效果项链，每个效果项在Qt Quick场景中作为单独的对象存在）

The new 2D-oriented effect enablers are going to be covered in other posts. Here we are going to focus on Qt Quick 3D and the effects that are performed on the 3D content in a View3D item.

新的面向2D的效果赋能器将在其他文章中介绍。在这里，我们将重点关注Qt Quick 3D以及对View3D项目中的3D内容执行的效果。

However, before moving on, let's quickly look at a question that tends to pop up quite often:

然而，在继续之前，我们先来看看一个经常出现的问题：

Q: How does one get 2D effects, such as blur or opacity mask, applied to the 3D scene as rendered by a View3D?

Q： 如何将2D效果（如模糊或不透明度遮罩）应用于View3D渲染的3D场景？

A: A View3D in its most commonly used form, that is with the default renderMode of Offscreen, is no different from any other QQuickItem when it comes to ShaderEffect, Graphical Effects, or MultiEffect.

A： View3D最常用的形式，即默认的渲染模式为屏幕外，与任何其他QQuickItem在ShaderEffect、Graphical Effects或MultiEffect方面没有区别。

(technically it would be more accurate to say that a View3D with the default Offscreen render mode is no different from texture-based items that are texture providers, such as Image, ShaderEffectSource or an Item with layer.enabled: true on it, but that's besides the point here)

（从技术上讲，更准确的说法是，具有默认屏幕外渲染模式的View3D与作为纹理提供程序的基于纹理的项目没有什么不同，例如Image、ShaderEffectSource或具有layer.enabled: true的项目，但这不是重点）

For example, to get a 3D scene rendered by a View3D blurred, the same approach can be used as with any other item in the Qt Quick scene: apply a (2D) blur effect, for example via MultiEffect, to either the View3D or an ancestor of it.

例如，要获得由View3D模糊渲染的3D场景，可以使用与Qt Quick场景中的任何其他项目相同的方法：例如通过MultiEffect将（2D）模糊效果应用于View3D或其祖先。

Now, doing a similar effect using the 3D post-processing facilities of the 3D renderer is possible via a custom 3D post-processing Effect or the now-deprecated Blur or GaussianBlur effects from the QtQuick3D.Effects module, but this is unnecessary in the vast majority of cases since performing a 2D blur on the output of the 3D renderer provides the same result.

现在，使用3D渲染器的3D后处理功能可以通过自定义3D后处理效果或QtQuick3D.effects模块中现已弃用的Blur或GaussianBlur效果实现类似效果，但在大多数情况下，这是不必要的，因为在3D渲染器输出上执行2D模糊会提供相同的结果。

The same is true for a number of the now-deprecated effects in QtQuick3D.Effects, such as Flip or Emboss: they are really just 2D-oriented effects that are better performed via Qt Quick's facilities, instead of reinventing them with the 3D post-processing system.

QtQuick3D中的许多现在已被弃用的效果也是如此，如翻转或浮雕：它们实际上只是2D方向的效果，通过QtQuick的设施可以更好地执行，而不是通过3D后处理系统重新设计它们。

Let's see a short example. We will assume that the Sponza and Suzanne models are imported from the glTF 2.0 assets in the Khronos glTF Sample Models repository via the balsam tool (and so the generated Sponza.qml and Suzanne.qml components are available and instantiable). For simplicity, image based lighting is used.

让我们看一个简短的示例。我们将假设Sponza和Suzanne模型是通过balsam工具从Khronos glTF Sample models存储库中的glTF 2.0资产中导入的（因此生成的Sponza.qml和Suzanne.qml组件可用且可实例化）。为了简单起见，使用了基于图像的照明。

import QtQuick
import QtQuick3D
import QtQuick3D.Helpers

Item {
    width: 1280
    height: 720

    View3D {
        id: v3d
        anchors.fill: parent
        environment: SceneEnvironment {
            backgroundMode: SceneEnvironment.SkyBox
            lightProbe: Texture { source: "00455_OpenfootageNET_field_low.hdr" }
        }

        Node {
            scale: Qt.vector3d(100, 100, 100)

            Sponza {
            }

            Suzanne {
                y: 1
                scale: Qt.vector3d(0.5, 0.5, 0.5)
                eulerRotation.y: -90
            }
        }

        PerspectiveCamera {
            id: camera
            y: 100
        }

        WasdController {
            controlledObject: camera
        }
    }
}

Running this with the qml tool gives us something like the following:

使用qml工具运行此命令可以提供如下内容：

Now, what if we need to blur this, for example because we are implementing a background blur which becomes active when some UI elements are shown on top?

现在，如果我们需要模糊这一点，例如，因为我们正在实现一个背景模糊，当一些UI元素显示在顶部时，该背景模糊将变为活动状态，该怎么办？

With Qt 6.5's MultiEffect this is easy, and no different from applying the blur to any other item in the 2D scene. First we add the import for QtQuick.Effects, where MultiEffect lives.

使用Qt 6.5的多重效果，这很简单，与将模糊应用于2D场景中的任何其他项目没有区别。首先，我们为QtQuick.Effects添加导入，MultiEffect位于其中。

import QtQuick.Effects

Then the following is added to the root Item (after the View3D), inspired by the sample snippet in the MultiEffect documentation. As a bonus we make the blur toggleable with the Tab key.

然后，根据MultiEffect文档中的示例片段，将以下内容添加到根项目（View3D之后）。作为奖励，我们使用Tab键可以切换模糊。

MultiEffect {
        id: effect
        source: v3d
        anchors.fill: v3d
        blurEnabled: true
        blurMax: 64
        blur: 1.0
        autoPaddingEnabled: false
    }

    focus: true
    Keys.onTabPressed: effect.blurEnabled = !effect.blurEnabled 

This gives us:

效果如下：

It is all working as expected, which is splendid. Now let's move on to talk about 3D postprocessing with depth buffers, multiple passes, and the joys of tonemapping, i.e. things that Qt Quick does not typically deal with, or at least not in the same form.

一切都按预期进行，这太棒了。现在，让我们继续讨论具有深度缓冲区的3D后处理、多次传递以及色调映射的乐趣，即Qt Quick通常不处理的事情，或者至少不以相同的形式处理。

Post-processing in the 3D world

3D世界中的后期处理

Qt Quick 3D's new ExtendedSceneEnvironment provides an approach principally similar to MultiEffect for post-processing effects that are performed by the 3D renderer.

Qt Quick 3D的新ExtendedSceneEnvironment 为3D渲染器执行的后处理效果提供了一种主要类似于MultiEffect的方法。

This includes effects such as:

这包括以下效果：

• glow and bloom,

• 发光和开花，

• depth of field,

• 景深，

• lens flare,

• 透镜光斑，

• vignette,

• 渐晕，

• color grading via a lookup table,

• 通过查找表进行颜色分级，

• color adjustments

• 颜色调整

• and another antialiasing method (FXAA).

• 以及另一种抗锯齿方法（FXAA）。

In addition, SceneEnvironment's existing tonemapping system is enhanced with a few new properties in ExtendedSceneEnvironment, giving more control over how the mapping of high dynamic range values to the 0-1 range is performed.

此外，SceneEnvironment的现有色调映射系统在ExtendedSceneEnergy中通过一些新的财产得到了增强，从而可以更好地控制如何将高动态范围值映射到0-1范围。

Qt 6.5 also introduces simple fog support via the new Fog type and SceneEnvironment's fog property. Technically, this type of fog is implemented within the materials (i.e. performed automatically, when enabled, by any PrincipledMaterial or CustomMaterial with shadingMode set to Shaded) and not done via post-processing, but from the API and a designer's perspective it falls into the same category.

Qt 6.5还通过新的雾类型和SceneEnvironment的雾属性引入了简单的雾支持。从技术上讲，这种类型的雾是在材质中实现的（即，在启用时，由任何PrincipledMaterial或CustomMaterial自动执行，shadingMode设置为Shaded），而不是通过后处理完成的，但从API和设计师的角度来看，它属于同一类别。

The screen space ambient occlusion support already available via SceneEnvironment in previous Qt versions continues to be available as well.

在之前的Qt版本中，已经通过SceneEnvironment提供的屏幕空间环境遮挡支持也将继续提供。

Compared to the previous approach of listing instances of the pre-made Effect subtypes in the SceneEnvironment's effect list property, using ExtendedSceneEnvironment has the following benefits:

与之前在SceneEnvironment的效果列表属性中列出预先制作的效果子类型的实例的方法相比，使用ExtendedSceneEnvironment具有以下优点：

• The enabled effects get combined efficiently, reducing the number of render passes involved, instead of processing the effect chain one-by-one.

• 启用的效果得到有效组合，减少了渲染过程的数量，而不是逐个处理效果链。

• Tonemapping is taken into account and handled correctly on every level in the post-processing stack. With the standalone post-processing effects built on top of Effect this was handled incorrectly in some cases.

• 在后处理堆栈的每个级别上都考虑并正确处理色调映射。由于在“效果”之上构建了独立的后期处理效果，因此在某些情况下处理不当。

• The quality and the level of configurability for effects such as depth of field or glow/bloom is significantly better, because the implementation is quite different from the older effect shaders.

• 景深或光晕/开花等效果的质量和可配置级别明显更好，因为实现与旧的效果着色器截然不同。

• Lens flare!

• 镜头光斑！

To use ExtendedSceneEnvironment, import QtQuick3D.Helpers and add the word Extended before SceneEnvironment when creating the object that is going to be referenced by the View3D's environment property.

要使用ExtendedSceneEnvironment，请导入QtQuick3D.Helpers，并在创建将由View3D的环境属性引用的对象时在SceneEnvironment之前添加单词Extended。

If we take the example application code from the previous section and replace the environment on the View3D to look like the following:

如果我们使用上一节中的示例应用程序代码，并将View3D上的环境替换为如下所示：

environment: ExtendedSceneEnvironment {
            backgroundMode: SceneEnvironment.SkyBox
            lightProbe: Texture { source: "00455_OpenfootageNET_field_low.hdr" }

            tonemapMode: SceneEnvironment.TonemapModeFilmic
            sharpnessAmount: 0.4

            vignetteEnabled: true
            vignetteRadius: 1.5

            glowEnabled: true
            glowStrength: 1.25
            glowBloom: 0.25
            glowBlendMode: ExtendedSceneEnvironment.GlowBlendMode.Additive
        }

the result is something like this:

结果是这样的：

We will refrain from diving into each and every property here. Instead, check out the pre-release documentation. It now comes with screenshots for many properties, to better illustrate what they do.

我们将避免潜入这里的每一处房产。相反，请查看预发布文档。它现在提供了许多资产的屏幕截图，以更好地说明它们的功能。

The sceneeffects example

场景效果示例

In some cases it may not always be obvious how a slightly different value for a given property is going to affect the rendering results, and if the development workflow consists of 1. change property value in .qml file 2. run application 3. goto 1, that can then quickly become a frustrating experience. This is why it is recommended to perform such tuning interactively.

在某些情况下，如果开发由工作流组成，则给定属性的稍微不同的值将如何影响渲染结果并不总是显而易见的：

1. 更改.qml文件中的属性值

2. 运行应用程序

3. 回到1

这会很快成为令人沮丧的经历。这就是为什么建议以交互方式执行此类调整。

Designers working within Qt Design Studio will typically adjust a few sliders in the properties panel and see the results live, right away. When a design tool is not involved, it can be helpful to experiment with the help of a testbed application first. Qt 6.5 comes with a new example, called sceneeffects. This offers interactive controls for every new property in ExtendedSceneEnvironment, and also offers the settings for the new depth/height fog.

在Qt Design Studio中工作的设计师通常会在属性面板中调整几个滑块，然后立即查看结果。当不涉及设计工具时，首先在测试台应用程序的帮助下进行实验可能会有所帮助。Qt 6.5附带了一个新的例子，叫做场景效果。这为ExtendedSceneEnvironment中的每个新属性提供了交互式控件，还提供了新深度/高度雾的设置。

As there are limitations for shipping with large assets, the best way to use this example is to load assets via the Import button, for example from the Khronos glTF Sample Models repository. Pictured here is the Sponza model loaded directly from the glTF 2.0 assets in the mentioned repository, with a few effects, such as an obligatory lens flare enabled and some parameters altered.

由于运送大型资产存在限制，使用此示例的最佳方法是通过导入按钮加载资产，例如从Khronos glTF Sample Models存储库加载资产。图中所示为Sponza模型，该模型直接从上述存储库中的glTF 2.0资产加载，具有一些效果，例如启用了强制镜头光斑，并更改了一些参数。

It is recommended to play and experiment with this application first when getting started with using ExtendedSceneEnvironment.

在开始使用ExtendedSceneEnvironment时，建议先播放并试用此应用程序。

Performance and resource usage investigation: DebugView

性能和资源使用情况调查：DebugView

In later phases of development it will likely become important to better understand the cost of doing post-processing on the 3D scene. The amount of resources used, and especially the number of additional render passes will depend on the exact settings. While the various effects are combined together as much as possible, some of them involve multiple render passes and intermediate buffers by nature.

在开发的后期阶段，更好地了解在3D场景上进行后期处理的成本可能变得很重要。使用的资源量，特别是附加渲染过程的数量，将取决于确切的设置。虽然各种效果尽可能组合在一起，但其中一些效果本质上涉及多个渲染过程和中间缓冲区。

In Qt 6.5 there is now an easy way to get some quick insights. Notice the "Click here for DebugView" label in the top-right corner. Clicking this makes a DebugView item visible. While it is built in to some of the Quick 3D examples, any application can place a DebugView in the Qt Quick scene, associate it with a View3D, and then control its visibility as seen fit.

在Qt 6.5中，现在有一种简单的方法可以快速获得一些见解。请注意右上角的“单击此处查看DebugView”标签。单击此按钮将显示DebugView项。虽然它内置在一些Quick 3D示例中，但任何应用程序都可以在Qt Quick场景中放置DebugView，将其与View3D关联，然后根据需要控制其可见性。

In addition to the typical timing information, it shows live lists of textures, meshes, and the render passes it takes to render the final output of the View3D.

A View3D with the default renderMode of Offscreen would report one render pass. With post-processing effects added, more passes will appear, the number and render target size of which can depend on a number of factors and often affected by the effect's properties in ExtendedSceneEnvironment, and additional textures may get loaded.

默认renderMode为Offscreen的View3D将报告一个渲染过程。添加后处理效果后，将显示更多过程，其数量和渲染目标大小取决于许多因素，通常受ExtendedSceneEnvironment中效果的属性影响，并且可能加载其他纹理。

Custom 3D post-processing effects

自定义3D后处理效果

How does all this affect custom effects implemented via the Effect QML type? In most ways it does not. Custom effects will continue to work like before. One important things to keep in mind is that any Effect added to the effects list of the environment gets to run before the built-in post-processing steps.

所有这些如何影响通过Effect QML类型实现的自定义效果？在大多数情况下，情况并非如此。自定义效果将继续像以前一样工作。需要记住的一件重要事情是，添加到环境效果列表中的任何效果都必须在内置后处理步骤之前运行。

Effect {
    id: simpleEffect
    property real someUniformValue
    property TextureInput someTexture: TextureInput {
        texture: Texture { source: "image.png" }
    }
    passes: Pass {
       shaders: Shader {
           stage: Shader.Fragment
           shader: "effect.frag"
       }
    }
}

environment: ExtendedSceneEnvironment {
    effects: [ simpleEffect ]
    vignetteEnabled: true
}

Here the custom 'simpleEffect' will get to do its processing before the vignette effect. When it comes to tonemapping, the custom effect can assume its output is in linear space (HDR), tonemapping is performed automatically at the end of the pipeline.

在这里，自定义“simpleEffect”将在渐晕效果之前进行处理。当涉及色调映射时，自定义效果可以假设其输出在线性空间（HDR）中，色调映射在管道末端自动执行。

Summary

总结

• When it comes to 2D effects, get familiar with MultiEffect. For the most common effects, such as blur or drop shadow, this should be the primary choice

• 当涉及到2D效果时，请熟悉“多重效果”。对于最常见的效果，如模糊或阴影，这应该是首选

• ...even if the goal is to apply the effect on the 3D scene. As shown above, using MultiEffect on a typical View3D is no different from applying the effect to other Items.

• …即使目标是将效果应用于3D场景。如上所示，在典型的View3D上使用“多重效果”与将效果应用于其他项目无异。

• Avoid pulling in the legacy Qt Graphical Effects types from the Qt5Compat module. MultiEffect and/or Qt Quick Effect Maker should enable implementing the same effects in a better way.

• 避免从Qt5Compat模块引入传统的Qt图形效果类型。MultiEffect和/或Qt Quick Effect Maker应该能够以更好的方式实现相同的效果。

• When MultiEffect is not sufficient, take a look at the Qt Quick Effect Maker tool, before considering implementing a ShaderEffect with custom shader code.

• 当MultiEffect不够时，在考虑使用自定义着色器代码实现ShaderEffect之前，请查看Qt Quick Effect Maker工具。

• In the 3D world, always consider using the post-processing features of ExtendedSceneEnvironment first.

• 在3D世界中，始终考虑首先使用ExtendedSceneEnvironment的后处理功能。

• Avoid using the old, standalone post-processing effects from the QtQuick3D.Effects module.

• 避免使用QtQuick3D.effects模块中的旧的独立后处理效果。

• Some of these, e.g. Blur, GaussianBlur, Emboss, Flip, etc. can (and should) be implemented as a 2D effect via MultiEffect or Qt Quick Effect Maker instead.

• 其中一些，例如模糊、高斯模糊、浮雕、翻转等，可以（也应该）通过MultiEffect或Qt Quick effect Maker实现为2D效果。

• A number of them, such as DepthOfFieldHQBlur, HDRBloomTonemap, ColorMaster, Fxaa, or Vignette have better alternatives in ExtendedSceneEnvironment.

• 其中一些，如DepthOfFieldHQBlur、HDRBloomTonemap、ColorMaster、Fxaa或Vignette，在ExtendedSceneEnvironment中有更好的选择。

• Some other effects are present solely for historical reasons (e.g. inherited from older products as-is), with limited practical value. These continue to be available in Qt 6 for the time being, but it can be advisable to pull these in to the application instead, as a custom Effect so that the application has full control over them now and in the future.

• 一些其他影响仅出于历史原因（如从旧产品继承而来），实际价值有限。目前Qt 6中仍然可以使用这些效果，但最好将它们作为自定义效果拉到应用程序中，以便应用程序现在和将来完全控制它们。

• When getting familiar with post-processing effects and their parameters, it can be hugely helpful to experiment with them in an interactive environment first. This can be the sceneeffects example, Qt Design Studio, or Qt Quick Effect Maker.

• 当熟悉后处理效果及其参数时，首先在交互式环境中进行实验会非常有帮助。这可以是场景效果示例，Qt Design Studio或Qt Quick Effect Maker。

• Be aware of the DebugView enhancements in Qt 6.5. Whereas in earlier versions it displayed only a few timings, it now has asset and renderer information as well.

• 请注意Qt 6.5中的DebugView增强功能。在早期版本中，它只显示少数计时，而现在它也具有资源和渲染器信息。

That's it for now, and thanks for reading this far. It is not unlikely that more blog posts will surface in the coming months around the topic of effects in Qt Quick and Qt Quick 3D, so stay tuned.

现在就到此为止，感谢您阅读本文。在接下来的几个月里，围绕Qt Quick和Qt Quick 3D中的效果主题不会出现更多的博客文章，所以请继续关注。