.unity3d反编译_Unity 4.5中的着色器编译

.unity3d反编译

A story in several parts. 1) how shader compilation is done in upcoming Unity 4.5; and 2) how it was developed. First one is probably interesting to Unity users; whereas second one for the ones curious on how we work and develop stuff.

一个故事，分为几个部分。 1)在即将发布的Unity 4.5中如何完成着色器编译； 2)它是如何发展的。 第一个可能对Unity用户很有趣； 而对于那些对我们如何工作和开发东西感到好奇的人则是第二本书。

Short summary: Unity 4.5 will have a “wow, many shaders, much fast” shader importing and better error reporting.

简短摘要：Unity 4.5将具有“哇，许多着色器，速度更快”的着色器导入功能，并提供更好的错误报告。

当前状态(Unity <= 4.3) (Current state (Unity <=4.3))

When you create a new shader file (.shader) in Unity or edit existing one, we launch a “shader importer”. Just like for any other changed asset. That shader importer does some parsing, and then compiles the whole shader into all platform backends we support.

当您在Unity中创建新的着色器文件(.shader)或编辑现有文件时，我们将启动“着色器导入器”。 就像其他任何更改的资产一样。 该着色器导入器进行一些解析，然后将整个着色器编译为我们支持的所有平台后端。

Typically when you create a simple surface shader, it internally expands into 50 or so internal shader variants (classic “preprocessor driven uber-shader” approach). And typically there 7 or so platform backends to compile into (d3d9, d3d11, opengl, gles, gles3, d3d11_9x, flash – more if you have console licenses). This means, each time you change anything in the shader, a couple hundred shaders are being compiled. And all that assuming you have a fairly simple shader – if you throw in some multi_compile directives, you’ll be looking at thousands or tens of thousands shaders being compiled. Each. And. Every. Time.

通常，当您创建一个简单的表面着色器时 ，它会在内部扩展到50个左右的内部着色器变体(经典的“预处理器驱动的uber-shader”方法)。 通常，大约有7个平台后端可以编译为这些后端(d3d9，d3d11，opengl，gles，gles3，d3d11_9x，闪存–如果您拥有控制台许可证，则更多) 。 这意味着， 每次您在着色器中进行任何更改时， 都会编译数百个着色器。 假设您拥有一个相当简单的着色器，所有这些–如果您抛出一些multi_compile指令 ，您将看到正在编译的成千上万个着色器。 每。 和。 每一个 时间。

Does it make sense to do that? Not really.

这样做有意义吗？ 并不是的。

Like most of “why are we doing this?” situations, this one also evolved organically, and can be explained with “it sounded like a good idea at the time” and “it does not fix itself unless someone works on it”.

就像大多数“我们为什么要这样做？” 在这种情况下，这也是有机发展的结果，可以用“在当时听起来像是个好主意”和“除非有人为之努力，否则它不会自我修复”来解释。

A long time ago, Unity only had one or two shader platform backends (opengl and d3d9). And the amount of shader variants people were doing was much lower. With time, we got both more backends, and more variants; and it became very apparent that someone needs to solve this problem.

很久以前，Unity只有一个或两个着色器平台后端(opengl和d3d9)。 人们所做的着色器变体的数量要少得多。 随着时间的流逝，我们将拥有更多的后端和更多的变体。 很明显，有人需要解决这个问题。

In addition to the above, there were other problems with shader compilation, for example:

除上述内容外，着色器编译还存在其他问题，例如：

• Debugging generated surface shader code involved quite some voodoo tricks (#pragma debug etc.).

  调试生成的表面着色器代码涉及许多伏都教技巧( #pragma debug等)。

• Shader importer process was running out of memory for really large multi_compile variant counts.

  Shader导入程序的内存不足，无法处理大量的 multi_compile变体。

So we’re changing how shader importing works in Unity 4.5. The rest of this post will be mostly dumps of our internal wiki pages.

因此，我们正在更改着色器导入在Unity 4.5中的工作方式。 这篇文章的其余部分将主要是我们内部Wiki页面的转储。

在Unity 4.5中导入着色器 (Shader importing in Unity 4.5)

• No runtime/platforms changes compared to 4.3/4.5 – all changes are editor only.

  与4.3 / 4.5相比，运行时/平台没有任何更改-所有更改仅是编辑器 。

• No shader functionality changes compared to 4.3/4.5.

  与4.3 / 4.5相比，着色器功能没有任何变化 。

• Shader importing is much faster; especially complex surface shaders (Marmoset Skyshop etc.).

  着色器进口快得多 ; 特别是复杂的表面着色器(Marmoset Skyshop等)。

  Shader importing is much faster; especially complex surface shaders (Marmoset Skyshop etc.).

  着色器进口快得多 ; 特别是复杂的表面着色器(Marmoset Skyshop等)。

• 
  Errors in shaders are reported on correct lines; errors in shader include (.cginc) files are reported with the filename & line number correctly.

  着色器中的错误报告在正确的行上 ； 着色器包含(.cginc)文件中的错误会正确报告文件名和行号。

  • On d3d11 backend we were reporting error column as the line, hah. At some point during d3dcompiler DLL upgrade it changed error printing syntax and we were parsing it wrong. Now added unit tests so hopefully it will never break again.

    在d3d11后端，我们将错误列报告为该行，哈哈。 在d3dcompiler DLL升级期间的某个时刻，它更改了错误打印语法，并且我们将其解析为错误。 现在添加了单元测试，因此希望它永远不会再次损坏。

  
  Errors in shaders are reported on correct lines; errors in shader include (.cginc) files are reported with the filename & line number correctly.

  着色器中的错误报告在正确的行上 ； 着色器包含(.cginc)文件中的错误会正确报告文件名和行号。

• 
  Surface shader debugging workflow is much better.

  表面着色器的调试工作流程要好得多 。

  
  Surface shader debugging workflow is much better.

  表面着色器的调试工作流程要好得多 。

• Shader inspector improvements:

  着色器检查器的改进：

• Misc bugfixes

  其他错误修正

工作原理概述 (Overview of how it works)

• Instead of compiling all shader variants for all possible platforms at import time:

  而不是在导入时为所有可能的平台编译所有着色器变体：

• At player build time, compile all the shader variants for that target platform

  在播放器构建时，编译该目标平台的所有着色器变体

  • So at player build time, only not-yet-ever-compiled shaders are compiled; and always only for the platforms that need them. If you never ever use Flash, for example, then none of shaders will be compiled for Flash (as opposed to 4.3, where all shaders are compiled to all platforms, even if you never ever need them).

    因此，在播放器构建时，仅会编译尚未编译的着色器。 并且始终仅针对需要它们的平台。 例如，如果您从未使用过Flash，则不会为Flash编译任何着色器(与4.3相对，在4.3中，所有着色器都会编译到所有平台，即使您从不需要)也是如此 。

• Shader compiler (CgBatch) changes from being invoked for each shader import, into being run as a “service process”

  着色器编译器(CgBatch)从每次着色器导入被调用转变为作为“服务进程”运行

它是如何发展的 (How it was developed)

This was mostly a one-or-two person effort, and developed in several “sprints”. For this one we used our internal wiki for detailed task planning (Confluence “task lists”), but we could have just as well use Trello or something similar. Overall this was probably around two months of actual work – but spread out during much longer time. Initial sprint started in 2013 March, and landed in a “we think we can ship this tomorrow” state to 4.5 codebase just in time for 1st alpha build (2013 October). Minor tweaks and fixes were done during 4.5 alpha & beta period. Should ship anyday now, fingers crossed!

这主要是一两个人的努力，并发展成几个“冲刺”。 为此，我们使用内部Wiki进行详细的任务计划(Confluence“任务列表”)，但是我们也可以使用Trello或类似工具。 总体而言，这可能是大约两个月的实际工作-但分布时间更长。 最初的冲刺于2013年3月开始，并及时进入4.5代码库的“我们认为明天就可以发货”状态，以便及时进行第一个alpha版本的构建(2013年10月)。 在4.5 alpha和beta期间进行了较小的调整和修复。 现在应该每天发货，但愿！

Surprisingly (or perhaps not), largest piece of work was around “how do you report errors in shaders?” area. Since now shader variants are imported only on demand, that means some errors can be discovered only “some time after initial import”. This is a by-design change, however – as the previous approach of “let’s compile all possible variants for all possible platforms” clearly does not scale in terms of iteration time. However, this “shader seemed like it did not have any errors, but whoops now it has” is clearly a potential downside. Oh well; as with almost everything there are upsides & downsides.

令人惊讶地(或可能不是)，最大的工作是围绕“如何报告着色器中的错误？”。 区。 由于现在着色器变体仅按需导入，这意味着只能在“初始导入后的某个时间”发现一些错误。 但是，这是设计更改，因为以前的“让我们为所有可能的平台编译所有可能的变体”的方法显然不能在迭代时间上扩展。 但是，这种“着色器似乎没有任何错误，但是现在却有问题”显然是潜在的不利因素。 那好吧; 与几乎所有内容一样，都有优点和缺点。

Most of development was done on a Unity 4.3-based branch, and after something was working we were sending off custom “4.3 + new shader importer” builds to the beta testing group. We were doing this before any 4.5 alpha even started to get early feedback. Perhaps the nicest feedback I ever got:

大部分开发工作都是在基于Unity 4.3的分支上完成的，在工作正常之后，我们将自定义的“ 4.3 +新着色器导入程序”构建发送给Beta测试小组。 在进行任何4.5 alpha测试之前，我们就已经开始这样做了。 也许是我收到的最好的反馈：

I’ve now used the build for about a week and I’m completely blown away with how it has changed how I work with shaders.

现在，我已经使用了约一个星期的构建，而对于它如何改变着色器的工作方式，我完全感到震惊。

I can try out things way quicker. I am no longer scared of making a typo in an include file. These two combine into making me play around a LOT more when working. Because of this I found out how to do fake HDR with filmic tonemapping [on my mobile target].

我可以更快地尝试。 我不再害怕在包含文件中打错字。 这两个因素使我在工作时玩得更多。 因此，我发现了如何使用电影色调映射(在我的移动目标上)进行伪造的HDR。

The thought of going back to regular beta without this [shader compiler] really scares me.

不使用此[shader编译器]返回常规beta的想法确实使我感到恐惧。

Anyhoo, here’s a dump of tasks from our wiki (all of them had little checkboxes that we’d tick off when done). As usual, “it basically works and is awesome!” was achieved after first week of work (1st sprint). What was left after that was “fix all the TODOs, do all the boring remaining work” etc.

顺便说一句，这是我们Wiki中的任务转储(所有任务都有一个小复选框，我们在完成后会选中它们)。 像往常一样，“它基本上有效并且很棒！” 在工作的第一周(第一次冲刺)之后就达到了。 之后剩下的就是“修复所有待办事项，完成所有无聊的剩余工作”等。

2013 March Sprint:

2013年3月：

• Make CgBatch a DLL

  使CgBatch成为DLL

• Shader importer changes

  着色器进口商变更

  • ShaderLab::Pass needs to know it will have yet-uncompiled programs inside, and able to find appropriate CGPROGRAM block:

    ShaderLab :: Pass需要知道它内部将有尚未编译的程序，并且能够找到适当的CGPROGRAM块：

• Compile shaders on demand

  按需编译着色器

  • GetMatchingSubProgram:

    GetMatchingSubProgram：

2013 July Sprint:

2013年7月：

• Player build pipeline

  玩家建立管道

  • Do that properly when building for a “no target” (everything in) platforms

    在构建“无目标”(所有内容)平台时正确执行此操作

  • Make building built-in resource files work

    使构建内置资源文件起作用

  • Multithread the “missing combinations” compilation while building the player.

    在构建播放器时多线程“缺少组合”编译。

• Misc

  杂项

2013 August Sprint:

2013年8月Sprint：

• Gfx test failures

  GFX测试失败

• Error reporting: Figure out how to deal with late-discovered errors. If there’s bad syntax, typo etc.; effectively shader is “broken”. If a backend shader compiler reports an error:

  错误报告：弄清楚如何处理最新发现的错误。 如果语法错误，错别字等； 着色器实际上是“损坏的”。 如果后端着色器编译器报告错误：

  • Add error to the shader, so it’s displayed in the editor. Can’t serialize shader at that time, so add shaders to some database under Library (guid­>errors).

    向着色器添加错误，使其显示在编辑器中。 当时无法序列化着色器，因此将着色器添加到“库”下的某些数据库中(guid>错误)。

• Misc

  杂项

• Make CgBatch again into an executable (for future 64 bit mac…)

  再次使CgBatch成为可执行文件(用于将来的64位mac…)

• Shader Inspector

  着色器检查器

2013 September Sprint:

2013年9月Sprint：

• Make ready for 4.5 trunk

  准备好4.5行李箱

• Make 4.3-based TeamCity green

  使基于4.3的TeamCity变得绿色

• Error reporting and #include handling

  错误报告和#include处理

• Misc

  杂项

• Shader Inspector

  着色器检查器

下一步是什么？ (What’s next?)

Some more in shader compilation land will go into Unity 5.0 and 5.x. Outline of our another wiki page describing 5.x related work:

着色器编译领域的更多内容将进入Unity 5.0和5.x。 另一个介绍5.x相关工作的Wiki页面的概述：

• Need to fix “New standard shader produces very large shader files” (due to lots of variants – 5000 variants, 100MB) problem.

  需要解决“新标准着色器产生很大的着色器文件” (由于许多变体– 5000个变体，100MB)的问题。

翻译自: https://blogs.unity3d.com/2014/05/06/shader-compilation-in-unity-4-5/

.unity3d反编译