unity测试_测试Unity第2部分

unity测试

Time for the second installment of what testing Unity is like (first one here). As mentioned in the previous post we have STEs (Software Test Engineers) working close with development teams on building high quality features, and SDETs (Software Development Engineers in Test) working with STEs and development teams promoting testability and making tools & frameworks for testing Unity. In this post I want to go more into details about the specific work we do and the tools we use.

是第二部分测试Unity的时间了( 这里是第一部分)。 如前一篇文章所述，我们让STE(软件测试工程师)与开发团队紧密合作，以构建高质量的功能，而SDETs(在测试中的软件开发工程师)与STE和开发团队合作，以促进可测试性以及制作用于测试Unity的工具和框架。 在这篇文章中，我想详细介绍我们所做的特定工作和使用的工具。

手动测试 (Manual testing)

The primary tool for any structured manual testing effort is the test case management system. Often the tool of choice is a spreadsheet, but once you get to more than a few people, you need a real tool for the job. After trying out a lot of different products, we have chosen a system called QMetry. Like I mentioned in the previous blogpost, the manual test has a lot to do with figuring out how to get coverage on an area and reporting to the development team on status. It’s all about visibility and feedback and using a tool like QMetry helps us track both. From within one place, we can organize the testing needed, prepare a new cycle of testing, track defects into our bugtracker and make reports for all to see how each part of Unity is being tested and what types of bugs come in.

测试案例管理系统是进行任何结构化的手动测试工作的主要工具。 通常，选择的工具是电子表格，但是一旦您吸引到了很​​多人，您就需要一个真正的工具来完成这项工作。 在尝试了许多不同的产品之后，我们选择了一个名为QMetry的系统。 就像我在前一篇博文中提到的那样，手动测试与弄清楚如何获得某个区域的覆盖范围并向开发团队报告状态有很大关系。 这一切都与可见性和反馈有关，使用QMetry之类的工具可以帮助我们跟踪两者。 在一个地方，我们可以组织所需的测试，准备新的测试周期，将缺陷跟踪到我们的错误跟踪器中，并为所有人提供报告，以了解如何测试Unity的每个部分以及出现了哪些类型的错误。

Besides the very structured testing, we also use a good amount of time doing exploratory testing to just find bugs in new features. It’s an artform in its own rights to attack an application and find bugs. Not everyone possesses the ability to do this effectively, but it is of incredible value. The most extreme form of exploratory testing is a bug bash, where everyone in the development pair up and try to find as many bugs as possible. The team finding the best bug and the team finding most bugs are rewarded with a very edible and/or drinkable award.

除了非常结构化的测试之外，我们还花费大量时间进行探索性测试，以发现新功能中的错误。 它本身就是一种攻击应用程序并查找错误的艺术形式。 并非每个人都具有有效执行此操作的能力，但是它具有不可思议的价值。 探索性测试的最极端形式是错误重击，开发中的每个人都会结伴，并尝试找到尽可能多的错误。 发现最佳虫子的团队和发现多数虫子的团队将获得非常可食用和/或可饮用的奖励。

It’s also worth mentioning that we have full integration between our bugtracker (FogBugz) and our source control system (Mercurial) meaning that we have traceability from bug-report to fix in sourcecode. This enables us to always know when and where fixes can be verified in different versions of Unity. We also build continuously and test on daily builds.

还值得一提的是，我们在bugtracker(FogBugz)和源代码控制系统(Mercurial)之间实现了完全集成，这意味着我们具有从错误报告到源代码修复的可追溯性。 这使我们能够始终知道何时可以在不同版本的Unity中验证修订。 我们还将不断构建并测试日常构建。

自动化 (Automation)

Much of the testing we do today is automated. For regression testing we have different frameworks which can help us make different types of test cases. For small, isolated tests of the Unity Runtime we have the Runtime Test Framework. With this framework we can produce very small, very fast to execute test cases which target a very specific part of Unity. It is capable of running the test cases on all of our targets, so one test case will be able verify the working on all supported platforms. This is of extreme value, since it gives very good coverage with little effort, and it is only possible because all platforms are required to implement an interface which enables us to communicate the same way to all of them. Further, the framework works like a sandbox so it is very easy to use and hard to do stupid things in.

我们今天进行的许多测试都是自动化的。 对于回归测试，我们有不同的框架，可以帮助我们制作不同类型的测试用例。 对于Unity运行时的小型隔离测试，我们具有运行时测试框架。 有了这个框架，我们可以产生非常小，非常快的执行测试用例，这些用例针对Unity的非常特定的部分。 它能够在我们所有目标上运行测试用例，因此一个测试用例将能够验证在所有受支持平台上的工作情况。 这具有极高的价值，因为它可以毫不费力地提供非常好的覆盖范围，并且这仅是可能的，因为所有平台都需要实现一个接口，该接口使我们能够以相同的方式与所有平台进行通信。 此外，该框架的工作方式类似于沙盒，因此非常易于使用，并且很难进行愚蠢的操作。

For larger tests we have the Integration Test Framework. With this we can make a larger scope in the test case, usually taking longer to execute and has a broader coverage. One example could be: Build an assetbundle, setup a webserver and deploy the assetbundle here, start Unity, build a player, download the assetbundle etc. The Integration Test Framework is not born with multiple target execution, so some bit of extra work needs to be done to utilize a test case on different platforms. Also, the framework is not sandboxed like the Runtime Test Framework, so it is somewhat easier to do things such as making processes hang, forgetting to release shared resources etc. This is the price for greater freedom and flexibility.

对于较大的测试，我们有集成测试框架。 这样，我们可以在测试用例中扩大范围，通常需要更长的时间来执行并且覆盖面更广。 一个示例可能是：构建资产捆绑包，在此处安装Web服务器并部署资产捆绑包，启动Unity，构建播放器，下载资产捆绑包等。Integration Test Framework并非具有多个目标执行能力，因此需要做一些额外的工作来可以在不同平台上利用测试用例。 另外，该框架没有像运行时测试框架那样沙盒化，因此它更容易执行诸如使进程挂起，忘记释放共享资源等操作。这是更大的自由度和灵活性的代价。

On the highest level we have our regression rig. Very simply put, this thing can run a set of pre-recorded webplayer games on any given changeset and then compare the output (screenshots, audio, logs) to any other changeset, effectively giving us a picture close to realtime of how much we broke existing games with the latest codebase. On top of this, the rig has a bisect feature that can pinpoint which exact changeset caused a regression by tracking down the change automatically. Obviously there’s a whole lot more to the details about this rig is working, but maybe that will come in a later post.

在最高级别上，我们有回归模型。 简而言之，这件事可以在任何给定的变更集上运行一组预先记录的网络玩家游戏，然后将输出(屏幕截图，音频，日志)与任何其他变更集进行比较，从而有效地为我们提供了接近实时的破损情况图片具有最新代码库的现有游戏。 最重要的是，该钻机具有二等分功能，可以通过自动跟踪更改来查明引起更改的确切更改集。 显然，有关此钻机正在运行的细节还有很多，但也许会在以后的文章中介绍。

The majority of test cases we have today have been written by the developers of each feature, which is a very good sign. As we staff up in QA there will be more test code written directly by developers in test, they will also make sure the entire suites are working well together and that tests are written in the most suitable framework.

我们今天拥有的大多数测试用例已经由每种功能的开发人员编写，这是一个很好的信号。 当我们进行质量检查时，开发人员将在测试中直接编写更多的测试代码，他们还将确保整个套件能够很好地协同工作，并确保测试以最合适的框架编写。

工具类 (Tools)

Tools are extremely important for an efficient development department and we have small team of 3 developers in QA working primarily with tools and frameworks.

工具对于高效的开发部门而言极为重要，我们有3个小组组成的质量检查小组，主要负责工具和框架。

We have what we call the Callstack Analyzer. This tool can extract the callstacks from the crash reports that the community files. Every time Unity crashes, you get a dialog with a request to file in a bit of information about what you were doing and then send us the project and the callstack. This callstack is then analyzed, broken into blocks which we have previously identified as being either Unity, Mono, windows OS, Mac OS etc etc. Then we match each block to all the other callstacks we have and we can identify those where we have duplicates on specific parts of the stack and start investigating them. This is where the additional information you give us comes into play, since the callstack itself rarely gives us a direction to look at; a callstack with 20 duplicates and 1 reproducible bug report is a great catch. So please press that button on a crash, even if you don’t want to write anything; even the callstack itself can be of value to us.

我们拥有所谓的Callstack Analyzer。 该工具可以从社区文件的崩溃报告中提取调用堆栈。 每当Unity崩溃时，您都会收到一个对话框，其中要求您提供有关您正在做的事情的一些信息，然后向我们发送项目和调用堆栈。 然后，对该调用堆栈进行分析，将其分成我们先前确定为Unity，Mono，Windows OS，Mac OS等的块。然后，将每个块与我们拥有的所有其他调用堆栈进行匹配，我们可以识别出重复的块对堆栈的特定部分进行调查。 这是您提供给我们的其他信息的地方，因为调用栈本身很少给我们提供指导的方向； 一个具有20个重复项和1个可重现的错误报告的调用栈非常有用。 因此，即使您不想写任何东西，也请在崩溃时按该按钮。 甚至调用栈本身对我们也很有价值。

We have some additional tools for e.g. for processing bugs, which resides on our servers. Tools for tracking our test projects, making code coverage analysis, making reports on cyclomatic complexity etc, are a part of the toolset we can bring in regularly to get a complete picture of how Unity is doing in the current development cycle.

我们还有其他一些工具，例如用于处理错误的工具，它们位于我们的服务器上。 跟踪我们的测试项目，进行代码覆盖率分析，编写关于圈复杂度的报告等工具，是我们可以定期引入的工具集的一部分，以全面了解Unity在当前开发周期中的运行状况。

还有更多… (More to come…)

I promised you the bug reporter in post 2, but I’ll leave it for the next installment of Testing Unity. In the meantime, go check out our job openings at http://www.unity3d.com/company/jobs.

我在第2部分中向您保证会提供bug报告，但是我将其留给下一期Testing Unity。 同时，请访问http://www.unity3d.com/company/jobs，查看我们的职位空缺。

翻译自: https://blogs.unity3d.com/2012/05/24/testing-unity-part-2/

unity测试