脚本文档
When I encounter an API in Unity that I am unfamiliar with the first thing I (and most of us) do is go to the Unity Scripting API Manual to see how it works via one of the examples. If that example will not compile when I try it, I assume that I must be doing something wrong. The example couldn’t possibly be broken, could it…?
当我在Unity中遇到一个我不熟悉的第一件事(以及我们大多数人)时，请去Unity Scripting API手册 ，通过其中一个示例了解它的工作方式。 如果在尝试时无法编译该示例，则认为我必须做错了什么。 这个例子不可能被打破，是吧？
This is how I discovered that we do indeed have examples in our scripting docs that do not compile, as a result of API changes over time and the odd case of an the example never compiling to start with. At Unity we have a lot of freedom in how we work; if we see a problem we can report it to the relevant team or fix it ourselves. At one of our recent Sustained Engineering team weeks we decided to do our own hackweek and picked several issues we wanted to tackle. Some of us chose to look into a solution for there being broken examples in the scripting docs.
这就是我发现脚本编写文档中确实存在无法编译的示例的原因，这是由于API随时间而变化以及该示例从未编译开始的奇怪情况。 在Unity，我们在工作方式上有很多自由。 如果我们发现问题，可以向相关团队报告或自行解决。 在我们最近的“ 可持续工程团队”中，有一个星期我们决定进行自己的“ 黑客周” 活动，并挑选了一些我们想解决的问题。 我们中有些人选择研究解决方案，以解决脚本文档中的破例。
There are about 15,000 scripting docs pages. Not all of them contain examples (a different problem which we are working to improve); however a large portion do. Going through each example and testing them manually would be unachievable in a week. It would not solve the problem of API changes or broken examples being written in the future either either.
大约有15,000个脚本文档页面 。 并非所有示例都包含示例(我们正在努力改善的另一个问题)； 但是很大一部分。 在一周之内无法完成每个示例并对其进行手动测试。 它也不会解决API更改或将来编写的示例不完整的问题。
Last year as part of the Unity 5.3 release we included a new feature called the Editor Test Runner. This is a unit test framework that can be run from within Unity. We have been using the Editor Test Runner internally for our own automated tests since its introduction. I decided to tackle the problem using an editor test. All our scripting docs are stored in XML files which we edit through an internal Unity project.
去年，作为Unity 5.3版本的一部分，我们加入了一项称为“编辑器测试运行器”的新功能。 这是一个可以在Unity内部运行的单元测试框架。 自推出以来，我们一直在内部将Editor Test Runner用于我们自己的自动化测试。 我决定使用编辑器测试解决问题。 我们所有的脚本文档都存储在XML文件中，我们可以通过内部Unity项目进行编辑。
The code to parse all these files is already available in this project so it made sense to add the editor test into the same project so we could reuse it.
解析所有这些文件的代码已经在该项目中可用，因此将编辑器测试添加到同一项目中是有意义的，以便我们可以重用它。
In our editor test framework (which is using NUnit) there is an attribute that can be applied to a test called TestCaseSource. This lets a test be run multiple times with different source data. In this case the source data would be our list of script examples.
在我们的编辑器测试框架(使用NUnit)中，有一个属性可以应用于名为TestCaseSource的测试。 这样就可以使用不同的源数据多次运行测试。 在这种情况下，源数据将是我们的脚本示例列表。
|
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
|
public class ScriptVerification
{
public static IEnumerable TestFiles
{
get
{
// Get all the xml files
var files = Directory.GetFiles("OurDocsApiPath/*.mem.xml", SearchOption.AllDirectories);
// Each file is a separate test.
foreach (var file in files)
{
string testName = Path.GetFileName(file).Replace(k_FileExtension, "");
yield return new TestCaseData(file).SetName(testName);
}
}
}
[Test]
[TestCaseSource("TestFiles")]
public void TestDocumentationExampleScripts(string docXmlFile)
{
// Do the test
}
}
|
|
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
|
public class ScriptVerification
{
public static IEnumerable TestFiles
{
get
{
// Get all the xml files
var files = Directory . GetFiles ( "OurDocsApiPath/*.mem.xml" , SearchOption . AllDirectories ) ;
// Each file is a separate test.
foreach ( var file in files )
{
string testName = Path . GetFileName ( file ) . Replace ( k_FileExtension , "" ) ;
yield return new TestCaseData ( file ) . SetName ( testName ) ;
}
}
}
[ Test ]
[ TestCaseSource ( "TestFiles" ) ]
public void TestDocumentationExampleScripts ( string docXmlFile )
{
// Do the test
}
}
|
Using this method now shows a list of all the tests that will be run in the test runner. Each test can be run individually or they can all be run using the Run All option.
现在，使用此方法将显示将在测试运行器中运行的所有测试的列表。 每个测试可以单独运行，也可以使用“全部运行”选项全部运行。
To compile the examples we use CodeDomProvider. It allows us to pass in one or more strings that represent a script, and it will compile and return information on errors and warnings.
为了编译示例，我们使用CodeDomProvider 。 它允许我们传入一个或多个表示脚本的字符串，它将编译并返回有关错误和警告的信息。
This is a cutdown version (XML parsing removed) of the first iteration of the test:
这是测试的第一次迭代的简化版本(删除了XML解析)：
|
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
|
using UnityEngine;
using NUnit.Framework;
using System.CodeDom.Compiler;
using System.Collections;
using System.Reflection;
using System.Xml;
using System.IO;
using UnityEditor;
public class ScriptVerification
{
public static IEnumerable TestFiles
{
get
{
// Get all the xml files
var files = Directory.GetFiles("OurDocsApiPath/*.mem.xml", SearchOption.AllDirectories);
// Each file is a seperate test
foreach (var file in files)
{
string testName = Path.GetFileName(file).Replace(k_FileExtension, "");
yield return new TestCaseData(file).SetName(testName);
}
}
}
CodeDomProvider m_DomProvider;
CompilerParameters m_CompilerParams;
[SetUp]
public void InitScriptCompiler()
{
m_DomProvider = CodeDomProvider.CreateProvider("CSharp");
m_CompilerParams = new CompilerParameters
{
GenerateExecutable = false,
GenerateInMemory = false,
TreatWarningsAsErrors = false,
};
Assembly unityEngineAssembly = Assembly.GetAssembly(typeof(MonoBehaviour));
Assembly unityEditorAssembly = Assembly.GetAssembly(typeof(Editor));
m_CompilerParams.ReferencedAssemblies.Add(unityEngineAssembly.Location);
m_CompilerParams.ReferencedAssemblies.Add(unityEditorAssembly.Location);
}
[Test]
[TestCaseSource("TestFiles")]
public void TestDocumentationExampleScripts(string docXmlFile)
{
// Parse the xml and extract the scripts
// foreach script example in our doc call TestCsharpScript
}
void TestCsharpScript(string scriptText)
{
// Check for errors
CompilerResults compilerResults = m_DomProvider.CompileAssemblyFromSource(m_CompilerParams, scriptText);
string errors = "";
if (compilerResults.Errors.HasErrors)
{
foreach (CompilerError compilerError in compilerResults.Errors)
{
errors += compilerError.ToString() + "\n";
}
}
Assert.IsFalse(compilerResults.Errors.HasErrors, errors);
}
}
|
|
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
|
using UnityEngine ;
using NUnit . Framework ;
using System . CodeDom . Compiler ;
using System . Collections ;
using System . Reflection ;
using System . Xml ;
using System . IO ;
using UnityEditor ;
public class ScriptVerification
{
public static IEnumerable TestFiles
{
get
{
// Get all the xml files
var files = Directory . GetFiles ( "OurDocsApiPath/*.mem.xml" , SearchOption . AllDirectories ) ;
// Each file is a seperate test
foreach ( var file in files )
{
string testName = Path . GetFileName ( file ) . Replace ( k_FileExtension , "" ) ;
yield return new TestCaseData ( file ) . SetName ( testName ) ;
}
}
}
CodeDomProvider m_DomProvider ;
CompilerParameters m_CompilerParams ;
[ SetUp ]
public void InitScriptCompiler ( )
{
m_DomProvider = CodeDomProvider . CreateProvider ( "CSharp" ) ;
m_CompilerParams = new CompilerParameters
{
GenerateExecutable = false ,
GenerateInMemory = false ,
TreatWarningsAsErrors = false ,
} ;
Assembly unityEngineAssembly = Assembly . GetAssembly ( typeof ( MonoBehaviour ) ) ;
Assembly unityEditorAssembly = Assembly . GetAssembly ( typeof ( Editor ) ) ;
m_CompilerParams . ReferencedAssemblies . Add ( unityEngineAssembly . Location ) ;
m_CompilerParams . ReferencedAssemblies . Add ( unityEditorAssembly . Location ) ;
}
[ Test ]
[ TestCaseSource ( "TestFiles" ) ]
public void TestDocumentationExampleScripts ( string docXmlFile )
{
// Parse the xml and extract the scripts
// foreach script example in our doc call TestCsharpScript
}
void TestCsharpScript ( string scriptText )
{
// Check for errors
CompilerResults compilerResults = m_DomProvider . CompileAssemblyFromSource ( m_CompilerParams , scriptText ) ;
string errors = "" ;
if ( compilerResults . Errors . HasErrors )
{
foreach ( CompilerError compilerError in compilerResults . Errors )
{
errors += compilerError . ToString ( ) + "\n" ;
}
}
Assert . IsFalse ( compilerResults . Errors . HasErrors , errors ) ;
}
}
|
And it worked! We needed to make some small changes in how we compile the examples, though, as some scripts are designed to go together as a larger example. To check for this we compiled them separately; if we found an error, we then compiled them again combined to see if that worked.
而且有效！ 但是，我们需要对示例的编译方式进行一些小的更改，因为某些脚本被设计为可以作为一个更大的示例一起使用。 为了对此进行检查，我们分别编译了它们。 如果发现错误，则将它们重新组合起来以查看是否可行。
Some examples are written as single lines of code which are not wrapped in a class or function. We could fix this by wrapping them in our test, but we have a rule that all examples should compile standalone (i.e. if a user copies and pastes it into a new file it should compile and work), so we count those examples as test failures.
一些示例是用单行代码编写的，这些代码没有包装在类或函数中。 我们可以通过将它们包装在测试中来解决此问题，但是我们有一个规则，即所有示例都应独立编译(即，如果用户将其复制并粘贴到新文件中，则可以编译并正常工作)，因此我们将这些示例视为测试失败。
The test was now in a state where it could be run as part of our build verification on the path to trunk. However there was one small problem: the test took 30 minutes to run. This is far too long for a test running in build verification, considering we run around 7000 builds a day.
现在，测试处于可以在通往中继路径的构建验证中运行的状态。 但是，有一个小问题：测试耗时30分钟。 考虑到我们每天运行大约7000个构建，对于构建验证中运行的测试来说，这太长了。
The test was running sequentially, one script after another, but there was no reason we could not run them in parallel as the tests were independent of each other and did not need to make any calls to the Unity API;and we are only testing that they compile, not the behaviour. Introducing ThreadPool, a .NET API that can be used to execute tasks in parallel. We push the tests as individual tasks into the ThreadPool and they will be executed as soon as a thread becomes available. This needs to be driven from a single function, meaning that we can’t have individual NUnit test cases for testing specific examples from the docs. As a result we lose the ability to run any one of the tests individually, but we gain the ability to run them all quickly.
测试按顺序运行，一个脚本接一个脚本运行，但是没有理由我们不能并行运行它们，因为测试彼此独立并且不需要对Unity API进行任何调用；而我们只是在测试那个它们会编译，而不是行为。 引入ThreadPool ，这是一个.NET API，可用于并行执行任务。 我们将测试作为单独的任务推送到ThreadPool中，并在线程可用时立即执行它们。 这需要从单个函数驱动，这意味着我们无法使用单独的NUnit测试用例来测试文档中的特定示例。 结果，我们失去了单独运行任何一个测试的能力，但获得了快速运行所有测试的能力。
|
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
|
[Test]
public void ScriptVerificationCSharp()
{
// Setup. Start all tests running on multiple threads.
s_ThreadEvents = new ManualResetEvent[s_DocInfo.Count];
for (int i = 0; i < s_DocInfo.Count; ++i)
{
// Queue this example up for testing
s_ThreadEvents[i] = new ManualResetEvent(false);
ThreadPool.QueueUserWorkItem(TestDocumentationExampleScriptsThreaded, i);
}
// Check for errors and build the error output if required.
bool testFailed = false;
StringBuilder results = new StringBuilder();
for (int i = 0; i < s_ThreadEvents.Length; ++i)
{
// Wait for the test to finish.
s_ThreadEvents[i].WaitOne();
if (s_DocInfo[i].status == TestStatus.Failed)
{
testFailed = true;
GenerateFailureMessage(results, s_DocInfo[i]);
}
}
// If a single item has failed then the test is considered a failure.
Assert.IsFalse(testFailed, results.ToString());
}
public static void TestDocumentationExampleScriptsThreaded(object o)
{
var infoIdx = (int)o;
var info = s_DocInfo[infoIdx];
try
{
TestScriptsCompile(info);
}
catch (Exception e)
{
info.status = TestStatus.Failed;
info.testRunnerFailure = e.ToString();
}
finally
{
s_ThreadEvents[infoIdx].Set();
}
}
|
|
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
|
[ Test ]
public void ScriptVerificationCSharp ( )
{
// Setup. Start all tests running on multiple threads.
s_ThreadEvents = new ManualResetEvent [ s_DocInfo . Count ] ;
for ( int i = 0 ; i < s_DocInfo . Count ; ++ i )
{
// Queue this example up for testing
s_ThreadEvents [ i ] = new ManualResetEvent ( false ) ;
ThreadPool . QueueUserWorkItem ( TestDocumentationExampleScriptsThreaded , i ) ;
}
// Check for errors and build the error output if required.
bool testFailed = false ;
StringBuilder results = new StringBuilder ( ) ;
for ( int i = 0 ; i < s_ThreadEvents . Length ; ++ i )
{
// Wait for the test to finish.
s_ThreadEvents [ i ] . WaitOne ( ) ;
if ( s_DocInfo [ i ] . status == TestStatus . Failed )
{
testFailed = true ;
GenerateFailureMessage ( results , s_DocInfo [ i ] ) ;
}
}
// If a single item has failed then the test is considered a failure.
Assert . IsFalse ( testFailed , results . ToString ( ) ) ;
}
public static void TestDocumentationExampleScriptsThreaded ( object o )
{
var infoIdx = ( int ) o ;
var info = s_DocInfo [ infoIdx ] ;
try
{
TestScriptsCompile ( info ) ;
}
catch ( Exception e )
{
info . status = TestStatus . Failed ;
info . testRunnerFailure = e . ToString ( ) ;
}
finally
{
s_ThreadEvents [ infoIdx ] . Set ( ) ;
}
}
|
This took the test time from 30 minutes to 2, which is fine for running as part of our build verification.
测试时间从30分钟缩短到2分钟，这对于作为我们的构建验证的一部分运行是很好的。
Since we couldn’t test individual examples with NUnit any more, we added a button to the scripting doc editor to allow developers to test the examples as they write them. The script with an error is now colored red when the test is run and error messages are displayed beneath.
由于我们无法再使用NUnit测试单个示例，因此在脚本文档编辑器中添加了一个按钮，以允许开发人员在编写示例时对其进行测试。 在运行测试时，带有错误的脚本现在显示为红色，并且错误消息显示在下方。
When the test was first run we had 326 failures which I whitelisted (so they could be fixed at a later date). We now have that down to 32, of which most are failures in the test runner mainly due to not having access to some specific assemblies. There have been no new issues introduced and we can rest assured that when we deprecate parts of the API the test will fail and we can then update the example to use the new API.
首次运行测试时，我将326个失败列入了白名单(因此可以在以后进行修复)。 现在，我们可以减少到32个，其中大多数是测试运行程序中的失败，主要是由于无法访问某些特定的程序集。 没有引入新的问题，我们可以放心，当我们弃用API的某些部分时，测试将失败，然后可以更新示例以使用新的API。
Overall I thought this was an interesting use of the Editor Test Runner. It does have some limitations: We only test C# examples, and I have not managed to get JS compilation working, although that won’t be an issue in the future.
总的来说，我认为这是Editor Test Runner的有趣用法。 它确实有一些局限性：我们仅测试C＃示例，尽管将来这不会成为问题 ，但我没有设法使JS编译正常工作。
Here is the full test.
这是完整的测试。
|
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
|
using System;
using System.CodeDom.Compiler;
using UnityEngine;
using NUnit.Framework;
using System.Collections.Generic;
using System.IO;
using System.Linq;
using System.Reflection;
using System.Text;
using System.Threading;
using System.Xml;
using Microsoft.CSharp;
using UnderlyingModel;
using UnityEditor;
public class ScriptVerification
{
const string k_PathToApiDocs = @"../../../../Documentation/ApiDocs/";
const string k_FileExtension = ".mem.xml";
const string k_WhiteList = "Assets/Editor/ScriptVerificationWhiteList.txt";
public enum TestStatus
{
Unknown, // Nothing has been done to this test yet.
Ignored, // Tests are ignored if they contain no example code
Failed, // The test failed to compile one or more of the examples.
Passed, // All examples were compiled successfully.
Whitelisted, // Test was ignored as the member is in the white list file.
}
public class ExampleScript
{
public string code;
public CompilerResults compileResults;
}
public class ScriptingDocMember
{
public TestStatus status = TestStatus.Unknown;
// Information on the test and the xml file it can be found in.
public string path;
public string parent;
public string name;
public string nspace;
public bool editor;
public List<ExampleScript> csharpExamples = new List<ExampleScript>();
// If we fail to compile multiple examples we also attempt to compile them as a single example.
public CompilerResults combinedResults;
// Error message if something caused the test runner to fail.
public string testRunnerFailure;
}
static List<ScriptingDocMember> s_DocInfo;
static ManualResetEvent[] s_ThreadEvents;
[SetUp]
public void SetupScriptVerification()
{
// Parse the scripting doc files and prepare the test data.
string path = k_PathToApiDocs;
if (!path.Contains(":"))
{
path = Application.dataPath + "/" + k_PathToApiDocs;
}
var files = Directory.GetFiles(path, "*" + k_FileExtension, SearchOption.AllDirectories);
s_DocInfo = new List<ScriptingDocMember>();
var whiteList = GetWhiteList();
for (int i = 0; i < files.Length; ++i)
{
var xml = new XmlDocument();
xml.Load(files[i]);
XmlNode xmlheader = xml.FirstChild;
XmlNode docsheader = xmlheader.NextSibling;
XmlNode namespaceTag = docsheader.FirstChild;
ParseMemberNode(namespaceTag, files[i], "", s_DocInfo, whiteList);
}
}
[Test]
public void ScriptVerificationCSharp()
{
// Setup. Start all tests running on multiple threads.
// This gets the test time down from 30 minutes to around 2 minutes.
s_ThreadEvents = new ManualResetEvent[s_DocInfo.Count];
for (int i = 0; i < s_DocInfo.Count; ++i)
{
if (s_DocInfo[i].csharpExamples.Count == 0)
{
// Ignore items with no examples
s_ThreadEvents[i] = new ManualResetEvent(true);
s_DocInfo[i].status = TestStatus.Ignored;
}
else if (s_DocInfo[i].status == TestStatus.Whitelisted)
{
// Skip white listed items
s_ThreadEvents[i] = new ManualResetEvent(true);
}
else
{
// Queue this example up for testing
s_ThreadEvents[i] = new ManualResetEvent(false);
ThreadPool.QueueUserWorkItem(TestDocumentationExampleScriptsThreaded, i);
}
}
// Check for errors and build the error output if required.
bool testFailed = false;
StringBuilder results = new StringBuilder();
for (int i = 0; i < s_ThreadEvents.Length; ++i)
{
s_ThreadEvents[i].WaitOne();
if (s_DocInfo[i].status == TestStatus.Failed)
{
testFailed = true;
GenerateFailureMessage(results, s_DocInfo[i]);
}
}
// If a single item has failed then the test is considered a failure.
Assert.IsFalse(testFailed, results.ToString());
}
static void GenerateFailureMessage(StringBuilder output, ScriptingDocMember info)
{
output.AppendLine(new string('-', 100));
output.AppendLine("Name: " + info.name);
output.AppendLine("Path: " + info.path + "\n");
// Print out the example scripts along with their errors.
for (int i = 0; i < info.csharpExamples.Count; ++i)
{
var example = info.csharpExamples[i];
if (example.compileResults != null && example.compileResults.Errors.HasErrors)
{
output.AppendLine("Example Script " + i + ":\n");
// Add line numbers
var lines = example.code.SplitLines();
int lineNumber = 0;
int startLine = 0;
// Find the first line of code so the line numbers align correctly.
// The compiler will ignore any empty lines at the start.
for (; startLine < lines.Length; ++startLine)
{
if (string.IsNullOrEmpty(lines[startLine]))
startLine++;
else
break;
}
for (; startLine < lines.Length; ++startLine)
{
// Does this line contain an error?
string lineMarker = " ";
foreach (CompilerError compileResultsError in example.compileResults.Errors)
{
// Add a mark to indicate this line has a reported error.
if (compileResultsError.Line == lineNumber)
{
lineMarker = "-";
break;
}
}
output.AppendFormat("{0}{1:000} | {2}\n", lineMarker, lineNumber++, lines[startLine]);
}
output.Append("\n\n");
output.AppendLine(ErrorMessagesToString(example.compileResults));
}
}
if (info.combinedResults != null)
{
output.AppendLine("Combined Example Scripts:\n");
output.AppendLine(ErrorMessagesToString(info.combinedResults));
}
if (!string.IsNullOrEmpty(info.testRunnerFailure))
{
output.AppendLine("Test Runner Failure: " + info.testRunnerFailure);
}
}
// Concatenates all the errors into a formated list.
public static string ErrorMessagesToString(CompilerResults cr)
{
string errorMessages = "";
foreach (CompilerError compilerError in cr.Errors)
{
errorMessages += string.Format("{0}({1},{2}): {3}\n", compilerError.ErrorNumber, compilerError.Line, compilerError.Column, compilerError.ErrorText);
}
return errorMessages;
}
public static void TestDocumentationExampleScriptsThreaded(object o)
{
var infoIdx = (int)o;
var info = s_DocInfo[infoIdx];
try
{
TestScriptsCompile(info);
}
catch (Exception e)
{
info.status = TestStatus.Failed;
info.testRunnerFailure = e.ToString();
}
finally
{
s_ThreadEvents[infoIdx].Set();
}
}
// Tests all scripts compile for the selected scripting member.
// First attempts to compile all scripts separately, if this fails then compiles them combined as a single example.
public static void TestScriptsCompile(ScriptingDocMember info)
{
var scripts = info.csharpExamples;
if (scripts.Count == 0)
{
info.status = TestStatus.Ignored;
return;
}
// Setup compiler
var providerOptions = new Dictionary<string, string>();
providerOptions.Add("CompilerVersion", "v3.5");
var domProvider = new CSharpCodeProvider(providerOptions);
var compilerParams = new CompilerParameters
{
GenerateExecutable = false,
GenerateInMemory = false,
TreatWarningsAsErrors = false,
};
Assembly[] assemblies = AppDomain.CurrentDomain.GetAssemblies();
foreach (var assembly in assemblies)
{
compilerParams.ReferencedAssemblies.Add(assembly.Location);
}
// Attempt to compile the scripts separately.
bool error = false;
for (int i = 0; i < scripts.Count; i++)
{
scripts[i].compileResults = domProvider.CompileAssemblyFromSource(compilerParams, scripts[i].code);
if (scripts[i].compileResults.Errors.HasErrors)
error = true;
}
if (error)
{
// Its possible that the scripts are all part of one example so we should compile them together and see if that works instead.
info.combinedResults = domProvider.CompileAssemblyFromSource(compilerParams, scripts.Select(s => s.code).ToArray());
if (!info.combinedResults.Errors.HasErrors)
error = false;
}
info.status = error ? TestStatus.Failed : TestStatus.Passed;
}
static HashSet<string> GetWhiteList()
{
var textAsset = AssetDatabase.LoadAssetAtPath(k_WhiteList, typeof(TextAsset)) as TextAsset;
var whiteList = new HashSet<string>();
if (textAsset)
{
foreach (var line in textAsset.text.Split('\n'))
{
whiteList.Add(line.Replace("\r", "").TrimEnd(' '));
}
}
return whiteList;
}
// Parses the scripting docs and generates our test data.
static void ParseMemberNode(XmlNode node, string file, string parent, List<ScriptingDocMember> infoList, HashSet<string> whiteList)
{
ScriptingDocMember info = new ScriptingDocMember();
info.path = file;
infoList.Add(info);
info.parent = parent;
foreach (XmlAttribute attr in node.Attributes)
{
// potential tag attributes: name, namespace, type
var attrLowercase = attr.Name.ToLower();
if (attrLowercase == "name") info.name = attr.Value;
else if (attrLowercase == "namespace") info.nspace = attr.Value;
}
if (whiteList.Contains(info.name))
info.status = TestStatus.Whitelisted;
if (!string.IsNullOrEmpty(info.nspace))
{
// trim down the namespace to remove UnityEngine and UnityEditor
if (info.nspace.StartsWith("UnityEngine"))
{
info.editor = false;
info.nspace = info.nspace.Remove(0, "UnityEngine".Length);
}
if (info.nspace.StartsWith("UnityEditor"))
{
info.editor = true;
info.nspace = info.nspace.Remove(0, "UnityEditor".Length);
}
if (info.nspace.StartsWith("."))
info.nspace = info.nspace.Remove(0, 1);
}
foreach (XmlNode child in node.ChildNodes)
{
var childNameLowercase = child.Name.ToLower();
if (childNameLowercase == "section")
{
// see if this section is undoc
for (int i = 0; i < child.Attributes.Count; i++)
{
if (child.Attributes[i].Name == "undoc" && child.Attributes[i].Value == "true")
{
infoList.Remove(info);
break;
}
}
foreach (XmlNode grandChild in child.ChildNodes)
{
var codeLangNode = GetExample(grandChild);
if (codeLangNode != null)
{
var scriptInfo = new ExampleScript();
scriptInfo.code = codeLangNode.InnerXml.Replace("<![CDATA[", "").Replace("]]>", "");
info.csharpExamples.Add(scriptInfo);
}
}
}
else if (childNameLowercase == "member")
{
ParseMemberNode(child, file, info.name, infoList, whiteList);
}
}
}
// Extract the cs example code.
static XmlNode GetExample(XmlNode node)
{
if (node.Name.ToLower() == "example")
{
for (int i = 0; i < node.Attributes.Count; ++i)
{
if (node.Attributes[i].Name == "nocheck" && node.Attributes[i].Value == "true")
return null;
}
return node.SelectSingleNode("code[@lang='cs']");
}
return null;
}
}
|
|
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
|
using System ;
using System . CodeDom . Compiler ;
using UnityEngine ;
using NUnit . Framework ;
using System . Collections . Generic ;
using System . IO ;
using System . Linq ;
using System . Reflection ;
using System . Text ;
using System . Threading ;
using System . Xml ;
using Microsoft . CSharp ;
using UnderlyingModel ;
using UnityEditor ;
public class ScriptVerification
{
const string k_PathToApiDocs = @ "../../../../Documentation/ApiDocs/" ;
const string k_FileExtension = ".mem.xml" ;
const string k_WhiteList = "Assets/Editor/ScriptVerificationWhiteList.txt" ;
public enum TestStatus
{
Unknown , // Nothing has been done to this test yet.
Ignored , // Tests are ignored if they contain no example code
Failed , // The test failed to compile one or more of the examples.
Passed , // All examples were compiled successfully.
Whitelisted , // Test was ignored as the member is in the white list file.
}
public class ExampleScript
{
public string code ;
public CompilerResults compileResults ;
}
public class ScriptingDocMember
{
public TestStatus status = TestStatus . Unknown ;
// Information on the test and the xml file it can be found in.
public string path ;
public string parent ;
public string name ;
public string nspace ;
public bool editor ;
public List < ExampleScript > csharpExamples = new List < ExampleScript > ( ) ;
// If we fail to compile multiple examples we also attempt to compile them as a single example.
public CompilerResults combinedResults ;
// Error message if something caused the test runner to fail.
public string testRunnerFailure ;
}
static List < ScriptingDocMember > s_DocInfo ;
static ManualResetEvent [ ] s_ThreadEvents ;
[ SetUp ]
public void SetupScriptVerification ( )
{
// Parse the scripting doc files and prepare the test data.
string path = k_PathToApiDocs ;
if ( ! path . Contains ( ":" ) )
{
path = Application . dataPath + "/" + k_PathToApiDocs ;
}
var files = Directory . GetFiles ( path , "*" + k_FileExtension , SearchOption . AllDirectories ) ;
s_DocInfo = new List < ScriptingDocMember > ( ) ;
var whiteList = GetWhiteList ( ) ;
for ( int i = 0 ; i < files . Length ; ++ i )
{
var xml = new XmlDocument ( ) ;
xml . Load ( files [ i ] ) ;
XmlNode xmlheader = xml . FirstChild ;
XmlNode docsheader = xmlheader . NextSibling ;
XmlNode namespaceTag = docsheader . FirstChild ;
ParseMemberNode ( namespaceTag , files [ i ] , "" , s_DocInfo , whiteList ) ;
}
}
[ Test ]
public void ScriptVerificationCSharp ( )
{
// Setup. Start all tests running on multiple threads.
// This gets the test time down from 30 minutes to around 2 minutes.
s_ThreadEvents = new ManualResetEvent [ s_DocInfo . Count ] ;
for ( int i = 0 ; i < s_DocInfo . Count ; ++ i )
{
if ( s_DocInfo [ i ] . csharpExamples . Count == 0 )
{
// Ignore items with no examples
s_ThreadEvents [ i ] = new ManualResetEvent ( true ) ;
s_DocInfo [ i ] . status = TestStatus . Ignored ;
}
else if ( s_DocInfo [ i ] . status == TestStatus . Whitelisted )
{
// Skip white listed items
s_ThreadEvents [ i ] = new ManualResetEvent ( true ) ;
}
else
{
// Queue this example up for testing
s_ThreadEvents [ i ] = new ManualResetEvent ( false ) ;
ThreadPool . QueueUserWorkItem ( TestDocumentationExampleScriptsThreaded , i ) ;
}
}
// Check for errors and build the error output if required.
bool testFailed = false ;
StringBuilder results = new StringBuilder ( ) ;
for ( int i = 0 ; i < s_ThreadEvents . Length ; ++ i )
{
s_ThreadEvents [ i ] . WaitOne ( ) ;
if ( s_DocInfo [ i ] . status == TestStatus . Failed )
{
testFailed = true ;
GenerateFailureMessage ( results , s_DocInfo [ i ] ) ;
}
}
// If a single item has failed then the test is considered a failure.
Assert . IsFalse ( testFailed , results . ToString ( ) ) ;
}
static void GenerateFailureMessage ( StringBuilder output , ScriptingDocMember info )
{
output . AppendLine ( new string ( '-' , 100 ) ) ;
output . AppendLine ( "Name: " + info . name ) ;
output . AppendLine ( "Path: " + info . path + "\n" ) ;
// Print out the example scripts along with their errors.
for ( int i = 0 ; i < info . csharpExamples . Count ; ++ i )
{
var example = info . csharpExamples [ i ] ;
if ( example . compileResults != null && example . compileResults . Errors . HasErrors )
{
output . AppendLine ( "Example Script " + i + ":\n" ) ;
// Add line numbers
var lines = example . code . SplitLines ( ) ;
int lineNumber = 0 ;
int startLine = 0 ;
// Find the first line of code so the line numbers align correctly.
// The compiler will ignore any empty lines at the start.
for ( ; startLine < lines . Length ; ++ startLine )
{
if ( string . IsNullOrEmpty ( lines [ startLine ] ) )
startLine ++ ;
else
break ;
}
for ( ; startLine < lines . Length ; ++ startLine )
{
// Does this line contain an error?
string lineMarker = " " ;
foreach ( CompilerError compileResultsError in example . compileResults . Errors )
{
// Add a mark to indicate this line has a reported error.
if ( compileResultsError . Line == lineNumber )
{
lineMarker = "-" ;
break ;
}
}
output . AppendFormat ( "{0}{1:000} | {2}\n" , lineMarker , lineNumber ++ , lines [ startLine ] ) ;
}
output . Append ( "\n\n" ) ;
output . AppendLine ( ErrorMessagesToString ( example . compileResults ) ) ;
}
}
if ( info . combinedResults != null )
{
output . AppendLine ( "Combined Example Scripts:\n" ) ;
output . AppendLine ( ErrorMessagesToString ( info . combinedResults ) ) ;
}
if ( ! string . IsNullOrEmpty ( info . testRunnerFailure ) )
{
output . AppendLine ( "Test Runner Failure: " + info . testRunnerFailure ) ;
}
}
// Concatenates all the errors into a formated list.
public static string ErrorMessagesToString ( CompilerResults cr )
{
string errorMessages = "" ;
foreach ( CompilerError compilerError in cr . Errors )
{
errorMessages += string . Format ( "{0}({1},{2}): {3}\n" , compilerError . ErrorNumber , compilerError . Line , compilerError . Column , compilerError . ErrorText ) ;
}
return errorMessages ;
}
public static void TestDocumentationExampleScriptsThreaded ( object o )
{
var infoIdx = ( int ) o ;
var info = s_DocInfo [ infoIdx ] ;
try
{
TestScriptsCompile ( info ) ;
}
catch ( Exception e )
{
info . status = TestStatus . Failed ;
info . testRunnerFailure = e . ToString ( ) ;
}
finally
{
s_ThreadEvents [ infoIdx ] . Set ( ) ;
}
}
// Tests all scripts compile for the selected scripting member.
// First attempts to compile all scripts separately, if this fails then compiles them combined as a single example.
public static void TestScriptsCompile ( ScriptingDocMember info )
{
var scripts = info . csharpExamples ;
if ( scripts . Count == 0 )
{
info . status = TestStatus . Ignored ;
return ;
}
// Setup compiler
var providerOptions = new Dictionary < string , string > ( ) ;
providerOptions . Add ( "CompilerVersion" , "v3.5" ) ;
var domProvider = new CSharpCodeProvider ( providerOptions ) ;
var compilerParams = new CompilerParameters
{
GenerateExecutable = false ,
GenerateInMemory = false ,
TreatWarningsAsErrors = false ,
} ;
Assembly [ ] assemblies = AppDomain . CurrentDomain . GetAssemblies ( ) ;
foreach ( var assembly in assemblies )
{
compilerParams . ReferencedAssemblies . Add ( assembly . Location ) ;
}
// Attempt to compile the scripts separately.
bool error = false ;
for ( int i = 0 ; i < scripts . Count ; i ++ )
{
scripts [ i ] . compileResults = domProvider . CompileAssemblyFromSource ( compilerParams , scripts [ i ] . code ) ;
if ( scripts [ i ] . compileResults . Errors . HasErrors )
error = true ;
}
if ( error )
{
// Its possible that the scripts are all part of one example so we should compile them together and see if that works instead.
info . combinedResults = domProvider . CompileAssemblyFromSource ( compilerParams , scripts . Select ( s = > s . code ) . ToArray ( ) ) ;
if ( ! info . combinedResults . Errors . HasErrors )
error = false ;
}
info . status = error ? TestStatus . Failed : TestStatus . Passed ;
}
static HashSet < string > GetWhiteList ( )
{
var textAsset = AssetDatabase . LoadAssetAtPath ( k_WhiteList , typeof ( TextAsset ) ) as TextAsset ;
var whiteList = new HashSet < string > ( ) ;
if ( textAsset )
{
foreach ( var line in textAsset . text . Split ( '\n' ) )
{
whiteList . Add ( line . Replace ( "\r" , "" ) . TrimEnd ( ' ' ) ) ;
}
}
return whiteList ;
}
// Parses the scripting docs and generates our test data.
static void ParseMemberNode ( XmlNode node , string file , string parent , List < ScriptingDocMember > infoList , HashSet < string > whiteList )
{
ScriptingDocMember info = new ScriptingDocMember ( ) ;
info . path = file ;
infoList . Add ( info ) ;
info . parent = parent ;
foreach ( XmlAttribute attr in node . Attributes )
{
// potential tag attributes: name, namespace, type
var attrLowercase = attr . Name . ToLower ( ) ;
if ( attrLowercase == "name" ) info . name = attr . Value ;
else if ( attrLowercase == "namespace" ) info . nspace = attr . Value ;
}
if ( whiteList . Contains ( info . name ) )
info . status = TestStatus . Whitelisted ;
if ( ! string . IsNullOrEmpty ( info . nspace ) )
{
// trim down the namespace to remove UnityEngine and UnityEditor
if ( info . nspace . StartsWith ( "UnityEngine" ) )
{
info . editor = false ;
info . nspace = info . nspace . Remove ( 0 , "UnityEngine" . Length ) ;
}
if ( info . nspace . StartsWith ( "UnityEditor" ) )
{
info . editor = true ;
info . nspace = info . nspace . Remove ( 0 , "UnityEditor" . Length ) ;
}
if ( info . nspace . StartsWith ( "." ) )
info . nspace = info . nspace . Remove ( 0 , 1 ) ;
}
foreach ( XmlNode child in node . ChildNodes )
{
var childNameLowercase = child . Name . ToLower ( ) ;
if ( childNameLowercase == "section" )
{
// see if this section is undoc
for ( int i = 0 ; i < child . Attributes . Count ; i ++ )
{
if ( child . Attributes [ i ] . Name == "undoc" && child . Attributes [ i ] . Value == "true" )
{
infoList . Remove ( info ) ;
break ;
}
}
foreach ( XmlNode grandChild in child . ChildNodes )
{
var codeLangNode = GetExample ( grandChild ) ;
if ( codeLangNode != null )
{
var scriptInfo = new ExampleScript ( ) ;
scriptInfo . code = codeLangNode . InnerXml . Replace ( "<![CDATA[" , "" ) . Replace ( "]]>" , "" ) ;
info . csharpExamples . Add ( scriptInfo ) ;
}
}
}
else if ( childNameLowercase == "member" )
{
ParseMemberNode ( child , file , info . name , infoList , whiteList ) ;
}
}
}
// Extract the cs example code.
static XmlNode GetExample ( XmlNode node )
{
if ( node . Name . ToLower ( ) == "example" )
{
for ( int i = 0 ; i < node . Attributes . Count ; ++ i )
{
if ( node . Attributes [ i ] . Name == "nocheck" && node . Attributes [ i ] . Value == "true" )
return null ;
}
return node . SelectSingleNode ( "code[@lang='cs']" ) ;
}
return null ;
}
}
|
翻译自: https://blogs.unity3d.com/2017/08/18/verifying-the-scripting-docs-fun-with-editortests/
脚本文档
2730
被折叠的 条评论
为什么被折叠?
到【灌水乐园】发言
