一个例子:
<?xml version="1.0" encoding="UTF-8"?> <project name="junit" default="createjar"> <property name="dir.src" location="src"/> <property name="dir.test.src" location="test"/> <property name="dir.dest" location="WebContent/WEB-INF/classes"/> <property name="dir.test.dest" location="${dir.dest}/test"/> <property name="dir.test.data" location="${dir.dest}/test/data"/> <property name="dir.test.reports" location="${dir.dest}/test/reports"/> <property name="dir.lib" location="WebContent/WEB-INF/lib"/> <property name="test.class" value="test.diary.core.entity.DiaryTest"/> <path id="src.classpath"> <fileset dir="${dir.lib}"> <include name="*.jar"/> <exclude name="junit-4.10.jar"/> </fileset> </path> <path id="test.src.classpath"> <path refid="src.classpath"/> <pathelement location="${dir.lib}/junit-4.10.jar"/> <pathelement location="${dir.dest}"/> </path> <path id="test.classpath"> <path refid="test.src.classpath"/> <pathelement location="${dir.test.dest}"/> </path> <target name="test" depends="compilesrc,compiletest" description="测试"> <junit printsummary="false" haltonfailure="true"> <classpath refid="test.classpath"/> <formatter type="brief" usefile="false"/> <test name="${test.class}"/> </junit> <echo>测试完成!</echo> </target> <target name="batchtest" depends="compilesrc,compiletest" description="批量测试"> <junit printsummary="false" haltonfailure="true"> <classpath refid="test.classpath"/> <formatter type="brief" usefile="false"/> <!-- 如果定义了单个测试类,则只测试定义的测试类 --> <test name="${testcase}" todir="${dir.test.data}" if="testcase"/> <!-- 如果未定义了单个测试类,则批量测试 --> <batchtest todir="${dir.test.data}" unless="testcase"> <fileset dir="${dir.test.dest}"> <include name="**/*Test.class"/> </fileset> </batchtest> </junit> <echo>批量测试完成!</echo> </target> <target name="batchtestwithhtml" depends="compilesrc,compiletest" description="批量测试并生成html"> <junit printsummary="false" haltonfailure="false"> <classpath refid="test.classpath"/> <formatter type="brief" usefile="false"/> <formatter type="xml"/> <test name="${testcase}" todir="${dir.test.data}" if="testcase"/> <batchtest todir="${dir.test.data}" unless="testcase"> <fileset dir="${dir.test.dest}"> <include name="**/*Test.class"/> </fileset> </batchtest> </junit> <junitreport todir="${dir.test.data}"> <fileset dir="${dir.test.data}"> <include name="TEST-*.xml"/> </fileset> <report format="frames" todir="${dir.test.reports}"/> </junitreport> <echo>批量测试并生成html完成!</echo> </target> <target name="batchtestwithfailandhtml" depends="compilesrc,compiletest" description="批量测试并生成html(遇到失败停止构建)"> <junit printsummary="false" haltonfailure="false" errorProperty="test.failed" failureProperty="test.failed"> <classpath refid="test.classpath"/> <formatter type="brief" usefile="false"/> <formatter type="xml"/> <test name="${testcase}" todir="${dir.test.data}" if="testcase"/> <batchtest todir="${dir.test.data}" unless="testcase"> <fileset dir="${dir.test.dest}"> <include name="**/*Test.class"/> </fileset> </batchtest> </junit> <junitreport todir="${dir.test.data}"> <fileset dir="${dir.test.data}"> <include name="TEST-*.xml"/> </fileset> <report format="frames" todir="${dir.test.reports}"/> </junitreport> <fail if="test.failed"> 测试失败。查看${dir.test.reports} </fail> <echo>批量测试并生成html(遇到失败停止构建)完成!</echo> </target> <target name="createjar" depends="batchtestwithfailandhtml" description="生成Jar"> <jar destfile="${dir.dest}/all.jar"> <fileset dir="${dir.dest}"> <exclude name="**/test/"/> </fileset> </jar> <echo>生成Jar完成!</echo> </target> <target name="compiletest" depends="compilesrc,copytest,maketestdir" description="编译Test中Java文件"> <javac srcdir="${dir.test.src}" destdir="${dir.test.dest}" includeAntRuntime="false" encoding="UTF-8"> <include name="**/*.java"/> <classpath refid="test.src.classpath"/> </javac> <echo>编译Test完成!</echo> </target> <target name="compilesrc" depends="copysrc" description="编译Src中Java文件"> <javac srcdir="${dir.src}" destdir="${dir.dest}" includeAntRuntime="false" encoding="UTF-8"> <include name="**/*.java"/> <classpath refid="src.classpath"/> </javac> <echo>编译Src完成!</echo> </target> <target name="copytest" depends="clean" description="复制Test中非Java文件"> <copy todir="${dir.test.dest}" includeEmptyDirs="false"> <fileset dir="${dir.test.src}"> <exclude name="**/*.java"/> </fileset> </copy> <echo>复制Test中非Java文件完成!</echo> </target> <target name="copysrc" depends="clean" description="复制Src中非Java文件"> <copy todir="${dir.dest}" includeEmptyDirs="false"> <fileset dir="${dir.src}"> <exclude name="**/*.java"/> </fileset> </copy> <echo>复制Src中非Java文件完成!</echo> </target> <target name="maketestdir" description="创建目录"> <mkdir dir="${dir.test.dest}"/> <mkdir dir="${dir.test.data}"/> <mkdir dir="${dir.test.reports}"/> <echo>创建测试目录完成!</echo> </target> <target name="clean" description="清理构建文件"> <delete includeEmptyDirs="true"> <fileset dir="${dir.dest}"> <include name="**/*"/> </fileset> </delete> <echo>清理完成!</echo> </target> </project>
几种输出,或以运行其他Target:
e:\workspace\Diary>ant
Buildfile: e:\workspace\Diary\build.xml
clean:
[echo] 清理完成!
copysrc:
[copy] Copying 1 file to e:\workspace\Diary\WebContent\WEB-INF\classes
[echo] 复制Src中非Java文件完成!
compilesrc:
[javac] Compiling 2 source files to e:\workspace\Diary\WebContent\WEB-INF\cl
asses
[echo] 编译Src完成!
copytest:
[echo] 复制Test中非Java文件完成!
maketestdir:
[mkdir] Created dir: e:\workspace\Diary\WebContent\WEB-INF\classes\test
[mkdir] Created dir: e:\workspace\Diary\WebContent\WEB-INF\classes\test\data
[mkdir] Created dir: e:\workspace\Diary\WebContent\WEB-INF\classes\test\repo
rts
[echo] 创建测试目录完成!
compiletest:
[javac] Compiling 2 source files to e:\workspace\Diary\WebContent\WEB-INF\cl
asses\test
[echo] 编译Test完成!
batchtestwithfailandhtml:
[junit] Testsuite: test.diary.core.entity.DiaryTest
[junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 0.108 sec
[junit]
[junit] ------------- Standard Output ---------------
[junit] diary.core.entity.Diary[id=3498a61c-da2b-46cd-a263-1e6978d74f30,name
=测试名称1,createTime=Thu Jan 17 16:00:43 CST 2013,content=测试内容1]
[junit] ------------- ---------------- ---------------
[junit] Testsuite: test.diary.core.service.DiaryServiceTest
[junit] Tests run: 2, Failures: 0, Errors: 0, Time elapsed: 0.005 sec
[junit]
[junitreport] Processing e:\workspace\Diary\WebContent\WEB-INF\classes\test\data
\TESTS-TestSuites.xml to C:\Users\xujing\AppData\Local\Temp\null86800682
[junitreport] Loading stylesheet jar:file:/D:/dev/ant/ant1.8.4/lib/ant-junit.jar
!/org/apache/tools/ant/taskdefs/optional/junit/xsl/junit-frames.xsl
[junitreport] Transform time: 618ms
[junitreport] Deleting: C:\Users\xujing\AppData\Local\Temp\null86800682
[echo] 批量测试并生成html(遇到失败停止构建)完成!
createjar:
[jar] Building jar: e:\workspace\Diary\WebContent\WEB-INF\classes\all.jar
[echo] 生成Jar完成!
BUILD SUCCESSFUL
Total time: 2 seconds
e:\workspace\Diary>ant batchtest -Dtestcase=test.diary.core.entity.DiaryTest
Buildfile: e:\workspace\Diary\build.xml
clean:
[echo] 清理完成!
copysrc:
[copy] Copying 1 file to e:\workspace\Diary\WebContent\WEB-INF\classes
[echo] 复制Src中非Java文件完成!
compilesrc:
[javac] Compiling 2 source files to e:\workspace\Diary\WebContent\WEB-INF\cl
asses
[echo] 编译Src完成!
copytest:
[echo] 复制Test中非Java文件完成!
maketestdir:
[mkdir] Created dir: e:\workspace\Diary\WebContent\WEB-INF\classes\test
[mkdir] Created dir: e:\workspace\Diary\WebContent\WEB-INF\classes\test\data
[mkdir] Created dir: e:\workspace\Diary\WebContent\WEB-INF\classes\test\repo
rts
[echo] 创建测试目录完成!
compiletest:
[javac] Compiling 2 source files to e:\workspace\Diary\WebContent\WEB-INF\cl
asses\test
[echo] 编译Test完成!
batchtest:
[junit] Testsuite: test.diary.core.entity.DiaryTest
[junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 0.081 sec
[junit]
[junit] ------------- Standard Output ---------------
[junit] diary.core.entity.Diary[id=5168fa53-91f5-4459-a6f2-70416a0f324a,name
=测试名称1,createTime=Thu Jan 17 16:03:56 CST 2013,content=测试内容1]
[junit] ------------- ---------------- ---------------
[echo] 批量测试完成!
BUILD SUCCESSFUL
Total time: 1 second
测试中用到的项目在附件中,里面包含了运行后的输出内容,项目很简单。
JUnit说明
Description
This task runs tests from the JUnit testing framework. The latest version of the framework can be found at http://www.junit.org. This task has been tested with JUnit 3.0 up to JUnit 3.8.2; it won't work with versions prior to JUnit 3.0. It also works with JUnit 4.0, including "pure" JUnit 4 tests using only annotations and no JUnit4TestAdapter
.
Note: This task depends on external libraries not included in the Apache Ant distribution. See Library Dependencies for more information.
Note: You must have junit.jar
available. You can do one of:
- Put both
junit.jar
andant-junit.jar
inANT_HOME/lib
. - Do not put either in
ANT_HOME/lib
, and instead include their locations in yourCLASSPATH
environment variable. - Add both JARs to your classpath using
-lib
. - Specify the locations of both JARs using a
<classpath>
element in a<taskdef>
in the build file. - Leave
ant-junit.jar
in its default location inANT_HOME/lib
but includejunit.jar
in the<classpath>
passed to<junit>
. (since Ant 1.7)
See the FAQ for details.
Tests are defined by nested test
or batchtest
tags (see nested elements).
Parameters
Attribute | Description | Required |
printsummary | Print one-line statistics for each testcase. Can take the values on , off , and withOutAndErr .withOutAndErr is the same as on but also includes the output of the test as written to System.out andSystem.err . | No; default isoff . |
fork | Run the tests in a separate VM. | No; default isoff . |
forkmode | Controls how many Java Virtual Machines get created if you want to fork some tests. Possible values are "perTest" (the default), "perBatch" and "once". "once" creates only a single Java VM for all tests while "perTest" creates a new VM for each TestCase class. "perBatch" creates a VM for each nested<batchtest> and one collecting all nested <test> s. Note that only tests with the same settings offiltertrace , haltonerror , haltonfailure , errorproperty and failureproperty can share a VM, so even if you set forkmode to "once", Ant may have to create more than a single Java VM. This attribute is ignored for tests that don't get forked into a new Java VM. since Ant 1.6.2 | No; default isperTest . |
haltonerror | Stop the build process if an error occurs during the test run. | No; default isoff . |
errorproperty | The name of a property to set in the event of an error. | No |
haltonfailure | Stop the build process if a test fails (errors are considered failures as well). | No; default isoff . |
failureproperty | The name of a property to set in the event of a failure (errors are considered failures as well). | No. |
filtertrace | Filter out Junit and Ant stack frames from error and failure stack traces. | No; default ison . |
timeout | Cancel the individual tests if they don't finish in the given time (measured in milliseconds). Ignored if fork is disabled. When running multiple tests inside the same Java VM (see forkMode), timeout applies to the time that all tests use together, not to an individual test. | No |
maxmemory | Maximum amount of memory to allocate to the forked VM. Ignored if fork is disabled. Note: If you getjava.lang.OutOfMemoryError: Java heap space in some of your tests then you need to raise the size like maxmemory="128m" | No |
jvm | The command used to invoke the Java Virtual Machine, default is 'java'. The command is resolved byjava.lang.Runtime.exec() . Ignored if fork is disabled. | No; default isjava . |
dir | The directory in which to invoke the VM. Ignored if fork is disabled. | No |
newenvironment | Do not propagate the old environment when new environment variables are specified. Ignored if fork is disabled. | No; default isfalse . |
includeantruntime | Implicitly add the Ant classes required to run the tests and JUnit to the classpath in forked mode. | No; default istrue . |
showoutput | Send any output generated by tests to Ant's logging system as well as to the formatters. By default only the formatters receive the output. | No |
outputtoformatters | Since Ant 1.7.0. Send any output generated by tests to the test formatters. This is "true" by default. | No |
tempdir | Where Ant should place temporary files. Since Ant 1.6. | No; default is the project's base directory. |
reloading | Whether or not a new classloader should be instantiated for each test case. Ignore if fork is set to true. Since Ant 1.6. | No; default istrue . |
clonevm | If set to true true, then all system properties and the bootclasspath of the forked Java Virtual Machine will be the same as those of the Java VM running Ant. Default is "false" (ignored if fork is disabled). since Ant 1.7 | No |
logfailedtests | When Ant executes multiple tests and doesn't stop on errors or failures it will log a "FAILED" message for each failing test to its logging system. If you set this option to false, the message will not be logged and you have to rely on the formatter output to find the failing tests. since Ant 1.8.0 | No |
enableTestListenerEvents | Whether Ant should send fine grained information about the running tests to Ant's logging system at the verbose level. Such events may be used by custom test listeners to show the progress of tests. Defaults to false .Can be overridden by a magic property. since Ant 1.8.2 - Ant 1.7.0 to 1.8.1 behave as if this attribute was true by default. | No |
By using the errorproperty
and failureproperty
attributes, it is possible to perform setup work (such as starting an external server), execute the test, clean up, and still fail the build in the event of a failure.
The filtertrace
attribute condenses error and failure stack traces before reporting them. It works with both the plain and XML formatters. It filters out any lines that begin with the following string patterns:
"junit.framework.TestCase" "junit.framework.TestResult" "junit.framework.TestSuite" "junit.framework.Assert." "junit.swingui.TestRunner" "junit.awtui.TestRunner" "junit.textui.TestRunner" "java.lang.reflect.Method.invoke(" "sun.reflect." "org.apache.tools.ant." "org.junit." "junit.framework.JUnit4TestAdapter" " more"
Nested Elements
The <junit>
task supports a nested <classpath>
element that represents a PATH like structure.
As of Ant 1.7, this classpath may be used to refer to junit.jar
as well as your tests and the tested code.
jvmarg
If fork
is enabled, additional parameters may be passed to the new VM via nested <jvmarg>
elements. For example:
<junit fork="yes"> <jvmarg value="-Djava.compiler=NONE"/> ... </junit>
would run the test in a VM without JIT.
<jvmarg>
allows all attributes described in Command-line Arguments.
sysproperty
Use nested <sysproperty>
elements to specify system properties required by the class. These properties will be made available to the VM during the execution of the test (either ANT's VM or the forked VM, if fork
is enabled). The attributes for this element are the same as for environment variables.
<junit fork="no"> <sysproperty key="basedir" value="${basedir}"/> ... </junit>
would run the test in ANT's VM and make the basedir
property available to the test.
syspropertyset
You can specify a set of properties to be used as system properties with syspropertysets.
since Ant 1.6.
env
It is possible to specify environment variables to pass to the forked VM via nested <env>
elements. For a description of the <env>
element's attributes, see the description in the exec task.
Settings will be ignored if fork
is disabled.
bootclasspath
The location of bootstrap class files can be specified using this PATH like structure - will be ignored if fork is not true
or the target VM doesn't support it (i.e. Java 1.1).
since Ant 1.6.
permissions
Security permissions can be revoked and granted during the execution of the class via a nested permissions element. For more information please seepermissions
Settings will be ignored if fork is enabled.
since Ant 1.6.
assertions
You can control enablement of Java 1.4 assertions with an <assertions> subelement.
Assertion statements are currently ignored in non-forked mode.
since Ant 1.6.
formatter
The results of the tests can be printed in different formats. Output will always be sent to a file, unless you set the usefile
attribute to false
. The name of the file is determined by the name of the test and can be set by the outfile
attribute of <test>
.
There are four predefined formatters - one prints the test results in XML format, the other emits plain text. The formatter named brief
will only print detailed information for testcases that failed, while plain
gives a little statistics line for all test cases. Custom formatters that need to implementorg.apache.tools.ant.taskdefs.optional.junit.JUnitResultFormatter
can be specified.
If you use the XML formatter, it may not include the same output that your tests have written as some characters are illegal in XML documents and will be dropped.
The fourth formatter named failure
(since Ant 1.8.0) collects all failing testXXX()
methods and creates a new TestCase
which delegates only these failing methods. The name and the location can be specified via Java System property or Ant property ant.junit.failureCollector
. The value has to point to the directory and the name of the resulting class (without suffix). It defaults to java-tmp-dir/FailedTests.
Attribute | Description | Required |
type | Use a predefined formatter (either xml , plain , brief or failure ). | Exactly one of these. |
classname | Name of a custom formatter class. | |
extension | Extension to append to the output filename. | Yes, ifclassname has been used. |
usefile | Boolean that determines whether output should be sent to a file. | No; default istrue . |
if | Only use formatter if the named property is set. | No; default istrue . |
unless | Only use formatter if the named property is not set. | No; default istrue . |
test
Defines a single test class.
Attribute | Description | Required |
name | Name of the test class. | Yes |
methods | Comma-separated list of names of test case methods to execute. Since 1.8.2 The
If the | No; default is to run all test methods in the suite. |
fork | Run the tests in a separate VM. Overrides value set in <junit> . | No |
haltonerror | Stop the build process if an error occurs during the test run. Overrides value set in <junit> . | No |
errorproperty | The name of a property to set in the event of an error. Overrides value set in <junit> . | No |
haltonfailure | Stop the build process if a test fails (errors are considered failures as well). Overrides value set in <junit> . | No |
failureproperty | The name of a property to set in the event of a failure (errors are considered failures as well). Overrides value set in<junit> . | No |
filtertrace | Filter out Junit and Ant stack frames from error and failure stack traces. Overrides value set in <junit> . | No; default ison . |
todir | Directory to write the reports to. | No; default is the current directory. |
outfile | Base name of the test result. The full filename is determined by this attribute and the extension of formatter . | No; default isTEST- name, where nameis the name of the test specified in the name attribute. |
if | Only run test if the named property is set. | No |
unless | Only run test if the named property is not set. | No |
Tests can define their own formatters via nested <formatter>
elements.
batchtest
Define a number of tests based on pattern matching.
batchtest
collects the included resources from any number of nested Resource Collections. It then generates a test class name for each resource that ends in .java
or .class
.
Any type of Resource Collection is supported as a nested element, prior to Ant 1.7 only <fileset>
has been supported.
Attribute | Description | Required |
fork | Run the tests in a separate VM. Overrides value set in <junit> . | No |
haltonerror | Stop the build process if an error occurs during the test run. Overrides value set in <junit> . | No |
errorproperty | The name of a property to set in the event of an error. Overrides value set in <junit> . | No |
haltonfailure | Stop the build process if a test fails (errors are considered failures as well). Overrides value set in <junit> . | No |
failureproperty | The name of a property to set in the event of a failure (errors are considered failures as well). Overrides value set in<junit> | No |
filtertrace | Filter out Junit and Ant stack frames from error and failure stack traces. Overrides value set in <junit> . | No; default ison . |
todir | Directory to write the reports to. | No; default is the current directory. |
if | Only run tests if the named property is set. | No |
unless | Only run tests if the named property is not set. | No |
Batchtests can define their own formatters via nested <formatter>
elements.
Forked tests and tearDown
If a forked test runs into a timeout, Ant will terminate the Java VM process it has created, which probably means the test's tearDown
method will never be called. The same is true if the forked VM crashes for some other reason.
Starting with Ant 1.8.0, a special formatter is distributed with Ant that tries to load the testcase that was in the forked VM and invoke that class'tearDown
method. This formatter has the following limitations:
- It runs in the same Java VM as Ant itself, this is a different Java VM than the one that was executing the test and it may see a different classloader (and thus may be unable to load the tast class).
- It cannot determine which test was run when the timeout/crash occured if the forked VM was running multiple test. I.e. the formatter cannot work with any
forkMode
other thanperTest
and it won't do anything if the test class contains asuite()
method.
If the formatter recognizes an incompatible forkMode
or a suite
method or fails to load the test class it will silently do nothing.
The formatter doesn't have any effect on tests that were not forked or didn't cause timeouts or VM crashes.
To enable the formatter, add a formatter
like
<formatter classname="org.apache.tools.ant.taskdefs.optional.junit.TearDownOnVmCrash" usefile="false"/>
to your junit
task.
ant.junit.enabletestlistenerevents
magic property
Since Ant 1.8.2 the enableTestListenerEvents
attribute of the task controls whether fine grained logging messages will be sent to the task's verbose log. In addition to this attribute Ant will consult the propertyant.junit.enabletestlistenerevents
and the value of the property overrides the setting of the attribute.
This property exists so that containers running Ant that depend on the additional logging events can ensure they will be generated even if the build file disables them.
Examples
<junit> <test name="my.test.TestCase"/> </junit>
Runs the test defined in my.test.TestCase
in the same VM. No output will be generated unless the test fails.
<junit printsummary="yes" fork="yes" haltonfailure="yes"> <formatter type="plain"/> <test name="my.test.TestCase"/> </junit>
Runs the test defined in my.test.TestCase
in a separate VM. At the end of the test, a one-line summary will be printed. A detailed report of the test can be found in TEST-my.test.TestCase.txt
. The build process will be stopped if the test fails.
<junit printsummary="yes" haltonfailure="yes"> <classpath> <pathelement location="${build.tests}"/> <pathelement path="${java.class.path}"/> </classpath> <formatter type="plain"/> <test name="my.test.TestCase" haltonfailure="no" outfile="result"> <formatter type="xml"/> </test> <batchtest fork="yes" todir="${reports.tests}"> <fileset dir="${src.tests}"> <include name="**/*Test*.java"/> <exclude name="**/AllTests.java"/> </fileset> </batchtest> </junit>
Runs my.test.TestCase
in the same VM, ignoring the given CLASSPATH; only a warning is printed if this test fails. In addition to the plain text test results, for this test a XML result will be output to result.xml
. Then, for each matching file in the directory defined for ${src.tests}
a test is run in a separate VM. If a test fails, the build process is aborted. Results are collected in files named TEST-
name.txt
and written to ${reports.tests}
.
<target name="test"> <property name="collector.dir" value="${build.dir}/failingTests"/> <property name="collector.class" value="FailedTests"/> <!-- Delete 'old' collector classes --> <delete> <fileset dir="${collector.dir}" includes="${collector.class}*.class"/> </delete> <!-- compile the FailedTests class if present --> <javac srcdir="${collector.dir}" destdir="${collector.dir}"/> <available file="${collector.dir}/${collector.class}.class" property="hasFailingTests"/> <junit haltοnerrοr="false" haltonfailure="false"> <sysproperty key="ant.junit.failureCollector" value="${collector.dir}/${collector.class}"/> <classpath> <pathelement location="${collector.dir}"/> </classpath> <batchtest todir="${collector.dir}" unless="hasFailingTests"> <fileset dir="${collector.dir}" includes="**/*.java" excludes="**/${collector.class}.*"/> <!-- for initial creation of the FailingTests.java --> <formatter type="failure"/> <!-- I want to see something ... --> <formatter type="plain" usefile="false"/> </batchtest> <test name="FailedTests" if="hasFailingTests"> <!-- update the FailingTests.java --> <formatter type="failure"/> <!-- again, I want to see something --> <formatter type="plain" usefile="false"/> </test> </junit> </target>
On the first run all tests are collected via the <batchtest/>
element. It's plain
formatter shows the output on the console. The failure
formatter creates a java source file in ${build.dir}/failingTests/FailedTests.java
which extends junit.framework.TestCase
and returns from a suite()
method a test suite for the failing tests.
On a second run the collector class exists and instead of the <batchtest/>
the single <test/>
will run. So only the failing test cases are re-run. The two nested formatters are for displaying (for the user) and for updating the collector class.