Get Started With Continuous Integration For Your .NET (C#) Projects - Test Your Project

In the previous post, the project is able to build when new code is checked in to the SCM.  In this post, I will explore how to run unit test cases for your .Net(C#) project.  In a nutshell, integrating the unit test case with Jenkins is nothing but adding one more windows batch command which invokes the mstest.exe command line tool.  There are still a few things to note during the process.

Prerequisites:

1. A unit test project must be created in the solution and there should have some unit test cases in it. This will not be covered in this post, and this post will only cover how to run the existing unit test cases in Jenkins.
   
2. The MSTest.exe command line tool is used to run those test cases. Installing visual studio will install the mstest.exe which is located at:C:\Program Files\Microsoft Visual Studio 10.0\Common7\IDE\MSTest.exe by default.
   
3. The projects including the unit test projects in the solution must be built before running the cases.
   

MSTest Tips

Specify the test cases to be run

The first thing of running unit test cases is to tell MSTest.exe which cases to run.  There are a few options can be used:

1. Use the /testcontainer: parameter. A test container is the dll file built from the unit test project including all the test cases. When using this parameter, all the test cases in the test container will be run.
2. Use the /testmetadata: parameter. Using test container is useful when all the test cases are needed to run.  However, in continuous integration environment, we may only want to run some quick test cases when code is being checked in.  And run other time-consuming acceptance test cases later. Using the test meta data file, you could create a list of test cases and tell mstest.exe which list to be run.  For more information about how to use the test meta date file, refer tothis blog post.

Specify TestResults folder

You may want to specify the test result folder for mstest.exe to store the results and the result file name.  If this is not specified, the result file will be named by mstest.exe using a default naming conversion (with user name, time stamp etc in it).  This is hard for the following steps in the continuous integration to locate the result file. Therefore, you'd better specify the test result folder and result file name using the/resultsfile:.

Note that: if the result file already exists, the mstest.exe will fail.  Therefore, before running the test cases, the folder must be cleaned up.

For more mstest.exe options, refer to the manual (for VS2010) at here.

Integrate MSTest with Jenkins

Now that we've learned about the options of mestest.exe, let's integrate it with Jenkins. Since the mstest.exe is a command line tool, using the Execute Windows Batch Command step is enough.

Below is my script to run the unit test cases, right now, all the cases are run.

:: Remove the directory
RD /S/Q TestResults

:: Create the directory
MKDIR TestResults

:: Run the tests
"C:\Program Files\Microsoft Visual Studio 10.0\Common7\IDE\MSTest.exe" /testcontainer:MyProject_UnitTest\bin\Release\
MyProject_UnitTest.dll /testsettings:Local.testsettings /resultsfile:TestResults\MyProjectTestResults.trx





Note: the project must be built before running the test cases, so this step is a subsequent step of the previous post.

Display the results in Jenkins
Now the unit test cases is able to run for every code check in.  The question now is how to view the test results. Jenkins has built-in support for JUnit test results.  With the JENKINS MSTest plugin, the mstest results can be converted to JUnit test results.
 Install the JENKINS MSTest plugin using the Jenkins Update Center and restart Jenkins.
Go to the job configuration page, you could see one more option in the Post-build actions section, see figure below.  In the text box, enter the test result file the mstest.exe generates in the previous step. That's the reason why we need to tell mstest.exe
 where to store the result file.  Otherwise, the it will not be easy for the MSTest plugin to find the result file here.



Now you should be able to view the test result for each build and the test result trend, like the test case count and the execution duration for the project. See figure below which shows the test case trend of my project.  You can that all the cases are
 passed on all the builds and the growth of the test case count.





Summary
In this post, we explored how to run C# unit test cases in Jenkins.  This is useful to run some quick unit test while code is being checked in.  We also explored how to display the test cases results in Jenkins.  In the next post, we explore integrate Sonar
 with Jenkins to display more quality metrics including the test results have already been presented in Jenkins.  Stay tuned.