Jun 17, 2008

Merging NUnit build results into TFS 2008 Team Build Logs - First Attempts

Paul Stovell was talking about TFS on the internal Readify mailing list recently and was bemoaning the fact that you can't get NUnit test results into the TFS Team Build log in Visual Studio 2008.

Well, I decided to have a crack at it today and see how hard it really was and I managed to get something going.

So here’s where we start.  A completed build with no test results showing.  Pretty normal situation.

clip_image002

Now I wanted to get something in there so I ran some unit tests (from a completely different solution as it turns out) using NUnit and saved the results to an XML file.  Then I wrote an XSLT to transform the nunit xml output into a test result file Visual Studio 2008 would load.

That took some work, and the XSLT is pretty ugly, but eventually I got it to the point where VS loaded the trx file and viewed the results. I then used MSTest to publish the results to the build log (using the /publish options) but even that was a bit problematic as it exposed some issues with GUIDs in the XSLT that I had to deal with, but in the end I got it working, albeit manually at this stage.

It does mean that the build server will need MSTest installed on it, but at least all the development team can have a bit more flexibility in their testing tools, and don't have to migrate all their Nunit tests to MSTest just to get them in the build log.

The process is pretty simple

  1. Run Nunit-console and produce an XML log file
  2. Convert the Nunit XML output to an MSTest test results file (with a .trx extension)
  3. Use MSTest /publish to push merge the trx file with an existing build

So here’s the result, and surprisingly enough it looks just like a normal TFS team build log:

clip_image004

And if I click the build results?  Yep – they load up as expected as well.

clip_image006

And clicking through to an individual test gives me something like this:

clip_image008

And because it's a published build, the test results appear in the data warehouse as well, available for reporting as seen here:

image

At this point this is all just proving a point, and it’s still very much a manual process.  Also I’ve only tried loading test results where all tests are passing but it's working and I think it's a good start for doing things properly.  Assuming there's enough interest I'll try and turn this into something that can be used in a build process - maybe write it as an MSBuild custom task or something like that.

Let me know if you'd like to see me put some more effort into this or if you've got any feedback.

3 comments:

  1. And if you want to know what the format of the .TRX and .VSMDI files are, have a look at Dominic's post:

    http://blogs.msdn.com/dhopton/archive/2008/06/12/helpful-internals-of-trx-and-vsmdi-files.aspx

    ReplyDelete
  2. Have you seen this? It's a plugin for VS2005 to add first class support for nunit in VS' test runner. It's a bit of a snapshot, and it's only for 2005, but it might be an interesting approach...

    Cheers
    Matt

    ReplyDelete
  3. Oh yes. A link would be useful...

    http://www.codeplex.com/NUnitForVS

    ReplyDelete