It’s widely recognised by those with a desire to do unit testing that there are better unit test frameworks out there than MSTest but given that Visual Studio has always been so tightly coupled with MSTest it’s always been more difficult than it should be to get other test frameworks working well in Visual Studio. The TestDriven.NET and ReSharper test runners have helped, but the integration back to visual studio was always lacking.
That all changes with the Visual Studio 11. Now it’s a case of “Use the framework that makes you happy. We don’t mind”.
You want to use XUnit? No problem!
MSTest (without the baggage it usually has)? Sure thing!
Something random that we’ve never heard of? Write an adapter and it’ll work just fine!
Dogs and Cats Living Together!Visual Studio 11 introduced a test adapter model so that any test framework can run inside Visual Studio if there is an adapter for it. The adapter model also means that you can not only run XUnit, NUnit, or any other type of test but you can run them in the same test assembly if you really wanted to! Why? I don’t know! But you can :-)
Maybe you have a suite of MSTest tests but you also want use XUnit’s data driven test features because MSTest sucks for unit testing. You can do just that. It’s really nice.
Note: In MSTest’s favour, in this release when MSTest is used in a plain old class library for unit testing the MSTest test adapter uses a cut down, light weight version of MSTest with just the features needed for unit testing and none of the baggage that it normally comes with, making it quite usable for most unit testing needs. For most developers with an existing investment in MSTest tests they will see an improvement in performance as a result.
Pre-RequisitesGet yourself started by loading the appropriate adapter for your unit test framework from the Visual Studio Extension Manager. MSTest is already in the box so you don’t have to worry about that one. In this screen shot I’ve loaded up the XUnit, NUnit and Chutzpah test adapters.
Running Unit TestsCreate a new C# Assembly project (NOT a test project) and add the XUnit and NUnit test frameworks to your project using NuGet
You can also add a reference to Microsoft.VisualStudio.QualityTools.UnitTestFramework so that you can do MSTest based unit tests.
Here, I’ve started by adding a simple XUnit test, then building the code and running the unit test as shown:
As you can see in the output window there is now a “Discover test started” phase where Visual Studio looks at the assemblies and determines what tests are in the system so that it can spin up the right framework and execute the tests.
The Unit Test explorer on the left shows the tests that were run, the time it took and any error information for failed tests.
In the same code I have then added MSTest and NUnit tests as shown, however at this stage I have not yet built the project – take note of the unsaved changes icon in the document tab, indicating that the project is neither saved nor built as yet:
Run Tests After Build (Almost Continuous Testing)Continuous Testing is the idea that as you do your work all the unit tests are constantly running in the background and giving you live feedback when there are problems code by highlighting where tests have failed and where your code is broken. The immediate feedback cycle makes test driven development an even faster development process since there’s no waiting around for all the tests to run.
Visual Studio has taken a step towards this ideal with the “Run Tests After Build” option as shown in the image below. Turn that setting on and as soon as you compile your code Visual Studio will run the tests automatically on a background thread so that you don’t end up with a blocked UI and can get on with coding the next thing on your list.
If a test failed the unit test explorer goes red and it’s obvious that there’s a problem. The interesting thing is that in smaller projects the tests often run so fast that you don’t even notice them happening!
As a tip, once you’ve been using the test after build feature for a while you will probably want to stop the Output window from popping up every time you build so that you don’t have to keep closing it. You can do this in the Visual Studio Options as shown
ConclusionSo there you have it! A brief overview of the new unit testing features in Visual Studio 11 Beta. Go and get it now and start playing with it.
Having Visual Studio automatically running your tests each time you do a build will change your development workflow for the better and keep you more focused on coding and help you stay in the mythical zone if you ever get there.
Remember that since all tests run in the background and this removes the time spent waiting for tests to finish that you can have tens of thousands of tests taking minutes to run each time and you won’t even feel a delay in your development activities. Fantastic!