Apr 6, 2010

How to Add Assert.Throws() to MSTest

One of the frustrations with MSTest is that it is falling behind the times for unit testing when compared to other testing frameworks.

I find it a real shame this this is happening and that MSTest didn’t gain an Assert.Throws method at any point along the way and that we still have to stick the ExpectedException attribute on our test methods, not to mention the fact that the expected message in the attribute still doesn’t get checked properly.

Now you may be asking “What’s wrong with just using the attribute on the test method?” Simple – consider what happens if the exception your looking for gets thrown by a line other than the one you were expecting it to be thrown on? You will get a false positive.  A test that passes even though it should be breaking.  Nasty stuff that.

I think the Assert.Throws() method from xUnit is so much better because it’s very explicit.  When you use it you know exactly what line the exception should be thrown on and you can also check that the message provided by the exception matches your expectations.

So, why not just use the xUnit asserts thent?  Well, we can.  We can actually keep the MSTest runner and use all the xUnit Asserts instead of the MSTest ones.  Not only do we gain the Assert.Throws() method but we also get xUnits Assert.Equals() method, which can compare arrays and lists, and more.

Here’s how…

  1. Add a reference to xUnit to your test project
  2. Add “using Assert=Xunit.Assert;” to your code
  3. Use the Assert.Throws method as you would in an xUnit test.

Pretty tough, huh?  Here’s an example of it in action:

using System;
using System;
using System.Collections.Generic;
using Microsoft.VisualStudio.TestTools.UnitTesting;
using Assert=Xunit.Assert;

namespace MyTests
{
[TestClass]
public class SomeSillyTests
{
[TestMethod]
public void ShouldThrowNullReferenceException()
{
string myNullString = null;
int length;
Assert.Throws(() => length = myNullString.Length);
}

[TestMethod]
public void ShouldCompareListEquality()
{
List<string> List1 = new List<string>() { "a", "b", "c" };
List<string> List2 = new List<string>() { "a", "b", "c" };
Assert.Equal(List1, List2);
}
}
}

Enjoy your new found Assert freedoms! :-)  P.S. The same techniques can be applied to using the NUnit or MBUnit Asserts or your own Assertion library of choice.

7 comments:

  1. That's insane to the point of brilliance!

    If you're doing some sort of AAA testing though, won't it be obvious if your exception is being thrown in the Act section, rather than in the Arrange or Assert?

    ReplyDelete
  2. Thanks David,

    Re AAA approaches...

    What happens if there's a problem in the arrange section because of, say, a null reference on an object initialiser?

    What if the act section contains multiple statements?

    What if the assert uses a Assert.IsNotNull(anObj.aLinkedObj.aProperty) and aLinkedObj is null?

    It's pretty easy to get things to break in unexpected places, especially when taking a red/green/refactor approach. You don't want the test being red for the wrong reasons :-)

    ReplyDelete
  3. But if you're using something like MSpec, then your code under test will almost certainly be a single line, something like

    Establish context = () => {
    ... lines of set up with possible null references in object initialisers
    }

    Because of = () => {
    Sut.FunctionUnderTest(some, params)
    }

    It should_return_valid_results() {
    Assert.Stuff(here);
    }

    Here, you will know if Sut.FunctionUnderTest throws the exception, through the stacktrace. Personally I find the MSpec stuff a little too clever in terms of its attempt to scan as an English sentence, but I find the general principle works well; if what your testing blows up, it should be the Because Of part (presumably what happens in the setup has already been tested elsewhere).

    But if you're not doing testing in that BDD style and your test code has multiple operations in it, then Assert.Throws is invaluable.

    ReplyDelete
  4. "not to mention the fact that the expected message in the attribute still doesn’t get checked properly" - The official documentation is poorly worded but the message parameter of the attribute is 'the message to display when the test fails' not the 'expected value of the Message property on the Exception that gets thrown'.

    ReplyDelete
  5. Thank you so much!

    ReplyDelete
  6. This is incredibly useful and just what I was looking for, many thanks!

    ReplyDelete
  7. It seems everyone thinks there's a bug with eth ExpectedExceptionAttribute attribute. There is no bug. The second parameter to the attribute is called noExceptionMessage. This message is the message that is displayed when the expected exception does not get thrown from the test. (Maybe the name of the parameter changed from VS2010 to VS2013--but that doesn't seem likely.)

    Don't get me wrong. I'm no fan of this attribute anymore because of the problems it may be hiding and/or the false positives that it may allow reporting. I just wanted to point out that there's no bug.

    ReplyDelete