Testing Exceptions in MSTest, NUnit, and xUnit

In this article, we will take a look at testing if our code throws exceptions, why we want to do that, and how to do it in the MSTest, NUnit, and xUnit frameworks.

Let’s start!

Why Do We Need to Test if Exceptions Are Thrown?

When writing software, there are use cases where we have to throw exceptions, and properly handle them, to prevent unwanted behavior in our applications. As we write unit tests during our development work, we need to ensure our tests assert proper exception handling, the so-called sad paths. This is as important as testing the happy paths, where everything executes as expected. Testing both the expected and unexpected ensures good code coverage.

Let’s lay the groundwork for our testing:

public class Hero
{
    private const int ExperienceNeeded = 1000;

    public int Level { get; set; }
    public int Experience { get; set; }

    public Hero(int experience)
    {
        Level = 0;
        Experience = experience;
    }

    public void LevelUp()
    {
        if (Experience - ExperienceNeeded < 0)
        {
            throw new ArgumentOutOfRangeException(
                nameof(Experience),
                "Not enough Experience to level up!");
        }

        Experience -= ExperienceNeeded;
        Level++;
    }
}

We define a Hero class with two properties – Level and Experience, which we pass to the constructor. We have a private constant, which indicates the experience needed to level up, and the LevelUp method. It throws an ArgumentOutOfRangeException if we have insufficient Experience otherwise, it increases the Level of our Hero.

Now, that we have things ready, let’s start testing.

Testing for Exceptions Thrown by Synchronous Code

We will look at the three different .NET testing frameworks – MSTest, NUnit, and xUnit. All three contain a version of the Assert class that we use to test our applications. The class itself has built-in methods for testing exceptions.

For each framework, we will test if an Exception of the ArgumentOutOfRangeException type is thrown when we try to invoke the LevelUp method when our Hero has insufficient Experience.

Testing Synchronous Code for Exceptions With MSTest

Test methods in MSTest are marked as such with the [TestMethod] attribute:

[TestMethod]
public void GivenInsufficientExperience_WhenLevelUpIsInvoked_ThenExceptionIsThrown()
{
    // Arrange
    var hero = new Hero(500);

    // Act
    Action act = () => hero.LevelUp();

    // Assert
    Assert.ThrowsException<ArgumentOutOfRangeException>(act);
    Assert.ThrowsException<ArgumentOutOfRangeException>(() => hero.LevelUp());
}

We only need to initialize our Hero with an Experience of less than 1000. The Assert class in MSTest has a generic ThrowsException<T> method that we use to test if an Exception is thrown. It takes an Action delegate as a parameter and we can either define it beforehand or directly inside the method using a lambda expression.

We can also use attributes to test exceptions:

[TestMethod]
[ExpectedException(typeof(ArgumentOutOfRangeException))]
public void GivenInsufficientExperience_WhenLevelUpIsInvoked_ThenExceptionIsThrownWithoutAssert()
{
    // Arrange
    var hero = new Hero(500);

    // Act
    hero.LevelUp();
}

After decorating the method with a [TestMethod] attribute, we add another attribute – [ExpectedException()] and pass it the type of Exception we expect to get. In our case this is ArgumentOutOfRangeException. We only need the Arrange and Act parts in the test method body, the attribute provides the assertion part. 

NUnit and xUnit don’t utilize attributes for testing exceptions.

Testing Synchronous Code for Exceptions With NUnit

We mark test methods in NUnit with the [Test] attribute:

[Test]
public void GivenInsufficientExperience_WhenLevelUpIsInvoked_ThenExceptionIsThrown()
{
    // Arrange
    var hero = new Hero(500);

    // Act
    TestDelegate act = hero.LevelUp;

    // Assert
    Assert.Throws<ArgumentOutOfRangeException>(act);
    Assert.Throws<ArgumentOutOfRangeException>(hero.LevelUp);
}

We have the same setup as with MSTest. The NUnit version of the Assert class has a Throws<T> method instead of ThrowsException<T> and it requires a TestDelegate. This is a custom type of delegate that the framework provides. We have the option to either declare it before or directly pass it, without a lambda expression, when invoking the Throws<T> method.

Testing Synchronous Code for Exceptions With xUnit

In xUnit, we use the [Fact] attribute to designate our test methods:

[Fact]
public void GivenInsufficientExperience_WhenLevelUpIsInvoked_ThenExceptionIsThrown()
{
    // Arrange
    var hero = new Hero(500);

    // Act
    Action act = () => hero.LevelUp();

    // Assert
    Assert.Throws<ArgumentOutOfRangeException>(act);
    Assert.Throws<ArgumentOutOfRangeException>(() => hero.LevelUp());
}

The method body is nearly identical to that of the other test frameworks. The Assert class has the same Throws<T> method we have in NUnit, but this time it takes an Action delegate. The delegate can be both pre-defined or passed directly to the Throws<T> method.

Testing for Exceptions Thrown by Asynchronous Code

Nowadays most applications rely on asynchronous code for better performance. Because of this, we need to be able to test if exceptions are thrown in async methods as well:

public async Task LevelUpAsync()
{
    if (Experience - ExperienceNeeded < 0)
    {
        throw new ArgumentOutOfRangeException(
            nameof(Experience),
            "Not enough Experience to level up asynchronously!");
    }

    Experience -= ExperienceNeeded;
    Level++;

    await Task.CompletedTask;
}

We expand our Hero class by adding an asynchronous version of our LevelUp method. The difference here is that the method is marked as async, has a return type of Task instead of void, and we await Task.CompletedTask.

Testing Asynchronous Code for Exceptions With MSTest

Writing tests for async code in MSTest is very similar to doing so for synchronous code:

[TestMethod]
public async Task GivenInsufficientExperience_WhenLevelUpAsyncIsInvoked_ThenExceptionIsThrown()
{
    // Arrange
    var hero = new Hero(500);

    // Act
    Func<Task> act = hero.LevelUpAsync;

    // Assert
    await Assert.ThrowsExceptionAsync<ArgumentOutOfRangeException>(act);
    await Assert.ThrowsExceptionAsync<ArgumentOutOfRangeException>(() => hero.LevelUpAsync());
}

We set up our test method as async and return a result of Task type. We use the Assert‘s ThrowsExceptionAsync<T> method that accepts a Func<T> delegate that, to no surprise, can be either initialized before we call the method, or directly inside the brackets. In our case, we have Func<Task> so we need to await the assertions.

Testing Asynchronous Code for Exceptions With NUnit

NUnit stays true to itself when it comes to asynchronous code by providing another custom delegate:

[Test]
public void GivenInsufficientExperience_WhenLevelUpAsyncIsInvoked_ThenExceptionIsThrown()
{
    // Arrange
    var hero = new Hero(500);

    AsyncTestDelegate act = hero.LevelUpAsync;

    // Assert
    Assert.ThrowsAsync<ArgumentOutOfRangeException>(act);
    Assert.ThrowsAsync<ArgumentOutOfRangeException>(hero.LevelUpAsync);
}

Note that we have a void method here due to the ThrowsAsync<T> method’s return type. It returns T? and not a Task, so we don’t need an async test method. Another particularity here is that the ThrowsAsync<T> takes in a custom type – AsyncTestDelegate, that can be both initialized or passed directly to the method.

Testing Asynchronous Code for Exceptions With xUnit

xUnit can handle async code as well:

[Fact]
public async Task GivenInsufficientExperience_WhenLevelUpAsyncIsInvoked_ThenExceptionIsThrown()
{
    // Arrange
    var hero = new Hero(500);

    // Act
    Func<Task> act = () => hero.LevelUpAsync();

    // Assert
    await Assert.ThrowsAsync<ArgumentOutOfRangeException>(act);
    await Assert.ThrowsAsync<ArgumentOutOfRangeException>(() => hero.LevelUpAsync());
}

The designated assertion method is the ThrowsAsync<T> method. It takes in a Func<Task> and returns a Task<T> where T is the Exception type we expect to get. Because of the return type Task<T>we need to await the assertion.

Testing for Specific Exception Messages

Exceptions can also carry very helpful messages. We, as developers, often have to throw such exceptions with custom messages, and here comes the necessity to ensure that we not only get the expected Exception but the correct message with it.

Next, we’ll see what the different frameworks have as options.

Testing for Specific Exception Messages in MSTest

Let’s see how MSTest handles checking messages:

[TestMethod]
public void GivenInsufficientExperience_WhenLevelUpIsInvoked_ThenExceptionIsThrownWithCorrectMessage()
{
    // Arrange
    var hero = new Hero(500);

    // Act
    var exception = Assert.ThrowsException<ArgumentOutOfRangeException>(() => hero.LevelUp());

    // Assert
    Assert.IsNotNull(exception);
    Assert.AreEqual("Not enough Experience to level up! (Parameter 'Experience')", exception.Message);
}

Here our test method is similar to previous work but instead, we assign the result of ThrowsException<T> to the exception variable. We first use the IsNotNull method to make sure we get the expected ArgumentOutOfRangeException. Then we use the AreEqual method which takes two string parameters – first the expected result and then the result we get.

For asynchronous methods:

[TestMethod]
public async Task GivenInsufficientExperience_WhenLevelUpAsyncIsInvoked_ThenExceptionIsThrownWithCorrectMessage()
{
    // Arrange
    var hero = new Hero(500);

    // Act
    var exception = await Assert.ThrowsExceptionAsync<ArgumentOutOfRangeException>(() => hero.LevelUpAsync());

    // Assert
    Assert.IsNotNull(exception);
    Assert.AreEqual("Not enough Experience to level up asynchronously! (Parameter 'Experience')", exception.Message);
}

The method can stay mostly the same, we just need to make it an async one that returns a Task, await the assignment of the exception variable, and update the expected error message.

Testing for Specific Exception Messages in NUnit

Let’s check how different this is in NUnit:

[Test]
public void GivenInsufficientExperience_WhenLevelUpIsInvoked_ThenExceptionIsThrownWithCorrectMessage()
{
    // Arrange
    var hero = new Hero(500);

    // Act
    var exception = Assert.Throws<ArgumentOutOfRangeException>(hero.LevelUp);

    // Assert
    Assert.That(exception, Is.Not.Null);
    Assert.That(exception.Message, Is.EqualTo("Not enough Experience to level up! (Parameter 'Experience')"));
}

The main difference here comes from the Assert class as we only use the That method. First, we pass the result we get, then we need to pass the expected result in the form of IResolveConstraint. This sounds very unfamiliar if you are new to the framework, but we use the NUnit‘s Is class to handle that.

For async methods we need minor adjustments:

var exception = Assert.ThrowsAsync<ArgumentOutOfRangeException>(hero.LevelUpAsync);

Here we replace the Throws<T> method with the ThrowsAsync<T> method. Then we replace the name of the method we pass, and finally, we update the expected message.

Testing for Specific Exception Messages in xUnit

Testing for specific Exception messages in xUnit is almost identical to MSTest:

[Fact]
public void GivenInsufficientExperience_WhenLevelUpIsInvoked_ThenExceptionIsThrownWithCorrectMessage()
{
    // Arrange
    var hero = new Hero(500);

    // Act
    var exception = Assert.Throws<ArgumentOutOfRangeException>(() => hero.LevelUp());

    // Assert
    Assert.NotNull(exception);
    Assert.Equal("Not enough Experience to level up! (Parameter 'Experience')", exception.Message);
}

Here we assign the result of Throws<T> to a variable and use it for the assertions. We use the NotNull and AreEqual methods. Here, the order of parameters in the AreEqual method is opposite to NUnit; first is the expected result followed by the one we get.

Let’s check the adjustments needed for async methods:

[Fact]
public async Task GivenInsufficientExperience_WhenLevelUpAsyncIsInvoked_ThenExceptionIsThrownWithCorrectMessage()
{
    // Arrange
    var hero = new Hero(500);

    // Act
    var exception = await Assert.ThrowsAsync<ArgumentOutOfRangeException>(hero.LevelUpAsync);

    // Assert
    Assert.NotNull(exception);
    Assert.Equal("Not enough Experience to level up asynchronously! (Parameter 'Experience')", exception.Message);
}

The updates here are identical to the ones in MSTest – we make the method async returning a Task, await the result of ThrowsAsync<T>, and assign it to a variable. Finally, we update the expected error message.

How to Improve Exception Testing

No matter how similar the three frameworks are, differences might make you stumble when switching between them. Here comes the FluentAssertions library. In this article, we will cover only the part that concerns exceptions, but for a detailed look, you can check out the full article on the topic.

Let’s see what improvements we can make:

[Fact]
public async Task GivenInsufficientExperience_WhenLevelUpIsInvoked_ThenExceptionIsThrownWithFluentAssertions()
{
    // Arrange
    var hero = new Hero(500);

    // Act
    Action actSync = () => hero.LevelUp();
    Func<Task> actAsync = () => hero.LevelUpAsync();

    // Assert
    actSync.Should().Throw<ArgumentOutOfRangeException>()
        .WithMessage("Not enough Experience to level up! (Parameter 'Experience')");

    await actAsync.Should().ThrowAsync<ArgumentOutOfRangeException>()
        .WithMessage("Not enough Experience to level up asynchronously! (Parameter 'Experience')");
}

Using what we already know, we create the actSync and actAsync variables that we will use to test both versions of our LevelUp method. The library uses the Fluent API approach so we can chain method calls on our two variables. We need to start with the Should method, then depending on what our actual method is, we call either the Throw<T> or ThrowAsync<T> method.

For a complete assertion, we finish with the WithMessage method, to which we pass the expected message that our ArgumentOutOfRangeException has. The example above uses xUnit, but this approach applies to the other two frameworks as well. 

Conclusion

In this article, we learned the value of asserting that our code throws the exceptions we expect and how we can achieve this in three different testing frameworks. Checking for expected Exception types and messages, no matter if the code is synchronous or asynchronous, is fully supported and can even be improved by leveraging the FluentAssertions library.