In this article, we are going to explore Shouldly. Shouldly is a library that improves the quality of our tests.
Before seeing this library in action, let’s see the problems of the default assertions.
Why Use Shouldly?
Let’s start with one of the most important properties of testing and in particular unit testing: readability.
We should care about readability because tests are a part of our software documentation. If a test immediately conveys its purpose, whoever uses the tested component will easily understand how to work with it. Shouldly improves the assertions’ readability thanks to a fluent interface.
This is how we would compare 2 numbers using the MSTest framework:
var result = 2; Assert.IsTrue(result < 5);
And this is how we can express the same assertion using Shouldly:
var result = 2; result.ShouldBeLessThan(5);
This is a very simple example, but the second method is clearer at first glance. Shouldly has many other fluent assertions that make assertions much more powerful and compact. This is accomplished with generic and ad-hoc extension methods that Shouldly implements.
Shouldly improves error messages as well. Starting from the above examples, with result == 6, the MSTest error message would be:
Error Message: Assert.IsTrue failed. Stack Trace: at ...
While this is what Shouldly tells us:
Error Message:
Test method ... threw exception:
Shouldly.ShouldAssertException: result
should be less than
5
but was
6
Stack Trace:
...
This should convince us that Shouldly is much better than MSTest in helping us find a bug in our code. Not only is the error message more descriptive, but Shouldly parses the source code to find where the error is coming from.
Let’s see it in action!
How to Refactor Our Tests Using Shouldly?
Let’s create a class Book:
public class Book
{
public string Title { get; init; }
private int _numEditions;
public int NumEditions
{
get => _numEditions;
}
private string _text;
public string Text
{
get => _text;
}
private IList<string> _authors;
public IList<string> Authors
{
get => _authors.ToList();
}
public Book(string title, string text, IList<string> authors)
{
Title = title;
_numEditions = 1;
_text = text;
_authors = new List<string>(authors);
}
}
New editions of a book could be published, so let’s add a method to do that:
public void PublishNewEdition()
{
_numEditions++;
}
And a test for it:
[TestMethod]
public void GivenABook_WhenANewEditionIsPublished_ThenEditionIsIncreased()
{
var book = new Book("title", "text", new[] { "author1" });
var oldEdition = book.NumEditions;
book.PublishNewEdition();
// Assert.AreEqual(oldEdition + 1, book.NumEditions);
book.NumEditions.ShouldBe(oldEdition + 1);
}
The ShouldBe() assertion method compares the objects for equality using the Equal() method. The commented line in the above test (and in the next ones) represents the corresponding assertions without Shouldly.
Now, let’s implement a method to add a new author to an existing book:
public void AddAuthor(string author)
{
if (_authors.Contains(author))
throw new ArgumentException($"{author} already present");
_authors.Add(author);
}
We have to create two tests, one for each branch:
[TestMethod]
public void GivenABook_WhenAnAuthorIsAdded_ThenItIsIncludedInAuthors()
{
var book = new Book("title", "text", new[] { "author1" });
book.AddAuthor("author2");
// CollectionAssert.Contains(book.Authors.ToList(), "author2");
book.Authors.ShouldContain("author2");
}
[TestMethod]
public void GivenABook_WhenThereIsDuplicatedAuthor_ThenThrowArgumentException()
{
var book = new Book("title", "text", new[] { "author1" });
// Assert.ThrowsException<ArgumentException>(() => book.AddAuthor("author1"));
Should.Throw<ArgumentException>(() => book.AddAuthor("author1"));
}
The first test method uses ShouldContain() to verify that book.Authors contains "author2". The second test method uses Should.Throw() to verify that the expected exception is effectively thrown when the code in the lambda body gets executed.
This time let’s add a method to append a new chapter to the book’s text:
public void AddChapter(string chapter)
{
_text += Environment.NewLine + chapter;
}
And a test for this method too:
[TestMethod]
public void GivenABook_WhenAddNewChapter_ThenChapterIsAppended()
{
var book = new Book("title", "text", new[] { "author1" });
book.AddChapter("new chapter");
// StringAssert.EndsWith(book.Text, "new chapter");
book.Text.ShouldEndWith("new chapter");
}
In this test, we use ShouldEndWith() to verify that Text ends with "new chapter".
As for now, we have used Shouldly to improve readability, but this library can offer a lot more.
How to Write Assertions on a Dictionary
Let’s introduce a new feature in our Book class: a bag of words. The BOW (Bag Of Words) consists in a dictionary where the keys are the words of a text and the values are their frequencies in the same text.
Let’s write a method to retrieve the BOW:
public IDictionary<string, int> GetBagOfWords()
{
var bow = new Dictionary<string, int>();
foreach (var word in Text.Split(' '))
{
if (!bow.TryAdd(word, 1))
bow[word]++;
}
return bow;
}
For example, if the book’s text was "first word and second word", the result of GetBagOfWords() would be:
{
"first": 1,
"word": 2,
"and": 1,
"second": 1
}
Because the word "word" appears twice, while the other words appear only once.
Let’s write a test for this method as well:
[TestMethod]
public void GivenABook_WhenAskForBow_ThenReturnBow()
{
var book = new Book("title", "word1 word2 word1", new[] { "author1" });
var bow = book.GetBagOfWords();
bow.ShouldContainKeyAndValue("word1", 2);
bow.ShouldContainKeyAndValue("word2", 1);
}
We can use ShouldContainKeyAndValue() exclusively with dictionaries to verify whether the specified key and value exist in the target dictionary.
We can improve this test even further!
Improving Our Tests With Soft Assertions
Soft assertions are different from hard assertions, the most frequently used, because they do not immediately throw an exception when an assertion fails. Instead, they wait until all the assertion results are available, then they show us the exceptions, if any.
For example, if the assertions were:
bow.ShouldContainKeyAndValue("word1", 3);
bow.ShouldContainKeyAndValue("word2", 2);
The test would throw an exception after the first assertion because these are hard assertions.
The problem is that we don’t see the result of the second assertion, which might still fail even after we fix the first one. In general, when we have multiple assertions in a test, we should convert them to soft assertions:
[TestMethod]
public void GivenABook_WhenAskForBow_ThenReturnBow()
{
var book = new Book("title", "word1 word2 word1", new[] { "author1" });
book.GetBagOfWords()
.ShouldSatisfyAllConditions(
bow => bow.ShouldContainKeyAndValue("word1", 2),
bow => bow.ShouldContainKeyAndValue("word2", 1)
);
}
ShouldSatisfyAllConditions() aggregates all the assertion results and then throws a ShouldAssertException that summarises the results if necessary. If the key-value pairs in the assertions were <"word1", 3> and <"word2", 2>, then the error shown would be:
Failed GivenABook_WhenAskForBow_ThenReturnBow [471 ms]
Error Message:
Test method Tests.BookUnitTest.GivenABook_WhenAskForBow_ThenReturnBow threw exception:
Shouldly.ShouldAssertException: book.GetBagOfWords()
should satisfy all the conditions specified, but does not.
The following errors were found ...
--------------- Error 1 ---------------
bow => bow
should contain key
"word1"
with value
3
but value was
2
--------------- Error 2 ---------------
bow => bow
should contain key
"word2"
with value
2
but value was
1
-----------------------------------------
Stack Trace:
...
We can do more with Shouldly, let’s continue to explore it!
Other Interesting Features of Shouldly
The first feature we are going to see concerns dynamic objects.
How to Assert Dynamic Properties
Let’s first create an expando object:
dynamic expando = new ExpandoObject(); expando.DynamicProperty = 1;
Let’s assert to check if the property exists:
DynamicShould.HaveProperty(expando, "DynamicProperty");
The HaveProperty() method accepts as the first argument any type of object. It uses reflection to inspect the object’s properties if this object has no dynamic behavior.
In this case, an expando object is cast to IDictionary<string, string> which it implements and the property name is searched among this dictionary’s keys.
How to Assert Our Code Runs Within a Certain Timespan
Let’s imagine we have a new requirement for the Book class. The customer wants to receive the BOW within 5 seconds. How can we do that? Shouldly provides the method Should.CompleteIn():
[TestMethod]
public void GivenABook_WhenAskForBow_ThenCompletesWithin5Seconds()
{
var book = new Book("title", "text", new[] { "author1" });
Should.CompleteIn(
() =>
{
book.GetBagOfWords();
},
TimeSpan.FromSeconds(5)
);
}
The first argument of CompleteIn() is an Action with the code that must be executed within the specified timespan.
We should use this method very carefully because a test must be deterministic and repeatable. This means that a test must always return the same result when we run it with the same configuration. If the book’s text in the test was much longer and the host is performing badly, the test could fail.
For this reason, it is a good idea to add a test like the one above if are checking for deadlocks, for example.
Conclusion
Shouldly can certainly improve the readability of our tests and the error messages we receive. It can do a lot more than what we have seen, it supports approval testing as well, for example. The documentation shows several examples of all the methods available.
