In this article, we are going to cover how and why we should test a REST API with .NET and xUnit. We’ll discuss why this is important, and how to actually test various components of an API response to ensure requirements are being met.
Before we jump into code, let’s spend a moment understanding why API testing is important.
Why Should We Test a REST API?
In current application development, the vast majority of client & server architectures involve APIs in some form or another. This is because it allows the client and server to be agnostic of any technical approaches, and simply agree on the contract that will be delivered. For example, we can build a client in ReactJS, and an API server in .NET. We can switch out the tech on either side as long as the contract is adhered to, which will give us an enormous amount of flexibility. It’s for some of these reasons that the popularity of APIs has skyrocketed in the last decade, and therefore we need to adapt our practices to these new software architectures.
The reason we should test a REST API is the same reason we test any software – to ensure expectations meet reality.
Testing a REST API is on both sides:
- Client (the API consumer). From the client side, we need to test the API to ensure it delivers the functionality we need to help us build our user experiences
- Server (the API producer). From the server side, we need to test our API to ensure it delivers the functionality for one or more clients that intend to consume it
Now that we understand why we should test a REST API, let’s talk about how we can do it at a high level.
How Can We Test a REST API?
As we mentioned, agreeing on a contract is usually the first step and provides the foundation for the testing strategy. This can be as simple as a document that both the API producer and consumer agree on, or it is something more automated like OpenAPI. The point is that the contract is the source of truth for the integration between the client and server, and is what we use to test our agreements. The contract includes everything from the URL where the API is hosted, security requirements, what the request and response payloads will look like, and what behavior is expected from each endpoint.
Once we define contracts, we can get down to testing them. In this article, we will test from the client side specifically, but in reality, the tests would be very similar from both sides. The main difference is the client-side tests only test the behavior that we are interested in for our use cases, whilst the server-side will test all use cases.
Let’s first set up a basic REST API to write tests against. It’s worth mentioning here that we will be testing a “live” API (hosted locally, but could be on the internet), so it makes things an “integration test”, as opposed to a “unit test”, which would make use of mocks. Integration tests will give us the best confidence in functionality since it’s hitting the real service we’ll integrate with. The trade-offs of course are speed (because we are going over the network), and not having a predictable state (which we will get into shortly). We could do unit tests by saving the API responses to a JSON file, then mocking HTTP Client to return the mocks. But then we aren’t really testing the API we are using, so let’s stick with integration tests here.
Setting up a Basic REST API
Let’s open up Visual Studio and select the “ASP.NET Core Web API” template type, and untick the box “Use controllers”. This will let us create a REST API using minimal APIs, which is a quick and efficient approach to setting up a very basic API.
Once we have the basic API project scaffolded, let’s begin setting up our REST API.
First, let’s remove all the code in Program.cs
and replace it with some basic setup logic:
using Microsoft.AspNetCore.Mvc; var builder = WebApplication.CreateBuilder(args); var app = builder.Build(); app.UseHttpsRedirection(); var books = new List<Book>(); void InitializeBooks() => books = Enumerable.Range(1, 5) .Select(index => new Book(index, $"Awesome book #{index}")) .ToList(); InitializeBooks(); app.Run(); internal record Book(int BookId, string Title);
The first few lines set up the basic web host, and then we are initializing a set of Book
records, which we’ll use to act as the resource for our REST API.
Creating Basic Endpoints
Now let’s set up the functionality of the REST API. Usually, a REST API has a set of common operations like Get, Create, Update and Delete, so let’s follow that approach and add a few methods:
app.MapGet("/books", () => { return Results.Ok(books); }); app.MapPost("/books", (Book book) => { books.Add(book); return Results.Created($"/books/{book.BookId}", book); }); app.MapPut("/books", (Book book) => { books.RemoveAll(book => book.BookId == book.BookId); books.Add(book); return Results.Ok(book); }); app.MapDelete("/books/{bookId}", (int bookId) => { books.RemoveAll(book => book.BookId == bookId); return Results.NoContent(); });
Nothing too complicated here, as we are just operating on the in-memory list of books and returning the correct status code & payload given the HTTP Verb in use.
Let’s now add an endpoint that can reset the state of our API:
app.MapDelete("/state", () => { InitializeBooks(); return Results.NoContent(); });
We need this as when we run tests against our live API, each test assumes a constant state of data. When we run a test that deletes a book, then we have a test that assumes that the book is present, and our tests can fail. We can use this endpoint to ensure we have a predictable state, which is always a critical goal of writing tests.
Creating a Secure Endpoint
So far the endpoints we’ve created are all anonymous. That is, anyone can access these endpoints without any credentials. In a lot of cases, we will create some operations that we would like secured. So, let’s add an endpoint with some basic security:
app.MapGet("/admin", ([FromHeader(Name = "X-Api-Key")] string apiKey) => { if (apiKey == "SuperSecretApiKey") { return Results.Ok("Hi admin!"); } return Results.Unauthorized(); });
In this endpoint, we are simply checking the value of an HTTP Header supplied against a constant value. If it’s a match we return some data and a successful status code, otherwise, we return an unauthorized response. Obviously in reality our security approach will be more robust than here, using something like JWT and Identity, but for our testing purposes, this will suffice.
If we run our API, we see the endpoints and behavior of our API as we expect.
Test a REST API
Now that we have our API up and running, we can write some basic tests. When it comes to testing an API, there are a few things we should be interested in:
Status Code – This is often the first signal for a successful or failed operation. If an operation succeeds, it’s usually a 2xx code, and a failed operation usually 4xx or 5xx. This is the first thing we will test for. If the status code isn’t as expected, likely the other parts of the response won’t be either, so we will short-circuit and fail fast.
Media Type – All responses should have the Content-Type
header, which tells the consumer what content to expect. This is critical information. It will inform us how to treat the response content and deserialize it into the objects we will operate on. Most APIs today return JSON, so they Content-Type
should be set to application/json
.
Payload / Response Content – Obviously the payload or response content is the most interesting aspect of our integration, so this is the portion of the response that helps us receive this information. Assuming the Content-Type
header is set to application/json
, we can safely assume the content itself is in that format and attempt to deserialize it into a model.
Performance – Depending on our business case, performance is a key focus. The quotas themselves will vary from business to business and even endpoint to endpoint. For example, it would be desirable for ‘hot paths’ to be executed a lot quicker than less used endpoints. These quotas allow us to deliver acceptable experiences to our users.
Setting up the Basics
To begin, let’s add a new xUnit test project to our solution, and replace the test class with our tests.
Next, let’s add a record:
internal record Book(int BookId, string Title);
We could have just referenced the API project and the Book
record directly in Visual Studio, but remember we are testing the API from a client perspective. In many cases, the API is in a different code base, so let’s have our own copies of the contracts.
Next, let’s set up the class:
public class BookLiveTest : IDisposable { }
Notice we are implementing the IDisposable
interface. This allows us to implement the Dispose()
method and clean up any resources after each test run, ensuring a clean and reliable state for our tests.
Let’s also add an instance of HttpClient
, to the running port of our API:
private readonly HttpClient _httpClient = new() { BaseAddress = new Uri("https://localhost:7133") };
Now, let’s create a new class and a few properties to help us with our tests:
public static class TestHelpers { private const string _jsonMediaType = "application/json"; private const int _expectedMaxElapsedMilliseconds = 1000; private readonly JsonSerializerOptions _jsonSerializerOptions = new() { PropertyNameCaseInsensitive = true }; }
Let’s break down each line. First, we set up the “application/json” content type, which is the format we expect our API responses should return. The value of _expectedMaxElapsedMilliseconds
is then set to 1000
. This is what we expect the max API response time to be, which obviously we can tweak as necessary. The JsonSerializerOptions
is set to be case-insensitive. By default, the System.Text.Json deserializer is case sensitive, but the serializer is camel case, causing payload mismatches. This line prevents these issues.
Now that we have the class members, we can implement Dispose()
:
public void Dispose() { _httpClient.DeleteAsync("/state").GetAwaiter().GetResult(); }
Here we are simply calling that special endpoint that resets the state.
Now that we have the basics set up, in the next section, we can look at writing some actual tests.
Common REST API Assertions
Since all our tests assert similar things (but different values), it’s worth creating a few helper methods for reuse.
Let’s add to our TestHelpers
class two new methods:
public static async Task AssertResponseWithContentAsync<T>(Stopwatch stopwatch, HttpResponseMessage response, System.Net.HttpStatusCode expectedStatusCode, T expectedContent) { AssertCommonResponseParts(stopwatch, response, expectedStatusCode); Assert.Equal(_jsonMediaType, response.Content.Headers.ContentType?.MediaType); Assert.Equal(expectedContent, await JsonSerializer.DeserializeAsync<T?>( await response.Content.ReadAsStreamAsync(), _jsonSerializerOptions)); } private static void AssertCommonResponseParts(Stopwatch stopwatch, HttpResponseMessage response, System.Net.HttpStatusCode expectedStatusCode) { Assert.Equal(expectedStatusCode, response.StatusCode); Assert.True(stopwatch.ElapsedMilliseconds < _expectedMaxElapsedMilliseconds); } public static StringContent GetJsonStringContent<T>(T model) => new(JsonSerializer.Serialize(model), Encoding.UTF8, _jsonMediaType);
The second method AssertCommonResponseParts
checks the status code is what is expected and also checks the ElapsedMilliseconds
on a provided stopwatch which helps with our performance assertions.
The first method AssertResponseWithContentAsync
checks the content type is JSON, deserializes it into the supplied type parameter, then calls into the second method for the other assertions.
The last method just helps serialize the JSON we want to submit to the API.
Now we’ve got everything set up, we can begin writing tests.
Writing CRUD Tests
Let’s begin with adding a test for the GetBooks
endpoint:
[Fact] public async Task GivenARequest_WhenCallingGetBooks_ThenTheAPIReturnsExpectedResponse() { // Arrange. var expectedStatusCode = System.Net.HttpStatusCode.OK; var expectedContent = new[] { new Book(1, "Awesome book #1"), new Book(2, "Awesome book #2"), new Book(3, "Awesome book #3"), new Book(4, "Awesome book #4"), new Book(5, "Awesome book #5") }; var stopwatch = Stopwatch.StartNew(); // Act. var response = await _httpClient.GetAsync("/books"); // Assert. await TestHelpers.AssertResponseWithContentAsync(stopwatch, response, expectedStatusCode, expectedContent); }
Thanks to our helper methods, the test is very simple. We set up the expected status code and model, start a stopwatch timer, call the API, and then perform assertions. Let’s run this test, and confirm it passes.
We can now quickly add tests for the other POST, PUT and DELETE:
[Fact] public async Task GivenARequest_WhenCallingPostBooks_ThenTheAPIReturnsExpectedResponseAndAddsBook() { // Arrange. var expectedStatusCode = System.Net.HttpStatusCode.Created; var expectedContent = new Book(6, "Awesome book #6"); var stopwatch = Stopwatch.StartNew(); // Act. var response = await _httpClient.PostAsync("/books", TestHelpers.GetJsonStringContent(expectedContent)); // Assert. await TestHelpers.AssertResponseWithContentAsync(stopwatch, response, expectedStatusCode, expectedContent); } [Fact] public async Task GivenARequest_WhenCallingPutBooks_ThenTheAPIReturnsExpectedResponseAndUpdatesBook() { // Arrange. var expectedStatusCode = System.Net.HttpStatusCode.OK; var updatedBook = new Book(6, "Awesome book #6 - Updated"); var stopwatch = Stopwatch.StartNew(); // Act. var response = await _httpClient.PutAsync("/books", TestHelpers.GetJsonStringContent(updatedBook)); // Assert. TestHelpers.AssertCommonResponseParts(stopwatch, response, expectedStatusCode); } [Fact] public async Task GivenARequest_WhenCallingDeleteBooks_ThenTheAPIReturnsExpectedResponseAndDeletesBook() { // Arrange. var expectedStatusCode = System.Net.HttpStatusCode.NoContent; var bookIdToDelete = 1; var stopwatch = Stopwatch.StartNew(); // Act. var response = await _httpClient.DeleteAsync($"/books/{bookIdToDelete}"); // Assert. TestHelpers.AssertCommonResponseParts(stopwatch, response, expectedStatusCode); }
Again the tests are very quick to write. The only difference is the method we call, the path, and the response type/content.
Testing Security
Now let’s write a passing test for the authenticated endpoint:
[Fact] public async Task GivenAnAuthenticatedRequest_WhenCallingAdmin_ThenTheAPIReturnsExpectedResponse() { // Arrange. var expectedStatusCode = System.Net.HttpStatusCode.OK; var expectedContent = "Hi admin!"; var stopwatch = Stopwatch.StartNew(); var request = new HttpRequestMessage(HttpMethod.Get, "/admin"); request.Headers.Add("X-Api-Key", "SuperSecretApiKey"); // Act. var response = await _httpClient.SendAsync(request); // Assert. await AssertResponseWithContentAsync(stopwatch, response, expectedStatusCode, expectedContent); }
The only difference with this test is we are setting a special HTTP Header, being the API Key.
We also need to write a failing test:
[Theory] [InlineData(null)] [InlineData("")] [InlineData(" ")] [InlineData("WrongApiKey")] public async Task GivenAnUnauthenticatedRequest_WhenCallingAdmin_ThenTheAPIReturnsUnauthorized(string apiKey) { // Arrange. var expectedStatusCode = System.Net.HttpStatusCode.Unauthorized; var stopwatch = Stopwatch.StartNew(); var request = new HttpRequestMessage(HttpMethod.Get, "/admin"); request.Headers.Add("X-Api-Key", apiKey); // Act. var response = await _httpClient.SendAsync(request); // Assert. AssertCommonResponseParts(stopwatch, response, expectedStatusCode); }
This ensures that when we authenticate incorrectly, we will receive the expected Unauthorized HTTP status code.
We now have a full set of API tests, asserting the behavior we expect. Why is this important? As we discussed earlier, the contract is the source of truth for client and server API integration. With this test coverage, we are now able to confidently deliver experiences that make use of this API. The next step is adding this to our CI system so that any changes to the contract trigger a failure. This is of course, however, outside the scope of this article.
Conclusion
In this article, we covered the reasons why it’s important to test a REST API. While we focus on testing the API via xUnit, the techniques of calling an API and asserting the response can apply to any programming language. Instead of written tests, paid tools like Postman offers a lot more features off the shelf. We might look at covering that in a future article.
It’s also worth noting the tests we are writing here focus on the basics and certainly aren’t exhaustive. There are other things we could assert from the API response, depending on our business use case. Hopefully, this article demonstrates how we can approach testing in a concise and reusable way.
shoud I create such(test) endpoints in production code for tests? I think than I cant wrtite such kind of test for real endpoints, because they changing database state. Am i right?
Hi Gerik, thanks for the comments! In theory, yes.. we can write tests against production. It will depend on the use case, and what you are testing. As you say, we don’t really want to be changing state of production data, unless we are certain we can roll it back or keep things in a transaction (often difficult things to do).
Since we are interesting in testing functionality, a better suggestion would be to run these tests on a prod-like environment (staging, for example), before proceeding with a production build.
We could also have ‘smoke tests’ on Production (probably better with a tool like Postman / Runscope / Assertible), but as mentioned it’s probably safer to limit those to basic availability tests, and not stateful operations.
Hope that helps!
Thank you for you answer Ryan
Can we set up a test database and use that for testing? I mean Working with two databases in one project? In Laravel (A PHP framework) when we run the test it creates the database and then we can seed the database and then we can write tests against this test database. Why don’t we have this approach here?
Hi Amir, thanks for your comment.
Yes, 100% we can do that approach in .NET. In this article because we are focused on testing the REST API itself and not the data behind it, we used an in-memory collection for simplicity. Therefore, there is no database to seed or test.
However, if we were using something like EF Core, it includes the ability to setup / seed an inmemory database similar to how you described with Laravel. Can see the MS docs about that here: https://learn.microsoft.com/en-us/ef/core/providers/in-memory/
Hope that helps!
Nice post, Ryan. Thank you.
I am a big fan of integration tests for .NET Web APIs although they sometimes are not easy to understand, write (for my case) and maintain over a longer time. I wrote integration tests in .NET Core 3.1 with WebApplicationFactory and xUnit for a post/poll driven web API with about 5 hosted services running internally to handle queues and data processing. It was an extra layer of tests on top of unit tests. Unfortunately it took quite some time to get right, understand the inner workings and overcome hidden pitfalls due to the polling, data seeding, temp SQLite DB with EF Core and hosted services in our system under test. When learning, I started off from the Pluralsight course made by Steve Gordon and I am glad for any article that covers the topic.
One thing I would suggest is to consider leveraging the IAsyncLifetime interface from xUnit for any asynchronous tear down logic to avoid the GetAwaiter().GetResult() that in our case was causing a lot of flakiness due to deadlocks in the Azure DevOps build pipelines. In addition we had to switch to file system based SQLite. And the IAsyncLifetime was the only option for us because we had to wait for all hosted services to be stopped so they release the DB file lock before moving on with the test clean up like deleting the DB file.
Anyways that’s a lot of details but difficult to figure out and work around when such issues happen. So I wanted to share for anyone coming around.
Hi Jiri, thanks for the comments! I haven’t used
IAsyncLifetime
too much, but definetely sounds reasonable. The other suggestion i would make is we could use Docker, to get a repeatable and clean state for all the tests. However, in this article I just wanted to focus on the principles of REST API testing, so i didn’t mention these challenges too much, other than touching on it for the DELETE state endpoint.Great suggestions and learnings though there for other readers, so thanks for reading and commenting 🙂
The main advantage of such written tests over Postman is that you can easily add them to source control within the project, share with others and work on them together. I’ve switched to some kind of standardized web request runners like: https://www.jetbrains.com/help/rider/Http_client_in__product__code_editor.html. You can prepare file with http requests, sync it with others, run all tests at once, change payloads using external files (based on requirements), automate such runs.
Good point! However note you can also collab/add to source control with Postman. You can save and sync the collection to source control, and use the Newman CLI tool to execute on CI. However there is definitely a bit more ceremony involved.
Thanks for reading and commenting:)