Using Streams with HttpClient to Improve Performance and Memory Usage

Up until now, we were using strings to create a request body and also to read the content of the response. But we can optimize our application by improving performance and memory usage with streams. So, in this article, we are going to learn how to use streams with HttpClient while sending requests and reading the content from responses. We are going to use streams with only GET and POST requests because the logic from the POST request can be applied to PUT, and PATCH.

You can also visit our HttpClient Tutorial page, to see all the articles from this tutorial.

So, let’s start.

More About Streams

The stream represents an abstraction of a sequence of bytes in the form of files, input/output devices, or network traffic. The Stream class in C# is an abstract class that provides methods to transfer bytes – read from or write to the source. Since we can read from or write to a stream, this enables us to skip creating variables in the middle (for the request body or response content) that can increase memory usage or decrease performance.

The vital thing to know here is that working with streams on the client side doesn’t have to do anything with the API level. This is totally a separate process. Our API may or may not work with streams but this doesn’t affect the client side. In the client application, we can use streams to prepare a request body or to read from a response regardless of the API implementation. This is an advantage for sure since we can use streams in the client apps to increase performance and decrease memory usage and still consume any API.

Using Streams with HttpClient to Fetch the Data

In the first article of this series, we have learned that while fetching the data from the API, we have to:

• Send a request to the API’s URI
• Wait for the response to arrive
• Read the content from the response body with the ReadAsStringAsync method
• And deserialize the content using System.Text.Json

As we said, with streams, we can remove that action in the middle where we use the ReadAsStringAsync method to read the string content from the response body.

So, let’s see how to do that.

First of all, we are going to create a new HttpClientStreamService in the client application:

public class HttpClientStreamService : IHttpClientServiceImplementation
{
    private static readonly HttpClient _httpClient = new HttpClient();
    private readonly JsonSerializerOptions _options;

    public HttpClientStreamService()
    {
        _httpClient.BaseAddress = new Uri("https://localhost:5001/api/");
        _httpClient.Timeout = new TimeSpan(0, 0, 30);
        _httpClient.DefaultRequestHeaders.Clear();

        _options = new JsonSerializerOptions { PropertyNameCaseInsensitive = true };
    }

    public async Task Execute()
    {
        throw new NotImplementedException();
    }
}

This is a standard configuration that we’ve seen a couple of times in this series.

Next, we can create a method to send a GET request using streams:

private async Task GetCompaniesWithStream()
{
    using (var response = await _httpClient.GetAsync("companies"))
    {
        response.EnsureSuccessStatusCode();

        var stream = await response.Content.ReadAsStreamAsync();

        var companies = await JsonSerializer.DeserializeAsync<List<CompanyDto>>(stream, _options);
    }
}

In this method, we use the GetAsync shortcut method to fetch the data from the API. But as we have explained in the first article of this series, you can use the HttpRequestMessage class to do the same thing with higher control over the request. Also, pay attention that this time we are wrapping our response inside the using directive since we are working with streams now.

After we ensure the successful status code, we use the ReadAsStreamAsync method to serialize the HTTP content and return it as a stream. With this in place, we remove the need for string serialization and crating a string variable.

As soon as we have our stream, we call the JsonSerializer.DeserializeAsync method to read from a stream and deserialize the result into the list of company objects.

Before we start our apps, we have to call this method in the Execute method:

public async Task Execute()
{
    await GetCompaniesWithStream();
}

And also, register this new service in the Program class:

private static void ConfigureServices(IServiceCollection services)
{
    //services.AddScoped<IHttpClientServiceImplementation, HttpClientCrudService>();
    //services.AddScoped<IHttpClientServiceImplementation, HttpClientPatchService>();
    services.AddScoped<IHttpClientServiceImplementation, HttpClientStreamService>();
}

That’s it. We can start both apps and inspect the result:

We can see, we have our result read from a stream.

Additional Improvements with HttpCompletionMode

In the previous example, we removed a string creation action when we read the content from the response. And as such, we made an improvement. But, we can improve the solution even more by using HttpCompletionMode. It is an enumeration having two values that control at what point the HttpClient’s actions are considered completed.

The default value is HttpCompletionMode.ResponseContentRead. It means that the HTTP operation is complete only when the entire response is read together with content. This is the case with our previous example.

The second value is HttpCompletionMode.ResponseHeadersRead. When we choose this option in our HTTP request, we state that the operation is complete when the response headers are fully read. At this point, the response body doesn’t have to be fully processed at all. This obviously means that we are going to use less memory because we don’t have to keep an entire content inside the memory. Also, this affects performance since we can work with the data faster.

To implement this improvement, all we have to do is to modify the GetAsync method in the GetCompaniesWithStream method:

private async Task GetCompaniesWithStream()
{
    using (var response = await _httpClient.GetAsync("companies", HttpCompletionOption.ResponseHeadersRead))
    {
        response.EnsureSuccessStatusCode();

        var stream = await response.Content.ReadAsStreamAsync();

        var companies = await JsonSerializer.DeserializeAsync<List<CompanyDto>>(stream, _options);
    }
}

And that’s all it takes.

If we run our application, we will see the same result as we had in a previous example. But this time, we improve our request even more.

Steve Gordon has an excellent article on this topic, so feel free to check it out to see the benchmark results as well.

Now, let’s see how to use streams with a POST request.

Sending a POST Request Using Streams with HttpClient

In our second article of the series, we have learned how to send a POST request using HttpClient. In that example, we were serializing our payload into a JSON string before we send the request. Of course, with streams, we can skip that part. Let’s see how.

First, let’s create a new method:

private async Task CreateCompanyWithStream()
{
    var companyForCreation = new CompanyForCreationDto
    {
        Name = "Eagle IT Ltd.",
        Country = "USA",
        Address = "Eagle IT Street 289"
    };

    var ms = new MemoryStream();
    await JsonSerializer.SerializeAsync(ms, companyForCreation);
    ms.Seek(0, SeekOrigin.Begin);

    var request = new HttpRequestMessage(HttpMethod.Post, "companies");
    request.Headers.Accept.Add(new MediaTypeWithQualityHeaderValue("application/json"));

    using (var requestContent = new StreamContent(ms))
    {
        request.Content = requestContent;
        requestContent.Headers.ContentType = new MediaTypeHeaderValue("application/json");

        using (var response = await _httpClient.SendAsync(request, HttpCompletionOption.ResponseHeadersRead))
        {
            response.EnsureSuccessStatusCode();

            var content = await response.Content.ReadAsStreamAsync();
            var createdCompany = await JsonSerializer.DeserializeAsync<CompanyDto>(content, _options);
        }	
    }	
}

In this method, we start by creating a new companyForCreation object with all the required properties. Then, we need a memory stream object. With the JsonSerializer.SerializeAsync method, we serialize our companyForCreation object into the created memory stream. Also, we use the Seek method to set a position at the beginning of the stream. Then, we initialize a new instance of the HttpReqestMessage object with the required arguments and set the accept header to application/json.

After that, we create a new stream content object named requestContent using the previously created memory stream. The StreamContent object is going to be the content of our request so, we state that in the code, and we set up the ContentType of our request.

Finally, we send our request using the SendAsync method and providing the HttpCompletionOption argument, ensure that the response is successful, and read our content as a stream. After reading the content, we just deserialize it into the createdCompany object.

So, as you can see, through the entire method, we work with streams avoiding unnecessary memory usage with large strings. Also, we are using the ResponseHeadersRead completion option because it makes sense while working with streams.

All we have to do now is to call this method in the Execute method:

public async Task Execute()
{
    //await GetCompaniesWithStream();
    await CreateCompanyWithStream();
}

Great.

Let’s run the app and inspect the result:

There we go. We have created a new company.

Conclusion

Using streams with HTTP requests can help us reduce memory consumption and optimize the performance of our app. In this article, we have seen how to use streams to fetch data from the server and also to create a StreamContent for our request body while sending a POST request. Additionally, we’ve learned more about completion options and how this can help us in achieving better optimization for our application.

In the next article, we are going to learn about the cancelation operations while sending HTTP requests.

Until then,

Best regards.