Hybrid Caching in ASP.NET Core

In this article, we’ll introduce hybrid caching in .NET, a new cache type that bridges the existing cache mechanisms in .NET: distributed cache and memory cache. We’ll start by briefly explaining a cache and how existing caches in .NET work. After that, we’ll explore hybrid caching and show how it can be used and configured in .NET applications. Finally, we’ll compare the mentioned types of cache. 

Let’s start!

Caching in .NET

Caching stores data in memory for faster access than getting data from its source. .NET implements two types of caching: in-memory and distributed caching.

In-memory caching uses server local memory to store data that we frequently access.

Conversely, distributed caching stores data using external services and enables multiple servers to use the same data.

Hybrid Caching in .NET / ASP.NET Core

Hybrid caching in .NET employs both types of cache storage. It introduces two-level caching:

• local in-memory caching as its primary storage (L1)
• distributed caching as its secondary storage (L2)

With this approach, hybrid caching keeps fast in-memory access and persistence and scalability of data through its secondary, external storage. 

Add and Configure Hybrid Caching in .NET

To add hybrid caching to our application, we have to install Microsoft.Extension.Caching.Hybrid NuGet package:

dotnet add package Microsoft.Extensions.Caching.Hybrid --prerelease

IMPORTANT NOTE: The hybrid cache feature is still in the preview stage and will be fully released in the upcoming minor release of .NET 9 extensions. 

To add the HybridCache service, we can call the IServiceCollection extension AddHybridCache():

builder.Services.AddHybridCache();

This extension comes with overload, which means we can configure different options for the HybridCache service.

As the feature is still in the preview, we have to disable the compiler warning:

#pragma warning disable EXTEXP0018 // Type is for evaluation purposes only and is subject to change or removal in future updates. Suppress this diagnostic to proceed.
builder.Services.AddHybridCache(options =>
{
    options.MaximumPayloadBytes = 1024 * 10 * 10; 
    options.MaximumKeyLength = 256;

    options.DefaultEntryOptions = new HybridCacheEntryOptions
    {
        Expiration = TimeSpan.FromMinutes(30),
        LocalCacheExpiration = TimeSpan.FromMinutes(30)
    };

    options.ReportTagMetrics = true;
    options.DisableCompression = true;
});
#pragma warning restore EXTEXP0018 // Type is for evaluation purposes only and is subject to change or removal in future updates. Suppress this diagnostic to proceed.

With the HybridCacheOptions class, we can configure:

• MaximumPayloadBytes – the maximum size of the cache items in bytes. In this example, we configure it to 10MB. The default value is 1MB. If we try to store an item bigger than this configured value, HybridCache logs the error and doesn’t store the item
• MaximumKeyLength – the maximum size of the cache item key in the number of characters. The default value is 1024, and we set it to 256
• ReportTagMetrics – indicating whether to use tags in metric reporting
• DisableCompression – disabling compression for the particular HybridCache instance
• DefaultEntryOptions – the HybridCacheOptions class with two properties: Expiration, which determines the duration of the distributed (L2) cache, and LocalCacheExpiration, which determines the duration of the local in-memory (L1) cache. If set in HybridCacheOptions, these values will be default values but can be overridden for each specific entry

When we use and configure any distributed cache, HybridCache will automatically detect it and use it as the L2 distributed cache.

Distributed cache requires serialization. HybridCache handles string and bytes[] internally. For everything else, it uses System.Text.Json.

Additionally, HybridCache can use other serializers. To configure them, we can chain AddSerializer<T>() to add the serializer for a specific type or AddSerializerFactory() to add the serializer for multiple types. 

Use Hybrid Caching in ASP.NET Core

The hybrid cache library exposes a unified API to get, add, or invalidate values in the cache. In our examples, we’ll use the simple CmCourse class:

public class CmCourse
{
    public int Id { get; set; }

    public required string Name { get; set; }    

    public required string Category { get; set; }
}

Next, we define the ICmCourseService that we will use in minimal API calls:

public interface ICmCourseService
{
    Task<CmCourse?> GetCourseAsync(int id, CancellationToken cancellationToken = default);

    Task PosttCourseAsync(CmCourse course, CancellationToken cancellationToken = default);

    Task InvalidateByCourseIdAsync(int id, CancellationToken cancellationToken = default);

    Task InvalidateByCategoryAsync(string tag, CancellationToken cancellationToken = default);
}

This interface describes reading, creating, and deleting the values from the data source.

The interface implementation class – CmCourseService requires the injection of HybridCache service in its constructor. Additionally, inside the CmCourseService class, you can find the private courseList field, which mocks the data storage for our example:

public class CmCourseService(HybridCache cache) : ICmCourseService
{
    public static readonly List<CmCourse> courseList = [
        new CmCourse
        { 
            Id = 1,
            Name = "WebAPI",
            Category = "Backend"
        },
        new CmCourse
        {
            Id = 2,
            Name = "Microservices",
            Category = "Backend"
        },
        new CmCourse
        {
            Id = 3,
            Name = "Blazer",
            Category = "Frontend"
        },
    ];
    
    ...
}

Let’s explore each of the HybridCache methods and their implementation in CmCourseService. 

Read the Entry From the Cache

The GetOrCreateAsync() method will try to read the value for a provided key from the cache:

public async Task<CmCourse?> GetCourseAsync(int id, CancellationToken cancellationToken = default)
{
    return await cache.GetOrCreateAsync($"course-{id}", async token =>
        {
            await Task.Delay(1000, token);
            var course = courseList.FirstOrDefault(course => course.Id == id);
                
            return course;
        },
        options: new HybridCacheEntryOptions
        {
            Expiration = TimeSpan.FromMinutes(30),
            LocalCacheExpiration = TimeSpan.FromMinutes(30)
        },
        tags: ["course"],
        cancellationToken: cancellationToken
    );
}

The GetOrCreateAsync() method first tries to read the value for the key $"course-{id}" from the cache. If the value is not in the local cache, HybridCache tries to get this value from the distributed cache.

The second parameter is a factory method that executes if the value doesn’t exist, and it retrieves the value from an external source, stores it in the configured caches, and returns it. 

The following parameters are optional. The options parameter allows us to override the default values for cache duration. The tags parameter defines the tag that HybridCache sets to the new entry it adds to the cache, and the last parameter is the cancellation token. 

Handling both caches happens in the library itself, and the developer doesn’t need to implement any additional logic. A very important feature of the HybridCache is that other concurrent requests for the same cache entry wait for the first one to finish. Such an approach prevents the cache stampede, the problem that occurs when there is no required entry in the cache and too many requests get the value from the source and try to repopulate the same cache entry. 

The GetOrCreateAsync() method has an overload that allows the definition of the state object for the factory method, reducing the overhead from captured variables. 

Write the Entry to the Cache

In the most use cases, the GetOrCreateAsync() method is sufficient. However, the HybridCache implements SetAsync() method, which stores an entry in the cache without trying to read it first:

public async Task PostCourseAsync(CmCourse course, CancellationToken cancellationToken = default)
{
    courseList.Add(course);
    await cache.SetAsync($"course-{course.Id}",
        course,
        options: new HybridCacheEntryOptions
        {
            Expiration = TimeSpan.FromMinutes(30),
            LocalCacheExpiration = TimeSpan.FromMinutes(30)
        },
        tags: [$"cat-{course.Category}"],
        cancellationToken: cancellationToken);
}

The first and the second parameters are mandatory, and they set the cache key and the cache entry, respectively. The next three parameters are optional and the same as those used for the GetOrCreateAsync() method: options, tags, and cancellation token. 

If the key already exists in the cache, its value will be overwritten. 

Invalidate Cache Entries by Key

The method RemoveAsync() removes the cache entry with the specified key from the cache:

public async Task InvalidateByCourseIdAsync(int id, CancellationToken cancellationToken = default)
{
    await cache.RemoveAsync($"course-{id}", cancellationToken);
}

The argument $"course-{id}" represents the key that will be removed from both L1 and L2 cache.  

The method has an overload, allowing us to pass the collection of the keys to it. 

Invalidate Cache Entries by Tag

As described in GetOrCreateAsync() and SetAsync() methods, we can set one or multiple tags for each cache entry. The RemoveByTagAsync() method removes all entries with specified tag: 

public async Task InvalidateByCategoryAsync(string tag, CancellationToken cancellationToken = default)
{
    await cache.RemoveByTagAsync($"cat-{tag}", cancellationToken);
}

Similar to the RemoveAsync() method, the RemoveByTagAsync() has an overload accepting the collection of the keys.

IMPORTANT NOTE: This method is still under development and currently doesn’t affect the cache state. 

Benefits of Hybrid Caching in .NET

The HybridCache implements a simple, unified API to handle local and distributed caches using the same API call. The local IMemoryCache can be replaced with HybridCache without much effort. 

The hybrid caching is designed with async flow in mind. As a result, the library handles concurrent operations on the cache, and the developers don’t have to implement any additional control or protection. By pausing the subsequent similar request until the first one is finished, the HybridCache offers cache stampede protection out of the box. The exact mechanism significantly lowers the requests for external data sources. 

The tag feature enables a fast and convenient way to invalidate multiple entries with a single call. With its configurable serialization, the cache can be easily adapted for usage with, for example, ProtoBuf. 

Built-in metrics and monitoring also greatly help with application observability and fine-tuning.

Conclusion

On the whole, HybridCache is a very valuable addition to the .NET ecosystem. It combines the benefits and advantages of two already existing types of caching and simultaneously solves the problem that each of them has. The two-level caching enables fast access to data and horizontal scalability of the cache when the demands on the application grow. 

To conclude, with a two-level approach to caching, HybridCache improves performance, allows scaling, and minimizes costs by optimizing data storage.