Memory Optimization With ArrayPool in C#

In this article, we will learn how to achieve memory optimization in C# through the utilization of the ArrayPool class. We’ll explore the significance of memory efficiency and validate its effectiveness via benchmarking, providing a comprehensive guide to enhance memory management in your C# applications.

Let’s start.

Why Do We Need Memory Optimization?

Efficient memory management is crucial for smooth-running software. We need more memory as we develop more complex applications or handle larger datasets. But if we do not manage it well, it can lead to problems.

One issue is memory fragmentation, where gaps form between blocks of memory. This can make it hard to find enough continuous space for large pieces of data, even if there’s enough free memory overall.

Another problem is excessive garbage collection (GC). In C#, the system automatically cleans up unused memory. But if it does not perform it efficiently, it can slow down the program and make it less responsive.

When the system allocates memory to a sizable object, specifically over 85,000 bytes, it moves it to the Large Object Heap (LOH). However, if the LOH becomes exhausted, the GC cleans the entire managed heap memory space, including Generation 0, Generation 1, and Generation 2, along with LOH. This thorough cleanup process, referred to as full garbage collection, is the most time-intensive.

One solution lies in buffer pooling using ArrayPool. This involves a collection of pre-initialized objects available for immediate use. Rather than creating new objects, we retrieve them from the pool when we need them and return them after use. Since large managed objects, such as arrays or array wrappers like strings, are at the core of this issue, array pooling becomes essential to face this challenge.

What is ArrayPool?

In C#, the ArrayPool is represented by the ArrayPool class. It is a thread-safe class, that provides a structured mechanism for efficiently managing and reusing arrays of type T. 

How to Create an ArrayPool

Now, let’s see how we can initialize an ArrayPool. For our demo purposes, let’s create a console application using the command dotnet new console -n ArrayPoolConsole.

Now, let’s initiate our first ArrayPool:

var defaultArrayPool = ArrayPool<int>.Shared;

Here, we create the defaultArrayPool variable and we populate it with an instance of the ArrayPool<int> class that is shared across the application. We choose to create an instance of the ArrayPool of int type for simplicity purposes.

The defaultArrayPool is a shared instance that can handle managed ready-to-use arrays of integer type. This pool has a default max array length, equal to 2^20 (1024*1024 = 1.048.576) bytes.

Another way to initialize an ArrayPool is with the use of the Create() extension method:

var maxArrayLength = 100;
var maxArraysPerBucket = 10;
var customArrayPool = ArrayPool<int>.Create(maxArrayLength, maxArraysPerBucket);

In this scenario, we create a pool of managed integer arrays, with specific custom settings. The first argument of the Create() method is an integer and sets the maximum length of an array that will exist in the ArrayPool.

The second argument is also of integer type. Each bucket in the pool can now hold a certain maximum number of arrays. The pool groups arrays with similar lengths into these buckets so that we can access them faster.

However, we do not recommend initializing an ArrayPool using the Create() method, unless absolutely necessary. When we create custom pools with a maximum length array, larger than the default one, we end up allocating memory when we use a new array.

How to Use an ArrayPool for Memory Optimization

Now, let’s see how we can use an array from the pool:

var pool = ArrayPool<int>.Shared;

var arraySize = 10;

var array = pool.Rent(arraySize);

for (var i = 0; i < arraySize; i++)
    array[i] = i * 2;

Console.WriteLine("Array elements:");

for (var i = 0; i < arraySize; i++)
    Console.Write(array[i] + " ");

pool.Return(array);

First, we initiate a variable pool as the instance of an ArrayPool. Then, we set the arraySize to 10, which is the number of elements of the array we want.

We use the Rent() method to receive an array from the pool. As an argument, we pass the minimum size of the array we want, which in our scenario is equal to 10.

For demo purposes, we use a loop to iterate through the first ten elements of the array and assign them with values derived from multiplying the loop index by 2.

Let’s print our result:

Array elements:
0 2 4 6 8 10 12 14 16 18

We can verify that the array elements consist of the expected values.

In the end, we must use the Return() method of the ArrayPool in order to send the array back to the pool of the managed arrays. Thus, we make the array available again for another call. If we do not Return() the array, we will not be able to reuse it from the same instance of the ArrayPool, and consequently, we will end up using a different one each time. Failing to return arrays to the pool can negate performance benefits which goes against the main purpose of ArrayPool usage through array recycling.

The Actual Size of the Array We Rent From the ArrayPool

We should mention here, that the actual size of the array we receive from the ArrayPool is at least the size we request. That means that the obtained array can have a greater length. Let’s inspect our previous case, by printing the length of the array:

Console.WriteLine($"Array size: {array.Length}");

The result:

Array size: 16

We can see that the array has 16 elements instead of 10. The last 6 elements have the default value of zero.

The ArrayPool does not consist of managed arrays of every size. Instead, it consists of arrays with sizes that are power of 2, starting from length 16. So, when we request an array of a certain size, we get the closest available array that’s at least that size.

Benchmark ArrayPool Performance

Let’s set up a benchmark in order to measure the efficiency of the ArrayPool:

private ArrayPool<int>? _arrayPool;

[Params(100, 1000, 10000, 100000)]
public int ArraySize { get; set; }

[GlobalSetup]
public void GlobalSetup()
{
   _arrayPool = ArrayPool<int>.Shared;
}

First, we define the benchmark parameters and assign them to the ArraySize property of our class. We will test arrays of lengths 100, 1000, 10000, and 100000 respectively. Then, let’s initialize our ArrayPool of int type that will be used in our benchmark methods:

[Benchmark]
public void CreateArrayWithArrayPool()
{
    var array = _arrayPool.Rent(ArraySize);

    _arrayPool.Return(array);
}

[Benchmark]
public void CreateArrayNewArray()
{
    var array = new int[ArraySize];
}

We create two methods that simply initialize an array. The first uses the ArrayPool, while the second uses the new operator.

Let’s inspect the results:

|                   Method | ArraySize |         Mean |      Error |     StdDev | Allocated |
|------------------------- |---------- |-------------:|-----------:|-----------:|----------:|
| CreateArrayWithArrayPool |       100 |     32.00 ns |   0.686 ns |   1.006 ns |         - |
|      CreateArrayNewArray |       100 |     45.56 ns |   0.647 ns |   0.605 ns |     424 B |
| CreateArrayWithArrayPool |      1000 |     28.18 ns |   0.614 ns |   0.992 ns |         - |
|      CreateArrayNewArray |      1000 |    403.36 ns |   7.864 ns |   7.724 ns |    4024 B |
| CreateArrayWithArrayPool |     10000 |     28.70 ns |   0.549 ns |   0.871 ns |         - |
|      CreateArrayNewArray |     10000 |  3,487.10 ns |  65.825 ns |  70.432 ns |   40024 B |
| CreateArrayWithArrayPool |    100000 |     28.45 ns |   0.612 ns |   0.680 ns |         - |
|      CreateArrayNewArray |    100000 | 61,816.37 ns | 513.016 ns | 428.391 ns |  400065 B |

The benchmark outcome clearly confirms that using an ArrayPool is not only significantly faster, but also memory-efficient. There is no memory allocation in the ArrayPool tests. When the size increases, the difference in time spent for array initialization, between the ArrayPool use and the classic new operator use, is exponential. 

This happens as array buffers are kept in memory in the ArrayPool scenario, and are reused to avoid new allocations.

The ArrayPool is an excellent choice when we need to optimize memory and reduce memory fragmentation, especially in performance-critical scenarios. It allows us to reuse existing memory blocks through array recycling. However, we have to explicitly return the borrowed arrays to the pool via the Return() method. Also, the rented array might be of a size greater than what we actually need, requiring special manipulation to fit our specific use case. 

Creating an array with new operator is simpler without needing manual memory management, making it a good choice when we do not frequently use multiple arrays in our application and thus we do not need fine-grained control over memory allocation.

In performance-critical applications where we use and initialize arrays often, ArrayPool is the preferred option, while for simpler use cases with less frequent array creation, the convenience of using new may be preferable.

Conclusion

In this article, we examined the array pool and how to use its extensions in our applications. Then, we evaluated the use of it in terms of performance and efficiency for array initialization in C#.