Compare Byte Arrays in .NET

In .NET, byte arrays are a common data type used to represent a sequence of bytes. They often represent binary data such as images, audio files, or serialized objects. Comparing byte arrays in .NET requires a good understanding of how the .NET framework handles arrays and the various comparison methods available. In this article, we are going to explore different techniques to compare byte arrays in .NET and discuss each method’s performance implications. 

Without further ado, let’s get started!

Use SequenceEqual to Compare Byte Arrays

We can use the inbuilt SequenceEqual() method to compare byte arrays. The method takes a byte array as its sole parameter and compares it against the current byte array instance. The method uses the default equality comparer to check whether both arrays have the same number of elements in the same order before returning true or false. 

Let’s learn how we can compare byte arrays by invoking the SequenceEqual() method:

public bool CompareUsingSequenceEqual(byte[] firstArray, byte[] secondArray)
{
    return firstArray.SequenceEqual(secondArray);
}

Here, we compare _secondArray against _firstArray and returns true or false. Next, we are going to define some byte arrays in our test class to test our implementations:

_compareByteArrays = new CompareByteArrays();
_firstArray = new byte[] { 0, 1, 2, 3, 4 };
_secondArray = new byte[] { 0, 1, 2, 3, 4 };
_thirdArray = new byte[] { 0, 1, 2, 3, 5 };

Finally, we can proceed to verify whether our SequenceEqual() method implementation is accurate:

Assert.IsTrue(_compareByteArrays.CompareUsingSequenceEqual(_firstArray, _secondArray));
Assert.IsFalse(_compareByteArrays.CompareUsingSequenceEqual(_firstArray, _thirdArray));

Compare Byte Arrays in .NET Using Except

We can use the Except() method to compare byte arrays. The idea is to find the set difference between the byte arrays we are comparing, i.e., checking if elements are in one array and not in the other. This method returns an IEnumerable<T> object with the set difference.

Let’s look at how we can use the Except() method to compare byte arrays:

var onlyFirst = firstArray.Except(secondArray);
var onlySecond = secondArray.Except(firstArray);

return !onlyFirst.Any() && !onlySecond.Any();

The onlyFirst variable holds the elements in  firstArray that are not in secondArray. On the other hand, onlySecond has the elements in  secondArray that are not present in the firstArray. The method returns true when onlyFirst and onlySecond are empty. 

Compare Byte Arrays Through Iteration

Another method for comparing to check whether byte arrays are equal is by iterating through them:

public bool CompareUsingForLoop(byte[] firstArray, byte[] secondArray)
{
    if (firstArray.Length != secondArray.Length)
    {
        return false;
    }

    for (int i = 0; i < firstArray.Length; i++)
    {
        if (firstArray[i] != secondArray[i])
        {
            return false;
        }
    }

    return true;
}

First, we check whether the arrays have an equal number of elements before checking whether each element in the first array is equal to the one in the second. This algorithm performs O(N), with N being the number of elements in an array. 

Use Binary Equality to Compare Byte Arrays in .NET

We can compare byte arrays using binary equality by iterating over each byte of the arrays and checking if they have the same values. Let’s learn how we can compare byte arrays using binary equality and the unsafe keyword in C#:

public unsafe bool CompareUsingBinaryEquality(byte[] firstArray, byte[] secondArray, string arrayName)
{
    if (firstArray == null || secondArray == null || firstArray.Length != secondArray.Length)
        return false;

    var arrayLength = firstArray.Length;
    var vectorSize = Vector<byte>.Count;

    fixed (byte* pbtr1 = firstArray, pbtr2 = secondArray)
    {
        var i = 0;

        for (; i <= arrayLength - vectorSize; i += vectorSize)
        {
            if (!VectorEquality(pbtr1 + i, pbtr2 + i))
                return false;
        }

        for (; i < arrayLength; i++)
        {
            if (pbtr1[i] != pbtr2[i])
                return false;
        }
    }

    return true;
}

private unsafe bool VectorEquality(byte* firstPointer, byte* secondPointer)
{
    var firstVector = *(Vector<byte>*)firstPointer;
    var secondVector = *(Vector<byte>*)secondPointer;

    return Vector.EqualsAll(firstVector, secondVector);
}

First, we perform preliminary checks to handle null arrays and arrays of different lengths, which avoids unnecessary comparisons. The method uses the fixed statement inside the unsafe block to pin the arrays in memory.

Next, we create and initialize two pointers bptr1 and bptr2 with the addresses of the first elements of the arrays. 

Our first iteration compares byte arrays in chunks using SIMD operations (processing multiple data elements with a single instruction), which improves its performance. The Vector class helps us compare multiple bytes at once and compare any remaining values individually when we invoke the Vector.EqualsAll() method. 

The second iteration covers processor architectures that do not support SIMD and AVX (Advanced Vector Extensions) operations. Here, we iterate using a loop and compare each byte of the arrays. The loop continues until we compare all the elements of firstArray and secondArray. 

Inside the loop, we compare the value at each memory location we are pointing using bptr1 and bptr2. If they differ, the method returns false, indicating that the arrays are unequal. After the loop finishes, the method returns true if no differences are found, indicating that the arrays are equal.

 Note that unsafe code bypasses some of the built-in safety features of C#.

Compare Byte Arrays With IStructuralEquatable Technique

In C#, we can compare byte arrays using the IStructuralEquatable interface, which provides a mechanism for comparing objects based on their structure rather than reference equality. This interface is available in the System.Collections namespace.

Let’s use the IStructuralEquatable  interface to compare byte arrays:

public bool CompareUsingIStructuralEquatable(byte[] firstArray, byte[] secondArray)
{
    if (firstArray == null || secondArray == null || firstArray.Length != secondArray.Length)
        return false;

    return StructuralComparisons.StructuralEqualityComparer.Equals(firstArray, secondArray);
}

First, we perform preliminary checks inside the method to handle null arrays and arrays of different lengths, similar to the previous example.

Next, we use the StructuralComparisons.StructuralEqualityComparer.Equals() method to compare the byte arrays. This method uses the IStructuralEquatable interface to perform the comparison.

The Equals() method of IStructuralEquatable interface compares the elements of the arrays structurally, considering the values rather than the references. It checks if the arrays have the same number of elements and if each corresponding pair is equal.

Platform Invocation Services

We can compare byte arrays in .NET using PInvoke to call native functions from a DLL (Dynamic-Link Library). One common approach is to use the memcmp() function from the C standard library to compare the memory blocks represented by the byte arrays.

Let’s learn how to compare byte arrays using PInvoke in C#:

#if NET
[DllImport("msvcrt", CallingConvention = CallingConvention.Cdecl, SetLastError = true)]
private static extern int memcmp(byte[] firstArray, byte[] secondArray, long size);
#else
[DllImport("libc", CallingConvention = CallingConvention.Cdecl, SetLastError = true)]
private static extern int memcmp(byte[] firstArray, byte[] secondArray, ulong size);
#endif
public bool CompareUsingPInvoke(byte[] firstArray, byte[] secondArray)
{
    if (firstArray == null || secondArray == null || firstArray.Length != secondArray.Length)
    {
        return false;
    }
    if (RuntimeInformation.IsOSPlatform(OSPlatform.Windows))
    {
        return memcmp(firstArray, secondArray, firstArray.Length) == 0;
    }
    else
    {
        return memcmp(firstArray, secondArray, firstArray.Length) == 0;
    }
}

After importing the System.Runtime.InteropServices namespace, we declare the memcmp() function from the C standard library as an external function using the DllImport attribute. We make sure that we account for other platforms, such as Unix, by performing a conditional check before importing the DLLs. 

Inside our method, we perform preliminary checks to handle null arrays and arrays of different lengths, similar to the previous examples. Next, we use the memcmp() function through PInvoke to compare the byte arrays. The function takes the two arrays to compare and the length of the arrays represented by a platform-specific pointer-sized integer.

The memcmp() function returns an integer value that indicates the result of the comparison. So, if the value is zero, the byte arrays are equal.

Let’s assess how different techniques for comparing byte arrays perform.

Performance Analysis of Comparing Byte Arrays

Let’s test how long each method takes to compare byte arrays with 10,000,000 elements each. To help us complete these tests, let’s implement a method to generate byte arrays from a given size:

private unsafe byte[] GenerateOrderedArray(bool firstElement, bool middleElement, int size)
{
    var byteArray = new byte[size * sizeof(int)];

    fixed (byte* bytePtr = byteArray)
    {
        int* intPtr = (int*)bytePtr;

        for (int i = 0; i < size; i++)
        {
            if (firstElement && i == 0)
            {
                intPtr[i] = _rand.Next();
            }
            else if (middleElement && i == (size / 2))
            {
                intPtr[i] = _rand.Next();
            }
            else
            {
                intPtr[i] = i;
            }
        }
    }

    return byteArray;
}

First, we define a byte array, which we are going to populate with values in ascending order. We use an unsafe block to pin the array in memory and cast bytePtr to an int* pointer intPtr. 

When firstElement is true, we insert a random first element into our intArray. We are going to use it when testing different performance scenarios.

The best-case scenario occurs when the first elements of the byte arrays are not equal. Assuming we pass two arrays of the same size, the algorithms will return false after comparing the first elements.

We can use arrays with different middle elements to simulate average-case scenarios. Here, the algorithms will compare the arrays and return false when encountering those elements and stop comparing the rest.  We set the middleElement to true to insert a random integer into the middle position of the array. We expect to notice performance differences when comparing arrays with millions of elements.

On the other hand, when both firstElement and middleElement are false, the method generates an ordered byte array, which comes in handy when simulating the worst-case scenario where we compare equal byte arrays (same size and order of elements). While the arrays will stop comparing the rest of the elements after encountering different elements in the average case, in this case, our examples will compare all the elements and return true. We expect the performance differences to become more apparent when comparing this scenario against the others. 

Next, let’s create an object that holds three different array pairs to simulate the three scenarios:

public IEnumerable<object[]> SampleByteArray()
{
    yield return new object[]
    {
        GenerateOrderedArray(true, false, 10000000),
        GenerateOrderedArray(true, false, 10000000),
        "Best Case"
    };
    
    yield return new object[]
    {
        GenerateOrderedArray(false, true, 10000000),
        GenerateOrderedArray(false, true, 10000000),
        "Average Case"
    };
    
    yield return new object[]
    {
        GenerateOrderedArray(false, false, 10000000),
        GenerateOrderedArray(false, false, 10000000),
        "Worst Case"
    };
}

Benchmark Results

Finally, let’s pass these arrays to the methods we have and assess the results:

|                          Method  |    arrayName |                 Mean  |    Allocated | 
|--------------------------------- |------------- |----------------------:|-------------:| 
|              CompareUsingForLoop |    Best Case |             0.8812 ns |            - | 
|       CompareUsingBinaryEquality |    Best Case |             3.1239 ns |            - | 
|              CompareUsingPInvoke |    Best Case |            24.2283 ns |            - | 
|        CompareUsingSequenceEqual |    Best Case |            24.6801 ns |            - | 
| CompareUsingIStructuralEquatable |    Best Case |           135.9513 ns |         48 B | 
|               CompareUsingExcept |    Best Case | 1,677,936,193.3333 ns | 1280011320 B |
|                                  |              |                       |              | 
|       CompareUsingBinaryEquality | Average Case |     2,537,428.5889 ns |          2 B | 
|              CompareUsingPInvoke | Average Case |     2,666,604.0495 ns |          2 B | 
|        CompareUsingSequenceEqual | Average Case |     2,764,673.1394 ns |          2 B | 
|              CompareUsingForLoop | Average Case |    10,213,870.7031 ns |          8 B | 
|               CompareUsingExcept | Average Case | 1,719,539,882.6087 ns | 1280011320 B | 
| CompareUsingIStructuralEquatable | Average Case | 2,459,233,937.5000 ns |  960000552 B | 
|                                  |              |                       |              | 
|        CompareUsingSequenceEqual |   Worst Case |     5,060,508.2031 ns |          4 B | 
|       CompareUsingBinaryEquality |   Worst Case |     5,115,654.3490 ns |          4 B | 
|              CompareUsingPInvoke |   Worst Case |     5,383,083.6496 ns |          4 B | 
|              CompareUsingForLoop |   Worst Case |    20,080,341.5865 ns |         16 B | 
|               CompareUsingExcept |   Worst Case | 1,707,928,842.8571 ns | 1280011320 B | 
| CompareUsingIStructuralEquatable |   Worst Case | 5,189,742,609.0909 ns | 1920000504 B |

In the best-case scenario, the iteration technique is the most efficient since it performs at O(N). Using Except() to calculate the set difference between arrays is quite memory-intensive as we repeat the same process on each array, making it the slowest. The same case applies to IStructural Equatable since it checks for structural equality, which is slower than the other techniques because it performs a deep comparison by recursively comparing each array element. This approach incurs additional overhead because it needs to handle various types and nested structures. 

Binary equality performs quite well in the average-case scenario since it relies on SIMD operations to compare byte arrays efficiently. The iteration technique is less efficient in this scenario as the algorithm compares more elements this time. 

The inbuilt sequence equal technique offers the best performance in worst-case scenarios because it compares array elements directly and uses compiler optimizations to improve its performance. Platform invocation services also perform well in this scenario because they rely on DLLs and shared libraries to compare arrays. 

Conclusion

We can compare byte arrays in .NET using various approaches, each with advantages and considerations. Which methods have you used to compare byte arrays? Please let us know in the comments below.