In this article, we will look at converting a byte array to a hexadecimal string. Converting between byte arrays and hexadecimal strings is a common occurrence in computer programming. Hexadecimal strings have the advantage of being both human-readable and easy to produce.
Let’s dive in.
Motivation for Converting a Byte Array to Hexadecimal
Because hexadecimal is a base 16 numbering system, conversion between byte arrays and hexadecimal strings is very straightforward. In a hexadecimal string, each digit (0-9A-F) represents exactly 4 bits. This means that we can represent a single byte as a 2-digit hex number. For example, the byte value 255 in hexadecimal would be written as FF, while 42 would be 2A.
A common use case for converting between byte arrays and hexadecimal strings is found within the realm of password hashing and salting. You can see this in action in our article on Hashing and Salting Passwords.
Another common usage of hexadecimal strings is as license key values. Because they are easy for humans to read and type, they make great candidates for use as license keys. In general, if we want to represent binary data in a human-readable form, converting it to a hexadecimal string is a great option.
Before We Begin
Each of the examples in our article will be focusing on how to convert a byte array into an uppercase hexadecimal string. If you are interested in lowercase implementations and options, you can visit the source code repository for this article. There you will find lowercase implementations of each method.Â
While looking at several different algorithms and approaches, we will also benchmark each of the options. This is important so that we have at least some idea of how the code will perform in a real-world scenario. While performance is always an important consideration when choosing a method, we also need to bear in mind the platforms we are targeting and the future maintenance of our code base. Sometimes a slower, but easier-to-implement solution may be the correct one.
For each of the examples in this article, we will be using the following byte array:
var source = new byte[] {222, 173, 190, 239, 222, 202, 251, 173};
Except for the first BitConverter example, all the samples shown in our article will produce the following output:
DEADBEEFDECAFBAD
Now we are ready, so, let’s get started converting byte arrays to hexadecimal strings!
Converting a Byte Array to Hexadecimal Using BitConverter
One of the most common recommendations seen on the internet using BitConverter to perform a byte-to-hex conversion. While this method is frequently recommended, as we will see later in our benchmarks, due to its lackluster performance, it is probably not a method we want to use.
Another issue with this method is that the resultant string will contain each pair of hex digits separated by a -. Let’s look at it in action:
BitConverter.ToString(source);
Which produces the following output:
DE-AD-BE-EF-DE-CA-FB-AD
To remove the dashes, we need to call String.Replace() on the result:Â
BitConverter.ToString(source).Replace("-", string.Empty);
Converting a Byte Array to Hexadecimal Using StringBuilder Append
Another popular method seen across the internet involves using StringBuilder.AppendFormat(). We can use the format string "X2" (or "x2" if we want lowercase) to format each byte as a 2-character hex string and append it to our StringBuilder. As we will see in the benchmarks, this approach is one of the worst in terms of performance.
We can improve the method slightly by taking advantage of Span, stackalloc and TryFormat(). TryFormat() allows us to perform the hex encoding into a pre-allocated (in our case, stack-allocated) buffer.
This improves performance by reducing memory allocations and thus pressure on the garbage collector:
Span<char> buffer = stackalloc char[64];
var sb = new StringBuilder(source.Length * 2);
var bufferIndex = 0;
ref var srcRef = ref MemoryMarshal.GetReference(source.AsSpan());
for (var i = 0; i < source.Length; ++i)
{
var b = Unsafe.Add(ref srcRef, i);
b.TryFormat(buffer[bufferIndex..], out _, "X2");
bufferIndex += 2;
if (bufferIndex == buffer.Length)
{
sb.Append(buffer);
bufferIndex = 0;
}
}
if (bufferIndex > 0) sb.Append(buffer[..bufferIndex]);
sb.ToString();
Now, let’s take a look at what’s going on here. The first thing we do is stackalloc a small buffer for holding our hex-encoded strings. Next, we create a StringBuilder of appropriate size to hold our result. bufferIndex is used to track our location within the stack-allocated buffer, and lastly we get a reference to a Span over our input source.
Next, we iterate over the source string, getting a managed reference to each item in the source array:
var b = Unsafe.Add(ref srcRef, i);
For each byte, we call TryFormat() to convert it to a two-character hex string, storing it in our buffer. After the conversion, we increment our buffer index. Next, we check to see if we have filled our buffer. If it is full, we append it to the StringBuilder and reset bufferIndex to 0.
The penultimate step, after iterating the source buffer, is to check if there is any unappended data in our buffer and append it if necessary.
if (bufferIndex > 0) sb.Append(buffer[..bufferIndex]);
Lastly, we call ToString() on our StringBuilder to get our converted string.
Performance Improvement Through TryFormat() And String.Create()
Related to the StringBuilder option above, we can combine both the TryFormat() and String.Create() methods to improve performance:
string.Create(_sampleData.Length*2, _sampleData, (chars, source) =>
{
ref var srcPtr = ref MemoryMarshal.GetArrayDataReference(source);
var destIndex = 0;
for(int i=0; i < source.Length; ++i)
{
var b = Unsafe.Add(ref srcPtr, i);
b.TryFormat(chars[destIndex..], out _, "X2");
destIndex += 2;
}
});
Here, instead of creating a StringBuilder, we create the string directly and modify it in place. This saves us both an extra buffer allocation for the StringBuilder and a copy operation, when calling StringBuilder.ToString(), to get the result. String.Create() gives us direct access to the underlying characters via a Span. This allows us to set each character directly.
Converting a Byte Array to Hexadecimal Through Bit Manipulation
Another approach we can take when converting a byte array into a hexadecimal string is bit manipulation. By performing some simple bit shifts and mathematical operations, we can compute the resulting characters needed for hexadecimal encoding.
The Setup
First, we declare our offset constants, which we need when computing a character value. The letterOffset is the difference between the ASCII value of the letter A and the decimal value 10 (0xA). This offset is used to convert the numeric values 10-15 into the corresponding hexadecimal letters A-F. We also need a digit offset value for converting the digits 0-9 to the appropriate character values.
We can compute this by subtracting the letterOffset from the character 0:
const int letterOffset = 'A' - 0xA; // For lowercase: 'a' - 0XA const int digitOffset = '0'-letterOffset;
The Magic
Next, we declare a simple helper method to make the code more readable. This method performs the actual computation to convert a nibble (4 bits) into the appropriate hex character:
static char ComputeCharFromNibble(int nibble, int letterOffset, int digitOffset)
{
var digitCalculation = ((nibble - 10) >> 31) & digitOffset;
return (char) (letterOffset + nibble + digitCalculation);
}
Let’s walk through this code so that we can understand how the magic happens.
First, let’s focus on digit calculation: (((nibble - 10) >> 31) & digitOffset). nibble-10 will be negative when nibble < 10 and positive when nibble >= 10. Shifting this value by 31 extracts the sign, resulting in either 0 or -1. We do this because when nibble >= 10, to compute the correct hex character, we need to add digitOffset. Instead of conditionally performing the addition, we use the fact that -1 in binary, has all bits set to 1 and 0 has all bits set to 0. This means that performing bitwise &, results in 0 when nibble < 10 and digitOffset when it is >= 10.
Lastly, we add this value to the result of the first calculation (letterOffset + nibble) and we get the proper hex character.
The Complete Example
Now that we have computed our offsets and created our helper method to perform the bit manipulation, let’s put it all together:
return string.Create(source.Length * 2,
(Source: source, LetterOffset: letterOffset, DigitOffset: digitOffset),
(chars, args) =>
{
ref var srcPtr = ref MemoryMarshal.GetArrayDataReference(args.Source);
ref var destPtr = ref MemoryMarshal.GetReference(chars);
for (var i = 0; i < args.Source.Length; ++i)
{
var b = Unsafe.Add(ref srcPtr, i);
destPtr = ComputeCharFromNibble(b >> 4, args.LetterOffset, args.DigitOffset);
destPtr = ref Unsafe.Add(ref destPtr, 1);
destPtr = ComputeCharFromNibble(b & 0xF, args.LetterOffset, args.DigitOffset);
destPtr = ref Unsafe.Add(ref destPtr, 1);
}
});
First, we get references to the source array and our destination Span. Second, we iterate through the source one byte at a time. Then for each byte, we compute two characters, the first from the high nibble (b >> 4) and the second from the low nibble (b & 0xF). Between each of the calculations, we advance our reference pointer to the next location within the destination.
Converting a Byte Array to Hexadecimal via a Span Over an Alphabet String
Another method we can use to convert a byte array into a hexadecimal string is by initializing a small array containing each hex character and then performing a lookup within the array to compute the proper hex character. Once again, we are using the high and low nibbles from each byte, but instead of performing bit manipulations and offset calculations, we use them directly to index into our string.
First, we need to declare our lookup array. For performance reasons (the explanation is outside the scope of this article, but you can read about it here), we will declare it as a ReadOnlySpan<byte> over a UTF8 string:
static ReadOnlySpan<byte> HexAlphabet => "0123456789ABCDEF"u8;
After the declaration of HexAlphabet, we can have a closer look at the implementation:
string.Create(source.Length * 2, source, (chars, src) =>
{
ref var srcPtr = ref MemoryMarshal.GetArrayDataReference(src);
ref var hexSpan = ref MemoryMarshal.GetReference(HexAlphabet);
ref var destPtr = ref MemoryMarshal.GetReference(chars);
for (var i = 0; i < src.Length; ++i)
{
var b = Unsafe.Add(ref srcPtr, i);
destPtr = (char) Unsafe.Add(ref hexSpan, b >> 4);
destPtr = ref Unsafe.Add(ref destPtr, 1);
destPtr = (char) Unsafe.Add(ref hexSpan, b & 0xF);
destPtr = ref Unsafe.Add(ref destPtr, 1);
}
});
First, we get references to our source array, our hex alphabet array, and finally, our destination. Then iterating through the source bytes, we compute two characters for each byte. This is accomplished by calculating both the high and low nibbles from the byte and using those as indexes into the hex alphabet Span.
Lastly, using our destination reference, we store the returned character and increment the reference.
Converting a Byte Array to Hexadecimal Using a Lookup Table
One of the most performant methods we can use involves using a precomputed lookup table to compute the hex characters two at a time. In this article, we are only showing how to do this in .NET 6 and greater. If you need a solution that can target an older framework, you can check out the source code for the article, where we present a solution that targets .NET Standard 2.0.
The Lookup Table
When talking about increasing our application performance, there is usually a tradeoff we have to make between memory usage and performance.
In our case, when converting bytes to hex, we get a significant performance boost at the cost of a few additional kilobytes of memory (~1k for our lookup tables). Our lookup table has 256 entries, allowing us to index directly into it from our byte value. This in turn allows us to map each byte to a uint that represents the two UTF16 hex characters needed to represent the byte:
public static class LookupTables
{
private static readonly uint[] HexLookupLittleEndian = { // Omitted for brevity };
private static readonly uint[] HexLookupBigEndian = { // Omitted for brevity };
public static uint[] GetLookupTable() =>
BitConverter.IsLittleEndian ? HexLookupLittleEndian : HexLookupBigEndian;
}
One thing to notice is the fact that there are actually two lookup tables. Since our values are uint we must consider the endianness of the system where our code is running. This means that we have to have one lookup table for little-endian systems and one for big-endian. For clarity and readability, we add a simple helper method that returns the appropriate table based on the current system’s endianness.
For an example of how to compute the lookup tables, please check out the source code for this article.
The Code
Now that we have our lookup tables computed, let’s see how we can use them:
string.Create(source.Length * 2, source, (chars, src) =>
{
ref var sPtr = ref MemoryMarshal.GetArrayDataReference(src);
ref var lookupTable = ref MemoryMarshal.GetArrayDataReference(LookupTables.GetLookupTable());
ref var cPtr = ref MemoryMarshal.GetReference(chars);
for (var i = 0; i < src.Length; ++i)
{
var b = Unsafe.Add(ref sPtr, i);
Unsafe.As<char, uint>(ref cPtr) = Unsafe.Add(ref lookupTable, b);
cPtr = ref Unsafe.Add(ref cPtr, 2);
}
});
This code may look a bit familiar, as we have already seen several of the techniques in our earlier examples.
First, we get references to the source array, the lookup table, and the destination. Next, as we have done previously, we iterate over the source array. This time, we don’t need to compute nibbles or anything like that, as our lookup tables map directly from byte to uint.
It’s crucial to note that when setting the character values, we treat our destination reference as a uint as opposed to a char. This enables us to set two character values in the destination string at once. This is possible because uint is twice the size of char. Because of this, we must also increment our destination reference by 2 on each loop iteration.Â
Converting a Byte Array to Hexadecimal Using Convert.ToHexString()
With the advent of .NET 5.0, we now have a built-in method for converting a byte array to a hexadecimal string.
This method internally makes use of lookup tables but also takes advantage of hardware-level SIMD instructions where possible to gain even more performance:
Convert.ToHexString(source);
That’s all there is to it. Just a single line of code. That by itself is a big advantage. This method also has the advantage of being part of the core library, so we don’t have to worry about maintaining it or dealing with the implementation details.
For applications targeting .NET 5.0 or greater, this is the recommended method, with one caveat. This method only produces uppercase hex strings. If you require lowercase, you may want to consider using the lookup table method presented in this article or using another NuGet package such as HexMate or DotNext.
Benchmarks
Now that we have seen several ways to accomplish our goal of converting a byte array into a hexadecimal string, let’s look at some benchmarks. For our benchmarks, we are converting a 4096-byte buffer into a hex string.
Our benchmarks are running on a Windows 10 PC with an AMD Ryzen 7 4800H and 32 GB RAM:
| Method | Mean | Error | StdDev | Median | Gen0 | Allocated | |-------------------------------------- |----------:|----------:|----------:|----------:|--------:|----------:| | ConvertToHexUsingConvert | 1.262 us | 0.0230 us | 0.0344 us | 1.246 us | 7.8125 | 16.02 KB | | ConvertToHexUsingLookup | 2.176 us | 0.0174 us | 0.0145 us | 2.181 us | 7.8125 | 16.02 KB | | ConvertToHexUsingAlphabetSpanLookup | 3.955 us | 0.0280 us | 0.0248 us | 3.942 us | 7.8125 | 16.02 KB | | ConvertToHexUsingBitManipulation | 6.160 us | 0.1001 us | 0.1953 us | 6.078 us | 7.8125 | 16.02 KB | | ConvertToHexUsingBitConverter | 9.965 us | 0.2074 us | 0.6114 us | 9.949 us | 11.6272 | 24.02 KB | | ConvertToHexUsingBitConverterNoDashes | 64.032 us | 0.7382 us | 0.6906 us | 63.892 us | 19.4092 | 40.05 KB | | ConvertToHexUsingTryFormatAndCreate | 64.316 us | 1.1692 us | 1.8545 us | 63.857 us | 7.8125 | 16.11 KB | | ConvertToHexUsingStringBuilderAppend | 65.098 us | 0.5855 us | 0.5477 us | 65.090 us | 15.6250 | 32.09 KB |
As we probably expected, the built-in Convert.ToHexString() is the fastest method. Our lookup table method is a close second. Next up, and coming in at about 3x slower, is our conversion using ReadOnlySpan<byte> created from a UTF8 string. Next is our bit manipulation method at just a shade under 5x slower. After that is the BitConverter method at around 8x slower.Â
However, if we don’t want dashes in the result, then our method jumps up to about 50x slower. Almost equal with BitConverter and no dashes is our method using TryFormat() and String.Create(). Rounding out the final spot in our benchmark is the String.Append() method, at a whopping 51x slower than the built-in Convert.ToHexString() method.
Putting It All Together
The method we choose to implement to convert a byte array to a hexadecimal string will depend on many factors. We have to consider where our code will be running. Is it an embedded system with limited resources, or are we running in the server room? We need to think about the size of the data we are expecting to process. Are we converting small 32-byte arrays, or are we processing a stream of data that we have to convert on the fly? All of these things play an important role when choosing which method we are going to use.
Conclusion
In this article, we looked at different ways to convert a byte array to a hexadecimal string. We also conducted benchmarks of each of these methods and discussed some factors that may impact which method we choose to implement.