In this article, we’ll explore why we might need sortable unique IDs in .NET and how to create them with the NewId NuGet Package.
Let’s dive in!
Why Might We Need Sortable Unique IDs in .NET
We all know there are two main approaches when it comes to generating IDs for entities in our projects: either an int or Guid (globally unique identifier) values. But both approaches have their problems. Let’s explore them in an API that deals with customer orders.
Integers as Primary Keys
First, let’s look at integers as primary keys:
public class Order
{
public int Id { get; set; }
public required string CustomerName { get; set; }
public required IEnumerable<string> Products { get; set; }
public required decimal TotalAmount { get; set; }
}
We create an Order class and use an int as the ID.Â
With this approach, we leave the ID generation to the database provider. The good here is that we get short, nice, and consecutive IDs. But, with database-generated primary keys, we get one major downside – our applications get harder to scale up. Dealing with large amounts of insert statements forces our database to constantly use locks to deal with generating new primary keys.
Storing entities over several different databases and maintaining unique int IDs is also next to impossible. Another thing is that we might leak sensitive information to our competitors – do we want to have sequential IDs and let people know exactly how many orders we have?
Globally Unique Identifiers as Primary Keys
The other popular approach is to use Guid values:
public class OrderService(IUnitOfWork unitOfWork) : IOrderService
{
public async Task<OrderDto> CreateAsync(
OrderForCreationDto orderForCreationDto,
CancellationToken cancellationToken = default)
{
var order = new Order
{
Id = Guid.NewGuid(),
CustomerName = orderForCreationDto.CustomerName,
Products = orderForCreationDto.Products,
TotalAmount = orderForCreationDto.TotalAmount,
};
unitOfWork.OrderRepository.Insert(order);
await unitOfWork.SaveChangesAsync(cancellationToken);
return new OrderDto
{
Id = order.Id,
CustomerName = order.CustomerName,
Products = order.Products,
TotalAmount = order.TotalAmount,
};
}
// ommited for brevity
}
We start by changing the Order‘s identification from int to Guid, and then we create our OrderService class. Our project is based on the Onion architecture, so our service takes in a Dto object, processes it, and inserts the entity into the database using a repository.
With this approach, the OrderService class is responsible for generating the primary key value. The biggest plus here is that we can use Guid.NewGuid() and get unique IDs with zero effort. Having unique, but random, IDs makes scaling our application to several databases very easy. But this is their greatest problem as well: it makes our data unsortable based on the primary keys alone and can lead to potential indexing problems. This type of primary key also takes up four times as much space as a regular integer when stored in the database.
And this is where the NewId library comes into play. By utilizing it, we combine the good sides of both int and Guid primary keys and eliminate some of the downsides.
What Is the NewId Library
The NewId library is a NuGet package that we can use to generate unique, yet sortable IDs. It is based on the now-retired Snowflake: an internal service at X (formerly Twitter) for generating sortable unique primary keys. NewId is part of the distributed application framework MassTransit and was developed to tackle problems present with both int and Guid identifiers. It’s aimed at providing a way to generate unique and sortable IDs at scale.
The package generates IDs based on three things – a timestamp, a worker ID, and a process ID. This way we end up with unique IDs that are still sortable and won’t collide when we have multiple instances of our application and database.
Before we start generating IDs, we need to install the NewId package:
dotnet add package NewId
Using the dotnet add package command, we install the library.
Now that we have things ready, let’s start using the package to generate IDs.
How to Generate Sortable Unique IDs with the NewId Library in .NET
To generate our sortable unique IDs, we need to use the NewId class. It is located in the MassTransit namespace and has three methods.Â
First, the Next() method – generates a new instance of the NewId class:
00070000-ac11-0242-3d9b-08dc45bed613 00070000-ac11-0242-c9d3-08dc45bed614 00070000-ac11-0242-cafd-08dc45bed614 00070000-ac11-0242-cb2e-08dc45bed614 00070000-ac11-0242-cb69-08dc45bed614
Next, the NextGuid() method – generates a new Guid value:
00070000-ac11-0242-df20-08dc45bed614 00070000-ac11-0242-0c11-08dc45bed615 00070000-ac11-0242-0d30-08dc45bed615 00070000-ac11-0242-0d46-08dc45bed615 00070000-ac11-0242-0d58-08dc45bed615
Finally, the NextSequentialGuid() method – generates a new sequential Guid value:
08dc45be-d615-19b5-0242-ac1100070000 08dc45be-d615-1b01-0242-ac1100070000 08dc45be-d615-1b46-0242-ac1100070000 08dc45be-d615-1b66-0242-ac1100070000 08dc45be-d615-1ba4-0242-ac1100070000
We can see that with the Next() and NextGuid() methods, we get the same pattern where the NextSequentialGuid() method has a slightly different pattern. The latter two methods return a Guid value and no change will be needed to our class, but if we opt for the Next() method, we’ll have to change the ID type of our Order class.
Let’s use one of them:
public class OrderService(IUnitOfWork unitOfWork) : IOrderService
{
public async Task<OrderDto> CreateAsync(
OrderForCreationDto orderForCreationDto,
CancellationToken cancellationToken = default)
{
var order = new Order
{
Id = NewId.NextSequentialGuid(),
CustomerName = orderForCreationDto.CustomerName,
Products = orderForCreationDto.Products,
TotalAmount = orderForCreationDto.TotalAmount,
};
unitOfWork.OrderRepository.Insert(order);
await unitOfWork.SaveChangesAsync(cancellationToken);
return new OrderDto
{
Id = order.Id,
CustomerName = order.CustomerName,
Products = order.Products,
TotalAmount = order.TotalAmount,
};
}
// ommited for brevity
}
In our OrderService class, where we generate the IDs, we replace the Guid.NewGuid() method with NewId.NextSequentialGuid() method. This one thing is the only change we need to do.
Let’s run our API and add some orders:
[
{
"id": "08dc45c0-b5b5-0d5d-5811-22b038790000",
"customerName": "Marcel Waters",
"products": [
"Piano"
],
"totalAmount": 599.99
},
{
"id": "08dc45c0-d6f4-fbed-5811-22b038790000",
"customerName": "Elizabeth Doyle",
"products": [
"Vase",
"Mirror",
"Blanket"
],
"totalAmount": 49.39
},
{
"id": "08dc45c0-f143-a9f9-5811-22b038790000",
"customerName": "Rayford Lopez",
"products": [
"Headphones",
"Microphone"
],
"totalAmount": 86.06
}
]
We can see that our entities now have unique yet not completely random identifiers that can be sorted.
When Not to Use the NewId Library to Generate Sortable Unique IDs
While the NewId library offers us the convenience of generating sortable unique IDs, there are a few scenarios where we should avoid using them. It’s crucial to keep in mind that the IDs we generate have a level of predictability. They can be guessed when you know the algorithm they are created with.
Therefore, it’s a good idea to not use NewId-generated IDs in contexts where unpredictability and security are paramount. We shouldn’t use such IDs for sensitive information such as passwords, security tokens, or any other data requiring high levels of confidentiality. If we rely on the NewId library in such scenarios we could potentially compromise the security of our applications. This can expose them to vulnerabilities and unauthorized access. Therefore, it’s vital to be cautious when security is critical in our applications.
Conclusion
In this article, we explored the limitations of both integer and globally unique identifiers when used as primary keys and the need for a new approach. The NewId NuGet Package provides us with a solution by offering unique, sortable IDs based on a combination of timestamp, worker ID, and process ID. With the NewId library, we gain the benefits of both int and Guid primary keys while reducing their drawbacks. This enables us to easily achieve scalability, and efficient indexing, as well as improve our data organization. All of this ultimately enhances the robustness and functionality of our applications.