Updated on

Put the key we are matching on into a HashSet, then filter the other list against it. Two lines, and it beats every alternative by a wide margin once the lists are more than a few hundred items.

The reason is what each approach costs per element. A nested loop or a Where with an inner Any re-scans the second list for every item in the first; a HashSet looks each key up once, in constant time.

To download the source code for this article, you can visit our GitHub repository.

How Do We Compare Two Lists in C#?

It depends on which of three questions we are actually asking, and they have different answers.

The first is overlap: which items appear in both lists. Build a HashSet of the key from one list, then filter the other with Contains: constant-time lookups instead of a scan per element.

The second is difference: which items are in one list and not the other. That is ExceptBy, which takes the keys to exclude and the key selector, and it answers only one direction at a time. Run it both ways for a full delta.

The third is equality: whether the two lists hold the same things. SequenceEqual compares element by element in order; SetEquals on a HashSet ignores order and duplicates.

Everything below solves the first question, which is the common one, and compares five ways of doing it.

Create Comparison Application

To start, let’s create a new console app with the command dotnet new console -n App in the command window. In our example scenario, we have a Customer class with a unique field Id and an Order class with a field CustomerId that relates one entity to the other.

Let’s create the first appropriate class regarding Customer:

public class Customer
{
    public int Id { get; set; }
    public string Firstname { get; set; }
    public string Surname { get; set; }
}

And now let’s implement a simple Order class:

public class Order
{
    public int CustomerId { get; set; }
    public int OrderId { get; set; }
}

Before we test our methods, let’s populate two lists with sample records:

var customers = new List<Customer>()
{
    new() { Id = 1, Firstname = "Alice", Surname = "Smith"},
    new() { Id = 2, Firstname = "John", Surname = "Terry"},
    new() { Id = 3, Firstname = "Fred", Surname = "Staton"}
};

var orders = new List<Order>()
{
    new () {CustomerId = 1, OrderId = 101},
    new () {CustomerId = 2, OrderId = 102},
    new () {CustomerId = 2, OrderId = 103}
};

We initialize a List of Customer type and a List of Order type, each one with three instances. The CustomerId property in the Order class connects with the Id property of a corresponding customer in the Customer list, forming a relationship between customers and their orders. The objective is to obtain a list of distinct customers who have placed orders.

Every method here takes and returns a List, but nothing in the techniques depends on that β€” if we are unsure which abstraction to declare our parameters with, it helps to know how IEnumerable, ICollection, and IList relate.

Use of Foreach Loops to Compare Two Lists

Let’s start by implementing a method that uses two foreach loops:

public static List<Customer> ForEachMethod(List<Customer> customerList, List<Order> orderList)
{
    var customersWithOrders = new List<Customer>();

    foreach (var customer in customerList)
    {
        foreach (var order in orderList)
        {
            if (customer.Id == order.CustomerId && !customersWithOrders.Contains(customer))
            {
                customersWithOrders.Add(customer);
            }
        }
    }

    return customersWithOrders;
}

Every method we implement takes two list parameters, a List<Customer> and a List<Order>, respectively. The outcome is always a List<Customer> with the customer records that have placed orders.

Firstly, we initialize an empty List<Customer>, customersWithOrders, to hold the customers we identify. Then, using nested foreach loops, we iterate through each customer in the customerList and each order in the orderList. For each customer-order pair, we check if the customer’s Id matches the order’s CustomerId and if the customer does not already exist in the customersWithOrders list.

When we meet these conditions, we add the customer to the customersWithOrders list. We repeat this process until we examine all combinations and then we return the customersWithOrders list. While this approach accomplishes our objective, its time complexity is O(nΓ—m), where ‘n’ is the number of customers and ‘m’ the number of orders β€” and the Contains() guard adds a third scan, over the result list, on every match.

Finally, to check the result, let’s present the outcome list:

var customerWithOrders = ListCompareMethods.ForEachMethod(customers, orders);
Console.WriteLine(string.Join(',',customerWithOrders.Select(i=>i.Firstname)));

We print the objects in customerWithOrders list to the console. Next, to concatenate the Firstname values into a single string separated by commas, we use string.Join(). It is performed with the Select() method that projects each Customer object to their Firstname:

Alice,John

Is this material useful to you? Consider subscribing and get ASP.NET Core Web API Best Practices eBook for FREE!

We verify that these two objects were expected as a result, since the Order list includes Customer with Id 1 and 2.

LINQ and Where Extension

In this method, let’s make use of LINQ and the Where() extension:

public static List<Customer> WhereAnyMethod(List<Customer> customerList, List<Order> orderList)
{
    return customerList.Where(y => orderList.Any(z => z.CustomerId == y.Id)).ToList();
}

Here we utilize LINQ to filter the customerList based on a specific condition. For each Customer in customerList, we check if there is any Order in the orderList where the CustomerId of the Order matches the Id of the Customer.

If such an Order exists, we include the Customer in the result. Before we return it, we convert the filtered result to a List<Customer> type. The filtering is ordinary work for the Where method; what costs us here is the inner Any(), which walks orderList again for every customer.

Compare Lists With LINQ Query Syntax

Let’s now write the same filter in LINQ query syntax:

public static List<Customer> JoinMethod(List<Customer> customerList, List<Order> orderList)
{
    var customersWithOrders = (from customer in customerList
                               where orderList.Any(order => customer.Id == order.CustomerId)
                               select customer
                               ).ToList();

    return customersWithOrders;
}

This is the query-syntax form of the previous method, not a different algorithm β€” the from … where … select clauses compile down to the same Where() call with the same inner Any(), which is why the two benchmark within a whisker of each other below.

Here, we use LINQ to iterate through each element in customerList using the from clause. Then, we utilise the where clause to filter customers based on the existence of any Order in orderList whose CustomerId matches the Id of the current Customer.

The select clause then determines what is included in the result, selecting the Customer object that satisfies the filtering condition. Finally, we enclose the entire LINQ query in parentheses, and the ToList() method to convert the result into a materialized List<Customer>.

Compare Lists With Join Extension Method

Let’s utilize the Join() extension method of List type to find the customers with orders:

public static List<Customer> JoinListMethod(List<Customer> customerList, List<Order> orderList)
{
    return customerList.Join(
            orderList,
            customer => customer.Id,
            order => order.CustomerId,
            (customer, order) => customer
        ).Distinct().ToList();
}

Here, we perform an inner join operation between two lists, customerList and orderList. The method uses LINQ to join the lists based on matching keys: the Id property of each Customer in customerList and the CustomerId property of each Order in orderList. The result of the join is a sequence of paired elements, where each pair consists of a Customer and the corresponding Order.

The result selector (customer, order) => customer specifies that only the Customer part of each pair is included in the final result. Then, we use the Distinct() method to ensure that each Customer appears only once in the result, removing any duplicates.

Finally, the result is converted into a List<Customer> using the ToList() method.

Use HashSet to Compare the Lists

The next solution uses the HashSet dataset and uses it to retrieve the result we desire:

public static List<Customer> HashSetMethod(List<Customer> customerList, List<Order> orderList)
{
    var customerIds = orderList.Select(i => i.CustomerId).ToHashSet();

    return customerList.Where(i => customerIds.Contains(i.Id)).ToList();
}

This time, we create a HashSet collection customerIds, containing unique CustomerId values that we extract from the orderList. The HashSet ensures that we include only distinct values, promoting efficient containment checks.

Next, the method filters the customerList using the Where() method. It includes in the result only those customers whose Id is present in the customerIds HashSet. Finally, we convert the result into a List<Customer> using the ToList() method.

How Do We Find the Difference Between Two Lists?

With ExceptBy, which Microsoft’s LINQ reference defines as producing “the set difference of two sequences according to a specified key selector function”.

The distinction from Except matters here. Except compares whole elements, so two Customer objects with the same Id are still different unless the type overrides equality. ExceptBy takes the keys to exclude and a selector for our own key, so it compares on Id and leaves the rest of the object alone.

It answers one direction at a time. customers.ExceptBy(orderedIds, c => c.Id) gives us the customers with no orders; swapping the arguments around gives us the orders with no matching customer. A full two-sided delta means running it twice and keeping both results.

The performance argument is the same one as above: ExceptBy builds a set from the keys internally, so it is a single pass over each list rather than a scan per element.

ExceptBy has been part of System.Linq since .NET 6, so it needs no package and no helper:

var orderedIds = orders.Select(o => o.CustomerId);

var customersWithoutOrders = customers.ExceptBy(orderedIds, c => c.Id).ToList();

If we need to compare whole objects rather than one key, the Except method in LINQ covers that case in detail. And when all we want is a yes-or-no answer instead of a list, checking whether items of one list exist in another is the smaller job.

Is this material useful to you? Consider subscribing and get ASP.NET Core Web API Best Practices eBook for FREE!

Now, let’s compare our five overlap methods by running a set of benchmarks.

Benchmark Set Up

We proceed with evaluating these methods by performing a benchmark, in terms of efficiency and speed. Let’s set up two helper methods for our scenario for testing list comparison performance, in our benchmark class:

private List<Customer>? _customers;
private List<Order>? _orders;

[GlobalSetup]
public void GlobalSetup()
{
    var numberOfCustomers = 10000;
    var numberOfOrders = 500000;

    _customers = GenerateRandomCustomers(numberOfCustomers).ToList();
    _orders = GenerateRandomOrders(numberOfOrders, _customers).ToList();
}

private static IEnumerable<Customer> GenerateRandomCustomers(int count)
{
    return Enumerable.Range(1, count)
        .Select(i => new Customer
        {
            Id = i,
            Firstname = $"CustomerFirstname{i}",
            Surname = $"CustomerSurname{i}"
        });
}

private static IEnumerable<Order> GenerateRandomOrders(int count, List<Customer> customers)
{
    var random = new Random();

    return Enumerable.Range(1, count)
        .Select(i => new Order
        {
            OrderId = i,
            CustomerId = random.Next(1, customers.Count + 1)
        });
}

In our benchmarking class, the GlobalSetup() method prepares the data we need for our performance evaluations. We mark it with the [GlobalSetup] attribute, to execute once before all benchmark methods.

Within it, we initialize _customers and _orders with realistic and randomized datasets. Specifically, we utilize the GenerateRandomCustomers() method to create a list of 10,000 customers, each having unique IDs, first names, and surnames. Subsequently, the _orders list is populated using the GenerateRandomOrders() method, generating 500,000 orders with unique OrderIds and associating each order with a randomly selected customer from the _customers list.

Our GenerateRandomCustomers() method facilitates the creation of a sequence of random Customer objects based on the specified count. Using Enumerable.Range() to produce a sequence of integers, we employ the Select() method to generate a new Customer object for each integer, ensuring distinct Ids.

Similarly, our GenerateRandomOrders() method generates a sequence of random Order objects, considering the desired count and the list of customers. Using Enumerable.Range() and the Select() method, we create Order objects, setting the CustomerId property of each order to a randomly selected value between 1 and the total count of customers. With this, we establish a valid association between orders and customers.

Together, these helper methods enable us to establish a realistic dataset for benchmarking methods designed to compare and filter lists of customers and orders based on a specific property.

Evaluation Results

Let’s evaluate the benchmark results:

| Method         | Mean         | Error        | StdDev     | Rank | Gen0      | Allocated   |
|--------------- |-------------:|-------------:|-----------:|-----:|----------:|------------:|
| HashSetMethod  |     16.02 ms |     3.303 ms |   0.858 ms |    1 |         - |   732.91 KB |
| JoinListMethod |     52.06 ms |    55.950 ms |  14.530 ms |    2 | 1000.0000 | 13302.52 KB |
| JoinMethod     |     80.63 ms |     7.172 ms |   1.862 ms |    3 |         - |   440.88 KB |
| WhereAnyMethod |     84.47 ms |    11.028 ms |   2.864 ms |    4 |         - |   440.93 KB |
| ForEachMethod  | 24,058.62 ms | 3,555.263 ms | 923.291 ms |    5 |         - |   256.41 KB |

HashSetMethod() is the fastest at 16 ms, and it is fastest for a structural reason rather than a micro-optimization: it projects the order keys into a HashSet once, then answers each membership question in constant time. Its cost tracks the sum of the two list lengths instead of their product. JoinListMethod() is second at 52 ms and works the same way β€” Join() builds a lookup over the inner sequence before it starts matching β€” though it pays for the pairs it materializes, which is where its 13 MB of allocations come from.

JoinMethod() at 80.63 ms and WhereAnyMethod() at 84.47 ms land within 5% of each other, and that is not a coincidence: they are the same Where plus Any filter written in query syntax and in method syntax, so both re-scan orderList for every customer. ForEachMethod() is roughly 285 times slower again at just over 24 seconds, because on top of that nested scan its !customersWithOrders.Contains(customer) guard walks the growing result list on every match β€” three scans where the others do two. Measured on .NET 10.0.10 with BenchmarkDotNet 0.13.11, in a monitoring run of five iterations.

Which Method Should We Use to Compare Two Lists?

Use the HashSet approach. On the benchmark above it finishes in 16 milliseconds where the nested foreach takes 24 seconds: the same work, three orders of magnitude apart.

The ranking is not really about five techniques, it is about two algorithms. HashSetMethod and JoinListMethod build a lookup once and consult it, which puts them in the top two places. WhereAnyMethod and JoinMethod re-scan the second list for every element of the first, which is why they land within 5% of each other and why they fall further behind as the lists grow.

ForEachMethod is a case on its own, not a third member of that group. Its duplicate guard scans the result list too, so it makes three passes where the others make two, and it finishes about 285 times behind the next slowest.

Join is worth knowing for another reason. It returns pairs, so when we need the matching order alongside the customer, it is the only one of the five that gives us both.

Log-scale bar chart of benchmark execution times for five list-comparison methods in C#, showing the two lookup-based methods fastest, two scan-based methods close behind, and the nested-loop method as a single far outlier.

What we needUseShape
Items in A that also exist in B, matched on a keyHashSet of B's keys, then Wherevar keys = b.Select(x => x.Key).ToHashSet();
Items in A that do not exist in BExceptBya.ExceptBy(b.Select(x => x.Key), x => x.Key)
Items in B that do not exist in AExceptBy, arguments swappedRun it both ways for a two-sided delta
Both lists paired up on the keyJoinReturns the pairs, not just one side
Whether the two lists hold the same items, same orderSequenceEqualElement equality, order-sensitive
Whether the two lists hold the same items, any orderHashSet.SetEqualsIgnores order and duplicates
Small lists, matching whole objectsAny of the aboveBelow a few hundred items the difference is unmeasurable

Conclusion

Five ways to match two lists on a single property, and the benchmark separates them into two algorithms rather than five techniques: build a lookup once, or re-scan the second list per element. The HashSet approach is the one to reach for, Join() when we need both sides of the match, and ExceptBy when the question is the difference rather than the overlap.

The same key-based thinking transfers directly to comparing two dictionaries, where the lookup is already built for us.

Tested with .NET 10.0.10.