Updated on

To await multiple tasks in C#, we start all of them first and then await Task.WhenAll(), which completes when every task has completed. If we await each call as soon as we make it, the calls run one after another.

Depending on the business logic, we may need either of the two. We’ll build both against a small Web API, catch the exceptions that Task.WhenAll() hides, try Parallel.ForEach, Parallel.ForEachAsync and Task.WhenEach, and benchmark everything but Task.WhenEach on .NET 11.

To download the source code for this article, you can visit our GitHub repository.

Creating a Demo REST API

Let’s consider that we are developing an application for employee management in an organization. The client/UI application consumes a REST-based API that will return the employee details, salary, and appraisal rating for an employee from three different endpoints.

Here, we will use a simple ASP.NET Core Web API. The controller actions in this Web API return dummy data for this demo:

[HttpGet("details/{id}")]
public async Task<IActionResult> GetEmployeeDetails(Guid id)
{
    return await Task.FromResult(Ok(new
    {
        Id = id,
        Name = $"Sam_{id}",
        DateOfBirth = DateTime.Now.AddYears(-1 * Random.Shared.Next(20, 30)).Date,
        Address = "Employee Dummy Address"
    }));
}

[HttpGet("salary/{id}")]
public async Task<IActionResult> GetEmployeeSalary(Guid id)
{
    return await Task.FromResult(Ok(new
    {
        Id = id,
        SalaryInEuro = 25000
    }));
}

[HttpGet("rating/{id}")]
public async Task<IActionResult> GetEmployeeRating(Guid id)
{
    return await Task.FromResult(Ok(new
    {
        Id = id,
        Rating = 4
    }));
}

Now, let’s call the three endpoints with curl and look at their responses:

curl https://localhost:7145/api/Employee/details/7119e779-3054-493c-8cf7-c617b4aa0f4e
{"id":"7119e779-3054-493c-8cf7-c617b4aa0f4e","name":"Sam_7119e779-3054-493c-8cf7-c617b4aa0f4e","dateOfBirth":"2001-11-13T00:00:00+01:00","address":"Employee Dummy Address"}

curl https://localhost:7145/api/Employee/salary/7119e779-3054-493c-8cf7-c617b4aa0f4e
{"id":"7119e779-3054-493c-8cf7-c617b4aa0f4e","salaryInEuro":25000}

curl https://localhost:7145/api/Employee/rating/7119e779-3054-493c-8cf7-c617b4aa0f4e
{"id":"7119e779-3054-493c-8cf7-c617b4aa0f4e","rating":4}

We have made the Web API simple on purpose as we don’t want to deviate from the topic of this article by diving into the intricate implementation details of an ASP.NET Core Web API.

Execute Multiple Tasks in Sequence using async and await

The client application has an EmployeeProfile class which we use as a presentation model to the end-user:

using System.Globalization;
using System.Text;

public class EmployeeProfile
{
     public EmployeeDetails EmployeeDetails { get; }

     public decimal Salary { get; }

     public int Rating { get; }

     public EmployeeProfile(EmployeeDetails employeeDetails, decimal salary, int rating)
     {
         EmployeeDetails = employeeDetails;
         Salary = salary;
         Rating = rating;
     }

     public override string ToString()
     {
         var sb = new StringBuilder();
         sb.AppendLine($"Name: {EmployeeDetails.Name}");
         sb.AppendLine($"DOB: {EmployeeDetails.DateOfBirth.ToShortDateString()}");
         sb.AppendLine($"Address: {EmployeeDetails.Address}");
         sb.AppendLine($"Salary: {Salary}{HexToChar("20AC")}");
         sb.AppendLine($"Appraisal Rating: {Rating}");

         return sb.ToString();

         char HexToChar(string hex)
             => (char) ushort.Parse(hex, NumberStyles.HexNumber);
     }
}

Here, we override the ToString() method to append all the employee data to a single string for printing it to the console.

Since this class acts like a view model, the properties of this class are constructed from the contract classes:

public class EmployeeDetails
{
    public Guid Id { get; set; }

    public string Name { get; set; }

    public DateTime DateOfBirth { get; set; }

    public string Address { get; set; }
}

public class Salary
{
    public Guid Id { get; set; }
    
    public decimal SalaryInEuro { get; set; }
}

public class AppraisalRating
{ 
    public Guid Id { get; set; }

    public int Rating { get; set; }
}

The Executor class in the client application keeps an EmployeeApiFacade instance in its _employeeApiFacade field and consolidates employee data from all three REST API endpoints while constructing the EmployeeProfile instance:

public async Task<EmployeeProfile> ExecuteInSequence(Guid id)
{  
    var employeeDetails = await _employeeApiFacade.GetEmployeeDetails(id);
    var employeeSalary = await _employeeApiFacade.GetEmployeeSalary(id);
    var employeeRating = await _employeeApiFacade.GetEmployeeRating(id);    

    return new EmployeeProfile(employeeDetails, employeeSalary, employeeRating);
}

Here, we are executing the tasks for fetching employee details, salary, and rating in sequence using asynchronous programming.

First, we are awaiting the Task instance returned by the GetEmployeeDetails() method. Then, we do the same with GetEmployeeSalary() in the continuation of the GetEmployeeDetails() method. Finally, we repeat the action with the GetEmployeeRating() method in the continuation of the GetEmployeeSalary() method. Hence, in this case, each task execution step waits for the previous step to finish without blocking the thread.

In these three methods (GetEmployeeDetails(), GetEmployeeSalary(), GetEmployeeRating()) all we do is send a simple HTTP request and process the response. Also, we are simulating a network latency in all three methods using the Task.Delay() method. Real calls have delays like this too, from slow database queries to servers on the other side of the world.

You can find the implementation inside the EmployeeApiFacade class in the client project.

Typically, we do such sequential execution in a workflow where each task execution step is dependent on the previous one, unlike the above steps. However, here we are anyway doing it to demonstrate in a later section the performance improvement that we can achieve by executing these tasks in parallel.

With the API running, let’s run the client application with dotnet run:

Name: Sam_7119e779-3054-493c-8cf7-c617b4aa0f4e
DOB: 11/13/1997
Address: Employee Dummy Address
Salary: 25000€
Appraisal Rating: 4

The controller picks each date of birth with Random.Shared, so the dates change from run to run.

We get back the consolidated result. Let’s now see if we can improve the performance of this functionality.

How Does Task.WhenAll Run Independent Tasks at Once?

Task.WhenAll() takes several tasks and returns one task that completes when all of them have completed.

We call GetEmployeeDetails(), GetEmployeeSalary() and GetEmployeeRating() first and keep the tasks they return, without awaiting any of them. At that point all three HTTP requests are already running. Then we await Task.WhenAll() once.

The whole method now waits about as long as the slowest call, not as long as all three added up. In our benchmark on .NET 11, the sequential version took about 62 ms and the Task.WhenAll() version about 31 ms.

For I/O work like HTTP calls, this needs no extra threads. While a request waits for the network, no thread is blocked on it, so all three requests can wait at the same time.

When all the tasks return the same type, await Task.WhenAll() also returns their results as an array, in the order we passed the tasks in.

If the difference between asynchronous code and threads is still blurry, our article on async vs multithreading goes through it.

Since the tasks for fetching employee details, salary and rating are independent of each other, it is easy to execute them in parallel to improve the overall performance of the workflow:

public async Task<EmployeeProfile> ExecuteInParallelUsingTaskWhenAll(Guid id)
{
    var employeeDetailsTask = _employeeApiFacade.GetEmployeeDetails(id);
    var employeeSalaryTask = _employeeApiFacade.GetEmployeeSalary(id);
    var employeeRatingTask = _employeeApiFacade.GetEmployeeRating(id);

    await Task.WhenAll(employeeDetailsTask, employeeSalaryTask, employeeRatingTask);

    var employeeDetails = await employeeDetailsTask;
    var employeeSalary = await employeeSalaryTask;
    var employeeRating = await employeeRatingTask; 

    return new EmployeeProfile(employeeDetails, employeeSalary, employeeRating);
}

Here, we are not awaiting the Task instances from any of the methods, so each call returns its Task right away, and the three requests run at the same time. However, we need to wait for all three methods to complete their execution before the consolidation of the result. This is achieved by using the WhenAll method from Task Parallel Library.

The Task.WhenAll method creates a Task that will be complete when all the supplied tasks have been completed. Once all three Tasks are complete, we await the individual Task instances to derive the result from them.

Alternatively, we can use the Result property from the Task instances instead of awaiting them to avoid some unnecessary state machine code generated during compilation:

...
...
var employeeDetails = employeeDetailsTask.Result;
var employeeSalary = employeeSalaryTask.Result;
var employeeRating = employeeRatingTask.Result;   

return new EmployeeProfile(employeeDetails, employeeSalary, employeeRating);
...

Though using blocking code like .Result is not considered good practice under normal circumstances, in this case, it is perfectly safe to use it as all the tasks have finished execution by the time this line is executed. In fact, in this case, it is more beneficial to use this approach as it reduces the size of the compiled code.

Now, when we execute the client code, we get back the consolidated result but with an increase in the overall performance.

Improving the Exception Handling in Task.WhenAll

Handling exceptions plays a major part in real-life application development. There is a problem in the above implementation in regards to handling exceptions. Let’s consider a scenario where both GetEmployeeDetails() and GetEmployeeSalary() methods throw exceptions:

public async Task<EmployeeDetails> GetEmployeeDetails(Guid id)
{    
    var response = await _httpClient
                         .GetStringAsync($"https://localhost:7145/api/employee/details/{id}");
    var employeeDetails = JsonSerializer.Deserialize<EmployeeDetails>(response, _serializerOptions);
    await Task.Delay(2000);
    throw new Exception("Test Exception 1");    
}

public async Task<decimal> GetEmployeeSalary(Guid id)
{    
    var response = await _httpClient
                         .GetStringAsync($"https://localhost:7145/api/employee/salary/{id}");
    var salary = JsonSerializer.Deserialize<Salary>(response, _serializerOptions);
    await Task.Delay(1000);
    throw new Exception("Test Exception 2");    
}

In this case, when we run dotnet run -- parallel-when-all, we only get one exception in the console, and the other one is ignored. It is not the first one in our list, though. await throws the exception of the task that failed first, and here that is GetEmployeeSalary(), which fails after one second:

System.Exception: Test Exception 2
   at MultipleTasksDemo.Client.EmployeeApiFacade.GetEmployeeSalary(Guid id) in EmployeeApiFacade.cs:line 33
   at MultipleTasksDemo.Client.Executor.ExecuteInParallelUsingTaskWhenAll(Guid id) in Executor.cs:line 26
   at Program.<Main>$(String[] args) in Program.cs:line 22

So, if other Task instances throw any exceptions, we will never know about it. If we want to get hold of all the exceptions for logging purposes, we need to handle Task.WhenAll a bit differently:

public class TaskExtensions
{
    public static async Task<(T1, T2, T3)> WhenAll<T1, T2, T3>(Task<T1> task1, Task<T2> task2, Task<T3> task3)
    {
        var allTasks = Task.WhenAll(task1, task2, task3);
        try
        {
            await allTasks;
        }
        catch
        {
            Console.WriteLine("Task Exception");
            throw allTasks.Exception!;
        }

        return (task1.Result, task2.Result, task3.Result);
    }
}

This works because allTasks.Exception holds every exception, even though await allTasks threw only one of them. We catch that one and throw the full AggregateException instead.

This method wraps the Task.WhenAll method to return a ValueTuple with the results of the tasks. Most importantly, it also handles the issue with multiple exceptions by catching and rethrowing an aggregate exception on all tasks that are awaited.

Now, we can invoke the WhenAll method from the TaskExtensions class instead of the original Task.WhenAll in the calling method:

public async Task<EmployeeProfile> ExecuteInParallel(Guid id)
{
    var employeeDetailsTask = _employeeApiFacade.GetEmployeeDetails(id);
    var employeeSalaryTask =  _employeeApiFacade.GetEmployeeSalary(id);
    var employeeRatingTask =  _employeeApiFacade.GetEmployeeRating(id);   

    var (employeeDetails, employeeSalary, employeeRating) 
        = await TaskExtensions.WhenAll(employeeDetailsTask, employeeSalaryTask, employeeRatingTask); 
      
    return new EmployeeProfile(employeeDetails, employeeSalary, employeeRating);
}

If we execute the client code now with dotnet run -- parallel, we can see an aggregate exception in the console showing both exceptions:

Task Exception
System.AggregateException: One or more errors occurred. (Test Exception 2) (Test Exception 1)
 ---> System.Exception: Test Exception 2
   at MultipleTasksDemo.Client.EmployeeApiFacade.GetEmployeeSalary(Guid id) in EmployeeApiFacade.cs:line 33
   ...
 ---> (Inner Exception #1) System.Exception: Test Exception 1
   at MultipleTasksDemo.Client.EmployeeApiFacade.GetEmployeeDetails(Guid id) in EmployeeApiFacade.cs:line 24<---

Both exceptions are there now, in the order the two tasks failed.

Performance Comparison async/await vs. Task.WhenAll 

To compare the two approaches, we run the MultipleApiExecutorBenchmark class with the default iteration counts while the API is running (dotnet run -c Release -- --filter *MultipleApi*):

| Method            | Mean     | Error    | StdDev   | Ratio | Rank | Allocated | Alloc Ratio |
|------------------ |---------:|---------:|---------:|------:|-----:|----------:|------------:|
| ExecuteInParallel | 31.16 ms | 0.092 ms | 0.086 ms |  0.50 |    1 |  11.05 KB |        1.09 |
| ExecuteInSequence | 62.11 ms | 0.155 ms | 0.137 ms |  1.00 |    2 |  10.09 KB |        1.00 |

The ExecuteInParallel() method that uses Task.WhenAll clearly exhibits better overall performance compared to the ExecuteInSequence() method.

Both numbers follow the delays we simulate in EmployeeApiFacade: the sequential method waits for all three in a row, while the parallel one only waits for the longest.

Sequential Task Execution using async, await, and foreach Loop

Let’s consider another business workflow where we need to invoke the same REST API endpoint multiple times. For example, invoking the /details/{id} endpoint to fetch details for three different employees. Once again, depending on the business case, we may have to do this either sequentially or in parallel.

To execute the tasks sequentially, we can use a foreach loop:

public async Task<IEnumerable<EmployeeDetails>> ExecuteUsingNormalForEach(IEnumerable<Guid> employeeIds)
{
    List<EmployeeDetails> employeeDetails = [];

    foreach (var id in employeeIds)
    {
        var employeeDetail = await _employeeApiFacade.GetEmployeeDetails(id);

        employeeDetails.Add(employeeDetail);
    }

    return employeeDetails;
}

Here, for each employee id, we call the GetEmployeeDetails() method that invokes the /details/{id} endpoint in the REST API. We await the Task instance returned from the GetEmployeeDetails() method and then write the employee details to the console (dotnet run -- foreach in the sample):

Id: 7119e779-3054-493c-8cf7-c617b4aa0f4e
Name: Sam_7119e779-3054-493c-8cf7-c617b4aa0f4e
DOB: 11/13/2003 12:00:00 AM

Id: cbb9c89f-3718-43d9-8763-b3fa3765bcea
Name: Sam_cbb9c89f-3718-43d9-8763-b3fa3765bcea
DOB: 11/13/2000 12:00:00 AM

Id: 165bcfa8-3a4f-4850-a681-bc496616f830
Name: Sam_165bcfa8-3a4f-4850-a681-bc496616f830
DOB: 11/13/2001 12:00:00 AM

We see that all the tasks are executed. However, these tasks of fetching employee details for multiple employees can very easily be executed in parallel as they are independent of each other. So, executing them in parallel will improve the performance. Let’s see how we can achieve such improvement.

Each await inside the loop waits for one request to finish before the next one starts. We look at that cost in more detail in our article on why we should avoid await in a loop.

Parallel Tasks Execution using Parallel.ForEach

The Parallel.ForEach method executes a foreach operation in which the iterations may run in parallel.

Let’s re-write the example from the previous section using Parallel.ForEach, with using System.Collections.Concurrent; at the top of the file for the ConcurrentBag<T>:

public IEnumerable<EmployeeDetails> ExecuteUsingParallelForeach(IEnumerable<Guid> employeeIds)
{
    ParallelOptions parallelOptions = new()
    {
         MaxDegreeOfParallelism = 3
    };    

    ConcurrentBag<EmployeeDetails> employeeDetails = new();

    Parallel.ForEach(employeeIds, parallelOptions,  id =>
    {
        var employeeDetail = _employeeApiFacade.GetEmployeeDetails(id).GetAwaiter().GetResult();
                          
        employeeDetails.Add(employeeDetail);
    });   

    return employeeDetails;
}

We set the MaxDegreeOfParallelism property to 3. This property limits the number of concurrent operations by the Parallel.ForEach loop to the set value. By default, ForEach will utilize as many threads as the underlying scheduler provides. Here, we set it to 3 because we only have three ids to fetch.

Parallel.ForEach does not support asynchronous delegates. So, we are blocking the thread that executes the GetEmployeeDetails() method till it produces a result. That blocking call is the same trap as running async code synchronously anywhere else, which we cover in our article on how to run an async method synchronously.

In many cases, Parallel.ForEach can provide significant performance improvements over ordinary sequential loops. However, the work of parallelizing the loop introduces complexity that can lead to problems that we don’t encounter in sequential code. In certain cases, a parallel loop might run slower than its sequential equivalent.

It is better to keep in mind the below points before using the Parallel loops:

  • Parallel loops are designed for CPU-intensive operations. It’s not always the best solution for I/O-bound operations
  • Parallel loops are not appropriate for asynchronous code flow
  • How many iterations really run at once depends on the thread pool and the number of cores, so performance can vary

In the previous code block, each thread runs the independent I/O operations because of the lack of async support in the Parallel.ForEach. This is a big issue considering scalability and has the potential for thread pool starvation in an ASP.NET web application.

One more difference from the foreach version: the employees can come back in any order. Each iteration adds its result to a ConcurrentBag<T>, and a bag doesn’t keep the order in which items were added.

However, starting from .NET 6 we can use the Parallel.ForEachAsync method which is async aware.

Parallel Tasks Execution using Parallel.ForEachAsync

The Parallel.ForEachAsync executes a foreach operation on an IEnumerable<T> in which iterations may run in parallel. However, unlike the Parallel.ForEach, it returns a Task instance that represents the entire for-each operation. It also supports an async delegate. It has been available since .NET 6.

Now, let’s re-write the example from the previous section using Parallel.ForEachAsync:

public async Task<IEnumerable<EmployeeDetails>> ExecuteUsingParallelForeachAsync(IEnumerable<Guid> employeeIds)
{
    ParallelOptions parallelOptions = new()
    {
        MaxDegreeOfParallelism = 3
    };
    
    ConcurrentBag<EmployeeDetails> employeeDetails = new();
    
    await Parallel.ForEachAsync(employeeIds, parallelOptions, async (id, _) =>
    {
        var employeeDetail = await _employeeApiFacade.GetEmployeeDetails(id);

        employeeDetails.Add(employeeDetail);
    });   

    return employeeDetails;

}

Here, we set the MaxDegreeOfParallelism property to 3 like in the previous section. Without it, Parallel.ForEachAsync would run at most Environment.ProcessorCount iterations at once, while Parallel.ForEach has no limit of its own.

Now, on executing the client code, we see that the tasks are executed and  it  is approximately similar to the timing that we got using the Parallel.ForEach. 

For a short, fixed list of I/O calls like ours, we don’t need a parallel loop at all. We can start one request per id and await them together with Task.WhenAll():

public async Task<IEnumerable<EmployeeDetails>> ExecuteUsingTaskWhenAll(IEnumerable<Guid> employeeIds)
{
    var tasks = employeeIds.Select(id => _employeeApiFacade.GetEmployeeDetails(id));

    return await Task.WhenAll(tasks);
}

This version has no degree of parallelism to set, so every request starts at once, and the results come back in the order of the ids. With thousands of ids we wouldn’t want that, and Parallel.ForEachAsync with a limit is the better tool. We compare the two in more depth in our article on Parallel.ForEachAsync and Task.WhenAll.

Performance Comparison foreach vs. Parallel.ForEach vs. Parallel.ForEachAsync vs. Task.WhenAll

Here, we run the SameApiExecutorBenchmark class the same way (--filter *SameApi*):

| Method                           | Mean     | Error    | StdDev   | Ratio | Rank | Allocated | Alloc Ratio |
|--------------------------------- |---------:|---------:|---------:|------:|-----:|----------:|------------:|
| ExecuteUsingParallelForeachAsync | 31.09 ms | 0.137 ms | 0.122 ms |  0.33 |    1 |  13.33 KB |        1.21 |
| ExecuteUsingParallelForeach      | 31.13 ms | 0.067 ms | 0.063 ms |  0.33 |    1 |  14.37 KB |        1.30 |
| ExecuteUsingTaskWhenAll          | 31.16 ms | 0.106 ms | 0.100 ms |  0.33 |    1 |  11.93 KB |        1.08 |
| ExecuteUsingNormalForEach        | 93.06 ms | 0.226 ms | 0.201 ms |  1.00 |    2 |  11.07 KB |        1.00 |

So, the Parallel.ForEach and the Parallel.ForEachAsync approaches perform better than the normal foreach. We see that there is not much difference in terms of execution time between the Parallel.ForEach and the Parallel.ForEachAsync. However, the scalability of Parallel.ForEachAsync is much higher considering that it is not blocking the individual threads anymore during the I/O operations. Hence, it minimizes the chances of thread pool starvation.

Task.WhenAll() lands in the same group as the two parallel loops. With only three requests, starting them all at once costs nothing extra, and it allocates the least of the three.

How Do We Handle Each Task as Soon as It Finishes?

Task.WhenEach(), available since .NET 9, gives us the tasks back one by one in the order they complete. It returns an IAsyncEnumerable, so we loop over it with await foreach and await each task inside the loop. That inner await doesn’t wait, because the task has already finished.

Task.WhenAll() waits for the slowest task and returns the results in the order we passed the tasks in. WhenEach() hands us the fastest task first, so we can show or store a result while slower calls are still running.

Before .NET 9, we wrote the same loop with Task.WhenAny(): wait for any task, remove it from the list, repeat. That still works, but Microsoft Learn suggests it only for a small number of tasks.

If a task fails, awaiting it inside the loop throws its exception and ends the loop. The other tasks keep running unless we cancel them.

Let’s fetch our three employees this way:

public async Task<IEnumerable<EmployeeDetails>> ExecuteUsingTaskWhenEach(IEnumerable<Guid> employeeIds)
{
    var tasks = employeeIds.Select(id => _employeeApiFacade.GetEmployeeDetails(id));

    List<EmployeeDetails> employeeDetails = [];

    await foreach (var task in Task.WhenEach(tasks))
    {
        employeeDetails.Add(await task);
    }

    return employeeDetails;
}

The list fills up in the order the responses arrive. Our three calls take about the same time, so the order can change from run to run. The sample’s test makes it visible: it gives the first id the slowest fake response and the last id the fastest, and the details come back in reverse order.

Compare that with the Microsoft Learn page on Task.WhenAll(), which promises the results “in the same order as they were provided”. We use Task.WhenAll() when we need everything before we can carry on, and Task.WhenEach() when each result is useful on its own.

Conclusion

In this article, we have demonstrated a few scenarios where we may need to execute tasks in sequence and parallel. C# provides us with many approaches to achieve the same and we have covered most of them here. On .NET 9 and later, Task.WhenEach() adds one more: handling every result as soon as it arrives. One must keep in mind the general pitfalls of asynchronous programming in C# while implementing such task execution.

Tested with .NET 11.0.0, BenchmarkDotNet 0.15.8, NUnit 5.0.0 and Moq 4.21.0.