In this article, we’re going to learn how to implement paging in ASP.NET Core Web API. Paging (pagination) is one of the most important concepts in building RESTful APIs.
As a matter of fact, we don’t want to return a collection of all resources when querying our API. That can cause performance issues and it’s in no way optimized for public or private APIs. It can cause massive slowdowns and even application crashes in severe cases.
The source code for this article can be found on the GitHub repo. If you want to follow along with the article, you can use the start branch and if you want to get the final solution or if you get stuck, switch to the end branch.
NOTE: Some degree of previous knowledge is needed to follow this article. It relies heavily on the ASP.NET Core Web API series on Code Maze, so if you are not sure how to set up the database or how the underlying architecture works, we strongly suggest you go through the series.
We’ll discuss what paging is, the easiest way to implement it, and then improve on that solution some more to create a more readable and flexible codebase.
So let’s see what we’re going to talk about in this article exactly:
- What is Paging?
- Initial Implementation
- Paging Implementation
- Testing the Solution
- Improving the Solution
Let’s start.
What is Paging?
Paging refers to getting partial results from an API. Imagine having millions of results in the database and having your application try to return all of them at once.
Not only that would be an extremely ineffective way of returning the results, but it could also possibly have devastating effects on the application itself or the hardware it runs on. Moreover, every client has limited memory resources and it needs to restrict the number of shown results.
Thus, we need a way to return a set number of results to the client in order to avoid these consequences.
Let’s see how we can do that.
Initial Implementation
Before we do any changes to the source code, let’s inspect how it looks right now, and how you would probably begin with any project.
In our case, we have the OwnerController which does all the necessary actions on the Owner entity.
One particular action that stands out, and that we need to change is the GetOwners() action:
|
1 2 3 4 5 6 7 8 9 |
[HttpGet] public IActionResult GetOwners() { var owners = _repository.Owner.GetAllOwners(); _logger.LogInfo($"Returned all owners from database."); return Ok(owners); } |
Which calls GetOwners() from OwnerRepository:
|
1 2 3 4 5 |
public IEnumerable<Owner> GetOwners() { return FindAll() .OrderBy(ow => ow.Name); } |
FindAll() method is just a method from a Base Repository class that return the whole set of owners.
|
1 2 3 4 |
public IQueryable<T> FindAll() { return this.RepositoryContext.Set<T>(); } |
As you can see it’s a straightforward action, meant to return all the owners from the database ordered by name.
And it does just that.
But, in our case, that’s just a few account owners (five). What if there were thousands or even millions of people in the database (you wish, but still, imagine another kind of entity). End then add to that, a few thousand of API consumers.
We would end up with a very long query that returns A LOT of data.
The best-case scenario would be that you started out with a small number of owners that increased slowly over time so you can notice the slow decline in performance. Other scenarios are far less benign for your application and machines (imagine hosting it in the cloud and not having proper caching in place).
So, having that in mind, let’s modify this method to support paging.
Paging Implementation
Mind you, we don’t want to change the base repository logic or implement any business logic in the controller.
What we want to achieve is something like this: https://localhost:5001/api/owners?pageNumber=2&pageSize=2. This should return the second set of two owners we have in our database.
We also want to constraint our API not to return all the owners even if someone calls https://localhost:5001/api/owners.
Let’s start by changing the controller:
|
1 2 3 4 5 6 7 8 9 |
[HttpGet] public IActionResult GetOwners([FromQuery] OwnerParameters ownerParameters) { var owners = _repository.Owner.GetOwners(ownerParameters); _logger.LogInfo($"Returned {owners.Count()} owners from database."); return Ok(owners); } |
A few things to take note here:
- We’re calling the
GetOwnersmethod from theOwnerRepository, which doesn’t exist yet, but we’ll implement it soon - We’re using
[FromQuery]to point out that we’ll be using query parameters to define which page and how many owners we are requesting OwnerParametersclass is the container for the actual parameters
We also need to actually create OwnerParameters class since we are passing it as an argument to our controller. Let’s create it in the Models folder of the Entities project:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 |
public class OwnerParameters { const int maxPageSize = 50; public int PageNumber { get; set; } = 1; private int _pageSize = 10; public int PageSize { get { return _pageSize; } set { _pageSize = (value > maxPageSize) ? maxPageSize : value; } } } |
We are using constant maxPageSize to restrict our API to a maximum of 50 owners. We have two public properties – PageNumber and PageSize. If not set by the caller, PageNumber will be set to 1, and PageSize to 10.
Now, let’s implement the most important part, the repository logic.
We need to extend the GetOwners() method in the IOwnerRepository interface and in the OwnerRepository class:
|
1 2 3 4 5 6 7 8 9 |
public interface IOwnerRepository : IRepositoryBase<Owner> { IEnumerable<Owner> GetOwners(OwnerParameters ownerParameters); Owner GetOwnerById(Guid ownerId); OwnerExtended GetOwnerWithDetails(Guid ownerId); void CreateOwner(Owner owner); void UpdateOwner(Owner dbOwner, Owner owner); void DeleteOwner(Owner owner); } |
And the logic:
|
1 2 3 4 5 6 7 8 |
public IEnumerable<Owner> GetOwners(OwnerParameters ownerParameters) { return FindAll() .OrderBy(on => on.Name) .Skip((ownerParameters.PageNumber - 1) * ownerParameters.PageSize) .Take(ownerParameters.PageSize) .ToList(); } |
Ok, the easiest way to explain this is by example.
Say we need to get the results for the third page of our website, counting 20 as the number of results we want. That would mean we want to skip the first ((3 – 1) * 20) = 40 results, and then take the next 20 and return them to the caller.
Does that make sense?
Testing the Solution
Now, in our database we only have only a few owners, so let’s try something like this:
https://localhost:5001/api/owners?pageNumber=2&pageSize=2
This should return the next subset of owners:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 |
[ { "id": "66774006-2371-4d5b-8518-2177bcf3f73e", "name": "Nick Somion", "dateOfBirth": "1998-12-15T00:00:00", "address": "North sunny address 102" }, { "id": "a3c1880c-674c-4d18-8f91-5d3608a2c937", "name": "Sam Query", "dateOfBirth": "1990-04-22T00:00:00", "address": "91 Western Roads" } ] |
If that’s what you got, you’re on the right track.
Now, what can we do to improve this solution?
Improving the Solution
Since we’re returning just a subset of results to the caller, we might as well have a PagedList instead of List.
PagedList will inherit from the List class and will add some more to it. We can also, move the skip/take logic to the PagedList since it makes more sense.
Let’s implement it.
Implementing PagedList Class
We don’t want our skip/take logic implemented inside our repository. Let’s create a class for it:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 |
public class PagedList<T> : List<T> { public int CurrentPage { get; private set; } public int TotalPages { get; private set; } public int PageSize { get; private set; } public int TotalCount { get; private set; } public bool HasPrevious => CurrentPage > 1; public bool HasNext => CurrentPage < TotalPages; public PagedList(List<T> items, int count, int pageNumber, int pageSize) { TotalCount = count; PageSize = pageSize; CurrentPage = pageNumber; TotalPages = (int)Math.Ceiling(count / (double)pageSize); AddRange(items); } public static PagedList<T> ToPagedList(IQueryable<T> source, int pageNumber, int pageSize) { var count = source.Count(); var items = source.Skip((pageNumber - 1) * pageSize).Take(pageSize).ToList(); return new PagedList<T>(items, count, pageNumber, pageSize); } } |
As you can see, we’ve transferred the skip/take logic to the static method inside of the PagedList class. We’ve added a few more properties, that will come in handy as metadata for our response.
HasPrevious is true if CurrentPage is larger than 1, and HasNext is calculated if CurrentPage is smaller than the number of total pages. TotalPages is calculated too, by dividing the number of items by the page size and then rounding it to the larger number since a page needs to exist even if there is one item on it.
Now that we’ve cleared that out, let’s change our OwnerRepository and OwnerController accordingly.
First, we need to change our repo (don’t forget to change the interface too):
|
1 2 3 4 5 6 |
public PagedList<Owner> GetOwners(OwnerParameters ownerParameters) { return PagedList<Owner>.ToPagedList(FindAll().OrderBy(on => on.Name), ownerParameters.PageNumber, ownerParameters.PageSize); } |
And then the controller:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 |
[HttpGet] public IActionResult GetOwners([FromQuery] OwnerParameters ownerParameters) { var owners = _repository.Owner.GetOwners(ownerParameters); var metadata = new { owners.TotalCount, owners.PageSize, owners.CurrentPage, owners.TotalPages, owners.HasNext, owners.HasPrevious }; Response.Headers.Add("X-Pagination", JsonConvert.SerializeObject(metadata)); _logger.LogInfo($"Returned {owners.TotalCount} owners from database."); return Ok(owners); } |
Now, if we send the same request as we did earlier https://localhost:5001/api/owners?pageNumber=2&pageSize=2, we get the same exact result:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 |
[ { "id": "f98e4d74-0f68-4aac-89fd-047f1aaca6b6", "name": "Martin Miller", "dateOfBirth": "1983-05-21T00:00:00", "address": "3 Edgar Buildings" }, { "id": "66774006-2371-4d5b-8518-2177bcf3f73e", "name": "Nick Somion", "dateOfBirth": "1998-12-15T00:00:00", "address": "North sunny address 102" } ] |
But now we have some additional useful information in X-Pagination response header:
Response headers
As you can see, all of our metadata is here. We can use this information when building any kind of frontend pagination functionality. You can play around with different requests to see how it works in other scenarios.
There is one more thing we can do to make our solution even more generic. We have the OwnerParameters class, but what if we want to use it in our AccountController? Parameters that we send to the Account controller might be different. Maybe not for paging, but we’ll send a bunch of different parameters later on and we need to separate the parameter classes.
Let’s see how to improve it.
Creating a Parent Parameters Class
First, let’s create an abstract class QueryStringParameters. We’ll use this class to implement mutually used functionalities for every parameter class we will implement. And since we have OwnerController and AccountController, that means we need to create OwnerParameters and AccountParameters classes.
Let’s start by defining QueryStringParameters class inside the Models folder of the Entities project:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 |
public abstract class QueryStringParameters { const int maxPageSize = 50; public int PageNumber { get; set; } = 1; private int _pageSize = 10; public int PageSize { get { return _pageSize; } set { _pageSize = (value > maxPageSize) ? maxPageSize : value; } } } |
We’ve also moved our paging logic inside the class since it will be valid for any entity we might want to return through the repository.
Now, we need to create AccountParameters class, and then inherit the QueryStringParameters class in both the OwnerParameters and the AccountParameters classes.
Remove the logic from OwnerParameters and inherit QueryStringParameters:
|
1 2 3 4 |
public class OwnerParameters : QueryStringParameters { } |
And create AccountParameters class inside the Models folder too:
|
1 2 3 4 |
public class AccountParameters : QueryStringParameters { } |
Now, these classes look a bit empty now, but soon we’ll be populating them with other useful parameters and we’ll see what the real benefit is. For now, it’s important that we have a way to send a different set of parameters for AccountController and OwnerController.
Now we can do something like this too, inside our AccountController:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 |
[HttpGet] public IActionResult GetAccountsForOwner(Guid ownerId, [FromQuery] AccountParameters parameters) { var accounts = _repository.Account.GetAccountsByOwner(ownerId, parameters); var metadata = new { accounts.TotalCount, accounts.PageSize, accounts.CurrentPage, accounts.TotalPages, accounts.HasNext, accounts.HasPrevious }; Response.Headers.Add("X-Pagination", JsonConvert.SerializeObject(metadata)); _logger.LogInfo($"Returned {accounts.TotalCount} owners from database."); return Ok(accounts); } |
And due to the inheritance of the paging parameters through the QueryStringParameters class, we get the same behavior. Neat.
That’s about it, let’s summarize.
Conclusion
Paging is a useful and important concept in building any API out there. Without it, our application would probably slow down considerably or just plain drop dead.
The solution we’ve implemented is not perfect, far from it, but you got the point. We’ve isolated different parts of the paging mechanism and we can go even further and make it a bit more generic. But you can do it as an exercise and implement it in your own project. You can also find one frontend application of paging in our Angular Material paging article.
In this article we’ve covered:
- The easiest way to implement pagination in ASP.NET Core Web API
- Tested the solution in a real-world scenario
- Improved that solution by introducing
PagedListentity and separated our parameters for different controllers
Hope you liked this article and that you’ve learned something new or useful from it.
hi, all your calls are synchronous. Would it be better to implement asynchronous in repository or just simply do
var owner = await Task.Run(() => { return _repository.Owner.GetOwners(ownerParameters);});. Thank you!Hey vidriduch, yes they are. We intentionally siplified the series, but if you can find our take on asynchronous implementation here: http://34.65.74.140/async-generic-repository-pattern/
We also have a few different “improvement” articles related to Web API, so you can try the search function to find out if we’ve already covered it.
Thanks for the feedback!
Hi Code Maze,
I am beginner, can you please explain about RepositoryBase.cs, where this can be write if following repository pattern with modelview, and how can use it if I have other lists like List of Student, List of Owner, List of Teacher.
Hey Vipin Jha,
As we mentioned at the beginning of the article, some of the concepts are explained in our basic ASP.NET Core Web API series:
http://34.65.74.140/net-core-series/
Make sure to go through that series first to be able to follow this one, since you’ve just begun your journey.
For the concrete answer to your question on what RepositoryBase is, I would go directly to:
http://34.65.74.140/net-core-web-development-part4/
And if you manage to get it down and understand the concept, you can improve it even more by making it asynchronous:
http://34.65.74.140/async-generic-repository-pattern/
Hope this helps and if you have any more questions, feel free to leave us a comment and we’ll try to help you as soon as we can.
Thanks Vladimir Pecanac for help..
In your blog the scenario defined as Generic repository, wrapper classes. Where as in my case it is not generic classes for repository. Here is example given below:
public interface ISchoolsRepository{
Task<ICollection> GetAllSchoolAsync();
}
and Repository Class:
public class SchoolsRepository : ISchoolsRepository{
public readonly learning_gpsContext _GpsContext;
public SchoolsRepository(learning_gpsContext GpsContext)
{
_GpsContext = GpsContext;
}
public async Task<ICollection> GetAllSchoolAsync()
{
List results = null;
var result = PagdeList.Create(_GpsContext.School, 1, 10);
results = await _GpsContext.School.AsNoTracking().ToListAsync();
return results;
}
}
and my controller is look like this:
[HttpGet]public async Task GetSchoolAll()
{
var schools = await _schoolsRepository.GetAllSchoolAsync();
List schoolsVms = new List();
foreach (Schools school in schools)
{
schoolsVms.Add(new SchoolsVm
{
Id = school.ID.ToString(),
Name = school.Name,
creatDate = school.CreatedAt.ToString()
});
}
return Ok(schoolsVms);
}
Any suggestion to do pagination in the given situation. Although your tutorial is very clear and well articulated but if I will follow your blog, I will have to do lots of changes in my projects.
Shouldn’t be the first parameter in ToPagedList method IQueryable instead of IEnumerable ? I checked it in my postgres database with pg_stat_statements and when it was IEnumerable I’ve got all of the records and only then it was sliced into smaller parts to retrieve it in the view. When I changed to IQueryable then it was fine, the query returned only the elements I requested and an additional COUNT(*) request was send. 🙂
Yes, you are exactly right. The query is executed in the ToPagedList method and it should have been IQueriable at that point of execution. Good catch, thanks! We’ll update the article and the source code ASAP.
Article “Searching in ASP.NET Core Web API” also needs to be updated, when list is passed as a parameter to ToPagedList method inside repository. Nice articles!
I like the service paging concept. Good way to prevent those folks that don’t do proper filtering on the front end web sites from returning the universe.
Hi Mr. Vladimir, I’ve got confuse about combine AutoMapper and Paging, in previous net-core-web-development-part5 GetOwners() method mapping from Owner->OwnerDto. Should I using paging for listOwner or listOwnerDto? Thanks.
You use paging functionality for the List but when you create such a list, you can map it to the List because you should always return a DTO object to the client. The model classes are here just for the database purpose.
https://media3.giphy.com/media/GCvktC0KFy9l6/giphy.gif
Thanks Marinko, I have a question:
public IActionResult GetAllServers([FromQuery] ServerParameters serverParams)
{
try
{
var servers = _repository.Server.GetAllServer(serverParams);
var metadata = new
{
servers.TotalRecords,
servers.PageSize,
servers.CurrentPage,
servers.TotalPages,
servers.HasNext,
servers.HasPrevious
};
Response.Headers.Add("X-Pagination", JsonConvert.SerializeObject(metadata));
_logger.LogInfo($"return {servers.PageSize} server from database.");
var lstServerResult = _mapper.Map<IEnumerable>(servers);
foreach (var serverDTO in lstServerResult)
{
var lastestComment = _repository.Comment.GetLastestCommentByServerID(serverDTO.ServerID);
if (lastestComment != null)
{
serverDTO.LastComment = lastestComment.Description;
}
}
return Ok(lstServerResult);
}
catch (Exception e)
{
_logger.LogError($"Error when get all servers: {e.Message}");
return StatusCode(500, "Internal server error");
}
}
—————
as excerpt from my code. Was I right when mapping Server with ServerDTO using this line:
var lstServerResult = _mapper.Map<IEnumerable>(servers);
It seems right. You have prepared everything, added the pagination header, and then mapped the result.
yep! thanks sir :D.
at first, i wonder about List doesn’t have some metadata about(next page, previous page,…) in header.
but I realize those metadata can be share between obj and objDTO.