In this article, we’re going to learn how to implement paging in ASP.NET Core Web API. Paging (pagination) is one of the most important concepts in building RESTful APIs.
As a matter of fact, we don’t want to return a collection of all resources when querying our API. That can cause performance issues and it’s in no way optimized for public or private APIs. It can cause massive slowdowns and even application crashes in severe cases.
The source code for this article can be found on the GitHub repo. If you want to follow along with the article, you can use the start branch and if you want to get the final solution or if you get stuck, switch to the end branch.
NOTE: Some degree of previous knowledge is needed to follow this article. It relies heavily on the ASP.NET Core Web API series on Code Maze, so if you are not sure how to set up the database or how the underlying architecture works, we strongly suggest you go through the series.
VIDEO: Paging in ASP.NET Core Web API Video.
We’ll discuss what paging is, the easiest way to implement it, and then improve on that solution some more to create a more readable and flexible codebase.
So let’s see what we’re going to talk about in this article exactly:
- What is Paging?
- Initial Implementation
- Paging Implementation
- Testing the Solution
- Improving the Solution
Let’s start.
What is Paging?
Paging refers to getting partial results from an API. Imagine having millions of results in the database and having your application try to return all of them at once.
Not only that would be an extremely ineffective way of returning the results, but it could also possibly have devastating effects on the application itself or the hardware it runs on. Moreover, every client has limited memory resources and it needs to restrict the number of shown results.
Thus, we need a way to return a set number of results to the client in order to avoid these consequences.
Let’s see how we can do that.
Initial Implementation
Before we do any changes to the source code, let’s inspect how it looks right now, and how you would probably begin with any project.
In our case, we have the OwnerController
which does all the necessary actions on the Owner
entity.
One particular action that stands out, and that we need to change is the GetOwners()
action:
[HttpGet] public IActionResult GetOwners() { var owners = _repository.Owner.GetAllOwners(); _logger.LogInfo($"Returned all owners from database."); return Ok(owners); }
Which calls
GetOwners()
from OwnerRepository
:public IEnumerable<Owner> GetOwners() { return FindAll() .OrderBy(ow => ow.Name); }
FindAll() method is just a method from a Base Repository class that return the whole set of owners.
public IQueryable<T> FindAll() { return this.RepositoryContext.Set<T>(); }
As you can see it’s a straightforward action, meant to return all the owners from the database ordered by name.
And it does just that.
But, in our case, that’s just a few account owners (five). What if there were thousands or even millions of people in the database (you wish, but still, imagine another kind of entity). End then add to that, a few thousand of API consumers.
We would end up with a very long query that returns A LOT of data.
The best-case scenario would be that you started out with a small number of owners that increased slowly over time so you can notice the slow decline in performance. Other scenarios are far less benign for your application and machines (imagine hosting it in the cloud and not having proper caching in place).
So, having that in mind, let’s modify this method to support paging.
Paging Implementation
Mind you, we don’t want to change the base repository logic or implement any business logic in the controller.
What we want to achieve is something like this: https://localhost:5001/api/owners?pageNumber=2&pageSize=2
. This should return the second set of two owners we have in our database.
We also want to constraint our API not to return all the owners even if someone calls https://localhost:5001/api/owners
.
Let’s start by changing the controller:
[HttpGet] public IActionResult GetOwners([FromQuery] OwnerParameters ownerParameters) { var owners = _repository.Owner.GetOwners(ownerParameters); _logger.LogInfo($"Returned {owners.Count()} owners from database."); return Ok(owners); }
A few things to take note here:
- We’re calling the
GetOwners
method from theOwnerRepository
, which doesn’t exist yet, but we’ll implement it soon - We’re using
[FromQuery]
to point out that we’ll be using query parameters to define which page and how many owners we are requesting OwnerParameters
class is the container for the actual parameters
We also need to actually create OwnerParameters
class since we are passing it as an argument to our controller. Let’s create it in the Models folder of the Entities project:
public class OwnerParameters { const int maxPageSize = 50; public int PageNumber { get; set; } = 1; private int _pageSize = 10; public int PageSize { get { return _pageSize; } set { _pageSize = (value > maxPageSize) ? maxPageSize : value; } } }
We are using constant
maxPageSize
to restrict our API to a maximum of 50 owners. We have two public properties – PageNumber and PageSize. If not set by the caller, PageNumber will be set to 1, and PageSize to 10.
Now, let’s implement the most important part, the repository logic.
We need to extend the GetOwners()
method in the IOwnerRepository
interface and in the OwnerRepository
class:
public interface IOwnerRepository : IRepositoryBase<Owner> { IEnumerable<Owner> GetOwners(OwnerParameters ownerParameters); Owner GetOwnerById(Guid ownerId); OwnerExtended GetOwnerWithDetails(Guid ownerId); void CreateOwner(Owner owner); void UpdateOwner(Owner dbOwner, Owner owner); void DeleteOwner(Owner owner); }
And the logic:
public IEnumerable<Owner> GetOwners(OwnerParameters ownerParameters) { return FindAll() .OrderBy(on => on.Name) .Skip((ownerParameters.PageNumber - 1) * ownerParameters.PageSize) .Take(ownerParameters.PageSize) .ToList(); }
Ok, the easiest way to explain this is by example.
Say we need to get the results for the third page of our website, counting 20 as the number of results we want. That would mean we want to skip the first ((3 – 1) * 20) = 40 results, and then take the next 20 and return them to the caller.
Does that make sense?
Testing the Solution
Now, in our database we only have only a few owners, so let’s try something like this:
https://localhost:5001/api/owners?pageNumber=2&pageSize=2
This should return the next subset of owners:
[ { "id": "66774006-2371-4d5b-8518-2177bcf3f73e", "name": "Nick Somion", "dateOfBirth": "1998-12-15T00:00:00", "address": "North sunny address 102" }, { "id": "a3c1880c-674c-4d18-8f91-5d3608a2c937", "name": "Sam Query", "dateOfBirth": "1990-04-22T00:00:00", "address": "91 Western Roads" } ]
If that’s what you got, you’re on the right track.
Now, what can we do to improve this solution?
Improving the Solution
Since we’re returning just a subset of results to the caller, we might as well have a PagedList
instead of List
.
PagedList
will inherit from the List
class and will add some more to it. We can also, move the skip/take logic to the PagedList since it makes more sense.
Let’s implement it.
Implementing PagedList Class
We don’t want our skip/take logic implemented inside our repository. Let’s create a class for it:
public class PagedList<T> : List<T> { public int CurrentPage { get; private set; } public int TotalPages { get; private set; } public int PageSize { get; private set; } public int TotalCount { get; private set; } public bool HasPrevious => CurrentPage > 1; public bool HasNext => CurrentPage < TotalPages; public PagedList(List<T> items, int count, int pageNumber, int pageSize) { TotalCount = count; PageSize = pageSize; CurrentPage = pageNumber; TotalPages = (int)Math.Ceiling(count / (double)pageSize); AddRange(items); } public static PagedList<T> ToPagedList(IQueryable<T> source, int pageNumber, int pageSize) { var count = source.Count(); var items = source.Skip((pageNumber - 1) * pageSize).Take(pageSize).ToList(); return new PagedList<T>(items, count, pageNumber, pageSize); } }
As you can see, we’ve transferred the skip/take logic to the static method inside of the
PagedList
class. We’ve added a few more properties, that will come in handy as metadata for our response.
HasPrevious
is true if CurrentPage
is larger than 1, and HasNext
is calculated if CurrentPage
is smaller than the number of total pages. TotalPages
is calculated too, by dividing the number of items by the page size and then rounding it to the larger number since a page needs to exist even if there is one item on it.
Now that we’ve cleared that out, let’s change our OwnerRepository
and OwnerController
accordingly.
First, we need to change our repo (don’t forget to change the interface too):
public PagedList<Owner> GetOwners(OwnerParameters ownerParameters) { return PagedList<Owner>.ToPagedList(FindAll().OrderBy(on => on.Name), ownerParameters.PageNumber, ownerParameters.PageSize); }
And then the controller:
[HttpGet] public IActionResult GetOwners([FromQuery] OwnerParameters ownerParameters) { var owners = _repository.Owner.GetOwners(ownerParameters); var metadata = new { owners.TotalCount, owners.PageSize, owners.CurrentPage, owners.TotalPages, owners.HasNext, owners.HasPrevious }; Response.Headers.Add("X-Pagination", JsonConvert.SerializeObject(metadata)); _logger.LogInfo($"Returned {owners.TotalCount} owners from database."); return Ok(owners); }
Now, if we send the same request as we did earlier
https://localhost:5001/api/owners?pageNumber=2&pageSize=2
, we get the same exact result:[ { "id": "f98e4d74-0f68-4aac-89fd-047f1aaca6b6", "name": "Martin Miller", "dateOfBirth": "1983-05-21T00:00:00", "address": "3 Edgar Buildings" }, { "id": "66774006-2371-4d5b-8518-2177bcf3f73e", "name": "Nick Somion", "dateOfBirth": "1998-12-15T00:00:00", "address": "North sunny address 102" } ]
But now we have some additional useful information in
X-Pagination
response header:
As you can see, all of our metadata is here. We can use this information when building any kind of frontend pagination functionality. You can play around with different requests to see how it works in other scenarios.
There is one more thing we can do to make our solution even more generic. We have the OwnerParameters
class, but what if we want to use it in our AccountController
? Parameters that we send to the Account controller might be different. Maybe not for paging, but we’ll send a bunch of different parameters later on and we need to separate the parameter classes.
Let’s see how to improve it.
Creating a Parent Parameters Class
First, let’s create an abstract class QueryStringParameters
. We’ll use this class to implement mutually used functionalities for every parameter class we will implement. And since we have OwnerController
and AccountController
, that means we need to create OwnerParameters
and AccountParameters
classes.
Let’s start by defining QueryStringParameters
class inside the Models folder of the Entities project:
public abstract class QueryStringParameters { const int maxPageSize = 50; public int PageNumber { get; set; } = 1; private int _pageSize = 10; public int PageSize { get { return _pageSize; } set { _pageSize = (value > maxPageSize) ? maxPageSize : value; } } }
We’ve also moved our paging logic inside the class since it will be valid for any entity we might want to return through the repository.
Now, we need to create AccountParameters
class, and then inherit the QueryStringParameters
class in both the OwnerParameters and the AccountParameters classes.
Remove the logic from OwnerParameters
and inherit QueryStringParameters
:
public class OwnerParameters : QueryStringParameters { }
And create
AccountParameters
class inside the Models folder too:public class AccountParameters : QueryStringParameters { }
Now, these classes look a bit empty now, but soon we’ll be populating them with other useful parameters and we’ll see what the real benefit is. For now, it’s important that we have a way to send a different set of parameters for
AccountController
and OwnerController
.
Now we can do something like this too, inside our AccountController
:
[HttpGet] public IActionResult GetAccountsForOwner(Guid ownerId, [FromQuery] AccountParameters parameters) { var accounts = _repository.Account.GetAccountsByOwner(ownerId, parameters); var metadata = new { accounts.TotalCount, accounts.PageSize, accounts.CurrentPage, accounts.TotalPages, accounts.HasNext, accounts.HasPrevious }; Response.Headers.Add("X-Pagination", JsonConvert.SerializeObject(metadata)); _logger.LogInfo($"Returned {accounts.TotalCount} owners from database."); return Ok(accounts); }
And due to the inheritance of the paging parameters through the
QueryStringParameters
class, we get the same behavior. Neat.
That’s about it, let’s summarize.
Conclusion
Paging is a useful and important concept in building any API out there. Without it, our application would probably slow down considerably or just plain drop dead.
The solution we’ve implemented is not perfect, far from it, but you got the point. We’ve isolated different parts of the paging mechanism and we can go even further and make it a bit more generic. But you can do it as an exercise and implement it in your own project. You can also find one frontend application of paging in our Angular Material paging article.
In this article we’ve covered:
- The easiest way to implement pagination in ASP.NET Core Web API
- Tested the solution in a real-world scenario
- Improved that solution by introducing
PagedList
entity and separated our parameters for different controllers
Hope you liked this article and that you’ve learned something new or useful from it. In the next article, we’re going to cover filtering.
hi, all your calls are synchronous. Would it be better to implement asynchronous in repository or just simply do
var owner = await Task.Run(() => { return _repository.Owner.GetOwners(ownerParameters);});
. Thank you!Hey vidriduch, yes they are. We intentionally siplified the series, but if you can find our take on asynchronous implementation here: http://34.65.74.140/async-generic-repository-pattern/
We also have a few different “improvement” articles related to Web API, so you can try the search function to find out if we’ve already covered it.
Thanks for the feedback!
Hi Code Maze,
I am beginner, can you please explain about RepositoryBase.cs, where this can be write if following repository pattern with modelview, and how can use it if I have other lists like List of Student, List of Owner, List of Teacher.
Hey Vipin Jha,
As we mentioned at the beginning of the article, some of the concepts are explained in our basic ASP.NET Core Web API series:
http://34.65.74.140/net-core-series/
Make sure to go through that series first to be able to follow this one, since you’ve just begun your journey.
For the concrete answer to your question on what RepositoryBase is, I would go directly to:
http://34.65.74.140/net-core-web-development-part4/
And if you manage to get it down and understand the concept, you can improve it even more by making it asynchronous:
http://34.65.74.140/async-generic-repository-pattern/
Hope this helps and if you have any more questions, feel free to leave us a comment and we’ll try to help you as soon as we can.
Thanks Vladimir Pecanac for help..
In your blog the scenario defined as Generic repository, wrapper classes. Where as in my case it is not generic classes for repository. Here is example given below:
public interface ISchoolsRepository
{
Task<ICollection> GetAllSchoolAsync();
}
and Repository Class:
public class SchoolsRepository : ISchoolsRepository
{
public readonly learning_gpsContext _GpsContext;
public SchoolsRepository(learning_gpsContext GpsContext)
{
_GpsContext = GpsContext;
}
public async Task<ICollection> GetAllSchoolAsync()
{
List results = null;
var result = PagdeList.Create(_GpsContext.School, 1, 10);
results = await _GpsContext.School.AsNoTracking().ToListAsync();
return results;
}
}
and my controller is look like this:
[HttpGet]
public async Task GetSchoolAll()
{
var schools = await _schoolsRepository.GetAllSchoolAsync();
List schoolsVms = new List();
foreach (Schools school in schools)
{
schoolsVms.Add(new SchoolsVm
{
Id = school.ID.ToString(),
Name = school.Name,
creatDate = school.CreatedAt.ToString()
});
}
return Ok(schoolsVms);
}
Any suggestion to do pagination in the given situation. Although your tutorial is very clear and well articulated but if I will follow your blog, I will have to do lots of changes in my projects.
Shouldn’t be the first parameter in ToPagedList method IQueryable instead of IEnumerable ? I checked it in my postgres database with pg_stat_statements and when it was IEnumerable I’ve got all of the records and only then it was sliced into smaller parts to retrieve it in the view. When I changed to IQueryable then it was fine, the query returned only the elements I requested and an additional COUNT(*) request was send. 🙂
Yes, you are exactly right. The query is executed in the ToPagedList method and it should have been IQueriable at that point of execution. Good catch, thanks! We’ll update the article and the source code ASAP.
Article “Searching in ASP.NET Core Web API” also needs to be updated, when list is passed as a parameter to ToPagedList method inside repository. Nice articles!
I like the service paging concept. Good way to prevent those folks that don’t do proper filtering on the front end web sites from returning the universe.
What matters in the end is application security and only application security. This means that “Proper” filtering on the front end is MEANINGLESS. We must always implement server side paging (and of course validate any inputs only the server side, only there it matters)
Hi Mr. Vladimir, I’ve got confuse about combine AutoMapper and Paging, in previous net-core-web-development-part5 GetOwners() method mapping from Owner->OwnerDto. Should I using paging for listOwner or listOwnerDto? Thanks.
You use paging functionality for the List but when you create such a list, you can map it to the List because you should always return a DTO object to the client. The model classes are here just for the database purpose.
https://media3.giphy.com/media/GCvktC0KFy9l6/giphy.gif
Thanks Marinko, I have a question:
public IActionResult GetAllServers([FromQuery] ServerParameters serverParams)
{
try
{
var servers = _repository.Server.GetAllServer(serverParams);
var metadata = new
{
servers.TotalRecords,
servers.PageSize,
servers.CurrentPage,
servers.TotalPages,
servers.HasNext,
servers.HasPrevious
};
Response.Headers.Add("X-Pagination", JsonConvert.SerializeObject(metadata));
_logger.LogInfo($"return {servers.PageSize} server from database.");
var lstServerResult = _mapper.Map<IEnumerable>(servers);
foreach (var serverDTO in lstServerResult)
{
var lastestComment = _repository.Comment.GetLastestCommentByServerID(serverDTO.ServerID);
if (lastestComment != null)
{
serverDTO.LastComment = lastestComment.Description;
}
}
return Ok(lstServerResult);
}
catch (Exception e)
{
_logger.LogError($"Error when get all servers: {e.Message}");
return StatusCode(500, "Internal server error");
}
}
—————
as excerpt from my code. Was I right when mapping Server with ServerDTO using this line:
var lstServerResult = _mapper.Map<IEnumerable>(servers);
It seems right. You have prepared everything, added the pagination header, and then mapped the result.
yep! thanks sir :D.
at first, i wonder about List doesn’t have some metadata about(next page, previous page,…) in header.
but I realize those metadata can be share between obj and objDTO.
Hi, for me it’s not clear why you set the pagination properties in the Header why you use this approach instead of just return in Json as data? I is better use this approach? thanks for your help
Hi Joao,
This approach is preferred because each endpoint should return only the relevant data for the resource that’s requested. By putting the metadata inside the value of the response we would break the self-descriptiveness of the API (check the HATEOAS part to learn more). Metadata is not actually part of either Owner nor Account resources, so returning it as a result would be wrong. Having said all this, if your solution can benefit from having metadata in the response, you should implement it that way. Many big APIs do it like that. But you should now that it’s not RESTful, and you should be aware of the consequences if doing that.
Hope this helps.
So I’m not new to programming, but I am new to C# and ASP.NET and I’ve got to say: THANK YOU! These articles on the various aspects of building/designing/implementing a .NET CORE Web API are just awesome. Thank you so much!
Thank you so much fro the kind words Guy!
Nice informative article!
Paging is not a “nice to have”. It is mandatory.
Not implementing server side paging correctly is nothing less than a security breach (Unbound query).
You should always implement it from the beginning, unless the data by definition has a maximum low row count (e.g. days of a month, etc..)
We completly agree with you on that. Thank you for reading and commenting as well.
Hi, this is great. Could you please explain asynchronous implementation for paging. Thank you very much !
Hi. Well it is almost the same, just you need propertly to apply async await keywords. Please read this article: https://code-maze.com/async-generic-repository-pattern/ Everything is explained here and after it, you will have no problem in implementing async paging functionality for sure.
Can you tell bit more on the above statement? 😉 the solution looks perfect so I wonder what you can see can be improved.
( except going async ).
thank you
Hi Lukasz. Well, we updated this article couple of times, so, we should probably remove that sentence 🙂 And yes, going async would be even better, but at this point I am pretty sure that converting this soution to the async one should be a peace of cake.
Hi Marinko,
Thanks for your prompt reply. When working on this found a tiny spot to improve 😉
There is an error when pageSize or pageNumber is < 0. ( at least when using postgres db saying that offset / limit can not be negative. )
hi, have you met with an opinion that pagination metadata should also return link to the next and the previos page ?
Hi. Well from our expirience, these information are quite enough. You can see here how the frontend can utilize it: https://code-maze.com/blazor-webassembly-pagination/
well, that’s true about the information is enough, however it is the matter of server to create link to the resource, you just send link to UI
I am looking at that this way. If you have pagination span of 10 pages. So, you have previous, next and 1 to 10 buttons. And you are currently on a fifth page. With a links to previous and next pages, you have covered prev, next and 4th and 6th buttons, but what about 7th, 8th? You don’t have links for those, it is a front-end’s task to create it. Also, every time you change something on the page, like Sort option or Search, or number of items per page a new request is sent with different query parameters created by front end. That’s the main reason why I let FE deals with it. API resources could always be exposed through HATEOAS.
of cource, next, prev – it was shortcut for pagination as you mention, all page numbers need to be returned with links ….and for sorting, filtering, searching you have got the new pagination structure generated by server