I am building a REST API which would power a front end as well as other 3rd party apps and hence I want it to be as "standard" as possible. Right now, I am trying to stick to HATEOAS. The only place I am struggling is pagination.
The authorization layer of our application is centralised. Multiple other apps use the centralised auth service and so my API needs to use the same. This gives rise to obvious problems in pagination, namely:
- How to restrict the number of entries to a required number?
- How to return entries of any valid page?
- How to return the total number of pages?
Right now, I am using an ad-hoc solution that fetches all records from the database, according to the API filters, and then the authorization layer, filters the unauthorized records and then another layer (lets call it the "Pagination Layer") filters according to the page parameters.
This works for now as our dataset is relatively less but I don't think this will scale well. What are my options?
P.S. There are a few things I have thought about but have no idea how good of an idea they are:
- The frontend can be switched into a lazy loading mechanism so that returning exactly a certain number of entries is no longer mandatory. The frontend is take care of querying the next page if required. But, this will give a bad experience to 3rd party devs using the API.
- The business layer gets only the number of records as in the page and the pagination layer decides if more queries are required to get more data. This looks like a bad idea in many ways as this won't solve getting a particular page.