Efficient API pagination with external authorization service?

Question

Let's say we want to return a paginated list of document id that a user can view. In DB, we have:

doc_id	user_id
ABC	user_1
def	user_1
...	...

We use an external authorization service, so we can query if the user has view permission on specific doc_id, by using doc_id and user_id

<doc_id>::<user_id>::view == true

example:

has_view_permission = check_permission("abc::user_1::view")

My question is, how to make the query efficient?

Without the permission check, I can take advantage of DB Keyset pagination. However for external authorization service, DB doesn't have data to do data join.

I have come up with a few ways, but I still thinking the solutions are complicated.

1. Store a local copy of permission data into DB. We need to sync the data regularly (or based on event), we then can do a table join - table join is expensive too.
2. Retrieve the data using keyset pagination, and send the result to AuthZ service. If some of the results don't have the view permission, we fetch the next data from db. This process repeat until we able to return enough data.
3. Every time there's a change in doc visibility, we pre-calculate the list and store in a different table, let's say, user_viewable_doc. This will use up a lot of storage.

What are the standard/efficient practice for dealing with this issue?

I have found this question, but I am not really satisfied

Answer 1

OpenFGA provides a document with guidance on this question: https://openfga.dev/docs/interacting/search-with-permissions

• Option 1: Search, Then Check
• Option 2: Build A Local Index From Changes Endpoint, Search, Then Check
• Option 3: Build A List Of IDs, Then Search

These roughly correspond to the possibilities you've already laid out. Which one is best for you will depend on the size of your data set, etc.