14
votes
Implementing paging with multiple data sources
Get the data into a single index
The simplest, no muss, no fuss solution.
But then again, why would anything Enterprise be simple?
Supply two result sets
The best you can do is to provide the ...
- 16.3k
14
votes
API pagination, how to implement page token paging method?
The page token approach that you are talking about here is more popularly known as the 'cursor-based-pagination' and depends upon marking the record where the last request left. It can be composed of ...
- 263
9
votes
API pagination, how to implement page token paging method?
Note: This is purely speculation, but I had to implement something similar before.
Consider a large time-series dataset stored in a big table like HBase. The data doesn't have a constant frequency so ...
- 9,217
5
votes
Accepted
Pagination with API
There has been quite a lot written on the subject of api pagination. I think it's fair to say that the naive approach 'get page 5, 20 items per page' doesn't work well with changeable data, filters, ...
- 79.8k
5
votes
Is it possible to cache data on a REST service that returns paginated data?
Isn't it possible to simply cache the thousands of records somewhere to avoid calling the database each time
Yes, but it is the exact opposite of being stateless, which REST is. You're trying to go ...
- 56.3k
5
votes
Implementing paging with multiple data sources
I don't think that you can come up with anything better here - basically, you're trying to solve an unsolvable math problem, which is joining two sets... without joining them.
Technologies like ...
- 239
4
votes
Accepted
How to avoid showing results of page 1 in page 2?
There is another kind of pagination that is harder to implements but will do what you want, it usually fits for infinite scrolling and it's more performant.
Basically you use the data from the last ...
- 3,536
4
votes
Improve performance for HTTP Request with huge body
I think it is pretty normal that unrealistic use cases lead to unrealistic issues.
Lets says you have 100k edited records, and different edits were done in some more or less complex or arbitrary ...
- 214k
3
votes
Accepted
Exceeded limit for API pagination
In general, explicit errors are good. But for the specific issue of a pagination limit, silent truncation of the result is probably better.
This is not really a fatal error – you will still be ...
- 135k
3
votes
Accepted
Is it RESTFul for an endpoint without a path parameter to always have a single resource as a result?
I think that a rest endpoint without path params should always have the potential of returning multiple resources
I'm new to REST so I don't know if my view on this is correct or not.
It isn't ...
- 33.7k
3
votes
Accepted
How to segregate blog posts into seen and unseen?
I would recommend not to tamper with ID ranges, this becomes overly complicated and makes unjustified assumptions about the ID value ranges or the order in which a users looks at the blog posts.
The ...
- 214k
3
votes
Accepted
API pagination with external or centralised authorization
For an efficient solution you need to be able to put the authorization and pagination constraints into the database query, and have the proper indexes for those aspects. Anything else will potentially ...
- 2,846
3
votes
Implementing paging with multiple data sources
It's often very difficult or impossible to combine multiple data sources into a single index. For example, if the underlying sources change often, you end up having to maintain essentially a ...
- 165
2
votes
How to avoid showing results of page 1 in page 2?
Fundamentally, your query needs to consider which items have already been seen. Since the position of items in your sort order is not stable (new items can be inserted at the beginning), you need to ...
- 1,742
2
votes
How should I balance memory with network latency on a paginated table?
The only way to know this for sure is to run some performance tests. There are too many factors to make an arbitrary judgment without testing.
Many web pages allow you to choose the level of ...
- 200k
2
votes
When to use pagination from database with ajax request
This is performance tuning. You have to try it and see. Try the same way your users will use it when it's deployed. Write code that makes it easy to adjust this setting by only expressing your ...
- 115k
2
votes
Pagination with API
Paging in multi-user environments where data can change between requests for pages is tricky at best, pointless at worst.
Say your first request got you a page of 10 items out of a total of 100. Now ...
- 10.1k
2
votes
Remember initial order of list while editing it
Step back a bit and look at situation from a cost/benefit perspective:
How much is gained by implementing the feature (or how much is actually lost by not implementing it) versus how much does it cost ...
2
votes
Is it RESTFul for an endpoint without a path parameter to always have a single resource as a result?
I don't consider your example to be a single resource. It returns an array of orders plus helpful paging information. This is typical of operations which return multiple entities.
While there is no ...
- 3,123
2
votes
How do I avoid tightly coupling one microservice to another microservice's feature that depends on specific views of the first's data?
As is often the case, efficiency comes at the cost of having to make things efficient.
To maximize efficiency, you'd develop these separate endpoints. This ensures that you get the best query ...
- 56.3k
2
votes
Accepted
DDD, Aggregates and paginate collections (specific case)
Few takes:
Pagination is not a domain related concern. It's a technical concern and it should be treated as it is, namely non taking it into account during the design phase
besides the point 1, you ...
- 687
2
votes
Accepted
Improve performance for HTTP Request with huge body
For the vast majority of developers, you'll never encounter a situation where the user needs to update 100K records at once from a web client, as noted in Doc Brown's answer.
If you're one of the ...
- 407
2
votes
Efficient API pagination with external authorization service?
OpenFGA provides a document with guidance on this question: https://openfga.dev/docs/interacting/search-with-permissions
Option 1: Search, Then Check
Option 2: Build A Local Index From Changes ...
- 201
1
vote
Pagination with API
What is best or common way to handle pagination on rest api?
RFC 5005 describes link relations for communicating the semantics of paging links to generic clients. It looks about as you expect - each ...
- 33.7k
1
vote
Accepted
Normalising pagination regardless of the target source
Ok, So; I've managed to figure this out. I'll post the solution here for future reference.
The Math:
recordNumber = pageNumber x itemPerPage
targetPageWithOffset = recordNumber / targetItemPerPage
...
- 349
1
vote
Normalising pagination regardless of the target source
You will most likely want to buffer results from the external source, which might be advisable anyway when your service will be used by multiple users and you want to avoid scraping the source every ...
1
vote
Accepted
How to handle or design review system with pagination?
Don't do premature optimization. Filtering reviews based on the ID of a user takes microseconds: I'm pretty sure the bottleneck in your app is at somewhere else.
Query the review of the current user, ...
- 136k
1
vote
Grouping search results on a map
An approach to this is to use Geohashes as a way of encoding locations.
This encoding technique subdivides a space into squares and assigns each of those squares an alphanumeric code.
Subdivide each ...
- 265
1
vote
How to do pagination while page gets populated dynamically
Let's review your options :
Option 1 is unpracticable if the result set is very large, the page might become unresponsive and the user prompted to stop the script. This is only true above a few ...
- 2,106
1
vote
Is it possible to cache data on a REST service that returns paginated data?
I'v never worked with ASP.Net, but first of all, why not have the data access layer pull only the necessary records? Most relational databases allo you to paginate with something like LIMIT and OFFSET,...
- 1,425
Only top scored, non community-wiki answers of a minimum length are eligible