A web application I'm working on has a tool that queries a database and returns up to several thousand rows. When considering pagination, at what point does it make more sense to get the entire result set at once, and store it in memory, as opposed to making asynchronous calls each time a user changes pages on the table?

The app is written using Angular 1.X, so I'm trying to use browser memory as sparingly as possible. For the API to return a typical result set takes about 2 seconds. The API is written in node, and the database is PostgreSQL. Should I consider returning a readable stream from the API call?

  • This depends a lot on how your database performs when getting partial results vs. all results and how often the end-user use case actually visits all the pages vs. just visits the first page and how big the result set is. If the overhead of N separate requests to the DB is high and the user often visits multiple pages in the same session and the whole result set it not too huge, then you probably want to get all the results and cache the other pages so they can be served from cache when requested. But, as you can see, if those factors go in the opposite direction, then use separate requests.
    – jfriend00
    Commented Oct 1, 2016 at 0:24

1 Answer 1


The only way to know this for sure is to run some performance tests. There are too many factors to make an arbitrary judgment without testing.

Many web pages allow you to choose the level of pagination (i.e. the number of records to return) in the UI; do that, and you can easily find out what the sweet spot is with your usual tools.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.