A web application I'm working on has a tool that queries a database and returns up to several thousand rows. When considering pagination, at what point does it make more sense to get the entire result set at once, and store it in memory, as opposed to making asynchronous calls each time a user changes pages on the table?
The app is written using Angular 1.X, so I'm trying to use browser memory as sparingly as possible. For the API to return a typical result set takes about 2 seconds. The API is written in node, and the database is PostgreSQL. Should I consider returning a readable stream from the API call?