So lets say I have a SQL query made by a server that fetches up to thousands of records at a time and I need to display those records in a grid(similar to AG Grid) on the client for the user to review. The user then has the ability to flip through the records (the grid is set up for both server side and client side pagination) and "check" or "uncheck" the items they do or don't want. Once finished, they submit what they have checked off for the server to process.

Right now, the way I'm handling this is to send all the records to the client for the user to review and then have the user send all the records back to the server for processing. In situations where there are thousands of records, this method seems less than ideal.

I would like to set up server side pagination but I'm trying to figure out how to keep track of the total set of records that need processing. For example, what happens if the query fetches 10,000 records and displays them in the grid in groups of 50-100 at a time? I would need to keep a reference to the entire set should the user decide they want to process all 10,000 rows. Additionally, what happens if a user decides to "uncheck" one of those items? How do I keep track of what they have unchecked.

I'm thinking the best way to handle this would be to make the initial query and store the records retrieved in a redis instance. Then, set up the grid on the client side to paginate on the data in redis. From there, perhaps I can set up some time of callback for when the user unchecks a row to update the data set in redis to have the record removed.

Am I on the right track or is there a better way to handle this? If I am on the right track, is there a specific strategy I should be using to handle these requirements in redis?

Thank you for your time.

  • Have you considered doing it client side? If one record is 100 bytes, 10000 records is 1MB which is totally doable in 2019 if you have ok bandwidth. – Esben Skov Pedersen May 10 '19 at 15:47
  • I'm currently doing it client side. I've noticed with larger sets, the application becomes quite sluggish. The particular grid that I'm using to display the rows was built in house using react.js and I'm thinking perhaps maybe the grid itself may not be optimized to handle such large amounts of data? – Aman Saran May 10 '19 at 16:04
  • Yeah. 10000 rows should not be a problem but maybe try and do the pagination client side before it hits the grid – Esben Skov Pedersen May 10 '19 at 16:09

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.