I have a monolith CRUD application with a feature - possibility to export some DB entires to an Excel file. The export is done within a background task. Often, however, when user presses the button and expects to get a link with completed download, they may trigger a crash for the server where the webapp is hosted (not the DB server). Exporting reserves a lot of memory, the more rows one exports - the higher the chance to get Out of Memory.
How should I design my system around that?
I presume, it has something to do with chunking (load just a part of data from DB, append to the excel-file, then load and append another, etc), but don't quite understand - if I use chunking, shouldn't the chunked objects still persist in memory for some time (until, obviously, being garbage-collected) making the monolithic application still prone to Out of Memory crashing?
What are the best practices and common architectural solutions in regards to such stuff?