In my current Java EE project I've got a quite common task: Load objects from database using
Hibernate, transfer them in another object structure (based on a XSD) and serialize it using
JAXB. To simplify the scenario imagine the XML I want so serialize is a file with information records to requests we get. In theory there can be infinite information records to the requests, but I estimate a pessimistic but still realistic number of records I have to serialize.
I first made a rough estimate of the resulting XML file size and got to 292MB (about 9KB per single record). I already expected that the file size will outrange the maximium heap size of 50MB a java request is allowed to gather by our operators / server admins. So I need to find a limit where to split the records into several processes and files. I know that I can't just cut down the number of records to serialize in one file by the resulting file size, meaning I can't calculate "okay x records are 200 MB then x/4 records are 50MB and that's fits with the maximum heap". For sure I have to keep the loaded objects and the object structure I serialize in mind.
But how can I roughly estimate the needed heap size by the final size a single record so I can serialize a maximum number of records without running into memory issues? As said a rough calculation is enough, like "if your final size per record is 6KB then you need 12KB (ratio 1:2) to process it in your process.