Background: I'm writing micro controller C code to write an EBML file. EBML is like a binary XML with nested elements, but instead of start and end tags, there is a start ID, length, and then the data. I am writing this into external Flash in a low power application, so I'd like to keep the flash accesses to a minimum. Memory is also limited, because nothing is ever easy.
When I can keep the whole EBML element in memory, then generating it is easy because I can go back and fill in the length of each element after I know what that length is. The problem is what to do when I can't hold the whole element in memory. The options I see are:
- Write what I know, then go back and add in the lengths (easiest, but adds more flash access than I want)
- Calculate each element's length before I start writing it (relatively easy, but a lot of processor time)
- Switch modes once my memory fills up, so that I then continue through the data, but only to calculate the lengths for elements already reserved in memory. Then write what I have in memory, and go back and continue processing the data from where I left off. (My favorite option so far)
- Give elements a maximum or worst case length when they need to be written and their final length is not yet known. (Easier than above, but could backfire and waste space)
Question: It seems like this should be a relatively common issue that people have thought about. I know it can also happen when forming some data packets. Is there a better / more common / more accepted technique I'm missing here? Or just some terms for the issue that I can search for?