I'm storing some temperature, voltage and current data every 10 seconds to a log file. This file is supposed to be retrieved later so we can read the data and display it. Time requirement is 6 month worth of logs and the space requirement is under 6 MiB.
The protocol I came up with looks something like a fixed record of 6 bytes:
- PID 2 bytes
- Data 4 bytes
if PID is a timestamp data is either date (4 bytes) or time (4 bytes). So I put one timestamp and whatever measurement that follows at that time.
I tried storing a timestamp followed by the three readings linearly with fixed records but quickly ran out of space. Then I tried only tracking changes of values, storing the last value and the current value if there was a change of data.
Since the timeline is tree structured (YYMMDDhhmmss) I was thinking of sorting the data in a tree structure sorted on a timeline instead, making the record size more packed, but how would you go about storing this data for easy request based retrieval later if it is stored in a tree?
/edit related questions here: How to store data that is recorded with different frequency? answer suggestions are similar to what I have come up with