I've developed a data acquisition system for a scientific experiment. An FPGA buffers the scientific data in a FIFO and a C# program empty this FIFO at a rate of ~45MB/s.
Because the duration of the experiment suddenly changed from seconds to days, the data now needs to be stored as binary data.
Not being a programmer or computer scientist, it seems to me that there is a huge potential for data corruption. Previously every packet was written to a new line. If, for some reason, one bit went missing it would corrupt one packet.
However, for one missing bit in binary data we might loose TBs of irreplaceable data.
Is there a way to assure that the data does not turn to garbage? My first thought is to write a separate ASCII file of hamming code data, but I don't know if it's feasible.