I have a rails app where users can upload data and then the backend processes it and creates a bunch more data that gets stored with the record. The data processing is async and handled outside of the rails app. Currently I have been storing the unprocessed data in the same table as the processed data and then adding the rest when it is processed but I want to stop the rest of the app touching the data before it's finished processing and also make all the columns on the table not null.
The simplest solution I can think of is creating a new table for the data to be processed and moving it over when it's ready but because this data will only be sitting unprocessed for a maximum of a few minutes it doesn't really make sense to store it on disk and then delete it shortly after.
The data itself is just some strings, ints, timestamps and an XML file attachment.
What methods would be good here for storing this temporary data? Just as a guess I think Redis might be good for the job but I haven't used it enough to know properly.