A mobile back-end I am building should receive a larger amount of data (some sensor recordings) from an Android phone. The phone does not have much use of it so it was most efficient to store the data into plain files (SQLite can get quite slow with larger amounts). Now when we upload the data to a REST service backed with a distributed database, we have another issue. Storing to it takes some time, since it is really a lot. I thought of uploading the file to the web service as fast as possible, leave the phone alone, cache the file somewhere on the server and then some long-running workers will pick the data up and chunk it into databases. We have some other mechanisms to verify the data is properly stored and so on, but that is not important.
I would like to know is the REST->file->database approach valid? My concern is where to store the file? Disk, some in-memory database, a cache? Web servers fail (we can request a re-upload, but I would rather mitigate the risk earlier), or the server can get cluttered. I'm afraid the local storage on the web servers does not scale really well, if we have multiple workers and servers.
Thank you in advance.