Upload files from mobile device to Amazon S3 architecture

Question

I have an iOS app that communicates to a REST API I developed. I am using Amazon S3 for storage of image and audio files. Right now I am able to upload/download the files directly from the mobile app (client). I am currently facing some issues with the speed of upload/download of small files (15-20 kB), as well as other problems such as Amazon S3 not allowing a time skew of more than 15 min (meaning if my customers have their time zone or date-time configurations incorrect, they would not be able to upload/download these files from S3.

I am now facing an architectural decision towards these issues, that is, uploading/downloading the files from S3 to my own API, store them temporarily and upload/download them to the mobile client. In other words change from this:

S3 <----> iOS <----> REST API

to this:

S3 <----> REST API <----> iOS

Advantages:

• All data is processed and handled by API
• No problems if client and storage (S3) are out of sync (time skew)
• Customer's phone communicates with only one web service

Disadvantages:

• All file transfers are a two way process
• API's web traffic increases substantially
• Increased chance of data/packet loss thus having to repeat the whole process

What would be the best choice? Leave it as is, or start developing file transfers through my own API?

Answer 1

The speed of downloading and uploading small files can easily be circumvented by putting several files into an archive (eventually with level 0 compression if files don't need to be compressed, which is the case for example for JPEG files). If the user uploads hundreds of files, this will result in two-three large files. If the user needs to upload only one or two files, then the speed won't matter much anyway.

The incorrect date and time configuration is the problem of the user (shouldn't synchronization be automatic on nearly every mobile device connected to internet?) Your application may verify if the time on the device is correct using NIST ITS. If, for some reason, this is impossible, use your own REST API to check if the time is correct. If the time is not correct, inform the user that the synchronization is impossible until the time is wrong, asking if he wants your app to correct the device's time automatically.

The choice between letting the app to interact directly with Amazon's S3 or using your REST API as a middle man depends mostly on those factors:

• Who owns Amazon's API keys. Are you ready to pay S3 usage for your users or do you prefer that they pay themselves directly? In the first case, the middle man API makes sense. In the second case, go with a direct connection to S3.

• Privacy, data sensitivity and legal issues. If the data is transitioning through your servers, this means that you can access it, and also means that you are uploading it to S3.

  The first point is problematic if the data is sensitive, and you may end up being sued if customers were not informed properly that the data ends on your servers instead of Amazon's (which may become even more interesting, legally speaking, if your servers are in a different country than Amazon's).

  The second point is problematic if users are uploading content which is against Amazon's policies. Imagine someone uploading child's pornography. For Amazon, you were uploading it, so you'll be the accountable person.

• Bandwidth. If your servers bandwidth is expensive (and why wouldn't it be?), you'll end up paying much more for the bandwidth than you'll pay to Amazon. More importantly, you pay twice the actual size of the files, since you are first receiving them, and then sending them.