0

How to upload large(mb/gb) of video files from client to app server?

The solution as of now I know is pretty simple and widely used is to break file in chunks at at client side and send http post request for each chunk.

Now if I take this to distribute environment, where a app server getting lot of parallel requests from different clients for different chunks or may be same user uploading different files from different endpoints(Browser/app etc.), So how does server know which chunk is for which file?Since server has not generated any filename or fileId on itself ,it has to rely on client to provide this, so does that mean client has to provide a file name and then server will merge all the chunks with same file name per user basis?Does that mean that a user cannot upload two videos with same name from his/her account?

The app server will here generate metadata for each chunk and put the chunk to s3

Map this use case to youtube upload.

Improve this question

1 Answer 1

Reset to default
1

On the first request, the server creates a random string for the file, and tells the client. On the subsequent requests, the client sends the same random string back so the server knows it's about the same file.

Exactly the same way S3 works.

Improve this answer
3
  • thanks this is what i was looking for. But regarding the S3 part, i wonder why they need upload id, the "key" itself can be used to uniquely identify object.
    – stkUser
    Feb 16, 2021 at 16:20
  • @stkUser Does it? I don't think they actually replace the object until the upload finishes, which means you can have old and new objects. Maybe you can upload the same object more than once at the same time, too. You wouldn't want to mix them up.
    – user253751
    Feb 16, 2021 at 16:27
  • yeah may be internal working of s3:)
    – stkUser
    Feb 16, 2021 at 20:49

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.