-3

I'm wanting to store lots of data in cloud storage - like S3.

I want some data to only be accessed by authorised users, one way I could do this is to authenticate the user on my service, download the data from the cloud to my server, and serve the data to the user. This seems innefficient - as my server has to both upload/download the content.

I'd rather the user directly download the data from the cloud - is there a standard technique for handling this?

I was hoping maybe there was a way to hook in my Oauth2 provider to my cloud storage so they can easily authenticate downloads - cant find anything like this. Maybe I'll use short-lived pre-signed URLs, that I'd have to generate for every download

Improve this question
2
  • I did this ages ago and back then the only way was to do as you described. An api with access wrapping the s3 store
    – Ewan
    May 8, 2021 at 20:58
  • @Ewan I'm thinking about rotating cached pre-signed download urls. Updated every hour or so, to minimise sharing/piracy
    – Tobi Akinyemi
    May 8, 2021 at 21:16

1 Answer 1

Reset to default
2

If you're happy with a small window where anybody who gets a specific "secret" URL can download the content, you can use presigned S3 URLs. If you're not happy with that, you probably want to think about whether you should be sharing the content to your users at all.

This seems innefficient - as my server has to both upload/download the content.

Oh no! Is this really an issue for you though - are your servers bandwidth limited? If not, why are you worrying about this?

Improve this answer
6
  • Because I will have to pay egress fees to my cloud storage provider, as well as ingress and egress fees to my server provider. Bandwidths never free - both in terms of money and time
    – Tobi Akinyemi
    May 8, 2021 at 19:03
  • I was hoping maybe there was a way to hook in my Oauth2 provider to my cloud storage so they can easily authenticate downloads - cant find anything like this. Maybe I'll use short-lived pre-signed URLs, that I'd have to generate for every download
    – Tobi Akinyemi
    May 8, 2021 at 19:11
  • That's an issue only if you're not running your servers in the same cloud as your block storage. The synergies of doing that usually outweigh everything else.
    – Philip Kendall
    May 8, 2021 at 19:12
  • you probably want to think about whether you should be sharing the content to your users at all what do you mean by this? Some security issue?
    – Tobi Akinyemi
    May 8, 2021 at 20:02
  • You've allowed the "sensitive" data onto a machine controlled by someone other than you; therefore you no longer control the data.
    – Philip Kendall
    May 9, 2021 at 8:53

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.