Prevent download of static files referenced only from pages a user is not authorized to access

Question

Let's say a user is authenticated to a website and can access a given page only if authorized to access it specifically, e.g. if the website has only these 2 pages

https://my-classified-docs.com/page=a
https://my-classified-docs.com/page=b

my user will be able to access page "a" but not page "b". Pages are static html files that are served via a single GET "page" method, that will first check (via db, redis o smth else) if the user is authorized to get the requested page.

Let's also say that page-a references image-a.png and page-b references image-b.png.

I don't want my user to download image-b.png by e.g. getting

https://my-classified-docs.com/images/image-b.png

Of course, I can store image-b.png in a folder other than the folder image-a.png is stored in, but the user might "guess" its url anyway, as unlikely as it is.

How do I prevent that?

Let's suppose my file system structure is

wwwroot
    a
     img
     a.html
    b
     img
     b.html

and that I handle all requests to static files using the same method and evaluating the same authorizations I use for pages: would that work? Would that make sense?

I guess checking authorization in a database would be too slow, especially if I have several images in a page, so Redis might be the only solution I can think of.

Authorizations are managed dynamically, e.g. for each user I can edit the list of pages the user can access via a configuration page; each change then will have to invalidate the cache.

Is there a smarter way to manage all the requirements above? I guess technology might not be an issue here, but for implementation I would look at ASP.NET Core, Django or any other popular web framework.

Answer 1

You need authorization, of course.

Now, you may simply run this through your web server, and check database. While not ideal from performance perspective, it is simple, and often will be good enough. You might be surprised how much databases can handle.

The first optimization would be to use a fast in-memory cache instead of database. Something like Redis, to store permissions.

The proper optimization though, is to first stop serving static html, and start rendering it. With this approach, first of all you serve only those urls that the user has access to. But more importantly, you can do url signing. This is a technique used by all major cloud providers. How does this work? You take url, say U, you generate expiration date, say D. Something like 5 minutes into the future. Then you create U?e=D url. Next you need two cryptographically secure functions: create_signature(private_secret, text) and verify_signature(public_secret, text, signature). You apply create_signature(my_private_secret, U?e=d) and it will generate some blob of data S. You append that blob of data to url U?e=D&sign=S and serve that to the client.

Now the client calls your content server (which can be the same server) with that url. The content server then calls verify_signature(my_public_secret, U?e=D, S) which will tell us if the signature is valid or not. If it is valid, then the server checks D expiration time against current time. If it did not expire, it serves the content. Note that this is very efficient: the content server only has to do a bit of arithmetic to verify that you have access. It doesn't have to call external services, which in case of shared services (like database or cache) doesn't scale as well.

Note that we have public_secret and private_secret. This typically means some asymmetric cryptography. I used EdDSA in the past which is fast and creates relatively small signatures.

For simple purposes you can use the same secret for both and symmetric cryptography. In fact, the create_signature may simply be implemented as HMAC-SHA, and validate_signature by doing the same and then doing simple string comparison. This is slightly more risky than asymmetric cryptography, but simpler and faster.

With both approaches of course you have to generate secrets and store them somewhere. Note that the signing service has to know the private_secret, but the content serving service only has to know the public_secret. And in fact the public_secret can be publicly known. That's why asymmetric signing is more secure.

So this sounds a bit complicated, but in reality it is not that hard. Just be sure to use some crypto lib for that (do not write your own cryptography). In the case of .NET I used bouncy castle in the past. Newer .NET versions do have built-in cryptography functions, but I don't think they support EdDSA unfortunately. But they do support ECDSA, which is as secure, except it slightly slower.

Other option is to use JWT (also appended to url), which is pretty much the same I wrote above, except in a more standardized format. Plus there are standalone C# libs that generate and handle JWTs for you, like this one. The problem with JWT is that these tend to be a bit big, hundreds of bytes. You have to be careful to not exceed max url length limit (which is browser specific, as a common denominator you should assume 2000 bytes, see here). Of course it is not the end of the world. You may be forced to pass them as headers, which then simply means more JavaScript.

Answer 2

I handle all requests to static files using the same method and evaluating the same authorizations I use for pages: would that work? Would that make sense?

I'm going to disagree with @freakish's answer here and say: provided you're systematic about this, and you have an "authorization oracle" that tells you what pages a user is entitled to, this is completely fine. It's a very 1999-era approach to it, like the old .htaccess files.

Done correctly, it allows the client to cache the static image files. The cryptographic URL approach prevents this. You may also be able to implement this in your CDN, so you don't have to serve the requests at all. See e.g. Cloudflare workers - that describes basic auth, but more complicated solutions are possible with the same approach.

Doing authentication by folder rather than individual file is a good idea. I would also suggest that you define a series of roles, so you might map 1000 users to three or for roles and then assign which of those roles can read each folder.

Further detail depends on what web server you're using. Many (e.g. NGINX) don't support it directly but can proxy to somewhere that will decide.