129

I want to expose a resource on the web. I want to protect this resource: to make sure it is only accessible to certain individuals.

I could set up some kind of password-based authentication. For example, I could only allow access to the resource through a web server that checks incoming requests for correct credentials (perhaps against some backing database of users) before serving up the file.

Alternately I could just use a private URL. That is, I could simply host the resource at some impossible-to-guess path, for example https://example.com/23idcojj20ijf..., which restricts access to those who know the exact string.

From the perspective of an evildoer who wants to access this resource, are these approaches equivalent? If not, what makes them different? And as far as maintainability, are there pros and cons to either approach that I should be aware of before implementing one or the other?

  • 45
    This approach is generally only used without authentication for things like password resets. The unguessable link typically expires within a short period of time, and can only be used by someone already semi-authenticated (i.e. the website already knows the email address to which the link was sent). – Robert Harvey Jul 26 '16 at 15:34
  • 6
    Related discussion on security.SE: security.stackexchange.com/q/58215/37496 – Mark Jul 26 '16 at 20:39
  • 9
    @MonkeyZeus it is not security through obscurity. The secret, that normally is a password, in this case is a URL. – Davidmh Jul 27 '16 at 8:19
  • 16
    @MonkeyZeus: Security-through-obscurity refers to keeping the implementation of the mechanism secret, not using obscure keys. If unguessable urls are security through obscurity, then strong passwords are also. – JacquesB Jul 27 '16 at 8:30
  • 1
    @GladstoneKeep keep in mind the URL shorteners. Once someone malicious uses one of them the link will be much easier to guess / write down. – RookieTEC9 Jul 28 '16 at 13:46
202

A private URL is somewhat weaker than authentication with credentials, even if the bit size of the URL is the same as that of the credentials. The reason is the URL may more easily "leak". It is cached in the browser, logged on the server and so on. If you have outbound links, the private URL may show up in the referrer header on other sites. (It can also be seen by people looking over your shoulder.)

If it leaks (by accident or due to carelessness by the user), it may end up being public and even indexed by Google, which would allow an attacker to easily search for all leaked URLs to your site!

For this reason, private URLs are typically used only for one-shot operations like password resets, and typically they are only active for a limited time.


There is a related thread over at Information security: Are random URLs a safe way to protect profile photos? - one answer shares this story: Dropbox disables old shared links after tax returns end up on Google. So it is not just a theoretical risk.

  • 7
    Minor quibble, but "typically used only for one-shot operations" seems an over-statement given that Dropbox (and perhaps other cloudy services) are using them to "protect" access to files. – Steve Jessop Jul 27 '16 at 8:56
  • 4
    I would add that users are taught, with limited success, to protect their passwords, but not to protect their URLs. – svavil Jul 27 '16 at 10:33
  • 1
    You should add, that many of the technical concerns can be mitigated by using a parameter in the private URL so xzy.com/myDoc?auth=8502375 and an automated redirect to a plain url after the authentication checks out. - But this doesn't solve all possible problems – Falco Jul 27 '16 at 11:25
  • 3
    TL;DR - there's no such thing as a secret URL. There's a current attack that exposes URLs to a malicious actor at WiFi hotspots even if you normally send the URL over HTTPS. The attack abuses Web Proxy Autodiscovery (WPAD), forcing your browser to send all of your URLs to the attacker. See (e.g.) arstechnica.com/security/2016/07/… – Harald Jul 27 '16 at 18:19
  • 5
    @JacquesB Aren't some of the risks you identified mitigated by putting the private part in the fragment portion of the URL (i.e. after the "#", as e.g. Stack Exchange does for its Oauth authentication responses)? For example, the referer header is not allowed to include the fragment. – Jason C Jul 28 '16 at 2:38
48

Note:

A lot of people seem to be confusing a "private" URL with authentication. Also, there seems to be some confusion that sending the link via a trusted entity is an attempt at two-factor authentication. To be clear, we're talking about a publicly accessible resource, albeit one that is sufficiently hard to guess.

When using a private URL, you should always assume that it can be compromised -- you should design such a URL so that even if it is compromised, the resource will not leak information to the attacker.


Private/hard to guess URLs are not equivalent to password-based authentication. By nature, private URLs are not private at all -- they are publicly accessible resources. I think the term "private" URL is a misnomer, rather they're "obscure" URLs.

There are certain cases where using a "private" URL is acceptable, but they are inherently less secure than traditional authentication methods such as password authentication or key-based authentication.

Some of the places I've commonly seen "private" URLs used are:

  1. Password Reset emails
  2. Certificate Generation emails
  3. Account / email confirmation emails
  4. Delivery of purchased content (ebooks, etc)
  5. Other misc things like flight check-in, print boarding pass, use private URLs in addition to traditional authentication

The commonality here is that random URLs are typically only good for one-shot operations. Also, traditional authentication and random URLs are not mutually exclusive -- indeed, they can be used in conjunction with each other to provide additional security when delivering a resource.


As Robert Harvey has pointed out, the only way to securely use a random/private URL is to generate the page dynamically and submit the URL to the user in a way such that the user can be considered semi-authenticated. This could be email, SMS, etc.

A randomly generated/private URL typically has a few properties:

  1. It should expire after a reasonable amount of time
  2. It should be a single-use URL: IE it should expire after the first time it's accessed.
  3. It should defer the user's authentication to some other entity that it trusts to securely authenticate the user. (By sending the link via email or SMS, etc)
  4. It should be impossible for a modern computer to brute force the URL in the timeframe preceding expiration -- either by rate limiting the API that exposes the resource or by creating a url endpoint with sufficient entropy such that it cannot be guessed.
  5. It should not leak information about the user. IE: If the page is to reset a password: the page should not display the requestors account information. If Alice requests a password reset link and Bob somehow guesses the URL, Bob should have no way of knowing whose password he is resetting.
  6. If it does leak information about the user, it should be used on top of traditional authentication, for instance a page may consider a user authenticated if they have a cookie set or if their session_id is still valid.

Different resources require different levels of security. If you want to share a secret recipe with some friends, for instance, it would be acceptable to use a random/private URL to share it with them. However, if the resource could be used to steal somebody's identity or compromise their accounts with other service providers, you'd likely care much more about restricting access to that resource.

  • 4
    If I wanted to share the secret recipe for Coke with my product development team, that would require something somewhat different from if I wanted to share the recipe for the potato salad I served the neighbors during a neighborhood barbecue party. Again, context. :-) – a CVn Jul 26 '16 at 19:08
  • 7
    @MichaelKjörling I'm not sure how somebody would infer diff differently from my post. Quite clearly I stated that different resources require different levels of authentication. The recipe for coke would is much more valuable than the recipe for grandma's cookies. – Charles Addis Jul 26 '16 at 19:46
  • 9
    @CharlesAddis Clearly you've never tasted my grandma's cookies! – Brian Jul 26 '16 at 21:29
  • 1
    I think, although I might be wrong, that @Michael's saying your 5-point description of the properties a secret URL should have, is already overkill for sharing a secret recipe with friends. Making each one single-use (and therefore needing a separate URL per friend who accesses the recipe) in particular seems a lot of hassle for negligible benefit, especially if there is also an expiry time. I read your answer to mean, "it's acceptable to use a private URL, but private URLs should have these 5 properties", and IMO that's slightly wrong. – Steve Jessop Jul 27 '16 at 8:59
  • 1
    @BenjaminHodgson that is precisely the reason for item #5 => if the link/URL ends up in the wrong hands, it should not leak any information about the user who requested it. – Charles Addis Jul 27 '16 at 14:53
8

Pretty much all authentication schemes boil down to proving that you know a secret. You authenticate yourself to some service by proving that you know a secret password, or a secret URL or,...

Some more advanced protocols (e.g., OAUTH, Kerberos, ...) enable you to prove that you know the secret without actually transmitting the secret. This is important because there are more ways to obtain an "unguessable" secret besides guessing it.

I could be sitting in the same Internet cafe as yourself, eavesdropping on your WiFi connection when you type in your "unguessable" URL. At that point, if you weren't using SSL, or if I can exploit the latest new bug in your SSL implementation, then I would know your secret too.

  • 1
    To be fair, this also holds true for authentication, or any communication. – Andy Jul 26 '16 at 18:13
  • "eavesdropping on your WiFi connection" work on anything: URLs, CSRF protected <form>s, JavaScript military grade encrypted data (maybe active sniffing will be needed). It's simple to fix it: use HTTPS. – Gustavo Rodrigues Jul 26 '16 at 21:42
  • @GustavoRodrigues First of all, if eavesdropping really did "work on anything", then it would work on HTTPS. Otherwise, what does "anything" mean? Or, what magic do you think is in HTTPS that puts it above everything else? – Solomon Slow Jul 26 '16 at 22:11
  • 2
    ...but there is magic that wards off eavesdroppers: It's public-key cryptography. Here's a simple example: A service sends me a challenge containing a random number and a timestamp. I sign the challenge with my private key and return it. If they can validate my response with my registered public key, then that proves that I know the secret (the secret is my private key), but I proved it without ever revealing it to a potential eavesdropper. It won't help the eavesdropper to replay my response, because the service will never send the same challenge twice. – Solomon Slow Jul 26 '16 at 22:16
  • HTTPS (That is, HTTP over SSL) uses public-key crypto, but FYI, the most popular implementations of SSL have a history of bugs that have allowed the eavesdroppers to break in even in spite of the cryptography. New bugs (a.k.a, "exploits") seem to be discovered two or three times each year, and all of the developers of all of the products that use SSL have to run around with their hair on fire until the latest exploit is patched. (Don't ask me how I know!) – Solomon Slow Jul 26 '16 at 22:25
3

Lots of good answers already in this thread, but to directly address the question:

From the perspective of an evildoer who wants to access this resource, are these approaches equivalent? If not, what makes them different?

Let me establish a definition. "Authentication" is the providing of credentials to prove a claim of identity. Access control is usually based on the identification of the user.

Your secret URL is not bound to a specific user. As others have pointed out, it could end up in a proxy's log file, or a search request that gets indexed by google, or many other ways it could leak out.

On the other hand, a password is tied to a unique user account. You have the ability to reset it, or only allow it to be used form the user's home location, or known IP address, or any number of other controls.

A username/password gives you much more granular control of the access.

Access control allows an identity (subject) access to a resource (object). In your URL example the identity is "anyone who ever gets the URL, via any means."

Go with the username/password if you can. URLs leak out in all sorts of unexpected ways over time.

1

A secret URL is just as secure as a secret password. However, passwords are easier to keep secret than URLs, because everyone and their programs knows that passwords must remain secret.

For instance, your browser will not show a password on screen, only store passwords with your permission, and offer means to protect that password storage (such as encrypted storage unlocked by a master password). In contrast, it will always show URLs on screen, may possibly leak them through the referrer header, and store them automatically in your browsing history without any further protection.

Likewise, HTTP proxies will not usually log passwords, while URLs are commonly logged.

Using URLs for authentication also means that sharing URLs shares authentication, which makes it hard to individually revoke (or record) access.

And of course, secret URLs inherit all weaknesses of secret passwords. In particular, using a password for authentication will reveal that password to the server and anyone able to read your communication.

  • 3
    This answer makes a lot of assumptions which are wrong. If you log in to a site with HTTPS, and type in your username and password, the hops in between and the proxies won't know your password. – Pieter B Jul 27 '16 at 8:52
  • By "intercept your communication" I meant the ability to reconstruct its contents. This may or may not be prevented by HTTPS. In particular, trusting a single bad certificate (for instance by some corporate HTTPS proxy that uses the same private key for all installations) allows an attacker to man-in-the-middle the HTTPS traffic. – meriton - on strike Jul 27 '16 at 12:29
  • 2
    but by encoding the secret in the url you basically make HTTPS totally unusable. Every hop between the client and the server has the secret.No compromised certificates needed. – Pieter B Jul 27 '16 at 13:02
  • 4
    Nonsense; in HTTPS, the URL is only transmitted in the encrypted stream. You must be confusing it with the domain or IP, which are visible in DNS lookup and IP header fields, respectively. – meriton - on strike Jul 27 '16 at 20:28
1

Another item not noted anywhere is throttling of 'guesses'. For most password authentication systems, you get a limited number of attempts at guessing a password for a user before further authentication attempts are either locked out, or limited.

While you could do something similar with 'guesses' against your URL scheme, it would be somewhat harder to implement. If there is a recognizable pattern to your URL generation, then it may be hard to stop someone setting up to work their way through your 'random' URL space.

0

There's another aspect which I didn't see mentioned yet - URL shorteners.

In a recent publication (April 2016), it was claimed that URL shortener services completely nullify the increased security provided by random generated "unguessable" URLs. The URL space of the shorterner service is considerably smaller than your randomly generated URL - meaning that any such "secure" URL shared with a shortener service can be guessed in an easier fashion than anticipated.

To illustrate - let's assume your random URL is 128bit long (i.e a guid). Also, let's assume that your random number generator is really strong and that you generate those bits in a uniform way. Guessing a 128bit number is very hard and can take a considerable time - your URL is effectively 128bit key protected.

Then, let's assume someone shared this URL on the google URL shortener. Today that service emits a 10 character long ID, composed of letters and numbers. (26+10)^10 ~= 3.6*10^15 < 2^52 - so we've effectively halved the key strength from 128 bit to 52 bit.

Add to that fact that the generators do not use the entire space due to other consideration and you can mount an effective attack that combines brute force with side channels (most likely pre-allocated random URL buffers).

The original article: http://arxiv.org/pdf/1604.02734v1.pdf
A blog post summarizing the article (by the author): https://freedom-to-tinker.com/blog/vitaly/gone-in-six-characters-short-urls-considered-harmful-for-cloud-services/

  • 2
    Well, yeah, but one would hope anyone using such services for sensitive data would know better than to post the URLs anywhere, including to a shortening service. This isn't really any different from saying Gah! My password/private key is too long and complex. I know! I'll just write it in a text document and put that in a zip file with an easier password. Both are transparent fails, which one hopes against hope people wouldn't be silly enough to do. But yes, in reality, sadly your warning is probably needed ;) – underscore_d Jul 30 '16 at 15:59
  • @underscore_d yeah exactly - if you know this subject in enough detail to comment then this isn't a blog-worthy point. – Robert Grant Jul 30 '16 at 17:40

protected by gnat Jul 27 '16 at 15:39

Thank you for your interest in this question. Because it has attracted low-quality or spam answers that had to be removed, posting an answer now requires 10 reputation on this site (the association bonus does not count).

Would you like to answer one of these unanswered questions instead?

Not the answer you're looking for? Browse other questions tagged or ask your own question.