Is obscuring/obfuscating public-facing database ids really a "best practice"?

Question

I've heard people lecture here and there on the internet that it's best practice to obscure public facing database ids in web applications. I suppose they mainly mean in forms and in urls, but I've never read anything more than a mouthful on the subject.

EDIT: Of course, now that I ask this, I find some resources on the subject:

• https://stackoverflow.com/questions/2374538/obscuring-database-ids
• https://stackoverflow.com/questions/1895685/should-i-obscure-primary-key-values
• http://joshua.schachter.org/2007/01/autoincrement.html

These links satisfied some of my curiosity, but the SO posts don't have many votes and aren't necessarily centered around the topic in this context so I'm not sure what to make of it, and some of them claim that the third link is bogus. I'll leave the rest of my post intact:

---

I understand the differences between obscurity and security, as well as how the two can work together, but I can't imagine why this would be necessary.

Is there any truth to this, is it just paranoia, or is it just totally bogus altogether?

I can think of ways to do it, but of course it adds a lot of complexity to the application code. Under what circumstances would this be useful? If this is something people frequently do, how is it usually deployed? Hashing the identifiers? Something else? It seems like a lot of work for not much extra security. I'm not looking for real solutions, I just want to get an idea of how/why people would do this in the real world.

Is this really considered a "best practice" or is it merely a micro-optimization of little value?

NOTE: I think a few folks might have gotten the wrong idea: I'm not suggesting that difficult-to-guess ids would be the only security mechanism, obviously there would have be the usual access checks. Let's assume those are in place, and that simply knowing the id or hashed id of a record is not enough to grant access.

Answer 1

I don't think it's that important, but there are some scenarios when it could matter depending on other decisions you make.

For example, if you were to expose an order ID that's generated sequentially, and you had a social engineering attack with someone calling customer support and saying "hey, I just spent $2000 on a new computer and you guys sent me some other guy's order for a $15 cable, now I'm out $2000" you could spend a lot of time trying to vet the issue before you either conclude it's bogus or you send the faker a new computer.

There are similar, less sophisticated, but embarrassing variations on the theme; if a bad guy increments an order ID an emailed link to a receipt, and if no additional validations are made to verify that the person who clicked on the link has the right to view the order ID, suddenly you're unwittingly exposing private customer information to the wrong person.

In such cases, if the numbers are non-sequential the exposure is slightly mitigated because guessing is less likely to yield interesting results. On the other hand, now you need a convenient way to reference an order ID in customer support interactions that won't result in long back-and-forth conversations with telephone-based customer interactions while your rep tries to distinguish between B, P D and T in order number BPT2015D.

I'd say it's a stretch to call this obfuscation a "best practice", but in certain scenarios, it can reduce the ease of exploiting another weakness in your validation or authorization code. On the other hand, it doesn't really matter whether someone knows you wrote blog post #1 or #2559. If the ID isn't valuable information, even with additional knowledge, then the argument that obfuscating it is a best practice holds less weight.

There's a second potential argument, which is that your database identifier may wed you to a particular database implementation (or instance), and when your company gets bought out or picks up a competitor and now you have to merge two legacy systems, or the CEO goes out drinking with the rep from DynoCoreBase and they decide that you will now move all your data to DynoCoreBase version 13h and it wants all the primary keys to be guids, and you have to create some sort of mapping layer to translate old IDs to new IDs so that old URLs don't break, but whether these scenarios matter to you depend far more on the nature of your business (and the customer involvement with those IDs) than on any general best practice.

Answer 2

This is my take on it:

While "security through obscurity" obviously isn't enough, obscurity can aid security, even if just a little. You have to decide if that little bit of psuedo-security is worth the extra effort it takes to deploy something like this in your application.

There is another reason outside of security I can think of to implement this:

Privacy

Let's say we're dealing with user ids in the url. If Joe's user id is 100 and Bob's user id is 101, it's probably obvious that Joe's account was created first. While this may not matter in most applications, it may matter to some. This is an example of privacy stricly through obscurity, so unless you have a very sophisticated system of obfuscating the user ids, it's might be easy enough to dissolve and figure out if the user with id 3Js9kW3hTs7sa120 has had an account longer than the user with id Q8Hs73kks0hEg.

From the link I referenced:

The first is that given the URL for some object, you can figure out the URLs for objects that were created around it. This exposes the number of objects in your database to possible competitors or other people you might not want having this information (as famously demonstrated by the Allies guessing German tank production levels by looking at the serial numbers.)

Using an auto-increment id publicly exposes the number of objects in the database, and can expose which ones were created first and which ones are newer. This information can give away the fact that a business is new, or not doing well even. For example: let's say you order a book and your order id is 1. It might be apparent that your order was the first one in the system, which can be unnerving. Let's say you go back and order another and your order id is 9. This gives away the information that only 7 orders were placed in the time frame between your two orders. This might be valuable information to competitors. In this case, the numerical auto-increment id is probably better off obfuscated.

Answer 3

The word "Obscure" is probably slightly misleading here, and, may lead people to think "obfuscate" or "partially hide".

The recommendation is that you never ever include any internally generated database keys as part of a public URL if these database records contain any sensitive data.

Its just too easy to play with the numbers in the URL and access other records.

Answer 4

I will go against the mainstream, and tell you that using a random, long identifier is in my opinion a pretty decent form of security.

It must be made clear to whoever has access to that data that it is as much sensitive as a password. And of course it has the disadvantage that if it goes into the wild it may be harder to change.

But it has a couple of advantages over the usual username and password pair. First, the user has no choice over it, so you can be sure it is essentially impossible to guess. There is no point in designing a perfectly secure site when the administrator choses its first name as password. Second, each item has a different identifier, so if one gets access to one item, this does not help with the other.

Of course, the more layers of security there are, the better. But this method alone can be more secure than a couple of credentials.

Answer 5

There are two aspects to this: usability and security.

Security-wise, obscuring IDs is rather pointless; what's important is that the ID must never be the only 'key' to any non-public resource. This means that if you want to give selected users access to a certain page, just requiring the page's ID in the URL is not enough, you have to implement an actual security mechanism such as a username/password login or public/private key authentication, as well as proper authorization (that is, the system must be able to judge on a per-case basis whether user X is allowed to access resource Y, and act accordingly).

Usability-wise, the general advice is to keep artificial keys hidden: they don't have any meaning to the user, and by revealing them in obvious ways, you introduce an extra dependency, one that is particularly hard to handle because it lives outside the realm of software - people will write down, e-mail, fax, print, bookmark, etc., those ID's, and if you ever change them, you're in for some annoying support tickets.

Answer 6

No, you absolutely, positively don't want to allow access to arbitrary resources just by knowing their internal id. This is the internet, whatever malicious, criminal, or plain childish presence you might imagine to exist, actually exists, and sooner or later they will come to your site. Unless everything you could ever serve is completely public for everyone (and if you have even one customer, that is guaranteed not to be the case), you must restrict access to those who are actually authorized to have it.

The usual solution is to use authorization tokens of some kind that are stored in an encrypted session, and checks that verify whether something is supposed to be visible to the authenticated user before actually sending it out. This can be an actual security manager, such as the one that ships with the JDK, or any other component that fulfills the same role, as long as it does so consistently. (Whether to expose internal IDs to someone who is already authenticated or not is also an interesting question with various pros and cons, but it is less security-essential.)

The value of doing this is hard to calculate until you are actually attacked by someone who means business. Then it's usually the difference between going out of business and just shrugging off yet another foiled script kiddie.

Answer 7

Obviously it depends on how sensitive the data is but for medium security the very minimum I would do is to include the ID and a checksum value.

Generate the checksum by adding salt (i.e. a set collection of random characters) before and after the ID and doing an MD5 on the value.

On every page that reads the ID value it should generate the checksum value and compares it with the one passed, deny the request if it does not match.

If a potential hacker is able to get enough valid combinations then they may be able to work out the Salt value by brute force, so if you can add another layer of security such as checking the User ID that will also help.