7

I am well aware of best practices for storing user passwords:

  • Never store passwords in plaintext
  • Never store encrypted passwords; always store password hashes
  • Always salt your password hash to deter brute force attacks
  • Always use unique salts to deter attacks using rainbow tables.

However, I have recently came upon on a potentially legitimate use case for keeping plaintext (or plaintext-accessible encrypted) user passwords: fraud detection. For example, if we have a system that needs to restrict purchases to 1 (or a small number) per person, we have observed that fraudsters often use the same password over and over, so if we have detected fraudulent purchases on one account, we can use the fact that password matches as a useful signal in a fraud detection system.

But implementing such a system requires breaking all the best practices for password storage.

Question: If it is determined that the fraud-detection ability of being to retrieve passwords is necessary, what are the best practices for storing passwords securely but in a way that is plaintext-accessible?

25
  • 9
    When first entering the password, how intensive is it for you to check that password against all the hashed and salted passwords you have stored already? Jul 31, 2012 at 7:45
  • 6
    Just wanted to point out that if you do this, you don't have to store plain text passwords. You can do this by simply throwing away the last requirement you list and hashing all passwords with the same salt. Then you can simply compare hashes.
    – James
    Jul 31, 2012 at 7:48
  • 6
    Of course, allowing DB operators to know that two users have the same password is itself considered a security flaw.
    – Brian
    Jul 31, 2012 at 13:22
  • 6
    This is a terrible way to detect 'user collision'. Especially given the number of 'uneducated' users who will all use 'password'. If I found a website that required 'unique' passwords, I'd probably use that to find out who used an existing password, and log in as that person. Note that even physical locations have problems with this - on the internet, without some independent way of identifying someone, you're just toast. Jul 31, 2012 at 17:05
  • 4
    I know some ticketing services limit it by credit card number. Seems like getting multiple credit cards is a bigger problem than multiple passwords.
    – JeffO
    Jul 31, 2012 at 19:13

8 Answers 8

23

In this case you could, upon registration, generate a hash of the entered password for every salt used so far, then check these salted and hashed passwords against all the passwords already stored.

You could even cache all the salts used so far, this is potentially a smaller list than having to loop through all passwords every time to collect these.

If that is too expensive to compute on each registration, limit the number of salts you generate (pick randomly from a limited list) to reduce calculation time. That'll still be safer (by far) than storing plain text passwords.

Note that if this is just for the sake of fraud detection, a security breach and resulting (huge) negative publicity is going to cost you much more, wiping out any advantages you gained from catching a few fraudsters this way.

12
  • 5
    Just for perspective, a security breach is only a potential loss of revenue, whereas fraud is actual loss of revenue. If your business is losing, say, hundreds of thousands of dollars a month due to fraud, management may be willing to take risks to stem that loss.
    – nohat
    Jul 31, 2012 at 18:47
  • I don't think people give a damn. Otherwise password storage criteria would be outlined in EULA. But yes, comparing hashes is the same thing as comparing real source data, it can even be faster. Minus few theoretically possible collisions.
    – Coder
    Aug 1, 2012 at 19:38
  • 3
    @nohat The simplification of this issue down to concerns of revenue stream seems really crass and unprofessional to me. When you store your users' passwords in plaintext, you are not just putting your company's reputation (and revenue stream) at risk, you are putting your users' security at risk, too. If you care so little about your users that you are willing to risk their security on behalf of your own revenue stream, you should be ashamed of yourself.
    – Eric King
    Aug 1, 2012 at 22:16
  • 3
    @nohat Well I do happen to believe that morals come into play when talking about professionalism. And I also happen to believe that your professional responsibility extends not just to management ("I was just following orders") but also to the customers.
    – Eric King
    Aug 2, 2012 at 16:52
  • 2
    No business needs to store plaintext passwords. There are always other options. Sorry to be a Sith about it, but if you are presenting to management, why would you even present an option to them that is unnecessarily insecure? How do you expect to catch anyone anyway? Everyone uses the same 150 passwords anyway, so your possibility of false collision is very high; just go look through a passwords list.
    – Jordan
    Aug 2, 2012 at 19:18
13

Making the passwords accesible in plaintext in one way or another is already a HUGE security flaw. This is one of the most important security components on any site anywhere. I believe you already know that.

My recommendation is to find another way to deal with those frauds. Exposing user passwords like that is just not worth it. You win a little and lose a lot. If you do it, you'll regret it later.

EDIT: I guess it depends a lot on what people can purchase on that site. You can check against the same passwords, the same IP, the same browser etc. but if someone wants to make 2 or more purchases, they can. If the goods we're talking about are actual objects, then in my opinion the best way to detect such "frauds" are by comparing shipping addresses. Someone may use a different account/password/browser (that is easy), but they will rarely change their address to accomplish something like this.

7
  • 2
    +1 for just saying how it is "there are better ways for detecting fraud."
    – Ramhound
    Jul 31, 2012 at 16:14
  • Well, if you never have a security breach, and you prevent millions of dollars in fraud, I'd say you win a LOT and lose nothing.
    – nohat
    Jul 31, 2012 at 19:06
  • 6
    "never have a security breach" is one damn big bet.
    – DaveE
    Aug 1, 2012 at 20:15
  • 1
    @nohat If those accounts would be in use for a few months maybe, then the risk is probably lower. But maybe after 10 years that users table will still be around. Yes, probably nothing will happen. That would be great. But security is not a fairytale world... It's the same with your house: you have insurance. Odds are nothing will happen to it in those 20 years you live in it. But you still get insurance. Why ? Because if something does happen, that would be a catastrophy for you and you just don't want to suffer the consequences. Aug 2, 2012 at 19:18
  • 2
    @nohat: These consequences: mashable.com/2012/06/21/linkedin-lawsuit Aug 2, 2012 at 22:21
4

Is the risk of the bad PR if your passwords leak worth it for the slight fraud-fighting gain you get? Personally, I wouldn't.

If you really have to "have a system that needs to restrict purchases to 1 (or a small number) per person" (which sounds an impossible task) is there not something else you can use?

  • Same IP address?
  • Same browser?
9
  • Sure, those are also useful signals for fraud detection, but if you are dealing with a lot of fraud you need to use all available to signals to detect it and maximize the loss.
    – nohat
    Jul 31, 2012 at 19:08
  • 2
    Like I say, it's a trade-off. The cost of fraud may be high, but that's nothing compared to the cost to your business if you actually leak passwords. It's up to you. I wouldn't.
    – James
    Aug 1, 2012 at 11:45
  • Even in the case of worst-case scenario password security breaches (LinkedIn, eHarmony, Last.fm) it's not clear that there have been substantial revenue impacts.
    – nohat
    Aug 2, 2012 at 15:16
  • Would love to see some fiqures about that. Tho I suspect the situation would be very different for a small retailer than it has been for those big players, so you have to make sure your comparing like with like.
    – James
    Aug 3, 2012 at 7:10
  • @James - Don't bother...Its clear the user doesn't actually want to hear us.
    – Ramhound
    Aug 3, 2012 at 12:40
2

assuming the "same password" is a good way to check for fraud, and considering that giving a way to revert the stored password into the plaintext is just a very bad idea, here's how I would allow for duplicate detection : I would still go with the "per user unique salt" scheme in order to identify my users.

In addition to that, I would also have a unique salt that's used before performing another hash on the password. All the passwords would go through hashing and be stored in a DB (not in the same table as the user table), along with the number of times it's been used. This would still allow a rainbow attack on this specific table, but you wouldn't have the user<->password link.

In other word, your attacker would be reduced to use a bruteforce dictionary attack (which he would perform anyway, and which would probably give him access to a large number of your accounts). If your user base is small, nothing prevents you from putting dummy rows in that table, just to increase the size of the dictionary your attacker would have to use.

The key here is that you don't care about which user has the newly typed password. The only thing that you'll have access to is "how many users have this password", which will allow you to reject the last subscriptions (no matter what, you can't reject subscriptions before a certain number of users have used the same password).

1
  • 1
    This seems like a stop gap measure to prevent a problem that can be prevented. You can detect fraud in other ways, detecting the same password, won't prevent fraud.
    – Ramhound
    Aug 3, 2012 at 12:41
1

It's probably not a good idea to store plaintext passwords, but if you wanted to use "same password" as an indicator of fraud, here's how I'd do it:

  • When a user signs up, generate a hash using a different salt to the one you otherwise use.
  • Truncate to the first n bits.

So for each user you'd have two distinct hashes -- one with, say, 64 bytes. The other with 2 bytes, say. Theoretically, this will make your hashing 65,536 times weaker (not entirely confident with this; maybe somebody could support/refute).

You'd then be able to use this value as a weak indicator of fraud.

1

The only way I'd consider doing something like this is if you put a stand-alone machine in a locked room with only an RS232 connection between it and the web server. No network card, no wi-fi, no USB thumbdrives allowed.

Perform an unsalted hash on the web server, send only the hash and a user ID over the serial port, and get back a count of the number of users with matching hashes from the locked computer. Don't cache the result on the webserver!

The locked computer can have a database of all the hashes stored by user ID and can do the comparison to say how many match.

Even then, this is all dependent on the discipline of the company involved. You don't want to be poking holes in the air gap here.

0

If you are set on doing this, the ideal solution seems to me to use a Bloom filter of the known "bad passwords".

The advantage is that even in a compromise, you only leak information about the passwords of known/suspected bad actors, and you never directly leak any password hashes. In addition, a dense Bloom filter is very fast to search and can be sized to have a low false positive rate. Also, if the Bloom filter is sized for a 1 in a million false positive rate, then even an attacker who has been able to steal your Bloom filter will find all of the real "bad passwords" plus one out of a million of his/her random password guesses.

Let's say you have a 65,536 entry Bloom filter using 8 different hash functions. This will require just 8 kilobytes to store, and will hold more than exp(ln(1e-6)/8) * 65536 / 8 = 1,457 "bad" passwords before you hit a 1 in a million false positive rate.

The downside is that after you mark an account as bad, you'll need to wait until the next login into that account to add the password to the Bloom filter (this only time you see cleartext passwords), and after you add a "bad password" to the Bloom filter, you won't discover other accounts sharing the "bad password" until the next time they login.

The first problem can be easily overcome by expiring the cryptographic tokens/login cookies of a bad account, tricking the attacker into logging in immediately. Crypto tokens/login cookies should always contain a message-authentication-code-protected creation timestamp. This allows you to force periodic login and also keep a per-user earliestValidToken timestamp that you can bump up to the current time when they change their password (or if you need to expire their existing tokens for some other reason).

The second problem can be overcome using a combination of several methods. (1) It's good to make users login every 2 weeks to keep them from forgetting their passwords (2) you probably mostly care about new account creation using "bad passwords" (3) assuming most accounts become spammy soon after creation, when you discover a spammy account, you can expire the crypto tokens/login cookies of all accounts created in the past N days (or maybe all accounts created after the spammy account).

As far as how to get your 8 hash functions, you need 8 16-bit hashes, or 128 bits of hash output to cut into 16-bit chunks. You'll want your hash functions to use site-specific keys, to minimize the possibility of cross-site use of a stolen Bloom filter. Two invocations of Siphash-2-4 using two different keys is probably the fastest way to get your 128 bits of hash values. SHA-1 HMAC (truncate MAC to 128 bits) or AES CMAC are also perfectly acceptable.

1
  • "The downside is that after you mark an account as bad, you'll need to wait until the next login into that account to add the password to the Bloom filter" I would suspect that for almost all of these accounts that you wish to add to the Bloom filter, they will create a new account, make their purchase, and never login again. Nov 7, 2015 at 9:41
-3

If I understand you correctly you intend to use this information only internally within your organization to detect possible fraud?

Now I don't know if you encrypt and salt the password on the client side using JavaScript or something similar before sending it to the server. But if not, then you could store unencrypted password at another db server in an isolated network not accessible from outside and run the checks on that?

Regarding checking for IP and browser I would guess that most people doing these types of fraud work through different proxies to change their ip around and use incognito features to "hide" their browser.

Update: Some people here seem to miss the fact that a question was raised. It has nothing to do with our own personal believes of password treatment and best security practices.

If one agrees to what the person asking the question wants to do or not is not the point. The question was plain and simple:

Question: If it is determined that the fraud-detection ability of being to retrieve passwords is necessary, what are the best practices for storing passwords securely but in a way that is plaintext-accessible?

He doesn't ask IF it's 100% secure. He asks for practices of making it as secure as it can be.

Yes, it will most likely suck and be impossible to protect with a 100% success rate. But IF IT HAS TO BE DONE, how could he do it as secure as possible. There will ALWAYS be a way around anything not encrypted with a, as of yet, unbroken hash algorithm. It might be more or less dramatic and might require some C4 to get into a server room 100m bellow the surface and kill 10 armed guards hardwired to a system that kills them if they even try to leave the room (so that they themselves can't steal the information and leave).

But it's still not the point of the question. He wants to know how to store PLAIN TEXT passwords as securely as possible. From whom isn't very specified and if he is the only one handling this system (and we assume he can trust himself) the only ones he has to protect the data from are outside parties. If not, then internal parties will also have to be taken into account.

Does it suck to store sensitive data as plaintext? HELL YES! Does the person asking the question deserve to know that? Yes!

Do we answer his question by only stating that? NO!

7
  • 3
    And then the old, old backup of that internal server is leaked, and a load of plain-text passwords that haven't been changed by their users in eons is out in the open. Not a good idea still. Jul 31, 2012 at 8:08
  • There should of course be a rigorous internal security plan in place for this. But the hardest thing to protect against is the own employees, this is true. But if they work there they could probably gain access to user accounts, create back doors etc just as easy. A isolated server, with access only to a single user in a locked room etc. could add enough security. If only one person has access and security is upheld, only one person has to be trusted. But someone must indeed be trusted. When we have had other sensitive material we have had security check points and surveillance all over :)
    – inquam
    Jul 31, 2012 at 8:12
  • 1
    @inquam - No matter how much security there is, if the passwords are not hashed, they will be leaked. I have to down vote this answer for even trying to suggest storing passwords as plaintext. If a single person can access the user's passwords then its a security risk not worth having.
    – Ramhound
    Jul 31, 2012 at 16:17
  • I'm not saying plain text is great. But IF this is what the person asking the question wishes to do, then it has to be done as securely as possible. Then I don't know, he might be the only person working with the application in question and trusting other people isn't an issue. I myself has NEVER stored passwords as plain text but have instead used advanced algorithms to try to link accounts together with combinations of behaviour and account information. But that's a WAY harder approach :). Worked fine for large banks and online gaming anti-fraud use though.
    – inquam
    Jul 31, 2012 at 18:10
  • 1
    @inquam "rigorous internal security plan"? Try "No ... plan survives first contact with the enemy".
    – DaveE
    Aug 1, 2012 at 20:21

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.