65

This is kind of similar to the Two Generals' Problem, but not quite. I think there is a name for it, but I just can't remember it right now.

I am working on my website's payment flow.

Scenario

  1. Alice wants to pay Bob for a service. Bob has quoted her US$10.
  2. Alice clicks pay.
  3. Whilst Alice's request is flying through the ether, Bob edits his quote. He now wants US$20.
  4. Bob's request finishes before Alice's has reached the server.
  5. Alice's request reaches the server and her payment is authorized for US$20 instead of US$10.
  6. Alice is unhappy.

Whilst the chances of this are very low in practice, it is a possible scenario. Sometimes requests can hang due to network issues, etc...

Possible Mitigations

I don't think this problem is solvable. But we can do things to mitigate it.

This is not exactly an idempotency issue, so I don't think the answer is "idempotency token".

Option 1

Let's define:

  • t_0 as the time Alice click pay.
  • t_edit as the time Bob's edit request succeeds
  • t_1 as the time Alice's request reaches the server

Since we cannot know t_0 unless we send it as part of the request data, and because we cannot trust what the client sends, we will ignore t_0.

At the time Alice's request arrives in the server, we check:

if t_1 - t_edit < 1 minute: return "409 Conflict" (or some other code)

Would this approach work? 1 minute is an arbitrary choice, and it doesn't solve the problem entirely. If Alice's request takes 1 minute or more to reach the server, the issue persists.

This must be an extremely common problem to deal with, right?

  • 15
    This is called a "mid-air collision". And yes, it is a common problem with any distributed system. – BobDalgleish Jan 22 at 13:44
  • 83
    The wrong thing to do here is to jump to conclusions - ask the business how the price-setting process works, and what's their policy on this. For example, if a price has been published, and then it changes mid-request, the business may not care at all - it could be that they are perfectly fine with the price being the one that was listed when the request was made. That is, there may be no concept of "THE price" in the domain, but a concept of "price at a given date". Understand what they actually want and design a solution around that. – Filip Milovanović Jan 22 at 19:40
  • 21
    This is what it means to understand the domain - don't solve for fictional problems (and potentially introduce unintended real ones). Check how the domain actually operates instead. – Filip Milovanović Jan 22 at 19:40
  • 13
    Alice's request should include either the amount she intends to pay, a unique id of the (immutable) quote she intends to pay, or both. Then it would not be processed if the price is different or the quote has been rescinded and a different quote provided. – David Conrad Jan 22 at 22:55
  • 13
    @turnip - I read some of your comments to the answers below: think of the quote as of a contract proposal between Alice & Bob; if either of them changes anything, it's a new contract; Bob must provide another quote, before Alice can accept/pay. If that makes sense, then the issue that remains is controlling the dynamics (cadence) of the interaction. How often can changes be made? Is the agreement automatically invalidated on any change, or does the user have to explicitly "commit" a group of changes in an additional step before they take effect., etc – Filip Milovanović Jan 23 at 0:05

10 Answers 10

153
  • Alice wants to pay Bob for a service. Bob has quoted her $10.

Give this quote a unique token.

  • Alice clicks pay.

When this response is send to the server, it must go with the token of what is being paid. This also allows you to discard duplicate payments.

  • Whilst Alice's request is flying through the ether, Bob edits his quote. He now wants $20.
  • Bob's request finishes before Alice's has reached the server.

That has a new different token※. The server must invalidate the old one.

  • Alice's request reaches the server and her payment is authorized for $20 instead of $10.

No, it isn't. Alice token does not match.

※: The server must send Alice the new quote, with the new token. And alice must click pay again.


For user experience, you can also add a timeout. That prevents the token to be used right away. This timeout can either be only client side or networked. The purpose is to give some time to the user to notice the change.

This must be an extremely common problem to deal with, no?

Many online video games that allow players to trade face this problem. A simple 5 seconds timeout can save support a lot of headaches.

|improve this answer|||||
  • 5
    I think this is the correct answer because it addresses the core of the issue. Whilst the other answers provide useful suggestions, they don't fully tackle the problem. Your solution ensures consistency AND still leaves room for flexibility; Bob can still edit his quote as many times as he wants. I will implement @Hans-Martin's suggestion of invalidating quotes when Alice changes her requirements too. – turnip Jan 22 at 13:07
  • 13
    One way how some card payment protocols do it is by having the token include (or be) a cryptographically secure hash or signature of the key fields of the transaction, obviously including the amount, so that at both sides (and any intermediary systems) it is possible to check if it matches, and nobody can maliciously alter the quote corresponding to the token without invalidating it. – Peteris Jan 22 at 19:43
  • 29
    Token is a good idea if you want Alice to be able to purchase at the old price for a certain period even if Bob changes the price. If that's not a requirement, Alice can just send the price she agreed on. If it doesn't match the price Bob is currently quoting, you can return a failure code with the updated price so you can display a message about the price increasing... – Jason Goemaat Jan 23 at 2:01
  • 3
    Ah, just read all the comments. The video game example with a token is good. Like in WOW when you open a trade window, you each put items/money into the trade. When you are happy with it, you click 'accept'. If the other party makes any changes, your 'accept' is no longer valid. In this example you need a record of exactly what each person thought the agreement was. That would involve keeping track of updates on the server with some sort of token, or recording the last thing each party agreed to until it matched. – Jason Goemaat Jan 23 at 2:06
  • 18
    This makes sense, you shouldn't think of it as Bob changing a property of the quote - a quote should be immutable. Rather by changing the price Bob is withdrawing the original quote and creating a new one. Generally in systems I've seen this, the id is a 2-tuple - the first element is fixed when Bob creates the quote and the second increments each time Bob amends a field. That way you can track change over time but ensure that both parties are manipulating the same object. – David Waterworth Jan 23 at 3:05
56

A quote should be a write-once record.

Bob isn't allowed to edit it once it has been created and passed to Alice.

You can ensure this at different levels, from simply not offering an edit dialog to sophisticated digital signature algorithms.

|improve this answer|||||
  • 20
    New requirements, new quote (which likely invalidates the old one.) If Bob offer one Whatsit for 10$, Alice can either accept this quote as it is, or request a new quote for 2 Whatsits. – Hans-Martin Mosner Jan 22 at 11:35
  • 5
    @turnip If Alice has changed the request, the quote should be invalidated. Otherwise she could game the system: request a quote for one widget. Get a quote from Bob. Then change the request to 10 widgets and hit the "buy now" button before Bob has a chance to update the quote. – Simon B Jan 22 at 11:39
  • 1
    @Caleth yes, it looks like that could be the way to go. Currently "Alice" has more power when it comes to accepting an offer because accepting and payment is one step. If I introduce a separate "accept" step which both parties must carry out, then I can proceed with payment. What Hans is suggesting could work too. – turnip Jan 22 at 11:39
  • 2
    @turnip - who are you making this for? Can you ask them how they handle changing requirements, how much negotiation they want to allow, and how they might want to structure the process? It may turn out that they have a way of doing it that avoids this problem completely. – Filip Milovanović Jan 22 at 19:48
  • 4
    @turnip in that case it's a new offer to a new set of requirements, the original quote keeps linked to the original set of requirements. Do NOT allow editing of quotes and requirements/order forms while quotes are being processed. If Alice changes her order, cancel the quote and issue a new one. If Bob notices he made an error in his quote, cancel the quote and issue a new one (if Alice hasn't accepted the existing one yet, in which case Bob has entered a legal contract and will need to renegotiate to fix that error). – jwenting Jan 23 at 6:20
27

This must be an extremely common problem to deal with, no?

No, it isn't. I doubt you'll be able to find a payment processor that lets you change the amount after the customer has authorised a particular amount.

You sell to Alice at the price she authorised, because that's what your quote to her was, and what she authorised. You don't check that the money you received matches what you currently quote. If you do check, it's that you issued that quote to Alice in the first place.

|improve this answer|||||
  • 3
    @CurtJ.Sampson No, it just means that if Bob wants to change his quote, what he's really doing is cancelling the old quote, and creating a new one. Alice's bid is no longer valid, because the original quote has been cancelled. She can choose to bid on the new one. – Luaan Jan 23 at 8:26
  • 9
    @CurtJ.Sampson Depending on jurisdiction, Bob has a contractual obligation to Alice if she authorises payment, and can't change the quote without Alice's agreement. – Caleth Jan 23 at 8:31
  • 3
    @CurtJ.Sampson But not whilst you have the item in your (physical) basket, or your card in their eftpos – Caleth Jan 23 at 12:43
  • 5
    @CurtJ.Sampson actually, in Germany where I live, prices in shops are binding. If the price tag at the shelf says an item costs 4.99€ the shop owner is not allowed to change his mind in the time I take it to the register. Same for gasoline prices which change several times a day. Once I have started pumping the price is fixed. Of course, if I see a price quote on Monday and come back on Thursday to buy the item, I can't expect the price to be unchanged. – Hans-Martin Mosner Jan 23 at 12:43
  • 4
    @CurtJ.Sampson You are confusing different situations. Your argument (shop pricing changes, which happen irregularly and often with advance notice) does not work for the use case of the question at hand (price changing practically at the same time as the purchase). In a real shop, if I pick up an item and a clerk changes the price tag while I'm walking to the cash register, the shop might be legally obligated to honor the price tag that was there when I picked up the item. Not at all the same as coming Monday and returning Tuesday expecting the same price. – Aaron Jan 23 at 16:00
20

Just send the amount Alice agreed to pay along with the request. If the price has increased since Alice sent the request, you send a response indicating that the item could not be purchased at or below that price, and the current price is whatever it is. This is pretty much the same situation as when there's only one item available and, at the time the request to buy reaches you, the item has already been sold.

So your message flow would look like:

   Bob → Server  Set item price to $10
 Alice → Server  Tell me item price.
Server → Alice   Item price is $10.
   Bob → Server  Set item price to $20.
 Alice → Server  Purchase item for $10.
Server → Alice   Item not available for $10; current price is $20.

This is, in fact, exactly how things work in trading systems connected to exchanges. (I wrote one about ten years ago.) You never know what bids and offers are currently on the exchange; you know only what was there a few milliseconds ago. And even if what was there is still there now, it may not still be there when your order reaches the exchange.

|improve this answer|||||
  • Going along with the exchange example, an alternative solution would be to present different order types to the customer. Let them choose "just give me the current market price, I'll pay whatever it is" vs "I will pay $x or better, and I'm ok if my order isn't executed right away." (Though make sure to take precautions to prevent the first option from being abused, such as limiting the frequency of price updates from the seller.) – 0x5453 Jan 23 at 13:45
  • @0x5453 Yes, those are known as "market" and "limit" orders. I was trading cash-settled options on futures of an index, and market orders were almost never used. (I certainly never used them.) There's no "abuse" in changing your orders frequently; it's quite normal to cancel old orders before they've been filled. The only limit is on how fast you can send your orders to the exchange, which can easily be several dozen per second. – cjs Jan 23 at 14:11
  • 6
    There could be abuse for a website like this where there are not regulating bodies in place to prevent such. For example, sellers could repeatedly flash between a reasonable offer and some insane number, hoping that a few users will use market orders without paying attention to the price updates during "checkout". – 0x5453 Jan 23 at 14:22
  • @0x5453 only if all of them work together. Because a market order means to buy at the lowest available price. – Josef says Reinstate Monica Jan 23 at 16:15
  • 3
    @LorenPechtel - this answer doesn't require trusting user data. If Alice sends a message for Purchase item for $0.01 or even Purchase item for -$10. the Server will just respond with Item not available for that price; current price is $x. – Martin Smith Jan 24 at 8:31
8

Yes this is a common issue, and it is about transactional consistency.

To summarize your issues:

  • the quote is binding for the seller. In general it has a reference and an expiration date/time.
  • the buyer may buy under the condition of an accepted quote. The buyer cannot be forced to accept a price that was not agreed.
  • if a buyer finishes the transaction before the expiration of the quote, there is no ambiguity about which price to use. But if you allow the seller to change the quote, either the price should be the one that the buyer accepted when starting the purchase, or the user should give consent to the price of the new quote.
  • if the buying transaction is started before the price change, but not finished in time, we are in an ambiguous situation in which the seller could decide not to accept the purchase under the old price; but the buyer is not obliged to buy under the new price. If we are speaking of minutes and seconds, the practice is often to accept the initiated transaction.
  • finally, there might be no explicit quote for a buyer, in the scenario of a public price list for a catalogue.
  • the main problem in your specific situation, is that you have no certainty about the time at which Alice initiated the request: you only have a time for the completion of the payment.

One solution that works for both, quotes and public price lists, is to give a certainty to the start time of the transaction. So before proceeding to the payment, Alice must confirm her intention to buy. Exactly like with your Amazon basket. If at this moment the price was already changed, Alice could decide to accept and continue, or to cancel the purchase.

Since you now know the purchase initiation date, you can fine-tune your business rules (e.g. accepting a payment within 10 minutes, or more, or less).

From a technical point of view, this approach can be used to implement the saga pattern that fully solves the issue:

  • preparatory actions (request a quote, issue a quote, consult a quote) that are all cancellable, until a pivot event;
  • all actions happening after the pivot are just optimistically performed as if we’d be sure of a positive outcome;
  • but the these actions must stay reversible (status "in progress", with undo possibility) until the last event of the saga is performed (payment completed in your case).

The saga is a more flexible alternative to a distributed two phase commit.

And if Alice and Bob were on the same system, there wouldn’t be an issue at all thanks to ACID isolation.

|improve this answer|||||
4

Bob changed the price at time t. Alice ordered at a time t’ which is close to t. Bob would have had no problem to charge the lower price, had she ordered 10 minutes away from t.

So you record not only the current price, but also the previous price and when it was changed. In Alice’s order you include the price she has seen.

When the order arrives and doesn’t match the current price: If it doesn’t match the previous price either, you fail (some dodgy request). If the price increased, but more than ten minutes ago, you fail. Otherwise, that is the price was lowered or changed in the last ten minutes, you charge the lower of current and previous price.

All this of course if Bob agrees. The “ten minutes” can be made longer. Easy to implement, and it keeps customers happy. If Bob prefers to make customers unhappy, or never return to the shop, you implement something else.

To make the situation less common, let Bob edit the prices at any time, but apply all the changes at 3am in the night when (almost) nobody is ordering.

|improve this answer|||||
2

In an API world, this would be solved by resource versioning. The resource can be versioned internally, for example with a time stamp.

For a standardized mechanism, an ETag can be used.

This allows for semantics such as "the thing we talked about earlier, I'd like to perform an operation on it, if things are still the same"

|improve this answer|||||
1

While Theraot's solution of associating each price offer from Bob with a unique token works (and may have some additional benefits like preventing Alice from accidentally paying for the same service twice, assuming that's not something she'd normally ever want to do), for this particular problem there's an even simpler solution:

Include the price that Alice is willing to pay in her purchase request.

(Edit: This is essentially the same solution as suggested by Curt J. Sampson earlier. Somehow I failed to notice their answer before writing mine. I'll leave this answer here since it includes some additional details, but I encourage you to upvote their answer too if you like this one.)

With that single modification, your example scenario now works out like this (with changes in italics):

  1. Alice wants to pay Bob for a service. Bob has quoted her US$10.
  2. Alice clicks pay, sending a request to purchase Bob's service for US$10 to the server.
  3. Whilst Alice's request is flying through the ether, Bob edits his quote. He now wants US$20.
  4. Bob's request finishes before Alice's has reached the server.
  5. Alice's request reaches the server and is rejected, since the price of US$10 that Alice is offering to pay does not match the US$20 that Bob is now asking.
  6. Alice receives a message that her purchase failed due to a price mismatch, and she must now choose between repeating the purchase with the new price of US$20 or rejecting the new offer. Alice is mildly annoyed at Bob for suddenly switching prices like that.

Note that, since the price of US$10 included in Alice's request in step 2 comes from Alice's browser, which is under her control, she could fairly easily modify the request to try and get a cheaper price. However, the server-side comparison of the prices in step 5 will also protect you and Bob against any such attacks by Alice: any attempt by Alice to unilaterally lower the price she's paying will just give her the same notice of a price mismatch and force her to retry the purchase, just as if Bob had changed the price.

If you want, you can try to distinguish these two scenarios, e.g. by keeping track of recent price changes by Bob on the server and/or by using a cryptographic token passed from the server to Alice and back to verify that Alice's request indeed matches a legitimate prior offer by Bob. This could be useful if you wanted to know whether Alice was trying to cheat or just a victim of unfortunate circumstances, but it's not needed to prevent such cheating attempts from working in the first place.


Also note that, if Bob had decided to instead lower his price from US$10 to e.g. US$5, you would have several options for handling the mismatch:

  • a) reject the request and make Alice repeat it, just like above;
  • b) accept Alice's request, but only charge her the new price of US$5, just as if she had repeated the request and accepted the new price; or
  • c) accept Alice's request and make her pay the original price of US$10, just as if Bob's price change had only happened after Alice's purchase.

In some sense, none of these options is wrong — they all (eventually) result in Alice paying a price that both she and Bob had considered acceptable for the service. That said, going with option (c) seems likely to leave Alice quite unhappy if she realizes what has happened. Thus, in general, I'd recommend either option (b) or, just possibly, some low-overhead version of (a) where Alice is only shown a quick confirmation dialog where she can click "OK" to accept the new, lowered price. Anything more than that would be needless overhead for something that Alice almost surely does want to do.

Of course, any such confirmation request must also include the new price that Alice now wants to pay, and it must be verified against Bob's offer on the server in order to protect against further price changes by Bob and/or attempts to manipulate the request by Alice.


BTW, unless your app includes a real-time feed of price changes from the server to each potential customer's browser, a much more likely version of your scenario is that Bob changes the price after Alice has loaded the page with the price and the purchase button, but before she has actually clicked the button. That's typically a much wider time window than the actual time it takes from Alice's request to reach the server after she clicks the purchase button. However, it doesn't actually make any practical difference for this scenario whether the price change occurs before or after the button click — in general, neither Alice nor Bob nor the server can even tell anything except that the price has changed at some point after Alice loaded the page and before her request reached the server.

(If your app does include a real-time price change feed, you'll need to also consider the possibility of Bob changing the price — and this price change reaching Alice's browser — a fraction of a second before Alice clicks the button, too late for her brain to react to the change and stop the click. It would probably be a good idea to disable the purchase button for at least a few seconds after any price change, and to show some very conspicuous notification whenever such a live price change occurs.)

|improve this answer|||||
  • 1
    Sometimes lowering the price of something may create problems for a buyer. For example, on a site offers free shipping on purchases over $25, and $10 shipping on smaller purchases, cutting the price of something from $26 to $24 may increase by $8 the amount the buyer has to pay unless the buyer adds more items to the purchase. – supercat Jan 25 at 8:58
  • @supercat: Agreed, in the presence of such perverse incentives a price reduction should definitely require confirmation. – Ilmari Karonen Jan 25 at 14:39
  • There may not be any intentional perverse incentives on the part of the seller, and indeed the person setting the price might not have any idea of any ways in which lower prices might affect the buyer. For example, if the goods are being sold in an aggregated-shipping marketplace, the seller may have no monetary interest in the shipping charged by the marketplace, nor any way to know whether the buyer would be near a price threshold. Personally I think that marketplaces should handle concepts like "Free shipping on orders over $25.00" by making the shipping cost be the lower of... – supercat Jan 25 at 17:28
  • ...the "normal cost" or $25.01 minus the order total, so that if normal shipping would be $5.00, then an order of X goods would cost X+$5.00 for X of $20.01 or less, $25.01 exactly for X of $20.01 to $25.01, and X for orders of $25.01 or more. In such scenarios, it would still be good to ask people if they'd like to order (e.g.) up to $4.73 of free merchandise, but it would not be unreasonable to interpret a lack of a response as confirmation of the original order as given. Alternatively, in some scenarios it might be reasonable to process the order for the original amount but... – supercat Jan 25 at 17:33
  • ...include a "free merchandise" voucher for the price difference, but the feasibility of that would depend upon how the customer's payment was divided among the entities processing the sale. – supercat Jan 25 at 17:34
1

I would like to address two side issues. How to solve the issue has been addressed by many others.

Criticality of the issue

You missed steps 7 through 9 in the scenario:

  1. Alice wants to pay Bob for a service. Bob has quoted her US$10.
  2. Alice clicks pay.
  3. Whilst Alice's request is flying through the ether, Bob edits his quote. He now wants US$20.
  4. Bob's request finishes before Alice's has reached the server.
  5. Alice's request reaches the server and her payment is authorized for US$20 instead of US$10.
  6. Alice is unhappy.
  1. Alice talks to her credit card company, tells them she agreed to pay US$10, and they agree.

  2. Bob gets his US$20.

  3. You pay the remaining US$10. Now you are unhappy.

This makes it critical that you deal with it. Otherwise, some Alice and Bob will collude to take money from you.

Commonness of the issue

If Alice performs step one on Monday morning, and step two on Friday night, step three could have happened any time in between. This makes it ridiculously easy to happen.

|improve this answer|||||
0

Note that in most cases there's an intermediate step--shopping carts. You put the item in the shopping cart rather than buying it. If there's a price change at the wrong instant you can see it before you press pay. When you pay you're buying the shopping cart--the items in it already have prices attached.

For a more complete solution you can make making prices read-only. When Bob raised the price on widgets to $20 he really created a new type of widget that goes for $20, the $10 widget still exists but can not be found unless you know the item ID. Alice is attempting to purchase the $10 version, the transaction goes through at $10.

|improve this answer|||||

Not the answer you're looking for?Browse other questions tagged or ask your own question.