1

Lets say we have an API that receives requests to commit a money transaction. The API receives the requests from a certain client and calls for a certain 3rd party service (lets say provided by the bank - call it bank service) to commit the transaction.

We choose to implement the API in a way that all of its endpoints are idempotent.

Why? well lets imagine a case where we have a transaction that is committed, meaning that the API made a successful request to the bank service but before managing to send the client that made the request that everything is good, it crashed.

If our API is idempotent, the client will be able to send as many requests to perform the (same) transaction over and over until he receives an answer that he is satisfied with.

The problem: If my API has to be idempotent, I have to save the idempotency key and its result in some sort of DB so I will know what happen with that request. In the same time, I also have to perform the call for the bank service and see if it succeeds. This means that I have to perform an atomic action but with two point of failure that are not on the save server, some sort of distributed transaction. I cant let the API be in a position that the request that came back from the bank service doesn't go into the store of the idempotency keys because of some failure (be it in the hardware or software).

The questions: How can I make this work, making sure that the call for the committing of the money transaction and the inserting of the result of it to some sort of DB is atomic?

Maybe there is another way which I am not aware of to tackle this kind of problem?

Maybe all of the companies that handle money transaction already use a bank service that has an idempotent API which results in a much easier work? It is much easier to implement an atomic and idempotent API if you have the direct access to the money DB.

  • What research have you done into distributed transactions? IIRC there is mathematical proof that, under the restrictions imposed by protocols used for internet-communication (and very much all of data-center-internal communication), two separate machines can not couple their transactions. I think the actual wording is "can not agree on one of two choices suggested by either side" (or sth. like that). – marstato Nov 13 '19 at 10:31
1

This definitely looks like a distributed transaction. And the most common pattern for handling failure in a distributed transaction is saga pattern.

The idea is that for each action there should be a compensating action in case of failure. I definitely can't guarantee that all banking services have idempotent APIs but most of them have refund transaction endpoints which might be helpful.

Also regarding this

I have to save the idempotency key and its result in some sort of DB so I will know what happen with that request

I think you should store some operation logs anyway because money is a delicate subject and some history would be useful anyways in case if something goes wrong.

  • I am familiar with the Saga pattern but I feel like its an overkill for this kind of task. – Max Nov 7 '19 at 16:43
  • In this case, I only need to perform the committing of the money transaction in a 100% safe way. If I will have more tasks like that, but need 100% consistency and involve an external Api, it will be A LOT of work... Using the saga pattern, I will have to sustain an HA log for the Saga Execution Cordinatior and have compensating requests for each action. Is there any other popular pattern to deal with this kind of problem? – Max Nov 7 '19 at 16:49

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.