2

I'm faced with implementing interfaces for some rather archaic systems, for handling online deposits to stored value accounts (think campus card accounts for students).

Here's my dilemma: stage 1 of the process involves passing the user off to a thrid-party site for the credit card transaction, like old-school PayPal. Step two involves using a proprietary protocol for communicating with a legacy system for conducting the actual deposit.

Step two requires that each transaction have a unique sequence number, and that the requests' seqnums are in order. Since we're logging each transaction in Postgres, my first thought was to take a number from a sequence in the DB, guaranteeing uniqueness. But since we're dealing with web requests that might come in near-simultaneously, and since latency with the return from the off-ste payment processor is beyond our control, there's always the chance for a race condition in the order of requests passed back to the proprietary system, and if the seqnums are out of order, the request fails silently (brilliant, right?).

I thought about enqueuing the requests in Redis and using Resque workers to process them (single worker, single process, so they are processed in order), but we need to be able to give the user feedback as to whether the transaction was processed successfully, so this seems less feasible to me.

I've tried to make this application handle concurrency well (as much as possible for a Ruby on Rails app), but now we're in a situation where we have to interact with a system that is designed to be single process, single threaded, and sequential. If it at least gave an "out of order" error, I could just increment (or take the next value off the sequence), but it's designed to fail silently in the event of ANY error. We are handling timeouts in a way that blocks on I/O, but since the application uses multiple workers (Unicorn), that's no guarantee.

Any ideas/suggestions would be appreciated.

Improve this question
5
  • 1
    Failing silently is not good. Since there's no way to get status of a particular task if it fails, your only recourse is to time out the entire sequence after awhile if one of the results is still missing.
    – Robert Harvey
    Dec 1, 2012 at 19:55
  • Here's what I heard: You have a service which needs to get requests in order, though the starting order is non-deterministic, and that order must be maintained. The key here is it's non-deterministic, which means you simply cannot get guarantees on this. That means it's time to look at redundancies, if you can safely retry requests you may need to gaurantee requests are all retried at least once if not twice to hope that if the order it started with was seqnum: 2,4,3,5 and you hand them back in 2,3,4,5 it will accept 2,4, and the retry of all of them will accept 3,5. Start inspecting the API.
    – Jimmy Hoffa
    Dec 1, 2012 at 20:24
  • The other thing to search the API for is any form of status. If there is simply zero ways to find out if something was successful (which doesn't make sense; if it failed and that's a problem for the original user, they will be able to determine that, can you make the same check?) then you are screwed and redundancy is you only hope. Make sure users are notified they do not have a guarantee of success, if for no other reason than to CYA (though also to set user expectations therefore improving their experience)
    – Jimmy Hoffa
    Dec 1, 2012 at 20:29
  • I have the full API docs... having unsuccessful transactions fail silently is this vendor's idea of "security." (UGH) I should note, by "unsuccessful" I mean that the "packet" was malformed, out of sequence, or otherwise rejected. It will notify of a successful deposit, or a debit rejected due to insufficent funds. But there's no way to tell the difference between, for instance, an out-of-sequence seqnum, and a timeout.
    – Jason Lewis
    Dec 2, 2012 at 4:43
  • Also, the seqnums are absolute: once 4 has been accepted, nothing < 4 will ever be accepted again. They don't have to be sequential, i.e., 6 is fine after 4 (even if there's no 5), but after 6, the number ca nbe anything >= 7. My feeling is that every retry needs to grab a new seqnum until it is successful; but since we expect this to be used frequently, I can see this strategy exceeding 2^32 in no time, and given how long ago it was written, who knows what will happen then :)
    – Jason Lewis
    Dec 2, 2012 at 4:49

1 Answer 1

Reset to default
2

You aren't limited to one terminal. You can set up additional terminals and have your app block each one until the transaction is complete. The additional terminals don't use up additional resources in the system you are referring to. You could easily have 100 available to your app and iterate through them with each transaction.

Each terminal maintains its own sequence numbers. This is how we handle ours and it works fine.

Also, IIRC, you can process the transactions as "offline" and send sequence numbers out of order. Though you would really want to avoid running everything as an offline transaction.

Improve this answer
1
  • Jeff, hey! Thanks for the info. I might be emailing you to pick your brain more about this... Glad you saw this, it's nice that someone out there is dealing with the same vendor and the same issues.
    – Jason Lewis
    Dec 2, 2012 at 4:53

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.