0

I have a system design question that I'm looking for some guidance on. I have two different systems that need to have a basic level of communication. This is abstracted via message queues.

For example, System A is responsible for user account registration. A user access System B and enters the user registration flow. After the user submits their details, they are sent to System B's Loading / Waiting screen, and they register a long-polling request with System B awaiting the completion of their account. On the back-end, System B places a message with registration details onto a message queue. System A consumes this message, creates the account, then places a message onto a separate queue indicating that registration is completed. System B consumes this message, and needs to notify the user that the registration is complete. This is where the problem lies. System B is running a cluster of servers, and there is no guarantee that the server which is holding onto the long-polling request is the same server that processes the registration completed message.

This seems like it should be a common problem, but I haven't been able to find much discussion on it. I've explored some options, but they all seem to come with some level of downside.

Potential Solutions:

  1. When System B consumes the registration completed message, use something like JGroups to broadcast this to all other servers within the cluster
  2. Each instance dynamically creates its own SQS queue and use the SNS-fan out pattern to deliver the message to all of these queues so all servers receive a copy of the message
  3. Keep track of which server the request is on and find a way to route the message to the appropriate server (this is especially tricky as there is an edge case where during the routing, the request time outs and another request by the same user is initiated and handled by a different server)
  4. Just use standard polling to eliminate the aspect of server-state

Any insight or suggestions on this problem are greatly appreciated. Thank you.

3 Answers 3

0

One alternative is: you can use a concept called "partition key". When a user place a registration form to System B, you create a key. This key can be derived from user id or email or username. Every partition need to be subscribed to exactly one instance of System B (and this can be derived from the partition key). So, every user will end up served by the same instance of System B.

The idea is: for every user, you should be able to tell which instance serving that user.

0

If you use message brokers to implement a kind of remote procedure call, you normally create a fresh response queue for each request. Only the process which initiated a request will wait on this queue.

However, if for some reason the long-polling connection is interrupted and the client retry is routed to another service process, the completion message would now needed to be routed to this service process.

One possible approach would be to store the account creation request together with its status and the response queue id in a shared database, and use this to recover from a broken long polling request. It would require some caution to avoid race conditions, but if you use long polling with timeout you would eventually succeed, maybe with some more waiting time.

0

You could use a topic instead of a queue to deliver the registration message to Cluster/System B - that way all the machines in cluster B would receive the message and only the system that was interested in processing it would do so (the rest would discard the message).

However that solution is both:

  • Wasteful of resources (sending the messages to multiple machines only for them to throw the message away).
  • Has some race conditions, for example if the registration message arrives before the redirect is complete.

Frankly I think the best solution is simply to hold the redirect on System A - specifically system A doesn't return the redirect until it has completed the user registration, it can then do the redirect including a (secure) key that System B can use to lookup the registration. This solution avoids the need for the queue synchronization.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.