I am tasked with creating a development, design and architecture guide for a large multi-year project. I have to dictate best design practices for a number of architectural perspectives. See the below diagram to demonstrate how I see this looking.
So based on other architectural mandate, MQ based asynchronous services are preferred because of guaranteed message delivery and because not every business critical component guarantees to be High Availability. My experience in developing against MQ interfaces was rather low scale and not in a transactional COA design. Typically I would develop multiple threads that would routinely check for new messages to process, both messages to push into a queue for requests, temporarily persisting the state, then another thread that will consume responses from another queue.
When considering a COA style architecture that has a clearly defined Data Access layer and Business Logic layer, how is asynchronous data sources represented or implemented here typically? It generally makes sense for a Data Access layer to provide an interface to Business Logic where the data source is a REST or SOAP based web service. These services are synchronous, they receive a request and immediately process it, then return a response while the client waits a predetermined amount of time to recieve it.
Asynchronous requests may not be processed right away, they also might not return a satisfactory response. MQ only guarantees the target component will eventually receive the request message, it does not guarantee it will return a response.
I feel like the best way to handle this is to have two seperate interfaces required by the Data Access layer, request interface and response interface. A scheduled batch job that is then using the Business Logic component might be able to deal with this effectively using multiple threads, however there may be another component that requires a real time synchronous interface of our Business Logic component where we are required to interface through MQ that is NOT real time.
I am confused in how to reconcile the two. Is this the right design pattern here? How can I effectively provide an interface to a component that requires real time when my component requires an asynchronous interface?
Some relevant articles I have been reading:
It seems this situation is defined already as an Architectural Anti-Pattern called Sync over Async. It is clear to me that reading this people are saying it is an extraordinarily bad idea. Then others note that sometimes we do have no choice and we have to do this which I am finding myself in. I am not in a position to ask for a synchronous interface to lookup and search services for this one system, and I have no say with the others components requirement for a synchronous lookup and search service.