I'm trying to design a layer/component in my distributed application that will communicate between component A and component B.
Right now, this "communication" is accomplished by replicating an entire database from the server where component A lives over to component B. But this design has proved to be very brittle and too tightly coupled. I want to recommend to the team that we move to a messaging solution where A notifies B of a change of state of the data.
But here's the question. Depending on the size of the data, I can either
- send the data as a part of the message payload,
- or just notify B that's there something for B to do / to grab.
Clearly, option 1 is easier because I won't need to write too much code in my subscriber logic. However, the reality is that sometimes (not too often) the size of the data may exceed the max allowable size of the message. (i'm going to use mqtt to start my prototype which as a max payload size of 268435455 bytes)
As far as design is concerned, I realize it's best for each service / component (A and B) to be autonomous... and to just "know" what it has to do when it gets a message. But in practical terms, what does this mean?
For example, consider this use case:
"A" publishes a message that a new widget has just been created in it's database. "B" is a subscriber on the "widget" channel. It gets the message that includes the following:
- action:CREATE - widgetID: 123
To get the data about the widget, should B:
call a REST API running on server A?
be able to grab the widget data from the message payload?
Other questions would include:
- when should I consider the message broker to be successful? WHen the message is delivered? Or when B has a copy of the data in question?
- any good articles you know of that I can read to help with this type of design?
- what about if i just avoided messaging all together.. and wrote a CRUD interface for component B? "A" would call the CRUD interface whenever a widget was changed. what's the benefit of using message broker over this type of a solution?