I'm trying to build a distributed system and have been researching how best to manage dependencies between services. The common example in tutorials is an ordering system - let's say I have a Catalog, Order and Account service.
The Order service needs to accept an order containing a number of items, cost up the total value, do a credit check against the account, and complete the order assuming all is okay.
This single action requires data from the other two services in a synchronous fashion (e.g. we need to look up the cost of each catalog item since we can't trust the user to have not tampered with it, and we need to validate that the user has sufficient credit), and I've experimented with ways of dealing with this:
- Maintain a persistent cache of the required data in the Order service which is kept synchronised using messaging (e.g. events in RabbitMQ, Kafka such as
accountUpdated.). The Order service contains a list of accounts with only the information it requires (e.g. the current credit), and a list of items (again, with a price and quantity in stock) to allow it to work autonomously. I've seen this referred to as a 'data-pump' architecture.
- Synchronous calls (e.g. REST/RPC) from the order service to the Catalog and Account services for it to verify the information.
- Another service (I think sometimes referred to as an aggregation service) sits above these three services, and aggregates the data from the Catalog and Account services into a format that the Order service can accept without doing the query itself.
Each method has issues and I'm having difficulty deciding if any are correct:
- A lot of data will end up being cached. In future, an order may be dependent on a new Discount service - do I cache all of these as well? My actual domain model will depend on over 8 other microservices already and it is in its infancy. Would this new Discount service also need to cache catalog items?
- This seems to go against some chief use-cases for independent and autonomous microservices - I've simply taken a monolithic application and swapped fast in-process calls for slower http calls. If I change one service, I likely to need to update another. If either the Catalog or Account services go offline, the Orders service cannot accept orders.
- This technique simply moves many of the problems with 2. to another layer.
I like the first approach due to its simplicity and resilience - if either other service goes offline then I can still make an Order. However, I am not sure I can justify the amount of data 'duplication' (I realise it's not strictly duplication) - is caching all of that information for validation (and potentially many times over for different services) acceptable practice? I can foresee the catalog being cached in many different services for example.
Thanks in advance!