I'm developing a software system, which provides HTTP REST API, and I want to achieve a very modular and flexible design. So, there is a core functionality and feature modules, which handle only the logic related to each specific feature. The goal is to be able to add/remove modules in a flexible manner without the need to manually connect each module to the core and to other modules. Therefore modules should not have tight dependencies between each other and the core should be very small and it should not depend on any feature.
I've found that event-based architecture is a good candidate for my goals.
However, I have some difficulties in implementing it in a robust manner.
1. How to approach the information sharing?
Initially, when some logic is executed inside the application (e.g. during user request processing), there is a context around it. For example, some entities were loaded from the database (I'm using ORM library for this). And then I want to fire an event.
What data should I pass with the event? I can see two approaches:
- Pass related entities with the event
- Pass only entity IDs (i.e. primary key, which could be used to fetch the entity from the database)
With first approach, the data could be used from the event right away, this is convenient. However, what if some previous event made some modifications to the data? We can't know for sure, because event handlers are decoupled. So the passed data could be already outdated.
With the second approach we are forcing fresh data to be loaded from the database, because we are only sending IDs and not the real data. So each event handler will need to manually fetch the entities. This solves the problem of data freshness, but it leads to another problem in terms of performance. There could be dozens of event handlers reacting to a single user request and each one will issue requests to the database. This looks like a very ineffective approach, which will put stress on the database.
2. How to handle concurrency (transaction isolation)?
Another problem, is that we need to ensure that our code is safe from the concurrency standpoint. We don't want parallel requests or even different parts of the same process to modify the same data. In order to do this, we need to use database transactions and various locking features.
And again, we could start a transaction and pass it with the event, so the event handler will be able to use the same isolation layer. Or each event handler would have it's own transaction. But, this leads again to the question of database performance.
I was thinking about creating a new transaction for each user request and to pass it to each method, which requires database access (directly or with the events). Therefore each request with all possible event handlers will be executed in a single transaction. Is this a good approach?
Generally, this architecture raises many questions, I would be very grateful for any good books or articles on this subject. Or maybe there is a better approach to achieve my goals in terms of flexibility and modularity?