0

I have a monolith implementing what is basically an ETL process, receive data from external system, orchestrate the processing of it, and then pass the results back to an external system. (The monolith does not perform the actual processing, that is handled by external services This monolith is deployed on-premise to our customers, not in infrastructure we control.

As the company has grown, we have run into issues where we are finding it difficult for the multiple teams (4 at this point) working on this monolith to keep track of each others work and avoid interfering with each other. Additionally we are now facing a need to scale the monolith beyond a single instance. A request has come down from management that we should use microservices to resolve these two problems.

How can we decompose a single pipeline within a single bounded context into multiple services to gain the scaling and organizational benefits, without introducing tight coupling or losing our ability to maintain predictable latencies?

14
  • Whole books have been written on this topic. I recommend "Microservice Patterns", "Doman Driven Design". I also recommend hiring somone with lots of experience in building (micro)services.
    – Euphoric
    Mar 6, 2023 at 6:34
  • And I would start with that "single bounded context". You have 4 teams working on it for long time. Yet you still have a only single bounded context? I find that hard to believe.
    – Euphoric
    Mar 6, 2023 at 6:43
  • 1
    Is this a "proper" Bounded Context or just a shared resource? Does it have a Domain-Driven name, and purpose or is it closer to "the orchestrator"?
    – ZioBrando
    Mar 6, 2023 at 11:01
  • 1
    "A request has come down from management that we should use microservices to resolve these two problems." -> this one got me worried, is it biz-management or tech-management?
    – ZioBrando
    Mar 6, 2023 at 11:03
  • 1
    With that said, there are clearly some good, on-topic problems implied in the question - the interesting part to me is "4 teams struggling to work together"; there's really no inherent reason why having 4 teams working in the same single monolith codebase or mono-repo should necessarily have difficulties working with each other, I suspect you'd get a lot more by looking at the specific points of friction between those teams (e.g. problems such as merge conflicts, testing, build/release, requirements changing, communication, etc) and trying to get specific solutions to each of those Mar 7, 2023 at 8:05

1 Answer 1

1

Recognise the advantages of what you already have

  • Easy visibility of the entire system and codebase for all teams, including faster visibility of other teams' changes.
  • A codebase optimised for minimal duplication and maximum code reuse
  • Simple, minimal dependency management.
  • No need to manage and juggle separate versions of microservices - just a single centralised version
  • Atomic changes into a single codebase, and no need to manage the synchronisation of changes into multiple projects/repositories/services.

These are important strategic advantages to the business because they are all factors in keeping down the total cost of ownership of the system, so they need to be included in any cost calculation because losing them will generally add some additional time, cost and risk.

How can we decompose a single pipeline within a single bounded context into multiple services to gain the scaling and organizational benefits, without introducing tight coupling [...]?

Generally there is no way which does not involve either a fundamental rethink of your single Bounded Context(BC) such as defining new smaller BCs, or widespread violations of the D.R.Y. Principle.

Bounded Contexts aren't architecture

Remember that bounded contexts aren't a technical concept but a natural reflection of your domain - a boundary which encompasses all the knowledge relating to some particular aspect of your domain and serves as the authoritative source defining that knowledge. (Knowledge being the sum of all of behaviours, relationships, information and interactions within the BC).

To split into services implies creating barriers between fragments of knowledge which naturally belong together; breaking relationships, adding complexity to interactions, altering behaviours, reorganising information. These aren't just issues for the code or architecture but the underlying requirements, which then impacts how you measure and test your solution.

Splitting a Bounded Context is creating new Bounded Contexts

To split a Bounded Context you need to fall back all the way back to its origins and fundamentally re-examine all of the assertions which led to it being defined in the first place. Otherwise, even if you manage to split your codebase into services, your requirements have not changed, nor has the knowledge it needs to represent; so any change in one part should be expected to impact every other part unless you rethink the model which everything is built upon.

Splitting a codebase without a more fundamental rethink cannot possibly result in meaningfully independent services or projects - you can only get independence by defining new Bounded Contexts (a fact which would likely become all too clear as soon as you consider integration testing, release cycles and deployment).

At which point, consider warnings from history - fundamentally rethinking your domain model is not impossible, but must be done with extreme care not to throw away everything you already have, which as Joel Spolsky described in 2004, is "the single worst strategic mistake that any software company can make"

1
  • Whilst I believe this is the best way of handling the situation, due to organisational politics it's not one I can move forward with. But that is organizational dysfunction, not a technical question. Mar 7, 2023 at 10:20

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.