9

You can't please everyone. Some people want a lot of context and background on sites like this. Others do not. If you don't want the background, skip the first three paragraphs.

I am a software architect with about 25 years experience, starting with Amiga Basic, then C, then C++, VB6, delphi, C#, SQL (server), and more C#. Over the last 15 years my focus has been on back ends - databases, data models, and systems integration (not UI development, and certainly not modern web development with giant javascript libraries)

I currently work in a reasonably large "enterprise". By "enterprise" I mean "not a software development company". By "reasonably large" I mean our software ecosystem includes such things as an ERP (vendor code), a CRM system (vendor), an HR system (vendor), a few other vendor systems, a data warehouse, a BI stack, and a rapidly growing number of internally developed applications.

The number of internally developed applications is growing rapidly because the business wants to be able to add new functionality that is specific to us, to provide market advantages, or just to be able to move faster than the large vendors, thus providing our monolithic systems. I expect this story will be familiar to many, although it probably won't be quite so familiar to people working for pure software development companies, where you don't have to deal with the problem of integrating with large vendor systems. If you fall into the latter category, please keep this in mind.

Enough general background.

Full disclosure, I am about to conflate "bounded context" and "domain" to some degree. Some people swear that a particular business function - such as order entry - is a single bounded context and is a natural application boundary. Others say it's a domain which can have multiple bounded contexts, which are more granular. So, depending on which camp you fall into, read the following either as "domains" or as "bounded contexts".

I have been deeply studying just about everything to do with microservices, event driven architectures, ESBs, message brokers, and other integration elements at a hectic rate over the last several months, as well as rereading Evan's "DDD", Vernon's "Implementing DDD", Hohpe and Woolfe's "Enterprise Integraiton Patterns", and other famous books. And I have noticed a problem.

There are several different "primary sources" or "patterns" of advice on this topic. They all make good points. And they all contradict each other somewhere. I believe I can make the similarities and differences obvious with some simple diagrams.

Of course, the big question is "what do you want to achieve?". Well, let's settle on things everyone seems to agree on with distributed systems: Given CAP, we are very interested in A and P, not so much C. Eventual consistency is accepted, but we don't want one system to bring down all the rest, and we do want to partition the systems - for example into bounded contexts per Eric Evans' DDD.

So, I want you to picture what at first seems to be a fairly "ideal" architecture according to many high-profile sources, by which I mean it hits all the right notes. We have an order entry (point of sale) system. It's a bounded context. We're not "trying too hard to be microservicey" and creating nanoservices, and we're also not a distributed monolith. It's effectively agnostic about the existence of any other system in the enterprise. It's as decoupled as it can possibly be. It has no hard temporal, logical, or availability dependencies on any other system. It looks something like this:

Decoupled orders bounded context application

One day the business comes along and says "I want order entry (or quoting) functionality in the CRM system".

Oh dear.

Orders plz

Now I think I can describe everything else I need to describe purely with a set images which illustrate the various approaches I've seen advocated over countless books, blogs, articles, lectures, and videos, making the distinction between them clear. I have never seen the options laid out quite this way, and I think doing so demonstrates that as an industry we don't seem to have any "logically sound" solution which meets all of our principles of software architecture - except maybe the last. And I would like to hear people's opinions on what they see.

Personally, I think option 6 is the most - and perhaps only - sane choice. In a couple of places, I mention that shared library/schema definitions are "probably not a real objection". I say this because the business rules are the business rules. There's only one set of business rules for the bounded context of orders. If the business rules change, everyone using those rules has to change. This isn't a devops issue.

Option 1 - UI Integration

Option 2 - API Orchestration

Option 3 - Shared libraries and persistence

Option 4 - Shared libraries only

Option 5 - Shared data only

Option 6 - Shared immutable data only

  • 3
    "I want order entry (or quoting) functionality in the CRM system" - one thing to consider is what they mean by that. E.g., do they just want a unified front-end interface to existing services (to be used as originally conceptualized)? Or are the people using the CRM system (maybe a distinct department) actually going to work with orders in a way specific to them (a way that makes it awkward & undesireable to adapt the orders system for their purposes, so that it may be beneficial to treat it as a separate bounded context in Evans' sense, with a distinct model of orders and related processes) – Filip Milovanović Jun 30 '20 at 20:53
  • Fair point. Since it's my hypothetical, I will say that they mean they want the standard order functionality available from within the CRM application. It might be more illustrative to consider it in the other direction. Suppose we are decomposing a monolithic ERP, which is currently acting as point of sale. As the first step, we just want to take the existing ordering semantics and split them off into a separate application over which we have more control, but for some time at least orders must still be translatable into to the ERP model for the sake of invoicing etc. – allmhuran Jun 30 '20 at 23:58
  • In options 1, 2 and 3 if the Orders system goes offline, would that make the entire CRM system to be unavailable as well, or would most of the CRM still work normally and only the Order-related functionality becomes unavailable? There might be better (cheaper) ways to improve the availability of the Orders system than options 4 to 6. – Bart van Ingen Schenau Jul 1 '20 at 10:19
  • 1
    What exactly do you mean with one system bringing down the rest? When do you consider that a system bas been "brought down"? I think the answer to that is essential for which of your options are viable and which are not. – Bart van Ingen Schenau Jul 1 '20 at 10:28
  • 1
    I think there is one additional option to consider. Keep the sites functionally isolated in all respects, and build an SSO (Single SignOn, e.g. using OAuth) from CRM to order management. – John Wu Jul 1 '20 at 21:26
2

I'd go with Option 1.

While you listed some potential downsides, you forgot to mention the upsides of this approach: Primarily keeping the loosest coupling between your CRM and Ordering domains by placing that coupling in the most volatile CRM system (the UI). This means that when the Ordering API changes only the CRM UI needs to respond. This is ideal.

I'd also like to point out that the "downside" of having the CRM ordering system fail when your Ordering domain is down doesn't really sound like a downside to me. Presumably, if your Ordering systems can't take orders... well... you probably don't want other parts of your system to be taking orders. What does the Order UI show when it's down? I'd expect the CRM UI to mirror that.

Furthermore, if we simply rearrange your architecture so that there is only one UI (instead of each service having its own) I think the solution is a little clearer. My experience with micro-services is that they tend to be more representative of your service layer than anything "above".

  • The loosest coupling of the options presented are the 3 where messages are shared between the systems. The first three all involve a direct call from one system to the other, rather than communicating through messaging. I can absolutely have other systems taking orders when the actual order processing system is down, because the other systems are simply queuing up messages. When the order system comes up, it can ingest those queued messages. The "one UI" idea is a good one if you write all of your own software, but remember, CRM might be a big vendor system, like dynamics or something. – allmhuran Jul 2 '20 at 1:15
  • @allmhuran The only options you included above where the CRM simply pushes messages onto a queue also include tight coupling to the ordering logic (either via maintaining a copy to the package or by duplicating domain logic). This means changing the Ordering domain necessarily requires changes to the CRM domain. There are very tightly coupled systems regardless of how messages are orchestrated. And a message queue presents similar points of failure as direct communication. Now the CRM ordering functionality could stop working even if Ordering domain is working! (we've added coupling to a MQ) – king-side-slide Jul 2 '20 at 15:05
  • @allmhuran It's also possible that I am taking a wider view of what constitutes "coupling" than what is used above. I don't view "coupling" in simple terms like "lines of communication". For example, to my mind, Option 2 is more coupled than Option 1, Options 3,4,5 are an absolute disaster waiting to happen, and Option 6 is the least coupled in terms of our domains (not in absolute terms) but doesn't fulfill the use-case. – king-side-slide Jul 2 '20 at 16:57
  • I don't view "coupling" in simple terms as "lines of communication" either. That's why I specifically qualified it as "availability coupling". As I said in the opening, there can also be temporal, logical, and other forms of dependencies. You'll note I didn't mention any of the advantages of any of the options, except when stating my preference. Because the point was to show that none of the options, except perhaps the last, adhere to all of the principles to which we would like to adhere. – allmhuran Jul 2 '20 at 21:52
  • @allmhuran I must have misunderstood your statement about the loosest coupling being the ones where messages are shared. – king-side-slide Jul 3 '20 at 3:15
0

The question isn't totally clear, so I will summarise first: How to integrate two separate microservice systems. (OP please update your question title if you agree).

Service dependencies are the downfall of microservice architecture. You can actually use Microprocess Architecture between these two systems. see https://colossal.gitbook.io/microprocess/differences/compared-to-microservices. (I am a contributor to this draft standard)

As @king-side-slide said in this answer, the "downside" of an Order system being down isn't really a downside, but the CRM system will need to be able to handle that scenario.

Summarising your Situation:

  • The CRM should be assumed to be undocumented (at least the domain packages, but let's say it's all undocumented)
  • The CRM database is accessible to be read only
  • The Order system components are all fully modifiable and understood.

Microprocess Integration Design:

  • Create a new schema/database OCRM to be colocated with the CRM database. This way it's separate, and the vendor can make schema changes without overwriting any of your functionality.
  • Create OCRM.VIEW_NoDealOrders. eg. select * from CRM.Deals D where Closed=true and not exists (select 1 from OCRM.DealOrders DO where DO.DealID = D.ID)
  • Create a microprocess that batch creates new DealOrders driven by OCRM.VIEW_NoDealOrders with notification link to CRM.Deals#Insert. This microprocess will also connect to the Orders database and work according to order rules. This can double-up on OrderService(s) and start a per-subsystem migration to Microprocess architecture gradually.
  • If there is ever going to be a vendor update, or any UI update to display Order information inside the CRM, that should happen via Database-Web-Gateway and Views on the Order data.

If you are concerned about the Order database being offline at times. Simply have a read-replica of the whole Orders database. The order system should use that for read actions, while writes go to the master.

(Of course there would be more to this, but I had to already use my imagination to demonstration what I have)

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.