Does SOA service composition actually work in practice?

Question

One of the main SOA service design principles is Service Composability principle (https://en.wikipedia.org/wiki/Service_composability_principle).

The idea is that by composing new services using existing ones as building blocks, one can rapidly develop new services. Kind of analogously to how you call exising methods of objects when you implement new methods. This is where much of productivity boost from SOA is supposed to come from.

Does someone really do this in practice? I have seen this repeated endlessly in written text, but have not experienced working real world implementations myself. Most of the text also omits any mention of transaction handling, which seems to me to be the biggest obstacle in realizing composable services.

First, you really do need to tackle the transactions problem before you can compose any non-trivial services. Sure, if the example has services "findCurrentTime()" and "writeLogMessage()" its easy to not worry about transactions, but not when having real world examples such as "depositMoney()" and "withdrawMoney()".

I know of two options:

1. Implement real transactions with WS-Atomic Transaction or such
2. Implement compensation-based solution which compensates call to A with "cancelA()" or somesuch if call to B fails

Both of these seem very problematic / close to unusable to me:

• WS-Atomic Transaction
  • a lot of complexity, most advice that I found just warns "pain in the ass, dont do it"
  • support is limited, for example if you take open source ESBs, main alternatives ServiceMix, Mule or WSO2 do not support it
• compensations
  • implementing the handling of compensations seems very complex to me. What do we do if service A succeeds and we never get an answer from service B and do not know if it failed or succeeded? Handling such logic manually (as implementor of compositing services) makes me want to slit my wrists - this is the kind of work some tool should do for me!.
  • I also dont see how you can have compensation methods in non-trivial services. Say your service A is "depositMoney()" and that succeeds, some other action quickly transfers the money elsewhere, and then we receive "compensateDepositMoney()", what do we do now? Seems like a big can of worms.

To me it seems that service composition is such a fundamental SOA principle that you really do not get the benefits of SOA if you cannot (conveniently) compose services. What's the reality? 90% of SOA users use "criplled SOA" without real service componsition? Or most users do in fact use service composition and I am exaggerating the difficulty of it?

Answer 1

The short answer is Yes!

I have seen this done at several large financial organisations, and, it has worked well.

The transaction issues are complex but usually handled by (expensive) middleware such as Oracles WebLogic EAI or IBMs Websphere ESB.

Answer 2

Yes, it can be made to work in practice. However, it may not be the best approach and is perhaps used as the default option more than it should. In my opinion, SOA became popular as a way of integrating legacy systems as organisations evolved their IT to automate ever larger tasks. It can be very messy but possibly worth it if legacy systems can be re-used. If you are lucky enough to be starting a green field project, other approaches should be considered before assuming it is the best way to go.

To answer some of your more specific concerns...

You can use transactions:

1. WS-TX is a PITA and I would avoid it.
2. All your services may be running in a single application server, in which case you can span them all with an XA transaction. This is why such things as application servers were invented.

Considering the compensation based approach:

Compensating actions only need to be considered when there has been a failure. Does the service composition tool have a flag it can control that is cleared when running a workflow step the first time, but set on subsequent invocations? Like the possible resend flag in JMS. Then you can use what is called a 1.5 phase commit, which basically means go ahead if the flag is clear, but if the flag is set, first make a call to check if the state is already updated and needs to be made a second time. This still requires manual error handling, and can be complex or even impossible as you point out.

In some situations is doesn't matter:

This is the eventually consistent approach. Suppose one service sends an email. If the service composition fails and is re-started, the email gets sent again. That is slightly annoying for the recipient, but they probably realize its a duplicate and everything can continue without much of a problem.

You could also keep a log of emails sent, and use that to de-dup when the flag is set, and in that way only send the email once.

You can use asynchronous messaging to break transactions up into smaller pieces:

Consider a JMS queue which is transactional. Your service coordinator can update its state and post a message to the queue in a single Tx. Downstream services can take messages off, and update their state in a single Tx. Now you are coordinating services in multiple units of work, but each is atomic. If anything fails and has to be restarted no problem.

This still means that you are doing XA transactions over the database and a JMS queue, but this can still be efficient using the last resource gambit to avoid the full over-head of XA.

Alternatively you can use this pattern but with a 1.5 phase commit to the database and JMS. OR you can run JMS without transactions but make it more reliable by clustering.

Async messaging can also help to decouple systems, as producers and consumers can become more independent of each other, reducing the amount of coupling in the over-all system, and making it more flexible. This kind of decoupling is pretty much essential when it comes to large organisations with lots of diverse services.

Answer 3

No, it's a myth. This is wrong intention to have when designing service architecture. There is a whole bunch of problems:

1. Very tight coupling. If one service was changed you need to test the whole system.
2. Such services are very fine-grained so there is a lot of internal communication.
3. As a result of being fine-grained there are a lot of the services. System’s getting hard to understand, the queries are getting harder to track.
4. Entity-services are poorly encapsulated: none of the business rules are checked there, this logic is in operation services. So any service can call any entity-service and update its data with common update-query that is present in its interface. Such kind of entity services are often referred to as data-centric — as opposed to process-centric, behaviour-centric services.
5. The innate nature of communication between these services is synchronous. So the chances are that the transport chosen would be http. Hence all its drawbacks that I’ll talk about a bit later.

There are even more wrong ways to approach service architecture. So be cautious.