Alternatives to outbox pattern, does outbox pattern break clean-architecture?

Question

I've read and tried the Transactional Outbox pattern for communicating between services.

It is clear to me what are the benefits of this pattern, as it has two main parts:

• By using a transaction, we promise that if the resource was saved correctly to the DB, so does the event in the outbox table, so we know an event will be sent to the message broker.
• Using a message relay service that listens on the outbox table, guarantees the event will be sent, even if the message broker is down atm.

However, the first advantage seems to break the clean-architecture idea, since it puts the logic and responsibility in the data layer. While, as I see it, the decision to publish an event is BL.

The repo is responsible to know what to publish to the outbox table. It can be difficult to reuse code to publish events, as it needs to be encapsulated into each transaction.

edit: I've read this question: Is "Transactional outbox" an anti-pattern?, but the author asked about SRP, which pretty much hijacked the discussion toward this principle. (IMO it's not directly related to SRP, as this principle is often misused)

What I would like to do?

I would like to create the functionality to publish an event to the message broker which will be used by all the services.

Simply, the BL service will have to do:

• repo.SaveResource(resource)
• eventProvider.PublishEvent(event)

I could also use a decorator to publish events, which keeps the main BL service agnostic, as sometimes we need to publish events not necessarily when inserting the resource to the DB.

The downside of this is of course I have dual-write that the transactional outbox pattern fixes. But I wonder if it's worth it because I lose clarity and separation of concerns.

I wonder if it's not enough to just revert the operation in case the publishEvent failed

• repo.SaveResource(resource)
• eventProvider.PublishEvent(event) // fails
• repo.DeleteResource(resource)

Of course, the DeleteResource can also fail, but IMO it's not enough of an argument to use the outbox pattern.

Farther more, the transactional outbox pretty much says that whenever I save a resource in the DB, I want to publish an event as well. It's not always the case, as we have other dependencies and processes we need to do (before or after the insertion to the DB) for the resource to be active. I wouldn't want to create different methods in the repo for insertion with/without an event. We already usually have retry and revert mechanisms which can take place here as well.

Answer 1

I think the alternative is simply "don't worry about it".

Think about the problem its attempting to solve, the simplest version of which is, I update my database and crash before I send a "database updated" message, is

1. Something that should rarely if ever happen.
2. Is really just pushed down the chain by the solution
3. Can be solved with some auditing of your processes. which you will want anyway
4. Could be solved by any other "poor mans transaction" style method, say an additional status on the db which is updated after the message is sent.

For example, say my ecommerce app receives an order in the order api, saves that order to the db after checking it and then sends the "Process this order please" message.

Orders can fail for a number of machine and human reasons. Even if the program works perfectly, maybe a human in the warehouse forgets to move the parcel to the truck, maybe there is a bug in the code, maybe the network went down.

You will need some auditing reports, ie "Show all orders that haven't been processed 24h after being received". This will pick up on everything that's failed for whatever reason and allow you to remedy the issue.

Its a mistake to try and write your code to account for every failure edge case, instead. Build your solution on a reliable infrastructure and program so that you can tell where you got to in any given process and continue after a disaster.

Answer 2

I don't think the pattern breaks clean architecture... Why?

As far as I understand, the Transactional outbox pattern uses the same DB to store both models & events. That way it warranties events are published at least once only after changes to the models are committed.

In Clean Architecture, you have the concept of gateways which represent external services. DB repositories and event publishers are two examples of type of gateway.

In my opinion, when using Transactional outbox pattern, you are just choosing the implementation of the event publisher gateway.

The main issue is to keep both calls in the same transaction. However, if I remember correctly, in Clean Architecture use cases set the transaction boundaries. So, basically you just need to call both gateways inside a use case.

Additionally, (I'll quote @Greg Burghardt here):

it feels like operations which should not result in events being published are separate business operations.

That means you can have use cases that send events and others that don't (they use the event publisher gateway or not).

How can you implement this?

The straightforward solution is to just write the business logic and the event publication in the same class.

However, I believe you where on the right path when you talked about decoration.

The downside of this is of course I have dual-write that the transactional outbox pattern fixes

You could use the unit of work pattern to wrap the decorated use case and set the transactional boundary. This should prevent the double trip to the DB.

we have other dependencies and processes we need to do (before or after the insertion to the DB) for the resource to be active

Could you explain a little more about this? If the resource is active only after these dependencies & processes execute, why aren't they part of the use case?