How to avoid oscillation by async event based systems?

Question

Imagine a system where there are data sources which need to be kept in sync. A simple example is model - view data binding by MVC. Now I intend to describe these kind of systems with data sources and hubs. Data sources are publishing and subscribing for events and hubs are relaying events to data sources. By handling an event a data source will change it state described in the event. By publishing an event the data source puts its current state to the event, so other data sources can use that information to change their state accordingly.

The only problem with this system, that events can be reflected from the hub or from the other data sources, and that can put the system into an infinite oscillation (by async or infinite loop by sync). For example

A -- data source
B -- data source
H -- hub

A -> H -> A -- reflection from the hub
A -> H -> B -> H -> A -- reflection from another data source

By sync it is relatively easy to solve this issue. You can compare the current state with the event, and if they are equal, you don't change the state and raise the same event again.

By async I could not find a solution yet. The state comparison does not work by async event handling because there is eventual consistency, and new events can be published in an inconsistent state causing the same oscillation. For example:

A(*->x) -> H -> B(y->x)
    -- can go parallel with
B(*->y) -> H -> A(x->y)
    -- so first A changes to x state while B changes to y state
    -- then B changes to x state while A changes to y state
    -- and so on for eternity...

What do you think is there an algorithm to solve this problem? If there is a solution, is it possible to extend it to prevent oscillation caused by multiple hubs, multiple different events, etc... ?

update:

I don't think I can make this work without a lot of effort. I think this problem is just the same as we have by syncing multiple databases in a distributed system. So I think what I really need is constraints if I want to prevent this problem in an automatic way. What constraints do you suggest?

Answer 1

If you have designed your system such that a data source changes state (and fires an event) for every event that it receives, then you have an inherently instable (oscillating) system if that data source is involved in a loop of events.

The only ways to avoid oscillations in such a system are:

1. Somehow ensure that the events received by such a data source don't originate from itself (directly or indirectly through other events). This is harder than it seems and the trouble is that you generally don't see an error until the oscillations actually happen.

2. Ensure that each data source has some combinations of states and events where it either ignores the event or at least doesn't fire a new event.
   So, in the case of the simple reflection of event E, data source A should ignore event E if it is received in state x.
   In general, in every event-driven system there will be a number of events that don't make sense when received in certain states. It is those events that should dampen out any oscillations that occur.

The main source for oscillations that remains would be two (external) events that occur at the same time and try to put the system into different states. These kinds of oscillations are inherent to the system and will have to be addressed on a case by case basis. Generally, that would involve preferring one state over the other and ignoring the events that would trigger the oscillations when received in the preferred state.