What is the "Fix Everything" design pattern?

Question

In this 2003 article by Stephen Figgins on linuxdevcenter.com, Bram Cohen's BitTorrent is described as using the "Fix Everything" design pattern.

A less common approach that both makes BitTorrent harder to grasp, but worthy of study, is Cohen's use of idempotence. A process is idempotent when applying it more than once causes no further changes. Cohen says he uses a design pattern he calls "Fix Everything," a function that can react to a number of changes without really noting what all it might change. He explains, "you note the event that happened, then call the fix everything function which is written in this very idempotent manner, and just cleans up whatever might be going on and recalculates it all from scratch." While idempotence makes some difficult calculations easier, it makes things a little convoluted. It's not always clear what a call is going to change, if anything. You don't need to know in advance. You are free to call the function, just to be on the safe side.

This sounds quite nice on the face of it.

However, it seems to me that calling an idempotent "fix everything" function would improve robustness of the system at the cost of efficiency and potentially screwing up the containing system (that might prefer processes that carefully plan and execute.).

I can't say that I've used it before, though. I also cannot find the source for his application online (but I did find this one that claims to be based on it.). Nor can I find reference to it outside of this article (and I consider my google-fu to be pretty good) but I did find an entry for "Idempotent Capability" on SOApatterns.org.

Is this idea better known by another name?

What is the "Fix Everything" design pattern? What are its pros and cons?

Answer 1

Let's say you have an HTML page that is fairly complicated-- if you pick something in one dropdown, another control might appear, or the values in a third control might change. There's two ways you could approach this:

1. Write a separate handler, for each and every control, that respond to events on that control, and updates other controls as needed.

2. Write a single handler that looks at the state of all the controls on the page and just fixes everything.

The second call is "idempotent" because you can call it over and over again and the controls will always be arranged properly. Whereas the first call(s) may have issues if a call is lost or repeated, e.g. if one of the handlers performs a toggle.

The logic for the second call would be a bit more obscure, but you only have to write one handler.

And you can always use both solutions, calling the "fix everything" function as needed "just to be on the safe side."

The second approach is especially nice when state can come from different sources, e.g. from user input versus rendered from the server. In ASP.NET, the technique plays very well with the concept of postback because you just run the fix everything function whenever you render the page.

Now that I've mentioned events being lost or repeated, and getting state from different sources, I'm thinking it is obvious how this approach maps well to a problem space like BitTorrent's.

Cons? Well the obvious con is that there is a performance hit because it is less efficient to go over everything all the time. But a solution like BitTorrent is optimized to scale out, not scale up, so it's good for that sort of thing. Depending on the problem you are trying to solve, it might not be suitable for you.

Answer 2

I think the article is a bit dated because as I read it, this isn't really a unorthodox or new idea at all. This idea is presented as a separate pattern when it really is just a simple Observer implementation. Thinking back to what I was doing at the time, I remember working on logic to sit behind a somewhat complex interface with a number of different panels with data that were interdependent. The user could change values and/or run a optimization routine and based on those actions, events were generated that the UI would listen to and update as needed. There were a number of issues during development where certain panels wouldn't update when they should. The fix (staying within the design) was to generate events from other events. Ultimately, by the time everything was working right, almost every change resulted in all the panels to refresh. All the complexity of trying to isolate when a given panel needed to refresh was for naught. And it didn't matter anyway. It was effectively a premature optimization. I would have saved a ton of time and effort by simply collapsing it all into a single event that refreshed everything.

There are innumerable systems designed in the "fix everything" or refresh everything way. Think of all the CRUD interfaces that add/update a row and then requery the DB. This isn't an exotic approach, it's just the obvious non-clever solution. You have to realize that in 2003, it was the height of 'pattern fever'. From what I could tell, people thought naming new patterns was going to be their path to fame and riches. Don't get me wrong, I think the concept of a pattern is an extremely useful one for describing solutions in the abstract. Things just kind of went off the rails a bit. It's unfortunate because it created a lot of cynicism about the pattern concept in general. It's only in this context that it makes sense to talk about this as an 'unorthodox' solution. It's similar to the orthodoxy around ORMs or DI containers. Not using them is seen as unorthodox even though people had been building software long before these tools existed and in many cases those tools are overkill.

So back to 'fix everything'. A simple example is calculating means. The simple solution is to sum numbers and divide by the cardinality of the values. If you add or modify a number, you just do it again, from the beginning. You could keep track of the sum and the count of numbers and when someone adds a number, you increase the count and add it to the sum. Now you aren't re-adding all the numbers again. If you've ever worked with Excel with a formula that references a range and modified a single value in that range, you have an example of the 'fix everything' pattern i.e. any formula that has a reference to that range will recalculate regardless of whether that value was relevant (e.g. using something like sumif()).

This isn't to say this isn't a smart choice in a given context. In the mean example, lets say we now need to support updates. Now I need to know the old value somehow and only change the sum by the delta. None of this is really that challenging until you consider trying to do this in a distributed or concurrent environment. You now have to handle all kinds of thorny timing issues and you'll likely end up creating a major bottleneck which slows down things far more than recalculating.

The upshot here is that 'fix everything' or 'refresh everything' approach is much easier to get right. You can make a more sophisticated approach work but it's a lot more complicated and therefore more likely to be flawed. In addition in a lot of contexts, the 'refresh everything' approach can be more efficient. For example, copy on write approaches are generally slower for single-threaded approaches but when you have high concurrency, it can allow you to avoid locks and therefore provide better performance. In other cases, it can allow you to batch changes together in an efficient way. So for most problems, you probably want to start with a refresh everything approach unless you have a specific reason why you can't do it and then worry about doing something more complex once you have a need. A working implementation that you can regression test against is valuable in itself, even if it's slow.

Answer 3

Not sure it's a "design pattern", but I would classify that type of behavior as end state configuration or desired state configuration, in the vein of Puppet, Chef or Powershell DSC.

Those solutions typically operate at the systems management level, not a business logic level as the question describes, but it's the effectively the same paradigm, and although such tools are usually declarative in nature, the same principles can be applied in procedural code or scripting.

Answer 4

I have mostly used this within user interfaces. The nice thing is that you write it once, and it handles everything from the simplest to the hardest case equally well (for example if the user rotates the screen, or on a laptop/desktop if the user resizes a window, and virtually everything changes).

There isn't much reason to worry about efficiency. In the user interface, the expensive things are things like redrawing an item that has been moved. The calculation where each item goes and how big it is is usually quite fast. All you have to make sure of is that whenever you find that an item should stay in exactly the place where it belongs, no code is executed to move it. The real changes are all things that you had to do anyway.

Answer 5

Sounds like reactive programming principles. "Fix everything" looks at a the current "core" state and propagates everything else that should be affected - "computed states". If you optimize this derivation it may reach high efficiency, a-la React, if done naively performance may not be optimal although it could still be fast enough.