14

As a software developer, one of my main tasks is to keep complexity under control.

However, in some projects, there is a moment when the complexity level grows so high that it reaches some kind of "no return" point. Past this moment, you can never return the project to an acceptable level of complexity in less time than you would need to rewrite everything from scratch.

Does this particular moment have a name in programmers dialect (something similar than the Godwin's law for trolls)?

--edit--

Sorry if I'm not clear. I don't think this "moment" has an official name, or is a serious metric. I was thinking about something in the spirit of the "Ballmer peak" in xkcd.

Improve this question
3
  • 1
    I see one controversy in your definition of the point in question: ...in less time that you would need to rewrite everything from scratch, which implies those who are going to rewrite the project are good enough, or at least better than those who created the mess in the first place ;)
    – mojuba
    May 5, 2011 at 10:48
  • 1
    I think one reason there's no agreed on name is that it depends on who is looking at the code. What appears hopelessly complex or unmaintainable to one developer can appear pretty reasonable to another. In severe cases, I just compile into a DLL with an "ms" prefix and say it came from Microsoft.
    – Kevin Hsu
    May 18, 2011 at 21:42
  • 1
    Maybe this would do: Technical Debt Event Horizon
    – PeterAllenWebb
    May 19, 2011 at 0:55

8 Answers 8

Reset to default
20

It is more an issue of maintainability than complexity.

The phenomenon is called a "technical debt", and once it reaches a critical level the project is on the way to bankruptcy.

Is that what you meant?

Improve this answer
6
  • Thank for your answer. I'm aware of the concept of "technical dept". Every project have some kind of technical debt. What I mean is : how do you call the moment where this debt become so hight that you would prefer throwing the project to garbage and start again?
    – Thibault J
    May 5, 2011 at 9:33
  • 3
    I like your term 'technical bankruptcy'. It suggests that just like in a real bankruptcy, you have to look carefully which parts are salvageable and which should be left behind. And maybe some debt restructuring is all that's really needed :)
    – Jaap
    May 5, 2011 at 9:43
  • 2
    @Thibault J: I do not believe there is a specific term for that moment. It's more about realizing if you're still happily before that time or sadly passed beyond it.
    – user8685
    May 5, 2011 at 10:31
  • 1
    @ Developer Art: ...realizing if you're still happily before that time or sadly passed beyond it - I think that's the key in giving a good definition of the point: a project that's gone beyond the point is one that no engineer would take over voluntarily.
    – mojuba
    May 5, 2011 at 11:43
  • 3
    I'll go for the "technical bankruptcy" term, I like it. And I'll also use your definition.
    – Thibault J
    May 5, 2011 at 12:04
16

The "point of excess complexity" is referred in English as:

OH MY GOD WHAT IS THIS CRAP.

The trouble is, this can apply to something thats actually simple, but is implemented in such a horrible way that you have the same reaction.

So telling apart something very complex from something very horrible can be difficult.

HOWEVER: What actually tends to happen to all software is process a bit like this:

Step 1: Have a nice spec, do a nice design, implement nice stuff. Everybody happy.

At the end of step 1: the developers congratulate themselves on the wonderful elegance of their design, and go away happy thinking "I have a wonderful legacy here for others to add things to in future, it will be wonderful and the world will be a better place."

Step 2: Some changes get made, things get added, new functions are included. The architecture and structure from Step 1 made this a fairly painless process. [But oops, the "cruft factor" just increased a bit.]

At the end of step 2: the developers congratulate themselves on the wonderful elegance of their design, and go away happy thinking "Gee I am so clever to have made all those allowance in Step 1. This went so well. I have a wonderful legacy here for others to add things to in future, it will be wonderful and the world will be a better place."

Step 3: More changes get made, more things get added, more new functions, a bunch of stuff gets changed, user feedback is actually being listened to.

At the end of step 3: the developers congratulate themselves on the wonderful elegance of their design, and go away fairly happy thinking "Gee this architecture is pretty good to allow so many changes to just slot in easily. But I'm a little unhappy about X and Y and Z. They could be cleaned up a bit now. But!!! Ahhh!!! I am so clever to have made all those allowance in Step 1. This went so well. I have a wonderful legacy here for others to add things to in future, it will be wonderful and the world will be a better place."

Step 4: just like step 3. Except:

At the end of step 4: the developers think: "This stuff that was so good is getting UGLY to maintain. It really needs some serious changes. I'm not really liking working on this. It needs refactoring. I wonder what the boss will say when I tell him it needs 6 weeks and there will be nothing for users to see at the end of this... but I will have got another 5 years of yummy future modification scope by doing this.... hmmm... time to go to the pub for some beer."

Step 5: A bunch of changes need to be made.

And DURING step 5 the developers say to each other: "This code sucks. Who wrote this? They should be shot. Its horrible. We HAVE TO RE-WRITE IT."

Step 5 is fatal. This is where the cruft factor has got so bad that the code can't just have a few more changes, it needs to have some BIG changes.

The trouble at Step 5 is the desire to throw it away and start again. The reason this is fatal is "The Netscape Factor". Go google it. Companies DIE at this point, because starting again means you start with about 50% assumptions instead of facts, 150% enthusiasm instead of knowledge, 200% arrogance instead of humility ("Those guys were so stoooopid!"). And you introduce a whole bunch of new bugs.

The best thing to do is to refactor. Change a little at a time. If the architecture is getting a bit tired, fix it. Add, extend, improve. Gradually. At each step along the way, test, test, and test some more. Incremental changes like this mean that 10 years later the current and original code are like grandfathers axe ("its had 10 new heads and 3 new handles but it is still grandfathers axe"). In other words, there is not much left in common. But you moved from the old to the new gradually and carefully. This reduces risk, and for customers, it reduces pissed-off-factor.

Improve this answer
4
  • I bet you will get more up vote if you shorten your steps.
    – Codism
    May 5, 2011 at 14:37
  • I have to add that most businesses do not budget for this, so refactoring is always too little too late. To manage the increasing entropy of systems is to set up, that from day 1, a budget (10%-20%) is allocated from every job for housekeeping. It's not a bug fix budget. The budget spend is decided by engineering, not management or marketing or sales. It is only used to factor out the entropy created by development, and the spend reduced as the product approaches end of life.
    – mattnz
    May 8, 2011 at 22:26
  • Agreed. Management always want to trim that kind of thing. Sometimes you can get away with hiding it (Add about 20% to the development estimate for doing anything, and when refactoring is needed - DO IT).
    – quickly_now
    May 8, 2011 at 23:19
  • 1
    There's a point where you really can't refactor. If you have several disparate business applications that depend on the same lousy interface or database, you can't fix the underlying stuff very well without breaking all of the other apps that depend on the crappy foundation. At this point you really are screwed.
    – John Cromartie
    May 19, 2011 at 1:33
2

I agree that the moment is hard to recognize, and can be avoided by proper processes. However, the question was about what to call it. In real economics, there is the concept of "diminishing returns": the point at which increasing input for one resource in a process decreases your overall profit per unit. This certainly applies to coding, and even good things like abstraction, reuse etc. have such a point of diminishing returns. The general programming-specific term is "overengineering". For someone who is prone to doing this, I like Joel's term "architecture astronaut".

Improve this answer
1

Too often good code is discarded under the false impression that the new team with new tools can do it cheaper, faster with more reliabilty, only to find that

  • The complexity is in the undocument requirements
  • The new tools are harder to use then the flash website promised
  • The new team is not as 'hot' as they thought the were

Possibly the time you have described does arrive with some code bases (I used to think so). I have never personally experianced a case of old code causing a project to belly up, or re-written code saving a project.

I do not include in this cases where metrics have been used to identify specfic problematic modules or designs, which were then culled and replaced.

Improve this answer
3
  • Well, I've seen project so f**ed up that their maintenance budget was three or four time the initial developement budget. Anyway, the term I'm looking for is not an "official" and serious thing, but more something like the "Ballmer peak" in xkcd. Sorry if I'm not very clear.
    – Thibault J
    May 5, 2011 at 9:40
  • But how did it get so f**ed up? If its because of complex requirements, bad management, over optimisic engineers, why would a rewrite fix it?
    – mattnz
    May 6, 2011 at 9:25
  • Because the team rewriting it is not the same as tho one who write it at the beginning?
    – Thibault J
    May 6, 2011 at 10:59
1

The real problem with this theoretical "moment" is that it's only ever recognized after the fact. Unless your colleagues are psychopaths, every single commit into the codebase is done with the belief that it's an improvement on that codebase. It's only looking back at the ensuing mess that you can see you've passed that "moment".

But I like that we could give it a name. "Gents," you could say, drawing your fellow developers up around you, "We've crossed the Maintainability Hellespont. Text your wife and let her know you won't be seeing her for a while."

Improve this answer
5
  • "every single commit into the codebase is done with the belief that it's an improvement on that codebase." It seems we never worked in the same companies :)
    – Thibault J
    May 5, 2011 at 13:23
  • @ThibaultJ - Perhaps your colleagues have been psychopaths?
    – Dan Ray
    May 5, 2011 at 13:29
  • @Thibault J: I believe that every single commit is done with the belief that it's an improvement on that codebase. The belief is sometimes poorly researched and unfounded, of course.
    – David Thornley
    May 5, 2011 at 14:17
  • At my last work, I don't think there's any maintenance commit that anyone ever did with the belief that it was an improvement to the codebase.
    – Bobby Tables
    May 18, 2011 at 22:32
  • Sometimes a project's requirements may change sufficiently to force replacement with a new design, but it may nonetheless be necessary to make changes to the old version. If most former users of old version will use the new system and won't need the old one anymore, it may be perfectly reasonable to produce a version which meets the requirements of those few for whom the new system is unsuitable, even if it would make the system less usable for the people who no longer need it anyway.
    – supercat
    Jun 9, 2014 at 20:59
-1

I don't know if there is a name but if there isnt any I would propose calling it "meltdown point"

Improve this answer
1
  • Or to borrow another nuclear term: critical mass.
    – John Cromartie
    May 19, 2011 at 1:34
-2

This is not a very interesting question.

Indeed it's trivial.

It's so trivial that we've evolved numerous ways to cope.

  1. Waterfall methodologies. Lots of people spend lots of time reviewing requirements and design docs to be sure that complexity is managed.

  2. Agile methodologies. Fewer people spend less time discussing what's immediately applicable to solve someone's problem and release software to them. Complexity is managed because everyone is focused on getting something released.

The only time anyone wrestles with "complexity" is because of a failure to follow the methodology and manage their time properly.

  • No detailed supervision in a waterfall methodology. They aren't forced to review intermediate work products at requirements, architecture, high-level design or detailed design reviews.

  • No sprint deadline or proper use case priorities in an Agile methodology. They aren't focused on getting something released to the user as quickly as possible.

Complexity should be limited by setting goals.

Wrestling with complexity means that goals are not set or not rewarded properly.

There's no "turning point". If complexity management is somehow an issue, something's already wrong organizationally.

Improve this answer
6
  • 1
    I don't see the point. A well-run project is very unlikely to hit the point of no return, but not all projects are well run. Some badly-run projects will succeed anyway, and the rest will fail for various reasons, sometimes hitting the complexity point of no return and sometimes not.
    – David Thornley
    May 5, 2011 at 14:27
  • @David Thornley: That is my point. The "complexity point of no return" doesn't exist. It's just plain-old bad management. There's no need for a sophisticated name or a rule. Complexity is just a symptom of bad management. Not really very interesting.
    – S.Lott
    May 5, 2011 at 14:43
  • @S.Lott: I think it exists, although not in well-managed projects. There is a horde of badly-managed projects out there, and some of them will get inside the complexity event horizon and some don't. I really don't think it's useful to lump all bad management together.
    – David Thornley
    May 5, 2011 at 16:40
  • @David Thornley: I think it's very hard to disentangle bad management (which leads to horrifying complexity) from bad management (which leads to everyone quitting). I can't see a way to tell whether a project will become too complex or just late or just incompetent.
    – S.Lott
    May 5, 2011 at 16:46
  • @S.Lott: However, there is a distinction between a project where everybody quits or suffers a major health breakdown and a project where the complexity becomes too much. There are different ways to fail, and it may be interesting or even useful to categorize them.
    – David Thornley
    May 5, 2011 at 16:48
-2

There exist methods to visualize and monitor the risk of increasing complexitiy for (big) projects and code. When they are applied reasonably hopefully a new name for the point of no return is not needed. (There is a related MOOC at openHPI: https://open.hpi.de/courses/softwareanalytics2015/)

Structural Complexity is a general design problem - not only for software design in big teams. Visualisation is the first step in structural complexity management. Matrices and directed graphs can also be used for this purpose.

The some methods to reduce structural complexity are http://www.buch.de/shop/home/suche/?fq=3540878890:

  • modularisation
  • avoidance of feedback loops
  • triangulation

Furthermore there is the concept of axiomatic design: https:\en.wikipedia.org\wiki\Axiomatic_design\ With this concept reliabiltiy problems due to unneccesary complexity can be avoided.

Thus there are a bunch of methods available. In the end it is always about the decision to think about them because a project gets big enough.

Improve this answer
1
  • This does not answer the question.
    – Hulk
    Jun 29, 2016 at 12:58

Not the answer you're looking for? Browse other questions tagged or ask your own question.