What changes are too big to be made easy by proper design?

Question

This is a rather vague question, but it's something I've never felt has been answered in a satisfactory way when reading about proper design.

Generally, when learning about Object Oriented programming, abstraction, factoring out, etc, the holy grail of design - and the reason they always claim you're using the development techniques in question - is that it will make your program "easy to change", "maintainable", "flexible", or any of the synonyms used to express such a productive-sounding concept. By marking ivars private, splitting code up into many small, self-contained methods, keeping interfaces general, you supposedly gain the ability to modify your program with total ease and grace.

For relatively small changes, this has worked well for me. Changes to internal data structures used by a class to increase performance has never been a major difficulty, and neither have changes to the user-interface end, independent of the API, such as redesigning a text entry system or overhauling the graphics for a gameplay element.

All of these changes seem inherently self-contained. None of them involve any changes to the behavior or design of the component of your program being modified, as far as outside code it concerned. Whether or not you're writing procedurally or in an OO style, with large functions or small, these are easy changes to make even if you have only a moderately good design.

However, whenever the changes become big and hairy - that is, changes to the API - none of my precious "patterns" ever come to the rescue. The big change remains big, the affected code remains affected, and many hours of bug-spawning work lie ahead of me.

So, my question is this. How big a change does proper design claim to be able to facilitate? Is there some further design technique, either unknown to me or that I have failed to implement, that truly does make sticky-sounding modification simple, or is that promise (which I have heard made by so many different paradigms) merely a nice idea, completely disconnected from the immutable truths of software development? Is there a "change tool" I can add to my toolbelt?

Specifically, the problem I am facing that's led me to the forums is this: I've been working on implementing an interpreted programming language (implemented in D, but that's not relevant), and I've decided that my closures' arguments should be keyword-based, rather than positional as they currently are. This requires modifying all existing code that calls anonymous functions, which, luckily, is rather small because I am early on in the development of my language (< 2000 lines), but would be enormous if I had made this decision at a later stage. In such a situation, is there any way that by proper foresight in design I could have made this modification easier, or are certain (most) changes intrinsically far-reaching? I'm curious if this is in any way a failure of my own design skills - if it is, I'd be very eager to improve them with the necessary reading material if there is a way to ameliorate the severity of the problem.

To be clear, I am in no way skeptical of OOP or any of the other patterns commonly in use. For me, however, their virtues are in the original writing, rather than the maintaining, of code bases. Inheritance allows you to abstract repetitive patterns out well, polymorphism allows you to separate code by human-understood function (which class) rather than by machine-understood effect (which branch of the switch statement), and small, self-contained functions allow you to write in a very pleasant "bottom-up" style. However, I'm skeptical of their claims of flexibility.

Answer 1

Sometimes a change is large enough that you have to design a migration path. Even if the start and end points are well designed, you often can't just switch cold turkey. A lot of otherwise good designers fail to design good migration paths.

This is why I think every programmer should do a stint writing software for 24/7 production environments. There's something about losses being quoted on the order of your annual salary per minute that motivates you to learn to write robust migration code.

What people naturally do for a large change is rip out the old way, put in the new way, then spend a couple days fixing a bazillion compile errors, until your code is finally compiling again so you can start testing. Since the code was untestable for two days, you suddenly have a huge mess of entangled bugs to sort out.

For a large design change like changing from positional arguments to keyword, a good migration path would be to first add support for keyword arguments without removing positional support. Then you methodically change the calling code to use keyword arguments. This is similar to how you went about it before, except this time you can test as you go along, because the unchanged calling code still works. Because your changes are smaller between tests, the bugs are easier to fix earlier on. Then when all the calling code has been changed, it's trivial to remove the positional support.

This is why publicly published APIs deprecate the old methods instead of removing them cold turkey. It provides a seamless migration path for calling code. It might feel like more work to support both for a while, but you make up the time in testing.

So to answer your question as originally phrased, there are almost no changes that are too big to be made easier by proper design. If your change seems too big, you just need to design some temporary intermediate stages for your code to migrate through.

Answer 2

I'd recommend you to read the Big Ball of Mud essay.

Basically the point that the design tends to deteriorate as you progress with development and you have to dedicate work towards containing the complexity. The complexity itself can't be eliminated, only contained, because it's inherent in the problem you are trying to solve.

This leads to the main principles of agile development. The two most relevant are "you ain't gonna need it" telling you not to prepare the design for features you are not about to implement yet, because you are unlikely to get the design right anyway and "refactor mercilessly" telling you to work on maintaining sanity of the code throughout the project.

It seems that the answer is that no design is able to facilitate any change beyond contrived examples designed specifically to show that the design is stronger than it really is.

On a side-note you should be sceptical about object oriented programming. It's a great tool in many contexts, but you need to be aware of it's limits. Especially misuse of inheritance can lead to some unbelievably convoluted code. Of course you should be equally sceptical about any other design technique. Each has it's uses and each has it's weak points. There is no silver bullet.

Answer 3

In general terms, there's "pieces of code" (functions, methods, objects, whatever) and "interfaces between pieces of code" (APIs, function declarations, whatever; including the behaviour).

If a piece of code can be changed without changing the interface that other pieces of code depend on, then the change will be easier (and it doesn't matter if you use OOP or not).

If a piece of code can't be changed without changing the interface that other pieces of code depend on, then the change will be harder (and it doesn't matter if you use OOP or not).

The main benefits of OOP are that the public "interfaces" are clearly marked (e.g. an object's public methods) and other code can't use internal interfaces (e.g. the object's private methods). Because the public interfaces are clearly marked people tend to be more careful designing those public interfaces (which helps to avoid the harder changes). Because the internal interfaces can't be used by other code you can change them without worrying about other code depending on them.

Answer 4

There is no design methodology that can save you from the hassles of a big requirements/scope change. It is just impossible to imagine and account for all possible changes that could be thought of at a later date.

Where a good design does help you is in helping you understand how all the parts fit together and what will be affected by the proposed change.
Additionally, a good design can limit the parts that are affected by most of the proposed changes.

In short, all changes are made easier by a good design, but good design can't turn a big change into a small one. (a bad/no design on the other hand can turn any small change into a big one.)

Answer 5

Since design is intended to take a development project from zero blank-sheet nothing to a working reliable final product (at least the design for one), and that is the biggest change to a project there can be, I doubt there's such a thing as a change too big to handle.

If anything it is the reverse. Some changes are too small to bother with any "design" or fancy thinking. Simple bug fixes, for example.

Remember that design patterns are to help think clearly and find good design solutions to common situations, like creating huge numbers of small identical-type objects. No list of patterns is complete. You, and all the rest of us, will have design problems that aren't common, that won't fit any pigeon-hole. Attempts to make every little move in a software development process according to one official pattern or another, is an overly religious way to do things, and leads to unmaintainable messes.

All work in software is naturally bug-spawning. Football players get injuries. Musicians get calluses or lip spasms depending on their instrument. (If you get lip spasms while playing guitar, you're doing it wrong.) Software developers get bugs. We love to find ways to reduce their spawning rate, but it'll never be zero.

Answer 6

The cost of sweeping changes are often proportional to the size of the code base. Therefore, reduction of the size of the code base should always be one of the goals of any proper design.

In your example, proper design has already made your job easier: having to make changes throughout a 2000 lines of code base, instead of a 20000 or 200000 lines of code base.

Proper design reduces sweeping changes, but do not eliminate them.

Sweeping changes are quite easily automatable if there are proper support from language analysis tools. A blanket refactoring change can be applied to the whole code base in a search-and-replace style (while properly respecting the language rules). The programmer only need to make a yes/no judgement for every search hit.