What's the best way to prepare your design & code for those “unknown unknown” bugs from day one?

Question

I am just wondering, is there any practical method or technique or even tricks to get avoid those "unknown unknown" bugs especially those nasty and random ones which are often emerge at the last minutes or at least to keep these things at a minimum level. I ask this because when you work on a new platform or use certain new technology for the first time, how do you justify your design and code are robust enough? Or these things can only be learned by time and mistakes?

(I use C++ for most of my working time)

Thanks!

Answer 1

Almost twenty years ago, I got a lot of insight into this from David Thielen's excellent book "No Bugs: Delivering Error-Free Code in C and C++", which is now available as a free PDF.

He taught me two great ideas...

Bugs don't come from nowhere. All of us programmers sit down and write them into our code with our own fingers.

"Bug" connotes that some outside agency decided to infest your program with bugs and that if you live a clean life, and sacrifice small furry animals at the foot of your computer, they will go away... This concept is important because it colors your approach to debugging your code. If you view mistakes as "bugs," you hope none are found. (You hope the good fairy came by, sprinkled pixie dust, and the bugs left.)

Bugs should not be called bugs, they should be called Massive Fuck-Ups [MFUs]... MFUs exist because programs are written by people, and people make mistakes... You will write MFUs. You will sit down and with complete malice of forethought put MFUs in your code. Think about it - you know that you are the one putting the bugs in there. So if you sit down to code, you will be inserting some bugs.

Since it is all programmers' inescapable destiny to write bugs, I need to code defensively, including things that will jump up, scream, and wave red flags when they detect a bug.

Having been written in the early 90s, the specifics on this in Thielen's book are rather dated. For instance, on Linux and Mac OS X, you no longer need to write your own wrapper for the C++ new operator; you can use valgrind for that.

But there are a few things I routinely do for C/C++/ObjC:

1. When I reasonably can, turn on the compiler's "Warnings are errors" option, and fix them all. (I maintain one legacy project where fixing those all at once would take weeks, so I just fix a file every few weeks - and in a few years, I can flip that option on.)
2. Use a static code analysis tool, like Gimpel's PC-Lint or the very nifty one now built into Apple's Xcode. Coverity is even better, but the cost is for large corporations, not mere mortals.
3. Use dynamic analysis tools, like valgrind, to check for memory problems, leaks, etc.
4. As Thielen says (and the chapter is still worth reading): Assert The World. Of course, nobody but an idiot will call your function with a nil pointer - and that means somebody, somewhere, is an idiot who will do just that. It might even be you in three years when what you were doing today has gotten foggy. So just add an assert at the beginning of the function to validate that pointer argument - it takes three seconds to type, and goes away in the release executable.
5. In C++, RTTI is your friend. Again, nobody but an idiot will call your function with a pointer to the wrong kind of object - which means that, inevitably, some idiot will - and the cost to defend against that is negligible. In C-based code derived from GObject, you can do the same thing with the defensive dynamic cast macros.
6. Automated unit and regression tests are now a key part of my repertoire. On one project, they are an integral part of the release build system, and the build won't complete unless all of them pass.
7. Another key part is logging code in both the debug and release executables that can be enabled at runtime by something like an environment variable.
8. Write defensive tests so programmers running debug executables can't ignore them if they fail. Runtime messages to the console can be ignored. The program crashing with an assert cannot be ignored.
9. When designing, provide public APIs, and private implementations that outside code can't get at. That way, if you have to refactor, nobody's relying on some magic interior state variable or something. In C++ classes, I'm a big fan of protected and private for this. I also think proxy classes are great, but don't really use them myself.

Of course, what you'll do for a new language or technology will vary in the details. But once you take into your heart the notions that bugs are Massive Fuck-Ups You Wrote With Your Own Fingers, and your code is under constant assault from an army of idiots, with you at the head as the general, I'm sure you'll figure out suitable defensive techniques.

Answer 2

Well, if you know that, then they slip into the "known unknown bugs" catagory (IE you know something of "this" nature will occur). Any amount of unit tests aren't going to catch them, they are only really useful for known cases.

The way we deal with this is to put an error logging service on the running application, report back to base when it occurs and deal with it when it comes up. Otherwise you can spend years and still not cover anything ... at some point you just wait to see what happens in the real world and evolve quickly.

Design side, you design for maintainablity as one of the key factors.

• Learning patterns which work and patterns to avoid. The more consistant and coheseive patterns you have the more comfortable you can be that a specific class of issue will or won't occur.
• Make things obvious. Obsurity leads to confusion, leads to bugs.
• Strong naming conventions all the way through. If you name things well and consistantly there is a huge amount of benefit when you trying to change things or explain it to theres ... every thing called Factory will do X.
• Spell things out completely. We have autocomplete these days don't use acronymns when the full word removes confusion.
• Seperate out into layers or abstrations ... so specific styles of problem will occur in a specific layer rather than "somewhere"
• Isolate the classes of problem into layers and aspects. The less one part has to do with another part of the code the better in general. Even if it takes a little longer to write similar stuff out twice.

And the key one ... Find a mentor who has made all the mistakes previously OR make a mess out of several until you find out what does and doesn't work.

Answer 3

All of the above are good points. But there is something not mentioned. You need to make your modules and functions paranoid. Range test all the function parameters. Watch out for strings with blank beginnings or endings or which are too short or too long. Beware of booleans which are niether true not false. In untyped languages like PHP, watch out for unexpected variable types. Watch out for NULL.

This paranoid code is often encoded as asserts that can be disabled on a production build to speed things up. But it will definitely prevent last minute panic bugs.

Answer 4

Rob is correct in saying that unit tests will not save you from unknown bugs BUT unit tests will help save you from introducing bugs when you fix the unknown bugs and will save you from accidentally re-introducing old bugs. TDD will also forces you to design your software from the start to be testable and that has huge positive ongoing value.

Answer 5

Avoid status/"side effects" where possible. While computers are deterministic, and deliver the same output for the same input, out overview about the input is always incomplete. Sadly, most of the time we dont realize how incomplete it is.

When talking about web applications, the whole database, the current request, the user's session, the installed 3rd-party-libraries, and much more ist part of the input. When talking about threads, it's even worse: your whole operating systen, with all all other processes managed by the same scheduler is 'part of the input'.

Bugs are caused by misjudging the way the input is handled or by misjudging the input. The latter are, in my experience, the hard ones: you only can observe them 'live', often, you don't have the input anymore.

When learning new Technologies, Infrastructures, etc, imho it's best practice to get an overview, what components contribute to the input and then try to avoid as many of them as possible.

Answer 6

As your software gets more complex, it's inevitable that some bugs will occurs. The only way to avoid that completely is to only develop trivial software - and even then, you're bound to make a daft mistake from time to time.

The only practical thing you can do is to avoid unnecessary complexity - to make your software as simple as it can be, but no more simple than that.

That's basically what all the more specific design principles and patterns are all about - making things as simple as they can be. The trouble is that "simple in what way" can be subjective - do you mean the absolute simplest design for the current requirements, or simple to modify for future requirements. And there's principles for that too.

Unit tests are an example of this uncertainty. On the one hand, they are unnecessary complexity - code that must be developed and maintained, but which doesn't get the job done. On the other hand, they are a simple way to automate testing, reducing the amount of much more difficult manual testing that must be done.

No matter how much design theory you learn and how much experience you gain, the overriding principle (and sometimes the only guide you have) is to aim for simplicity.

Answer 7

There is nothing random about software bugs, root causes are perfectly deterministic in nature, incorrect instructions to the computer.

Threading bugs can be non-deterministic in behavior of execution, but the are not random in root cause.

They happen for the exact same reason just at seemingly un-predictable moments in time, but that doesn't make them random just apparently un-predictable, once you know the root cause you can deterministically predict when they will happen.

I said apparently and made the distinction for a reason. Random means one thing, Made, done, happening, or chosen without method or conscious decision, that implies there is some independent decision making on the part of the computer, there aren't it is doing exactly what it was told to do, you just didn't tell it to do the right thing in some very deterministic cases.

The semantics of words are there for a reason, random doesn't mean something different just because someone uses it in-correctly, it always means the same thing. A better term would be in unintentional or not obvious logic errors.

Considering bugs as random is almost like accepting that there is some other in-comprehensible force at work that is not fully understood acting independently of your input to the computer, and that isn't very scientific. I mean are the Gods angry and smiting your application on a whim?