Formalizing programmers errors

Question

Every one of us make errors leading to bugs. Once I wanted to start logging my errors for future analysis, probably mentioning project title, approximate time spent and the most important, the type of error. For example when I copy-pasted a fragment about 'x' and replaced every occurrence of 'x' with 'y' and forgot to replace a tiny piece, this goes to 'copy-paste error'. The usefulness of this approach depends on whether I can formalize my errors at all and probably minimizing the number of types to choose from. Otherwise I would start postponing, ignoring and so on so make this system useless.

Are there existing research in this area, probably a known minimum set of errors? Maybe some of you already tried to implement something like this and succeeded/failed?

Answer 1

What you're looking for is a taxonomy of software bugs. Here are several of such taxonomies:

• http://cs.nyu.edu/~lharris/content/bugtaxonomy.html
• http://spin.atomicobject.com/2011/05/01/taxonomy-of-software-bugs/
• http://www.mcr.org.in/sureshmudunuri/stm/unit1.php#bug_taxonomy

Here's even a thesis paper of someone who surveyed 26 different bug taxonomies.

You'll probably find that each taxonomy is a little different, with different categories, different levels of detail, and even completely different basic concepts.

I wouldn't worry too much about what categories to choose exactly, or what category to assign individual bugs too. It seems like a rather useless exercise in futility to aim for a perfect categorization of something as varied as software bugs.

Answer 2

Take a look at Brain Hayes' Debugging Myself. I'm not sure that he started out with error categories, or just determined them as time goes on.

I think this is an inherently difficult area. What do you classify as an error? I personally wouldn't classify a typo as an error, but if you're doing Personal Software Process, I'm pretty sure that you do classify misspelled key words as an error.

Answer 3

I would say that, if you want to understand your own programming errors, you would need to think about what matters to you. I personally think of my programming errors in the following categories:

1. Clerical errors. Examples are typo's, calling the wrong functions/methods, getting the order of arguments wrong etc. Typically compilers and type checkers catch such errors, or a simple exercising of the code does.

2. Misunderstandings of the program. I may not have a full understanding of the program (or its components, classes, libraries or whatever), or misunderstood them, and wrote the wrong code that doesn't fit. Usually these errors are caught by the first few cycles of routine testing.

3. Misunderstandings of the requirements. I was supposed to do X but did Y. Either I didn't understand the requirements fully or misunderstood them. These are caught by the first few cycles of user testing.

4. Logical flaws. These are the biggies. Assuming I understood everything correctly, I still ended up writing the wrong code. These are not simply "human" errors. I was either lazy, didn't think things through carefully, used the wrong algorithms or data structures that didn't fit, the problem was too hard and complicated and I didn't put enough effort into doing it right, or perhaps I wasn't up to the task. These are usually not caught by routine testing and go into deployment. Big problem. Every time I get one of these, I think long and hard about how to avoid such a thing in future.

5. Design flaws. Used the wrong data structure, wrong interfaces, wrong program structure or whatever. These things don't affect the program behaviour, but affect future development. They result in a brittle system that can't be extended or modified easily, result in more and more bugs of the previous kinds over a period of time, or simply result in a structure that I am not proud of. Some amount of painful and tedious refactoring is needed to fix these problems. Every time I get one of these, I learn something new and deep about the nature of programming.

Answer 4

The Personal Software Process emphasizes a lot of these things. Improving quality and estimation by keeping logs of the past/present. I recommend looking at sample forms for ideas.

Answer 5

None of the previous answers concern software bug causes in terms of human errors by the coder. There's lots of research into human errors in safety critical human operator areas, such as aviation pilot errors by the FAA and NTSB. There's also research into human memory and perceptual fallibility that might be applicable.

Unit testing is like the stall warning going off after the pilot has already approached a dangerous flight regime. A personal study of how not to create, or how to minimize, code that produces results for unit tests to find and fail seems at minimum interesting.

Answer 6

I did this to a very limited degree:

I had a production application with a lot of live users. The system had an in-built error logger. During critical errors, it would send detailed email reports.

Since everything was logged it was possible to see trends, and allowed developers to focus on different areas (sometimes ones we would not normally concentrate on).

SO

A more automated approach would be better.

Perhaps you could create a an application that can log and analyse compiler errors and cross reference them with Source control logs along with automatic error reports from live applications?