Should making debug-checklists be an essential part of development process? How it can be integrated with unit-tests?


Debugging checklist: Think about it as your troubleshooting checklist -- like what you do for your network connection, this time for your developers and your source code. For example if you're trying to access web via your web browser and you can't, then you'd probably go and check if you can load other websites or not, if not, then you'd check your internet/network connection, and so on.

Here if you have a team of multiple developers and you ran into a bug, you wouldn't just jump into the source code and try to debug it there, because someone else might changed the code and that might cause the problem. To spot the actual bug without a checklist, everybody needs to spend a lot of time looking at different things, probably in an unorganized way also.

For example we have a Map module in our software. If you're trying to use that module somewhere in the application and that doesn't work, then there is a small checklist to help you debug it faster:

  1. Check if the license exists in Dashboard/Settings/Map or in the database. Is that a valid license?
  2. What is the MapCenter? Is that a valid LatLng?
  3. What is the MapProjection?
  4. Can you reach the MapServer?

So, specially if you're new to the team/code, you can catch up with others much faster without spending hours trying to spot the cause of errors.

There are ways to do a better error handling -- like throw an exception for example if the MapServer is unreachable, however there are also situations that you still need to check different elements to make sure what exactly causing the error.

The question is: If I'm writing a sort function, and I know that you need to specify the correct encoding in order to get the correct result, should I write a checklist simply like this:

  1. Make sure you have set the proper encoding in configuration file.

If the above example could save myself or another developer 10-15 minutes of looking around to find the problem, should we make it mandatory for every developer to write this kind of checklists when they spot something that's potential to be source of a problem on a specific part of application later?

closed as too broad by gnat, amon, Konrad Morawski, user53019, user40980 Jan 18 '14 at 1:49

Please edit the question to limit it to a specific problem with enough detail to identify an adequate answer. Avoid asking multiple distinct questions at once. See the How to Ask page for help clarifying this question. If this question can be reworded to fit the rules in the help center, please edit the question.

  • I appreciate if somebody explains the downvote. – Mahdi Jan 16 '14 at 8:12
  • Explain what you mean by debug-checklist. You formatted this as code, so presumably you don't mean the developer just, while writing the code, making a list of possible things to check. Your question is very unclear. (It wasn't my downvote BTW.) – Ian Goldby Jan 16 '14 at 8:20
  • @IanGoldby Thanks for the explanation, I'll update it soon ... – Mahdi Jan 16 '14 at 8:26

It's a good idea, but the trick becomes where to put the checklist so troubleshooters are guaranteed to know about it and use it. For example, we follow an agile development process at work. That means we sometimes release features to testing that work, but aren't the full-featured user-friendly version we are ultimately working towards.

We invite everyone who will use the new feature to our demo. We present a troubleshooting checklist, we talk about the limitations, we answer their questions, we make the list available to everyone. Then a few days later when they actually start testing, we get a bunch of phone calls and emails and bug reports about stuff we put on the list.

As Julia pointed out, checklists in code comments or a documentation file somewhere aren't particularly helpful. That troubleshooting information needs to be baked into the code as much as possible. If the license is invalid, you at least need a big error message saying your license is invalid, and it's better if it says go here to purchase a valid license. If your map center is an invalid latitude/longitude, you at least need a big error message saying invalid latitude/longitude for map center, but it's better if you state what values are valid, like please specify a map center within the boundaries of Georgia.

In other words, it's your job to enable troubleshooting of your application without using a checklist.


I can see two aspects to this.

  1. With most functions there is a kind of contract that has to be fulfilled by the caller. This should be described in the documentation. This documentation can take on many different forms. It's very common to find comments in the source code explaining the purpose of the function and the parameters and return values. You can get software that extracts these comments and generates a hyperlinked help file if you want.

    If you can't get the Map module (from your example) to work, you'd start by looking at this documentation to see if you are using it properly.

  2. There's nothing wrong with writing a FAQ or trouble-shooting list for particular libraries, but this should be only if needed. The trouble with having a policy that every library must have such a list is that some simply don't need it and you spend effort to produce something of no value. What business would intentionally throw money away?

    By the way, checks 2 and 3 in your Map example would be covered by the function documentation. Anyone who doesn't think of check 4 probably shouldn't be working in software development in the first place. Check 1 is valid if it isn't obvious a licence is needed, and the error message from the module is useless. (But as Julia says in her answer, it's much better to make the error message more informative than to create a troubleshooting FAQ.)

I don't see any obvious way for unit testing to include checking for required documentation. This should be part of the code review, not unit testing.


There are ways to do a better error handling -- like throw an exception for example if the MapServer is unreachable, however there are also situations that you still need to check different elements to make sure what exactly causing the error.

True, but as a matter of course every time you need to delve in to find what's going wrong, you should add some code to provide an informative message by whatever means, in preference to adding the problem to a list. It makes very little sense to have to repeatedly check the same set of possible error causes, particularly as that list is likely to grow, when you can get your application to diagnose problems itself.

Not the answer you're looking for? Browse other questions tagged or ask your own question.