What process do you normally use when attempting to debug a problem/issue/bug with your software? [closed]

Question

Most people seem to treat debugging as an art, rather than a science. For those here which treat it as a science, rather than an art - what process(es) do you normally use when faced with a new issue/bug/problem?

Answer 1

In very general terms, what I do is:

1. Try to isolate the problem. Think of what has changed when the bug first appeared. What where you working on? What part of the code were you changing? 99% of my bugs are solved this way. It's usually something silly.

2. If I have a guess about where the problem is, take a good look at the code that seems to be the cause. Read it. Read it aloud even. Ask myself: "What am I trying to achieve?". For some types of problems: Could it have some side effects or could it be affected by code in some other place in a way I hadn't thought of?

3. Try in various ways to analyze what goes wrong, where and when (see below).

4. If I still have no clue, I check if an older version of my source has the same problem, try to find when in my development timeline the problem first appeared. To do this you need to work with a good version control system, such as git (git has a feature called bisect exactly for this kind of debugging).

5. If still no clue, take a break....it actually often helps.

6. Go back to the drawing board - review how your program is supposed to work and whether that actually makes sense.

It really depends on the kind of problem, but assuming I have a general idea of where the problem might be, then:

• If I suspect the problem is in some part of the code / recent change, I try first to remove / comment out / change or whatever to get the bug to disappear by making the code simpler, and then bring back the problematic code and take a good look at it.

• Run a debugger with breakpoints (if possible at all) and take a look at how my data looks trying to find when it starts acting bad, to get a better idea of where things go wrong.

Answer 2

I try to use test-driven development (TDD). I write a test that replicates the bug, then try to get the test to pass. Sometimes the act of writing the test helps to find the bug.

This keeps me out of the debugger most of the time, and provides regression tests to prevent reintroducing the bug.

Some links:

• Find a BUG, Write A TEST
• TDD and Debugging

Answer 3

There are a number of definitions for the word science, but it sounds like you are possibly referring to what may be more accurately termed the "scientific method". The scientific method might be summarized as observing some phenomena (presumably a bug or unexpected program behavior), formulating a hypothesis or hypotheses to explain the behavior, and the most likely experimenting to prove it (writing a test that reproduces the problem reliably).

The types of bugs (phenomena) that can occur are practically endless and some don't necessarily require a well-defined process. For example, sometimes you observe a bug and you instantly know what caused it simply because you are very familiar with the code. Other times, you know that given some input (action, series of steps, etc.), an incorrect result happens (crash, bad output, etc.). For those cases, it often does not require much "scientific" thinking. Some thought can help reduce the search space, but a common method is simply to step through the code in a debugger and see where things went awry.

The situations, though, that I find most interesting and possibly worthy of a scientific process are where you are handed some end result and asked to explain how it happened. An obvious example of these is a crash dump. You can load the crash dump and observe the state of the system and your job is to explain how it got in that state. The crash (or core) dump may show an exception, deadlock, internal error, or some "undesirable" state as defined by the user (e.g., sluggishness). For these situations, I generally follow steps along these lines:

• Narrow Observation: Study information directly surrounding the specific problem if applicable. The obvious things here are the call stack, the local variables if you can see them, the lines of code surrounding the problem. This type of specific location study is not always applicable. For example, studying a "slow" system may not have an obvious starting location like this, but a crash or internal error situation will likely have an immediate and obvious point of interest. One specific step here might be to use tools such as windbg (run !analyze -v on a loaded crash dump and look at what it tells you).

• Wide Observation: Study other parts of the system. Examine the state of all threads in the system, look at any global information (number of users/operations/items, active transactions/processes/widgets, etc.), system (OS) information, etc. If the user provided any external details, think about those in conjunction with what you have observed. For example, if they told you that the problem occurs every Tuesday afternoon, ask yourself what that could mean.

• Hypothesize: This is the truly fun part (and I'm not being facetious about it being fun). It often requires a great deal of logical thinking in reverse. It can be very enjoyable to think of how the system got into the current state. I suspect that this is the part that many people think of as being an art. And I suppose it might be if the programmer just starts randomly throwing things at it to see what sticks. But with experience, this can be a fairly well-defined process. If you think very logically at this point, it is often possible to define possible sets of paths that led to the given state. I know that we are in state S5. For that to happen, S4a or S4b needed to occur and maybe S3 before S4a, etc. More often that not, there can be multiple items that could lead to a given state. Sometimes it may help to write down on a scratch pad a simple flow or state diagram or a series of time-related steps. The actual processes here will vary greatly depending on the situation, but serious thought (and re-examination in the previous steps) at this time will often provide one or more plausible answers. Also note that an extremely important part of this step is to eliminate things that are impossible. Removing the impossible can help trim the solution space (remember what Sherlock Holmes said about what's left after you eliminate the impossible).

• Experiment: In this stage, try to reproduce the problem based on the hypotheses derived in the previous step. If you did the serious thinking in the previous step, this should be very straightforward. Sometimes I "cheat" and modify the code base to help a given test. For example, I recently was investigating a crash that I concluded was from a race condition. In order to verify it, I simply put a Sleep(500) between a couple of lines of code to allow another thread to do its bad stuff at the "right" time. I don't know if this is allowed in "real" science, but it is perfectly reasonable in code that you own.

If you succeed in reproducing it, chances are you are nearly done (all that's left is the simple step of fixing it ... but that is for another day). Be sure to check the new test into the regression test system. And I should point out that I intended that previous statement about fixing it being simple to be tongue-in-cheek. The finding of a solution and implementing it can require extensive work. It is my opinion that the fixing of a bug is not part of the debugging process but is, rather, development. And if the fix is at all involved, that it should require some amount of design and review.

Answer 4

Try to reduce the test case. When it's small enough it usually is easier to locate the corresponding code that is causing the problem.

It is likely that a new check-in is causing the problem and the previous daily build was fine. In that case your change-log from the source control should help you decide whom to catch.

Also, if you are into C/C++ consider running valgrind or purify to isolate memory related issues.

Answer 5

The hardest part of debugging is isolating the problem, particularly when the problem is burried beneath several layers. At college I studied music recording, and oddly enough there was a Studio Electronics class that directly applies here. I'm going to use debugging a studio environment as an illustration of the systematic debugging process.

1. Test your meters. Using a test tone at a known calibrated voltage, the meter should read "U" (unity gain). Translation: If your tools are broken, you can't use them to figure out what else is wrong.
2. Test each component/gain stage working backwards from the end. Using the same test tone applied to the input of the stage, there should be no change at the output of the stage. Translation: By isolating each object from the output backwards we are building trust in our code until we find the spot where it's messing up. If it takes a few layers for your tools to signal the problem, you need to know that the layers in between aren't contributing to it.

Debugging code really isn't so different. Debugging is a lot easier when the code is throwing an exception. You can trace backwards from that exception's stack trace and set break points at key positions. Usually just after you set a variable, or on the line that calls the method that throws the exception. You might find that one or more of the values are not right. If it's not right (a null when there shouldn't be, or the value is out of range), then its a process of discovering why it's not right. The break points in an IDE are the equivalent to electronic test points (designed for a meter's probe to check the circuit).

Now, once I've gone through that hard part of discovering where my real problem is, I'll write some unit tests to check for that in the future.

Answer 6

With the nasty bugs that I struggle to track down late in the afternoon, my most effective strategy is to stand up and walk away for a few minutes. Usually new ideas about possible sources of error start flowing in after just 30 seconds.

Answer 7

For a more practical aproach:

1. If the bug is related to an unhandled exception - look at the stack trace. Null reference, index out of bounds etc. and your own defined exceptions are the most common, you can assign this bug to a junior dev, its probably easy and a good learning exp.

2. If it doesn't happen on every machine, it's probably a form of race condition / threading issue. These are super fun to track down, put your bored senior programmer on it. Lots of logging, good knowledge and good tools gets this done.

3. Another big class of bugs is when the test team or the client(s) do not like a particular behavior. For example, they don't like that you decide to display user IDs or that when searching you don't get auto-complete. These are genuine bugs, consider having better product management and devs with a broader view. It should take a developer a relatively short time to "fix" this if he build the system with expansion in mind.

4. 80% of all other bugs are solved by having a good logging systems and collecting enough info to solve them. Use build-in tracing with multiple levels of , complex logging systems like Log4Net / Log4J

5. performance bugs are a category of their own, the golder rule here is "measure first, fix later!", and you'd be surprised to see how many devs just guess where the problem is and go right in to fix it only to see later a mere 3-4% decrease in response time.

Answer 8

I have flow two approaches:

1. Divide given problem in smaller parts and then conquer each smaller parts following Divide and Conquer Paradigm.
2. Whenever I am in doubt regarding any values than I just printout the values of the variables to see what exactly is coming in and going out of the variable.

This approaches have helped me most of the times.