7

I've encountered bugs that are extremely difficult to reproduce reliably and/or explain definitively, but that appear to be solved. When this happens, how much time should I spend chasing it down?

Example: this SO question and this related jQuery forum post, which offer differing solutions. The issue was reproducible intermittently until the change I discuss in the SO question, and not at all after the change.

If I don't conclusively understand what caused the bug, can I claim with confidence that it won't come back in the future by surprise?

Improve this question

3 Answers 3

Reset to default
10

If I don't conclusively understand what caused the bug, can I claim with confidence that it won't come back in the future by surprise?

No. So your next question should be: how bad will it be if it does come back by surprise?

If the answer is "pretty bad, it will cost us millions of dollars and customers" then you need to spend some time figuring it out. Best approach is to take a test environment and revert the change that you think fixed it and nothing else.

If the answer is "well, it'll be a bit embarrassing, but nothing we can't deal with" then call it a glitch for now.

Improve this answer
3

In your particular case you had a race condition. There are two ways to fix that. One is to make sure one side always wins the race (the $(window).load() solution). The other is to make it so it works properly no matter who wins the race (the explicit image dimensions solution).

Usually best practices and coding standards arise out of the need to avoid such problems from cropping up in the first place. In this instance, the best practice of always explicitly specifying image dimensions applies. You found one very good reason why the hard way, and there are others you haven't encountered yet. Adding that practice to your coding standards should satisfy the suits.

Improve this answer
3

If you've made a change but you're not sure why it could have fixed the bug then you can't really claim to have actually fixed the bug.

There are several ways that a change could affect a bug - assuming it doesn't actually fix the problem:

  1. You've changed the timing of some asynchronous call. If it now takes more time it's no longer trying to access the same data as some other code. The bug will reappear if the timing changes.
  2. You've moved some data around in memory so it no longer overruns a buffer into code. (Obviously this is language dependent). Again the bug could reappear at any time.
  3. The order in which variables are initialised has changed and it now works because B is initialised before A.

and so on.

You can only definitively claim a bug has been fixed if you know why the problem occurred in the first place. You admit that you "don't conclusively understand what caused the bug" so you will have to do more investigation of both the problem and the fix until you do.

Improve this answer
1
  • Long ago, a programmer found out why his program worked without a splash page and not with. He was using uninitialized memory, and the three additional bytes for the subroutine call pushed the uninitialized memory being used across a 256-byte boundary, and the system set 256-byte blocks alternately to all bits 0 and all bits 1 when starting a program.
    – David Thornley
    May 6, 2011 at 21:04

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.