Should a bug fixer know the overall structure what the program does?

Question

I heard that if you maintain a program, you have to study what the code does. Another engineer said it is not necessary. Just use test cases to find out the function that does not work well, modify the code so it handles the new case, as well as the previous ones, and release it. So when fixing bugs, do you have to focus on the bug or think about how the edited code affects the program overall?

Answer 1

In order to fix a broken program you need to know both what the program actually does and also what it is supposed to do. The latter comes from some sort of specification, which might, itself, be either executable or prose.

Finding the function that "does not work well" requires knowledge of what it is supposed to do.

It is also a bit (or more) naive to think that an error involves only a single function (or other component). Many errors, including the more sophisticated ones come from faulty interactions between parts.

But generally, a component "works well" if it contributes correctly to the overall design, which implies knowledge of the specification.

In very large and well designed programs, it may be possible to isolate the effect of errors to some subset of components, in which case only the specification of that subset may be necessary to make a correction.

But, small changes without considering the overall effect is quite likely to introduce additional bugs due to unforeseen and unintended interaction effects. This leads to "thrashing" where the bug "fixes" make it worse and worse. A microscopic view is very seldom enough.

FWIW, I'm working on such a situation now. A seemingly small change is leading to a lot of analysis.

Answer 2

Complete understanding of a large piece of software may be impossible if it is very large. So you end up having to fix bugs on the assumption that your local knowledge of a particular subsystem is adequate.

A major challenge of software architecture is how to structure a system so that a change intended to be local will have no unexpected non-local effects. Since this is hard to do, you may find that locally testing the function isn't sufficient.

In summary: do the best you can with the information available.

Answer 3

There are two parts to fixing a bug: One, find out what goes wrong, and change some code so it doesn’t go wrong. Two, make sure that your code change doesn’t affect any other code in your application and makes it go wrong.

For that you don’t need to know the complete structure of the program but you need some way to find all places affected. Sometimes a refactoring tool may help: You may be able to select “rename method” and a refactoring tool will show you all the places in your complete application where that method is used (and not an unrelated method that has the same name by coincidence). Doesn’t work well if you change code in a library.

Answer 4

You can sometimes just focus on the test cases, but in reality the one and only thing you care about is "does the program perform correctly for the user." Tests are only a proxy for this. They make it easier to locate and isolate the bugs, but the end goal is never just to make the tests pass.

As a redactio ad absurdum, consider how successful you would be if you simply recoded the entire unit under test to do nothing except spit out the right answers to the unit tests, effectively making it one big mock. Sure, it would pass the tests, but it would certainly not be an acceptable job.

We use tests to isolate the bugs as best as we can, and that makes your job easier. If most of the behavior you need to worry about is well defined in requirements on the unit under test, then you don't need to worry much about the behavior.

Other bugs are more subtle. What may pass in the tests may fail when you get to integration testing. As an example, one of the people I work with has been working on a library that is steeped in mathematics. All of the equations work flawlessly in math with real numbers, but we don't have real numbers in computing. We have floating point numbers. Many of the bugs he is working on are numeric stability issues. These are delicate holistic beasts. As he described it, he often is playing "whack a mole," where the resolution of one stability issue introduces another subtle one elsewhere in the pipeline. For the problems he is solving, he must be aware of the functionality of the whole application.

Answer 5

The short answer here is that it depends on the bug. Some bugs are of a technical nature (e.g. an index going out of range), whereas other bugs are a misapplication of the intended behavior. The former is probably solvable by any competent developer even if they don't know the application requirements, the latter is probably going to require intimate knowledge of the application requirements to solve.

Additionally, developer seniority and expertise makes a difference. A more experienced developer is likely going to be better able to learn the codebase on the fly than a more junior profile.

The cleanliness of the codebase can further help a developer make sense of the intention of the code without already having prior knowledge.

Overall, many things go into the consideration whether (a) a particular developer is able to go into (b) a particular codebase in order to solve (c) a particular bug. There is no "one size fits all" answer here.

Another engineering said it is not necessary. Just use test cases to find out the function that does not work well, modify the code that it handles the new case as well as the previous ones and release it.

Depending on how far they take this advice, there's two possible answers here.

One interpretation is that they're describing the process of studying what the code does. They're just doing it by reading the tests instead of talking to another dev or reading some documentation, but in actual fact they are "studying what the code does" (which is what they claim you don't need to do).

Another interpretation is that they're not reading the tests to gain understanding, they're just shotgun debugging:

The making of relatively undirected changes to software in the hope that a bug will be perturbed out of existence.

There's a lot to be said about shotgun debugging. It's stupid, but sometimes it does work. When you're stuck and you don't know what to do, shotgun debugging is a great way to try and shake something loose. I'm a firm believer of "if it's stupid and it works, it ain't stupid", which is why I picked the phrasing that I did.

However, just because it might not be stupid does not mean that it is therefore an efficient approach. When the alternative is being stuck, going slowly is still better than standing still. However, if there are smarter ways to skin this proverbial cat, they will almost always be faster and more reliable than shotgun debugging will be.

So it's not that you should never do it, but it should literally be your last resort when all else has failed you.

Additionally, your coworker's assumption that they can divine the source of the bug based on playing around with the test almost entirely relies on the assumption that the test was well written to begin with. This refers back to my original point: in addition to (a), (b) and (c), it very much depends on (d) the particular test suite you find yourself working with (which may or may not be a subset of (b)).

Answer 6

If there is high coupling, then you have to understand it all. If low coupling then you don't.

When you make a change, any change, reduce coupling a bit. Not usually a complete rewrite, but make the bit that you worked on better. You may introduce more bugs (5 or 6 compared to 1 or 2), but they will be easy to remove, and you probably won't introduce more bugs at this stage.