Software Engineering Stack Exchange is a question and answer site for professionals, academics, and students working within the systems development life cycle. It only takes a minute to sign up.
Sign up to join this community
Tools like pc-lint or QAC can be used to perform static code analysis on a code base.
In my experience the static analysis often yields a huge amount of noise, i.e. warnings on things that are not real bugs but somehow violate one of the rules in a given rule set. Turning off certain rules (either for good in the rule set or via special comments in the code) can be a real cumbersome process.
What are the real benefits of static code analysis?
I worked at a place that used a commercial static code analysis system called Coverity Prevent, and it was freaking amazing! It's really sophisticated and intelligent.
We threw about 18 GB of both open-source and proprietary C and C++ code at it, and it would trace through the code paths and quickly find subtle bugs that would take a human forever to track down. It was also great at pinpointing things that would usually be Heisenbugs.
It ran every few days against our code base, and a nice feature was that we could tell it, "This isn't really a bug," and it would remember that in the future.
The gotcha is, Coverity is really expensive. They don't publish the costs, but I get the sense that for commercial projects, it starts in the hundreds of thousands of dollars per year. But it probably saved us having to hire a whole bunch of developers and QA staff, so on the whole our management seemed to think it was a good buy.
Having had that experience, I look quite favorably on static code analysis.
When starting out with a new language it's nice to have a coach. That's how I think of static analysis tools. I write a lot of javascript and in the beginning I picked up a few bad habits mainly because I was transferring some things from earlier languages. Javascript is pretty flexible so you can get away with pretty much anything but if I'd had jslint warning me about certain patterns I would have picked up better coding patterns right from the beginning instead of having to relearn stuff later on.
Static analyzers are basically machine-assisted code reviews. They'll point-out questionable areas that might be missed during regular testing.
For example, did the author really mean to make an assignment in this conditional?
if (x = 1) {
...
}
Or perhaps a rookie confuses lexical casting:
char* number = "123";
int converted = number;
Certainly static analyzers require tweaking. Then again, revision control, wikis, bug trackers, pretty printers and other tools require some set-up as well. The bigger the project, the better the reward for initial labor.
From a consultant's perspective, every static analysis tool will have some noise but not all static analyzers are created equal. Static analysis tools like Coverity, Klocwork, Grammatech have good analysis techniques that should produce more accurate results. If you tune and tweak some more you get better results typically (after all, static analyzers have to be able to run on all different types of code from a tiny medical device to a network operating system). Defining "noise" is also dependent upon your criteria for what constitutes a fix-worthy report. On one end of the spectrum, some developers mark all reports they don't fix as "false" (even poorly written code that they don't have the time to fix) and on the other end, we worked with several mil-aero and med devices companies that made sure even false positives were "fixed" because if the code was confusing the tool, then it probably should be written more clearly so it would be more maintainable.
Some of these tools are more central analysis focused and others are more desktop focused - although all three have features that support both. As @Bob mentioned, they cost money.
In my previous company we have used the static analyzer by Parasoft. And it was believed within the team that at least 60% of the run time bugs were caught before compilation.
Static analysis can also be performed without tools, by performing manual reviews on the software code. This is often the most cost-effective way to improve code quality.
The second best option is to invest for one or more high-end static analysis tools (like earlier mentioned Coverity, or KLOCwork). Since these tools perform much deeper analyses than lint, for example, the signal-to-noise ratio is much better.
I consider using lint as the third option, because of the high noise level. Applying lint to an existing project can be a daunting task.
In general static program analysis has progressed a lot over the recent years. Current static analysis tools are capable of performing deep interprocedural analyses, an can automatically identify for example procedure pre- and post-conditions. This can be a great help for later code reviews as well.
Because of the high false positive rate, you should not be using a static analysis tool (like lint or FindBugs) for each compilation.
Rather, it's a nice sanity check to consult once you have a bug and can't figure it out. In that case, you can entertain the false positives, and you may have already narrowed down the possible sources of the error. For example, if you reproduce your bug without even executing some module, you can ignore what FindBugs says for them. This is particularly helpful when you are looking at a piece of code and think it says one thing, whereas the compiler reads it literally (such as in Java when you have an equals method that takes in the type of the class instead of Object).
You can also have static analysis tools be a part of your development process: When a developer gets a code review, they should also run FindBugs on it. In short, it's useful, but you won't use it as often as the compiler or the text editor.
