About 3 months ago, our engineering group rolled out Review Board to be used for all peer code reviews. Today, I had a discussion with one of the people involved in that process and found out that we are already looking for a replacement (possibly something commercial) because of several missing features.

One of the features that is apparently asked by many people is the ability to classify/categorize each code review comment (i.e. is it a style issue, coding convention, resource leak, logic error, crash... whatever).

For those teams that regularly practice code review, is this categorization a common practice? Do you do it? have you done it in the past? Is it good/bad?

On one hand, it gives the team some more metrics and possibly will indicate more specific areas where developers may potentially need to be trained in (at least that seems to be the argument). Are there other benefits? And on the other hand, and this is my concern, is that it will slow down code review process that much more. As a team lead, I've done a fairly large share of reviews, and I've always liked the ability, to highlight a chunk of code, hammer off a comment and move on as fast as possible. Although I haven't tried it personally, I have a feeling that expanding that combo box every time and scrolling/searching for the right category would feel like something is tripping you. Also if we start keeping metrics on this stuff, my other concern is that valuable code review meeting time will be spent on arguing whether something is a logic error or if it should be classified as a crash.

4 Answers 4


I believe that you have largely answered your own question as to the relative benefits and drawbacks.

Code reviews can often end up being quite laborious and drawn out affairs, and anything that adds to the time taken to review code is going to make it seem worse. The way I see it, you want to keep your review process very short, and not labor too heavily over the finer points. So the key is to decide what it is about the review process that offers business value to your team.

Issues of style are probably one of the items I would place as the lowest of priorities. Sure, keeping code neat and uniformly formatted can make it easier to understand, but fussing over style can also result in HUGE inefficiencies during the coding process, because worrying about how pretty the code is takes the developers thoughts away from the problems to be solved. If you are still concerned about style issues, then using a Style/Formatting checking tool (E.g.: StyleCop for C#) is a great way to leave the style-specific issues to to last moment, and take the decision-making process relating to style out of the developers hands, freeing their thinking for the more important stuff. If you don't have such a product for your language of choice, perhaps a simple parser can be written to quickly scan your code for such issues, but this should only be done if you can see that real value will be delivered as a result.

Memory Leaks and other performance specific issues should never be up to the review process to pick out. Sure, if you spot something that will obviously cause a major problem you should point it out, but it shouldn't be the purpose of the code review to track down every little memory/performance problem in your code. That's what a good profiling tool is for, and they are certainly worth every penny spent if you manage to locate a very good one for the language you are developing in.

Logic issues are problematical at best, and these are the things that can really suck up a lot of valuable time when you are reviewing code. Rather than leaving this all entirely to the code review, this is what your unit tests should be used for. Yes, even tests can be wrong, however if you develop test first, Stick to the SRP and DRY principals, refactor mercilessly and define your unit tests as a means to to validate your specifications, you'll find that you'll end up with far fewer logic-related issues. If you test after you code, you're less likely to deal with potential logic problems as they arise, and more likely to forget to test a particular pathway through your code.

So if you do everything as I have suggested here, what does that leave you to do in the code review? The simple answer is that your code review becomes a fairly simple process whereby the coder explains to the reviewer how a particular requirement has been captured in the tests, and how those tests have been applied to the problems solved. You tend to spot check the code more, and analyze the tests more thoroughly, as this is where the greatest business value can be measured, and particularly when that code needs to be maintained later. To make the test code review even more painless, using a good Behaviour Driven test framework can simplify the review greatly, as the specifications are captured in code as an almost plain English description about how the test will run. Detailed checks are performed on any code supporting the tests, and if your BDD framework produces nice text reports listing the tests in plain text story/feature statements, then the process becomes even easier still. This all adds up to an extremely efficient process that will be just as valuable as a more traditional code review, but can be conducted much more quickly and in a more focused manner, helping you to avoid becoming bogged-down in trivialities and double-checks that often lead you nowhere. This leaner approach means more time spent testing and coding, and less time on administrative processes.

So what about those metrics then? If all problems are treated simply as "bugs", then the only metric you really need concern yourself with is the number of bugs you encounter before and after release, over time, and whether the bugs identified are trending in any particular direction. If you are using even a half decent issue tracking system, all of this information will be practically at your fingertips, and you won't need to worry about whether your review process needs to identify them. At the end of the day, your team wants to do what they are good at, which is writing software, and not spend too much time on administrative issues that often only offer something of interest to 1 or 2 individuals in the company. Even the most simple cost/benefit analysis would show you that time spent in red tape is going to drive your expenditure up and tie up resources that you could better allocate to new tasks, and this is probably where the risk and drawbacks will be if your code review process becomes too inefficient.


...on the other hand, and this is my concern, is that it will slow down code review process...

If you want to keep reviews lightweight then effort consuming stuff like investigation, fixes and testing for discovered defects shouldn't be there.

  • Straightforward (dumb if you wish) option to handle that is, well, just drop it and wait until testers find and report the bug themselves. The major drawback of this approach is that some problems that are easily visible in source code may take much efforts to discover in black box testing.
    Threading issues and memory leaks often fall in this category. Data race that takes me a minute to detect by looking at code, may sit there for months, passing all the tests before it rears its ugly head.

Because of above, I never use and don't recommend an option like above.

  • Another option is to dequeue heavy work - in the sense that it goes of my reviewer working queue somewhere else. This option is one I use, the one I am quite comfortable with.
    The approach is quite simple. If I feel that problem(s) I found may take substantial effort to deal with, I simply create a ticket in issue tracker with title like "investigate and address issues pointed out in code review <review ID / URL>" and keep on going through code making comments in a free form. I don't waste time on stuff that will be done per ticket I opened - because it will enter team work queue and be prioritized and dealt with in a routine way.
    That way, I make sure that 1) review process remains as lightweight as possible and 2) problems discovered won't be forgotten.

It looks like you are looking for codestriker. It has a feature to classify problems both by mode, level and type.

For those teams that regularly practice code review, is this categorization a common practice?

Well, since I use the above tool, I am forced to do. The problem I see with that is that sometimes it can be difficult to get all right, because there are so many options.

The classification of problems provides little value to the code review, since the comment is more important. Actually, since I am forced to set 4 fields in the tool above for every comment, it became cumbersome.

Later when you check the summary of reviews, you can see that the distribution seems to have unique distribution. People tend not to make the same mistake several times.

  • thanks but I wasn't looking for a tool that can do it. I'm looking for people who used classification feature in their reviews and I want to hear their opinion on such a feature. Is it useful? Does it add value for developers and/or management? Or is attempting to classify things simply wastes time and shifts focus away from what's actually important?
    – DXM
    Mar 15, 2012 at 5:50
  • @DXM Ok, in that case I misunderstood the question. Hopefully I answered it in the edit Mar 15, 2012 at 5:57

We code review using a simple Major/Minor and Root Cause (Requirements/Design/Implementation). Keeping the number items down gives usable metrics, without the burden of being too fine grained. There's rarely a discussion over a category as there're too few to choose from.

Our review tracking is ancient, but has evolved over 20 years. The tool of "choice" is Excel, so it's already burdensome. We are switching to Gerrit for less formal code reviews (Outside of "The Process" as it fits developer needs better - with gerrit we just bang in the comment and move on. Not "metrics" capture.

I personally think that Metrics are over rated - review is about code improvement, and ask anyone who has sat in on a few, they will tell you the same things the metrics will. However, the metrics cannot be argued with. They can also be usefully manipulated as required, and management is none the wiser (I am a bit/lot of a cynic at times.)

  • Gerrit? Haven't heard of it before (apart from the fact that it is a Dutch first name for guys). Could you add a link so I could find more information? Mar 15, 2012 at 8:22
  • code.google.com/p/gerrit
    – mattnz
    Mar 19, 2012 at 5:39

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.