If you are doing code reviews

  • How much time do you spend on code reviews, compared to implementation?
  • How much of the changes undergo code review?
  • you think it's to much / should be more?

Are there any studies about effectiveness?

[edit] thank you all for the answers, it's hard to pick a "winner" for such a question, there is lots of valuable info in the other replies, too.

  • 2
    My answers to those three questions would be "none", "none of them" and "it should be more." :-) – Carson63000 Nov 9 '10 at 5:31
  • 4
    Good question, I'm in the process of starting a personal crusade to introduce code reviews to my work. So the answers to this could be helpful. – Kevin D Nov 9 '10 at 9:16
  • Next question is "Why" - Most think its about quality, but the big gains come when you understand the educational value of code exceeds the quality value. – mattnz May 14 at 9:26
up vote 6 down vote accepted

At my work we have the following procedure for code reviews. It has worked well for us so far, and we found it to be very time-efficient, especially in terms of man-hours. We do not really have any specific time allocated to the reviews. Every commit or merge to the trunk must be reviewed, and it takes as long as it takes for the reviewer to ok it.

Edit: The time it takes, of course, depends on the magnitude of the change. Small features and bug fixes take minutes. Large new features, refactorings, or changes that affect many parts of the system can take half a day to review and another day to address all the issues that come up as a result.

To make this system work it is crucial to commit to the trunk often, so that the changes are of manageable size. You do not want to have the situation when you have to review a year's worth of somebody's code.

  • 1
    Do you have any idea how much it takes? – peterchen Nov 9 '10 at 8:11

In regards to studies, Smart Bear software will send you a small book, Best Kept Secrets of Peer Code Review, for free. It has a number of articles in it about various aspects of code review, including studies on how much time they should take and how effective they are.

  • I've just ordered this book. Should hopefully be an interesting read. – Kevin D Nov 9 '10 at 20:43
  • 3
    Ordered it, too. It's strange to wait 20 days for a "free" book - from a company that sells software over the internet, no less. – peterchen Nov 10 '10 at 18:02
  • That book has done the rounds at my work place for a number of years. Our copy is well read and dogeared with a few postit's marking key points. Its small (Can read it in a hour) but packed with great information. The clear purpose is to convince you to do code review, then convince you to use a tool, then convince you to buy their tool - fair enough, they gave you the book. – mattnz May 14 at 9:23

In our project, every significant change to the system is reviewed by the team leader or together with another developer who's gonna be the main "consumer" of the new module. We talk on skype and either use Rudel in Emacs (a plugin for collaborative editing, basically it allows several users to edit the same file live), or TypeWith.me (Piratepad), or one of us shares his screen in skype.

It's hard to quantify this, because mundane changes, like new views, pages, etc. are not reviewed. We do review new modules, major updates and refactorings. As for big changes, code review can take from 10% to 30% of time, but it's worth that.

I can say pair programming, when 2 programmers do edit the same file at the same time, not just sit at the same computer, is a lot better than the usual office practice of sitting behind one's shoulder.

For simple things like naming conventions and scope errors we use our own or open source automatic tools (jslint, pylint, pyflakes, pep8). And we don't limit commits and pushes: we use Mercurial which has very easy branching and merging (I have to say, easier than in Git). Bugs are not a code review matter.

We do team meetings where the changes and new things are announced, but there, not everyone really pays attention. Probably we should do a bit more code reviews.

There is no right or wrong answer to this. But as many before me have suggested - if code review is done by an external team member [highly recommended method] it might amount to approximately 30% to 35% of the development effort. But if this is done by an internal project team member who was part of the development team [not recommended] this can be completed in approximately within 20% of the time taken for development effort.

Note: I came across this thread when searching for something else and I thought I might be able to add some value here. Btw, all this above effort estimate is based on my work experience as a engagement manager across multiple projects. Hope it helps.

  • thanks - and no sweat, I appreciate seeing numbers for a "how much" question! – peterchen May 14 at 8:17

Every organisation and code base is different, so it is difficult to get an industry wide value.
If you are really serious then you should start collecting metrics. I.e. Do the code review until is satisfactorily done including rework. Start collecting this in a database (LOC, code complexity, programming language, time etc.). Then also collect metrics on your defect rate during testing. As long as you can reduce this code review should pay by itself. If defect comes back from testing then collect metrics on how much time was spent on fixing defects. Build these data in your organisation, create baselines, and you can predict it quite accurately. The terms to search for further learning is Cost of Quality and Cost of Poor Quality.

The only caveat is that this can start to become bureaucratic and depends on organisation culture.

  • I am looking for some practical data, so I get any idea of the range expected. – peterchen Nov 9 '10 at 11:58

protected by gnat May 14 at 6:09

Thank you for your interest in this question. Because it has attracted low-quality or spam answers that had to be removed, posting an answer now requires 10 reputation on this site (the association bonus does not count).

Would you like to answer one of these unanswered questions instead?

Not the answer you're looking for? Browse other questions tagged or ask your own question.