9

I encountered a developer who wanted some outside advice on their teams project. I found out they're developing a huge software suite for the companies executives, project manager and developers that can calculate metrics automatically and graph them per iteration.

As a student from a computer science background I know very little on metrics and their importance, but my questions are:

  1. Do most companies have some way, doesn't have to be an elegant program, to measure meaningful metrics?
  2. Which metrics, single or combined, help you narrow down your projects scope and estimates?
  3. As a person who analyzes metrics, how often do you base decisions off of them? IE. Tests failed per week is increasing drastically?
  4. Do you feel that the introduction of studying metrics has helped you understand the project better?

Not sure why but the developers project intrigued me and I must know more. If y

|improve this question
6

Some books on metrics that your college library probably has include Software Metrics and Metrics and Models in Software Quality Engineering. Those 2 should give you a starting place. In the industrial world, very few companies have any sort of metric measurement program at all.

Do most companies have some way, doesn't have to be an elegant program, to measure meaningful metrics?

Visual Studio includes some code analysis tools that can get you started. Most companies don't even have something to measure the worst metric possible: lines of code. "Just get it done" appears to be the overwhelming driving force in the industry, and concerns of maintainability are given very short attention to managers' concerns of "will I get my bonus this year?" and "will this be done in the time I promised?" Even with products that carry over from year to year with incremental changes, those 2 concerns dwarfed the developers concerns of maintainability and bug detection/prevention.

Which metrics, single or combined, help you narrow down your projects scope and estimates?

I find that cyclomatic complexity and coupling are strong indicators of how buggy or how hard to maintain the code will be. If the cyclomatic complexity is around 20, I find that it will be almost impossible to test (as it will have up to 2^20 paths through the code) and should be decomposed into smaller pieces. You cannot eliminate complexity, but you can slice it up into more manageable chunks.

If you are looking for estimation, you probably want to investigate function points.

Code coverage % is drastically lowering each iteration, do you alert your developers of the issue

I find that most managers care about number of check-ins and number of bugs getting fixed. My current manager is opposed to unit testing (he thinks it is a waste of time) and my previous manager felt that time spent on unit tests was time that should have been spent writing it in the first place.

The canonical argument used by developers is that if you measure something, that's only what you will get. This argument comes from the idea that the only metric is lines of code.

|improve this answer
  • Thanks for detailed answer and relevant links. Just as a follow up: 1. Why would a manager care about the number of check-ins? Maybe our definition of a check-in is different. 2. What do you mean by lines of code being the worst metric? Worst as in it gives no good indication about the project? – Russ K Feb 7 '11 at 0:07
  • @Russ, a developer who is not checking code in, will be seen as not working. LOC is worst in that it is trivial to game. Take a look at the difference between K&R and Allman style of indenting code: en.wikipedia.org/wiki/Indent_style . The Allman style will give a higher LOC count just by putting the open { on a separate line. Disclaimer: I hate the K&R style as I can rarely find the matching open { without spending too much time playing Where's Waldo. – Tangurena Feb 7 '11 at 0:32
  • In the industrial world, very few companies have any sort of metric measurement program at all. Any company with a CMMI rating of 2 or higher will have a measurements/metrics analysis program in place. The gathering of measurements and metrics is a Maturity Level 2 requirement. CMMI Maturity Level 4 requires quantitative project management, based on those measurements and metrics, along with things like root cause analysis to act on identified problems. There are a large number of organizations rated at CMMI Level 4 (or 5). – Thomas Owens♦ Jan 16 '12 at 18:44
2

I was at a talk about software metrics where the speaker made some, IMHO, insightful points. Having little experience with these things myself, I was still intrigues and inspired, but I cannot say if it's wrong or right.

The main ideas were:

  • No singular metric is useful in it's own right.
  • Setting an absolute target (i.e. XX % code coverage) is not meaningful.
  • A metric without history is useful.

So, to solve this:

  • Show several metrics, such as:
    • # of lines total / changed
    • # of commits
    • % code coverage
    • # of tests
    • cyclomatic complexity
    • file/package/... dependency
    • ...
  • Show data from QA/CI:
    • # of bugs/enhancement/changes (I personally think this categorization is important)
    • # total/added/fixed
  • Show these metrics on graphs that show trends over time

This way, when tickets are fixed rapidly, one can see if the code quality goes down. Also, when not much seems to be happening with the bug database, code quality might go up, as refactorings are being done.

Summing it up: It's this type of dynamic behaviour that's important and what gives you information rather than raw data (which would be the value of a single metric).

I'm planning to put some graphs according to this scheme on a widescreen TV next to our CI connected lava lamps. ;)

|improve this answer
  • Well put. I read a lot of articles on how metrics alone are useless, as you mentioned, and history is important. Thanks for taking time to reply. – Russ K Feb 7 '11 at 0:03

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.