Given two software projects X and Y relating to some Product A -

  • Project X offers some new capabilities.
  • Project Y offers incremental capabilities.

Y's features can be measured: we know its uses, access patterns, etc. Product A understands how to monitor Y and quantify its improvements.

Project X offers totally new capabilities; these capabilities will combine with existing project A capabilities as well as other, unplanned, ways. How do you quantify the unexpected ways that users will use it in? In particular, what kind of ROI can you reckon for Project X, what kind of % improvement to your customer base will happen?

An example might be: do you maintain a library in your standard company components, or do you build a totally new one?

Or, do you continue working with your PHP installation, or do you make a leap to Haskell?

Or, do you deliver an improved feature, or a totally new feature?

A glance through the software engineering Google search seems to only give standard source code metrics; product management search gives only standard business metrics.

1 Answer 1


The way you measure the ROI depends on the effect of the projects on the overall product. For totally behind-the-scenes work (library improvement or replacement, implementation language, etc.), you would probably look at server load, system performance, defect rate, future/estimated engineering time savings, etc. For a user-facing feature, you would look at things like customer satisfaction, defect rate, user engagement rates, etc.

In your case, I think you would measure project X the same way you measured project Y when it was new. If you are unsure whether to do X or not, then you basically have to make an educated guess. This is risk, and it is unavoidable. You can do a cost/benefit analysis to determine if the rewards justify the effort.

If you cannot expect or predict the ways in which users will use the results of X, then there is no way to measure them beforehand. What you can do is mock up X, show it to some users, and observe their reaction. Use a prototyping tool or some quick and dirty code as a proof of concept. This may give you a sense of how the proposed feature will be received and used.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.