0

I am looking for a "test" to determinate which IDE is more productive.

I imagine this is a project in the desired language (my case would be javascript react, or Ruby) and I need to follow instructions like:

  • Refactor class A and rename it to B
  • Write a short function that does "this little job".
  • Search for a method called C and change it so that it returns an integer
  • Add a logging line to method F

Then you could determinate productivity by timing the time it takes to complete the changes.

But maybe there are other ways of achieving the same result, which I'm not aware of.

My goal is to determinate how productive I am using different IDEs, and even comparing how well two people can use their IDE of choice, possibly stopping a few discussions in the office, which would also increase productivity :)

|improve this question
  • 1
    You're going to run into trouble as such simple tests are not going to approach the depth of software development problems you may have to solve. How would you account for e.g. really infrequent, but important and diabolical challenges that IDE A provides the tools to solve, but IDE B provides none of the tools but is faster for more ordinary tasks. – whatsisname Sep 22 '17 at 3:55
  • I probably see your point, but for clarity, do you have an example for these "infrequent, but important and diabolical challenges" – ecoologic Sep 22 '17 at 5:13
  • If you like to fight an Editor war in your office, go on. If not, either let everyone use his favorite one, or let your boss make a final decision. – Doc Brown Sep 22 '17 at 5:59
  • Suppose one IDE user has mastered macros, or some other productivity tool, offered by the IDE. That particular user may be much more productive than others. Unless you have hundreds of potential users, with a wide range of skills and experience, any tests are producing random noise. – Frank Hileman Sep 22 '17 at 17:04
  • Is the IDE really The Bottleneck or is it the speed of thought? – RubberDuck Sep 22 '17 at 18:30
1

This doesn't seem to be a particularly complicated problem.

  • Collect metrics for performing various tasks.
  • Normalise the results per person.
  • Aggregate the results per IDE.

Are you asking what metrics to collect, or what task to perform?

The latter is largely irrelevant as long as it reflects 'day to day' tasks for people. The former is probably a mix of something like time taken, defects produced, LOC generated.

The Personal Software Process is an old, but easy, way to collect this sort of metric and there are plenty of digital tools that support it.

The biggest problem you'll face is gathering enough data for it to be statistically significant, and avoiding the 'I don't like this IDE so I am deliberately misreporting or misperforming when using it'.

...but if you want to show A is better/worse than B, the steps to take are the same:

  • Collect baseline data.
  • Collect test data.
  • Compare baseline to test data.
  • Generalize taking the error margin into account.
|improve this answer

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.