Today I was looking at some blogs on Ruby and Python. A Python loyalist wrote a few lines of code with a loop to show how Python is faster than Ruby, with a similar code. I also found a blog of a Ruby disciple who says that it is wrong and he too submitted a code sample and benchmarking results.

I am wondering whether background programs or background processes may sometime hinder the results and we may not get exact benchmarking scenario.

Please correct me if I am wrong.

  • 2
    Benchmarking the execution speed of loops in Ruby in Python is utterly missing the point. Benchmarking the productivity the programmer that uses them -- while much more difficult -- is entirely the point. Commented Nov 16, 2010 at 21:43
  • @Adam: Yes, you can just Google and find some samples.
    – RPK
    Commented Nov 16, 2010 at 22:03
  • Maybe the Ruby disciple just wrote better Ruby than the Python loyalist - better to let Ruby disciples wrote Ruby and Python loyalists write Python shootout.alioth.debian.org/u32/… And remember the phrase "time / space tradeoff"
    – igouy
    Commented Dec 12, 2010 at 0:47

2 Answers 2


Yes they can.

Normally just common sense in taking results from a couple of benchmarking runs, having some idea of what to expect, and being aware of what background services might fire up on your machine (beware indexing/search services in particular) and slow it down will generally avoid being seriously mislead by results. I've found this to be more of a problem with automated performance testing where there is no human in the loop to go "WTF... ? Oh the stupid foo service just started an update".

For purely compute benchmarks, I've found running the benchmark process at a higher than normal priority can usefully give more consistent, stable results on "busy" machines. This may not work if I/O is a factor though (depends whether your OS' notion of priority just affects CPU scheduling or I/O too).

  • I think for this type of tasks, command-line is best.
    – RPK
    Commented Nov 17, 2010 at 11:55

It's possible.

But if that's the case, running the benchmark again would almost certainly produce different results. That's why benchmarks are always run multiple times, to make sure that there are no background processes that are skewing the results.

  • 1
    More accurately: Multiple runs make the effects of background processes statistically insignificant.
    – greyfade
    Commented Nov 16, 2010 at 21:51
  • removing outlier timings is usually the best way of eliminating the effect of background processes.... of course this still needs multiple runs :-)
    – mikera
    Commented Jul 15, 2011 at 14:48

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.