Software Engineering Stack Exchange is a question and answer site for professionals, academics, and students working within the systems development life cycle. It only takes a minute to sign up.
Sign up to join this community
Today I was looking at some blogs on Ruby and Python. A Python loyalist wrote a few lines of code with a loop to show how Python is faster than Ruby, with a similar code. I also found a blog of a Ruby disciple who says that it is wrong and he too submitted a code sample and benchmarking results.
I am wondering whether background programs or background processes may sometime hinder the results and we may not get exact benchmarking scenario.
Please correct me if I am wrong.
Yes they can.
Normally just common sense in taking results from a couple of benchmarking runs, having some idea of what to expect, and being aware of what background services might fire up on your machine (beware indexing/search services in particular) and slow it down will generally avoid being seriously mislead by results. I've found this to be more of a problem with automated performance testing where there is no human in the loop to go "WTF... ? Oh the stupid foo service just started an update".
For purely compute benchmarks, I've found running the benchmark process at a higher than normal priority can usefully give more consistent, stable results on "busy" machines. This may not work if I/O is a factor though (depends whether your OS' notion of priority just affects CPU scheduling or I/O too).
It's possible.
But if that's the case, running the benchmark again would almost certainly produce different results. That's why benchmarks are always run multiple times, to make sure that there are no background processes that are skewing the results.
