Background: there'a a gazillion types of virtual machines in Microsoft Azure each having different performance and price. Such virtual machines are paid per hour. The goal is to decide how to get "the most for the money spent". This can be also reworded in terms of electricity spent for a certain amount of computation.

So I have a piece of code as here which is highly CPU-intensive - uses little memory and no disk. My goal is comparing processor speed only and I assume that the code I use mostly uses the same CPU instructions sequences as the production load I plan for so it makes sense to use this code in the first place. I select the parameter (the number of the value to compute) for that code and it's unchanged across runs - let it be "five million" - and so all runs compute exactly the same.

So I run this code multiple times on VM of type T1 (imaginary, no such VM type in Azure now) and I see that it runs for 5 minutes on average. T1 machines cost 10 cents per hour.

Then I run this code multiple times on VM of type T2 and I see that it runs for 7.5 minutes on average. T2 machines cost 8 cents per hour.

I then must somehow compare which is better. My logic is the following: we're interested in minimizing both the time and the cost. So we can multiply the time to run the code by the cost per hour which gives us "money-by-time" (cents-by-minutes in this specific comparison) measure. A VM with the smallest "money-by-time" value is the most efficient. In this case T1 machines has "5 minutes by 10 cents" (50) and T2 machine has "7.5 minutes by 8 cents" (60). T1 machine is 17 % more efficient then.

Does such way of comparing VMs in terms of performance for given price make sense? Is there perhaps any error in my logic?

  • 5
    "Dollars per time * time" is dollars, yes – Caleth Feb 7 '18 at 14:00
  • 6
    It makes sense to compare cost like you do, but you say 'we're interested in minimizing both the time and the cos' and you don't really check it. Suppose there is a free instance that will take half an hour. You'll choose it by this method while it may (or may not) be too slow – RiaD Feb 7 '18 at 14:34
  • if you can split the task over multiple machines then you would be right to always choose the free one – Ewan Feb 7 '18 at 15:54
  • You're currently measuring how much the thing costs to run, multiplying that by 60, and comparing those costs. I'd suggest you figure out your ideal ratio of time to cost (e.g. seconds divided by dollars), and then you can attempt to get as close as possible to that. – Robert Grant Feb 7 '18 at 15:58
  • 1
    To clarify the dimensional analysis, (Time-per-unit-of-work) * (Cost-per-time) = (Cost-per-unit-of-work). This doesn't actually favor machine speed at all. With this formula, a machine is freely substitutable (interchangeable) with another machine that is twice as fast but is twice as expensive when measured in hourly rate. – rwong Feb 7 '18 at 17:03

Amazon measure the processing power of their instances in "Amazon EC2 Compute Units".

It's a interesting experiment to run as you can check to see if their published rating is accurate.

Presumably you would find some rounding off happens in the calculation and maybe you can save some money by getting a particular instance type.

However. I think we can assume that broadly speaking the published measure is correct. Running a check against it probably wont give you any extra information

Check out this link https://www.ec2instances.info/

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.