1

Background: there'a a gazillion types of virtual machines in Microsoft Azure each having different performance and price. Such virtual machines are paid per hour. The goal is to decide how to get "the most for the money spent". This can be also reworded in terms of electricity spent for a certain amount of computation.

So I have a piece of code as here which is highly CPU-intensive - uses little memory and no disk. My goal is comparing processor speed only and I assume that the code I use mostly uses the same CPU instructions sequences as the production load I plan for so it makes sense to use this code in the first place. I select the parameter (the number of the value to compute) for that code and it's unchanged across runs - let it be "five million" - and so all runs compute exactly the same.

So I run this code multiple times on VM of type T1 (imaginary, no such VM type in Azure now) and I see that it runs for 5 minutes on average. T1 machines cost 10 cents per hour.

Then I run this code multiple times on VM of type T2 and I see that it runs for 7.5 minutes on average. T2 machines cost 8 cents per hour.

I then must somehow compare which is better. My logic is the following: we're interested in minimizing both the time and the cost. So we can multiply the time to run the code by the cost per hour which gives us "money-by-time" (cents-by-minutes in this specific comparison) measure. A VM with the smallest "money-by-time" value is the most efficient. In this case T1 machines has "5 minutes by 10 cents" (50) and T2 machine has "7.5 minutes by 8 cents" (60). T1 machine is 17 % more efficient then.

Does such way of comparing VMs in terms of performance for given price make sense? Is there perhaps any error in my logic?

Improve this question
7
  • 5
    "Dollars per time * time" is dollars, yes
    – Caleth
    Feb 7, 2018 at 14:00
  • 7
    It makes sense to compare cost like you do, but you say 'we're interested in minimizing both the time and the cos' and you don't really check it. Suppose there is a free instance that will take half an hour. You'll choose it by this method while it may (or may not) be too slow
    – RiaD
    Feb 7, 2018 at 14:34
  • if you can split the task over multiple machines then you would be right to always choose the free one
    – Ewan
    Feb 7, 2018 at 15:54
  • You're currently measuring how much the thing costs to run, multiplying that by 60, and comparing those costs. I'd suggest you figure out your ideal ratio of time to cost (e.g. seconds divided by dollars), and then you can attempt to get as close as possible to that.
    – Rob Grant
    Feb 7, 2018 at 15:58
  • 1
    To clarify the dimensional analysis, (Time-per-unit-of-work) * (Cost-per-time) = (Cost-per-unit-of-work). This doesn't actually favor machine speed at all. With this formula, a machine is freely substitutable (interchangeable) with another machine that is twice as fast but is twice as expensive when measured in hourly rate.
    – rwong
    Feb 7, 2018 at 17:03

2 Answers 2

Reset to default
2

Yes, but they are called dollars, not dollar-hours. You are not multiplying cents and minutes; you are multiplying cents per hour, and minutes. The time units cancel out and give you cents. 7.5 minutes (0.125 hours) times 8 cents per hour is 1 cent.

This makes sense if you are planning to run your code non-stop and you want it to run as many times as possible for a certain amount of money, or a certain number of times for as cheap as possible.

As 1201ProgramAlarm pointed out, this might not make sense if the VMs are going to be doing nothing most of the time. In your example, if you only run the program once, it's cheaper to use the T2 machine, even though the T1 machine is more efficient, because you have to pay for the whole hour. If you have to run the program 8 times per hour, the T2 machine will still be cheaper. If you have to run the program 9 times per hour, you have to use two T2 machines or one T1 machine, and the T1 is cheaper.

Improve this answer
2
1

Amazon measure the processing power of their instances in "Amazon EC2 Compute Units".

It's a interesting experiment to run as you can check to see if their published rating is accurate.

Presumably you would find some rounding off happens in the calculation and maybe you can save some money by getting a particular instance type.

However. I think we can assume that broadly speaking the published measure is correct. Running a check against it probably wont give you any extra information

Check out this link https://www.ec2instances.info/

Improve this answer

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.