1

I have simulations that evaluate a certain value X. I run the simulations several times and save the value of X in a vector V. When all the runs have finished I evaluate the mean and standard deviation for the vector V.

This approach works, but implies saving all the values for X. As my computer is quite old and with limited ram, I was wondering if there is a way to update the mean value M and the standard deviation S, knowing the value of X at the (n+1)-th run, and the values of M and S after n runs.

How can I update the mean value and the standard deviation as simulations are added to the set?

Please note that this is just a conceptual example, I don't save only one number X but thousands at each simulations, so I really have problems running a big number of runs if I have to keep all the past values into the memory.

Improve this question

2 Answers 2

Reset to default
2

For the mean, it is enough to save the sum and a counter of how many values you added. For the standard deviation, you need an online algorithm like this one: http://en.wikipedia.org/wiki/Algorithms_for_calculating_variance#Online_algorithm

Or here: https://stackoverflow.com/questions/11978667/online-algorithm-for-calculating-standard-deviation

Improve this answer
3
  • but storing the sum don't you risk to go in overflow? (I mean that the sum might become larger than a computer can actually handle).
    – lucacerone
    Commented Oct 1, 2012 at 13:18
  • I haven´t thought about that, if you work on a possibly infinite stream of data points, then this approach would not work. The mean should be calculated/updated with every incoming data point. I guess it would be called online mean calculation or similar.
    – Puckl
    Commented Oct 1, 2012 at 13:42
  • @LucaCerone, the Wikipedia article also shows an online mean algorithm that works by storing the mean to date, and updates it.
    – John R. Strohm
    Commented Oct 1, 2012 at 13:52
0

Old time calculators with statistics functions saved only the number of items, the sum of the items and the sum of the squares of the items.

  • mean is the sum of the items divided by their number:

enter image description here

  • standard deviation is the square root of the variance:

standard deviation

  • variance is obtained by substracting the square of the mean from the mean of the squares:

variance formula

Improve this answer

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.