# Advice on reconciling discordant data

Let me support my question with a quick scenario.

We're writing an app for family meal planning. We'll produce daily plans with a target calorie goal and meals to achieve it for our nuclear family. Our calorie goal will be calculated for each person from their attributes (gender, age, weight, activity level).

The weight attribute is the simplest example here.

When Dad (the fascist nerd who is inflicting this on his family) first uses the application he throws approximate values into it for Daughter. He thinks she is 5'2" (157 cm) and 125 lbs (56kg). The next day Mom sits down to generate the menu and looks back over what the bumbling Dad did, quietly fumes that he can never recall anything about the family, and says the value is really 118 lbs!

This is the first introduction of the discord. It seems, in this scenario, Mom is probably more correct that Dad. Though both are only an approximation of the actual value.

The next day the dear Daughter decides to use the program and sees her weight listed. With the vanity only a teenager could muster she changes the weight to 110 lbs. Later that day the Mom returns home from a doctor's visit the Daughter needed and decides that it would be a good idea to update her Daughter's weight in the program. Hooray, another value, this time 117 lbs.

Now how do you reconcile these data points? Measurement error, confidence in parties, bias, and more all confound the data. In some idealized world we'd have a weight authority of some nature providing the one and only truth. How about in our world though?

And the icing on the cake is that this single data point changes over time.

How have you guys solved or managed this conflict?

• Your software can be no better than the input it is given. It sounds like you need to get the family on the same page. Commented Nov 20, 2012 at 21:48
• This seems similar to stock (and similar) market prices. A moving average would probably provide a fairly accurate value over time, for an appropriate number of terms. Subtracting the moving average from the actual data points would then give you fluctuations you could treat as a stochastic process, to obtain variances and deviations and such.
– You
Commented Nov 20, 2012 at 22:24
• @You: If you elaborate a little more, that could be a pretty good answer! Commented Nov 20, 2012 at 22:27
• In addition the the time factor, a degree of certainty factor could figure in too--best guess vs. just-stepped-off-the-scale, for example. Commented Nov 20, 2012 at 22:42
• In the situation described, I'd just use the current value, whatever it is, and not try to outsmart the entries. The recipes would have variation in their calorie count due to different sources and preparation, and people have different matabolisms, so you're not dealing with a lot of precision to start with. I do think its a great idea for an app, though. Perhaps handle it in documentation, suggesting a family weigh-in. (I left this as a comment since it doesnt address the larger concept of handling such data) Commented Nov 21, 2012 at 7:21

Given both the actual data `y` and the average `y'`, you can regard the fluctuations `y-y'` as a stochastic process. This process has properties such as a standard deviation and variance, giving you useful information about the data in terms of its accuracy. This information could be used to discard some data points (those for which the fluctuation is larger than the standard deviation, for example), allowing you to recalculate the moving average from more accurate data or even using the data directly, but this may not necessarily be a good idea.
There are two ways that this data can change, as I see it: Corrections (where the wrong value was entered) and natural variation over time (because people's weight changes). So you may not want to record `weight` as a single value, but maybe record it as `weight_measure_at_point_in_time` which has a value for weight, and another for the date it was recorded. That way you know the measurement changes, and you know when. When you do calculations with this value, you could choose to look at only the most recent measure, and compare similar calculations to earlier measurements.