2

I'm writing some web socket (wamp) connectivity tests. basically one end pings, and the other end records it.

let's say I have the latency for each ping. So I'm tracking the min and the max latency figures.

the test however can run for a long duration of time, so a single min/max value won't tell me much about the nature of the connection generally.

Is there an canonical to obtain a useful figure. Any ideas?

  • 1
    You could try creating a histogram. Alternatively, calculate a median and an average. – Robert Harvey Apr 17 at 18:11
3

You will probably want to make several measurements here, because you'll want to understand how the system works both including and excluding the outliers. There are many possible solutions for exploring the data with and without the outliers.

Here are a few starting points for measurements I would be interested in:

Percentiles

You can try choosing a few meaningful percentiles, say 50, 90, and 99. For example, if you chose 90th percentile and that value ended up being 500ms, you would then be able to say: "90 percent of our latencies are below 500ms".

Trimmed Mean

Taking a trimmed mean will exclude the outliers so they don't skew your average. For example, in an extreme case where 90 percent of your requests are under 500ms, but you have one request on record that took three days, a trimmed mean would keep this extreme outlier from affecting your average.

Median

The median is another common measurement for drawing insights from your data set. Wikipedia describes its behavior well:

The basic advantage of the median in describing data compared to the mean (often simply described as the "average") is that it is not skewed so much by extremely large or small values, and so it may give a better idea of a "typical" value.

The median is generally equivalent to a very aggressive (50%) trimmed mean and the 50th percentile value.

Histogram

Finally, as Robert Harvey suggested in his comment, you may want to create a histogram and visually examine the data. This can provide inspiration as to where to focus your measuring efforts.

  • while a median can be a very good and robust measurement, presenting it like this might be a bit confusing: it's really just a fancy name for the 50% percentile. – amon Apr 18 at 8:18
  • @amon I updated the answer to clarify. – Kyle McVay Apr 18 at 15:27
1

By themselves the pings are unlikely to show much of anything interesting or useful. Neither are basic stats like median and mean for those. Percentiles are typically more interesting for this kind of thing. It's common to assume that you will see a normal distribution for your data but with this kind of thing it's unlikely. What you are more likely to see is that most of your pings come in a little higher than the minimum i.e. highly skewed towards the low end of your figures and a long tail of values that fall outside of the normal curve. For this reason, things like standard deviation will not be meaningful for your data. There are two measures that you can use to evaluate this mathematically: skewness and kurtosis. These operations are something that are built into common statistics packages and spreadsheet applications.

Typically, in this case your norm, or most common value/range, is the figure you want to use as the 'average'. The main question about that is typically whether that is low enough. the next thing that you'll want to consider is all the values that fall outside of the normal range of latency. If all you have is just a list of ping times, there's not much you can say about this.

So in order to have useful data, you need to store each data point with the time it was measured. This by itself is interesting. Avoid condensing this into aggregate data (means, medians, percentiles, etc.) until you have a good understanding of what you are seeing; saving historical data this way may be OK if you are short on space. Try plotting the latency data as a function of time as a simple scatter plot. Do you see any patterns e.g. every day a 9AM you see a spike in your latency. This might be interesting by itself but you might need to pull in other data to make sense of this. For example plotting the latency data against the volume data for the corresponding time may add some interest. You may find spikes or patterns that you can't explain resulting in a search for more types of data that you can plot against the latency.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.