I have a question regarding finding relevant events in a line graph. The following graphs show the views (y-axis) of a video over time (x-axis). Certain events lead to an huge increase in views and what I am trying to do is finding the timespan in which the event happened programatically.

The problem here is that there is a base noise of a few views all the time. I don't want those to be counted to the timespan of the event because the noise occurs almost over the total timespan.

enter image description here

This for example is a video with many views. The timespans I want to detect are marked in red. What I want to achieve is to calculate the position of the blue line which indicates that everything above the line is relevant data, everything below it is noise. In a chart like that the noise may vary between around 0 and 100.

Here it is pretty clear where the event appeared since the slope is rather radical, and getting that programatically would be pretty easy. However consider this graph:

enter image description here

The maximum on the y-axis here is not that high as in the first graph and it is much harder to find out where the noise is. Theoretically everything here could be an event, and there are also decreases which could be interpreted as the end of the event although right after that there is a rise again and logically they should be part of the timespan I want to detect.

So my question is: What algorithms could be used to efficiently and dynamically detect and filter out ambient noise in line graphs?

I thought about two approaches:

  • Compare every data point with it's neighbours to find if there's a significant rise. Here I think it's a terribly inefficient approach, especially for large data sets since you'd have to traverse to every data point AND it's neighbours.
  • Setting a fixed rate and discard every data point below it. The rate could be set relative to the y-maxium in the graph. That seems to be a nice approach but I'm not sure if it will work out for graphs with a relatively low y-maximum
  • 1
    Would it be fair to say that a video won't be "hot" more than 50% of the time (or time samples)? Otherwise it is ambiguous whether "hot" is the new "normal". – rwong Oct 23 '15 at 7:56
  • 1
  • 1
    The diagrams in this article about hysteresis switching helps explain the idea in zgnilec's answer. To summarize: output stays low until input hits the higher threshold. After that, output stays high until input falls below the lower threshold. – rwong Oct 23 '15 at 8:07

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.