0

What data structures and algorithms are suitable for event stream correlation? Specifically, I am looking at these two use cases:

  1. X occurrences within t seconds grouped by some variables (v1,v2). For example, 5 failed login attempts within a minute grouped by service such as SSH or FTP and IP address.
  2. Event A is followed by events B and C grouped by some variables (v1,v2) within a time-frame t.

In terms of actual implementation, I have found the Simple event Correlator to be approximately what I am looking for, but it's a big blob of perl...

2

Those are simple state machines.

For the first you implement a ring buffer with X entries that log the time. If the wrapped time is within the maximum time, you found your event.

Not sure about the second. It is just that you buffer the times for A, B and then the C combination with the variables. Maybe you elaborate on the conditions.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.