3

The issue

I am building a server-side system that handles streams of time-series events from multiple users and I'd like to perform an action after a certain quiet period has elapsed in the event stream of every user.

For example, a user emits this stream of data:

-[5]------[4]-[3]--[2][1]-->

My consumer will handle this stream of data, but will also perform an operation if X time has passed after [4] and before [5].

Rx?

When looking at the issue Rx debounce immediately comes to mind.

Rx + Multiple consumers?

But is Rx fit for an environment where I run more than one consumer for the streams?

To my understanding, multiple consumers means that individual consumers potentially may be exposed only to parts of the time-series, making it impossible to know when a specific user's stream has started or ended.

For example:

USER                         | Application
                             |
-[5]------[4]-[3]--[2][1]--> | ---> [4]--[2]-[1] ---> CONSUMER 1
                             | \
                             |  \-> [5]------[3] ---> CONSUMER 2

So can I handle streams with more than one consumer but still harness the power or Rx and debounce? Is there a more fitting solution for this issue?

Improve this question
5
  • If you mean "perform an action after a pause in the stream for each user", then the way to approach this is to ensure that any data from a particular user is always handled by the same consumer.
    – BobDalgleish
    Commented May 7, 2019 at 13:43
  • 1
    I think you are overestimating Rx here. Making your application work with Rx for this feature seems more effort than implementing the denounce yourself for this special case. Your own solution also allows you to not keep a thread per user and not tie a user to a consuner
    – marstato
    Commented May 8, 2019 at 12:29
  • @marstato It looks like streams are the way to go, but not Rx. I'm researching Kafka and Redis streams at the moment
    – noamt
    Commented May 8, 2019 at 15:32
  • 1
    @noamt if you don't have near real time constraints why not leaving it outside RX and delegate to an external process? I.e. elastalert.readthedocs.io/en/latest/ruletypes.html#flatline
    – Davide Cavestro
    Commented May 22, 2019 at 19:42
  • Help me understand... You have an Observable that emits -[5]------[4]-[3]--[2][1]- and you are wondering if you can debounce that observable? Because of course you can...
    – Daniel T.
    Commented Apr 12, 2023 at 15:08

0

Reset to default

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.