0

When would I use reactive programming libraries like RX Java and Project Reactor compared to stream processing engines such as Storm and Flink?

I am aware that these concepts might not be directly comparable, but I would like to understand the relationship between both concepts better.

My current undertanding is that both have a focus on unbounded streams of events/data.

Reactive programmin is a more low-level concept in the sense that a stream processing engine could be implemented with it. I think this is actually the case for Akka Streams.

Stream processing engines seem to aim to solve the problem of horizontal scaling stream processing tasks, i.e., clustering etc.

Please clearify the commonalities and differences between both concepts, typical use cases for each of them and how they relate to each other otherwise.

1

Reactive programming libraries often use streaming as a basic capability. From there, they add the capability of managing work across threads, and across time.

While a streaming library might have operations that parallelize computation, those operations are restricted to the stream they are working on. A reactive library might combine and split streams, moving computation between threads, and rejoining results into a single thread, etc.

| improve this answer | |
  • I am not sure whether we have the same understanding of what a stream processing engine or a streaming livrary is. Can you give examples of the libraries you are refering to? – Hyggenbodden Mar 2 at 17:00
  • I was primarily thinking of Java Streams, but my comments apply to Java Reactive Streams as well. The RxJava 2 and RxJava 3 libraries are built upon the Java Reactive Streams library, and make significant extensions into thread-safe and performant computing. – BobDalgleish Mar 2 at 18:36
  • Thanks for the answer so far, a main point of my question was to understand how rx java etc. relate to "stream processing engines" like storm, flink, kafka streams, samza etc. – Hyggenbodden Mar 2 at 19:08
  • Well, RxJava would be used inside a particular process, and as such, is very lightweight. In addition, it can be added gradually to a system. Personally, I started using RxJava as support for unit testing legacy code that was heavily threaded, and found it extremely powerful. – BobDalgleish Mar 2 at 19:31

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.