0

I don't fully understand reactive extensions, but my inital reading caused me think about the ETL code I have.

Right now its basically a workflow to to perform various operations in a certain sequence based on conditions it find as it progresses.

I can also imagine an event driven way such that only a small amount of imperative logic causes a chain reaction to occur.

Of course I don't need a new type of programming model to make an event driven collaboration like that.

Just the same I am wondering if ETL is a good fit for potentially exploring Rx further.

Is my connection in a valid direction even? If not, could you briefly correct the error in my logic?

  • note: i wasn't sure if this should go on SO or here – Aaron Anodide Jun 5 '12 at 23:31
  • If you're still interested in pursuing this, it looks like someone has had the same idea: reactiveetl.codeplex.com – shambulator Nov 30 '12 at 10:28
3

My first instinct is to say no: Rx is about observable collections and asynchronous programming, whereas ETL is about dynamic data mapping and extraction. I don't see how they are related at all.

1

They seem very related to me too. We're looking at Rhino ETL (of which that Reactive ETL library mentioned by @shambulator is a re-write), but it feels weird having to work with foreach and yield return all the time.

After having done the initial load from the source to the target database, when doing incremental loads using the same logic, the next step would be using your ETL logic on-the-fly, through queues or events: as new data is entered in your sources (databases, files, queues), it's pushed through the pipeline and inserted in the target database.

Seems an ideal candidate for Reactive programming to me!

Actually, I find it very surprising that the Reactive ETL library isn't at all supported or hardly used (there seems to be no NuGet package), or that there don't seem to be any similar libraries! Did you end up using it eventually (or something similar), and how were your experiences with it?

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.