When should a ESB vs ETL tool be used? I have worked on ESB projects using Tibco Businessworks quite a few years ago. The message bus that we built used to consume messages from a source system , applying validation , perform transformation and send it to appropriate destination. The source could be a db , service , file etc and same with the destination. I was under the impression that ETL is primarily to extract , transform and load large volume of data & source / destination are primarily different databases. I recently looked at one pentaho project and it pretty much supported all the functions that I have seen in Tibco Businessworks and the project that I looked at was performing data extraction from a DB , but doing the same activities such a filter / transformation and routing. I also believe it has facility to route messages and consume messages from queues. When should we choose one over the other?
1 Answer
Enterprise Service Bus is a "software architecture" model used for designing and implementing communication between mutually interacting software applications in a service-oriented architecture (SOA).
Extract, Transform and Load (ETL) refers to a data warehousing process that extracts data from various data sources, transforms it into the proper format, and loads it into the final target, usually a database.
The two things that they both have in common:
- They both work with databases, and
- They both allow heterogenous systems to be connected together,
But they have different purposes. ESB is used for ongoing operations, whereas ETL is intended for one-time, ad-hoc or batch-like imports of data, much like the purposes of OLAP and OLTP.
-
Thanks Robert. If ETL is primarily aimed at dealing with data transfer between DBs , why do the tools support polling from different data sources , calling we services etc? Nov 13, 2015 at 5:33
-
Because the emphasis is on the connecting together of disparate systems. ESB does that too, but on a more real-time basis. Nov 13, 2015 at 5:37