I am embarking on a business intelligence project which will require abstracting access to two existing data warehouses. I need to design an application architecture to allow self-service business intelligence to join up the data and provide a single view over the two existing warehouses. I have come up with something like this:

enter image description here

I am struggling with the virtualization/caching piece and wondering if there are any enterprise design patterns to solve my problem. Would an architecture like this work to abstract star schemas in data warehouses? I'm looking at products such as Red Hat JBoss Data Virtualization and Red Hat JBoss Data Grid (amongst others).

We are not using Hibernate currently and my understanding of Data Grids is that they are key-value stores, or object stores and therefore unsuitable for caching a relational model. I should also mention that we are keen to use vendor products for the Self-Service Dashboard part, but we may end up doing some custom-build in this area if vendors can't offer us everything we want.

  • 2
    Just found this book, which might be good for me amazon.com/Data-Virtualization-Business-Intelligence-Systems/dp/… – Mark Allison Mar 11 '15 at 10:19
  • 2
    I am not sure you've provided enough information about your project to advise on architecture. – Vladislav Rastrusny Mar 11 '15 at 11:52
  • Why relational data can't be cached in a key-value store as {key: pk, value: the_rest_of_the_row}? You'll probably want to cache tables metadata, too. – 9000 Oct 12 '15 at 20:08
  • 2
    What is the problem with the classical approach? – NoChance Oct 12 '15 at 21:12

There isn't a huge amount of detail about what you're trying to achieve here but from what you've described, it sounds like you could do with a data mart to abstract away the main repositories and expose a minimal subset of data to service the application.

Even if you could design a decent application layer, you're likely to hit performance issues due to load on one (or both) of the repository databases. The benefit of the mart approach is that the DB the application talks to is highly performant. Updates take place on the repository DBs behind the scenes and are pushed thru on whatever basis you see fit.

An additional benefit that you also only have one DB vendor to consider in your application layer.

|improve this answer|||||

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.