Guideline choosing between reporting solution or application front-end

Question

I am working in the data department of a middle-sized logistics company who is shifting from an 'old-school' on-premise BI data warehouse to a more 'modern' cloud-based data lake approach. Another piece of context: the data team is separate to the application development team. Probably quite common, but a precision that helps explain my following questions.

Most applications are now feeding their data in real-time to our new data platform.

On top of the typical 'analytical'/'long-term' reporting (technically: Redshift->PowerBI) which are typically viewed off-line, we have started creating more (near-)realtime solutions on this new platform (typical example: live operational dashboards used by field users to take operational decisions) (technically: lambdas->Angular JS)

I may be wrong, but there seems to be an overlap between such real-time reporting solutions and what the application team can build as visualization directly on top of the application DB, like any other application front-end.

I suspect there are no hard rules, and lot is a case-by-case basis, but are there any guidelines out there when one should choose one over the other? which factors play a role in the decision and what are the main pro's and con's?

I was thinking of the following, but suspect there is a lot more to it?:

• user-input that affects process -> application. But does this mean that visualization-only solutions should be on the data platform?
• multiple 'independent' application sources -> data platform. But does that mean that single-source reporting solutions should be on the application directly?
• business criticality: if report issues have severe business impact, probably best to reduce the number of links in the chain

Steve

Answer 1

There is a hard technical reason to run your reports on a data warehouse/lake/swamp rather than your transactional database, and that is because they can slow the application down.

Aside from that, the thing that always gets me is that companies spend thousands if not millions on these off the shelf reporting tools that are supposed to do anything and a team to write the reports on them, and then get the application devs to make them a website report because its prettier!

You have bought powerbi/kibana/splunk/whatever, use it! if its not doing what you need then examine your purchasing process. Work out why, don't just keep paying for it and not use it.