I have a website that is based around some external APIs related to a certain theme. It combines data of responses from multiple APIs before returning to the frontend.
With that said, my server side does not have a database. Instead, it requests data from the external APIs, and returns things to the frontend, without persisting anything. It does not need to persist since that would create an overhead from syncrhonizing data between my database and the external APIs.
But, I have a new requisite now that envolves creating a dashboard. This dashboard depends on a comparison of data between the entity being requested, and all the other ones that exists.
For example: Suppose entity A expends $ 10 per month. A dashboard field would inform how far away entity A is from the average money spent by all the other entities. I would have to read the expenses from all other entites before calculating the average.
I cant build this dashboard with transient data, like I have been doing. I will have to persist data from all entities in order to supply this dashboard.
What should be my approach to it?
- Should I create a feature on my server that periodically reads and persists all the data from the external APIs? I would then also need to calculate and persist the metrics that would be shown on the dashboard for each entity.
- Should I download that data to memory on my server and persist only the calculated results? For example I would download all the expenses from all entities, calculate the average money spent and persist it as a metric on the database.