I am working on a polyglot web app project that has functionalities that can be separated, but that rely on each other. I am unfamiliar with the following (what key words to even search for, what concepts are in play):
I have a few different sources of disparate data that are separately created/ingested into a data store. the data that is needed to back the actual web app requires clean-up and curation of the different types of data.
As a concrete example, perhaps you have a large book/article datastore and a user datastore. You want to curate and push data from both to a graph store for recommender purposes, and a paired down version of the book store to actually serve the web app and search functionality.
How do you usually go about this? It's not fast changing data, so I wouldn't think a full event-based reactive system is quite necessary. Is the concept to somehow schedule jobs at a specified period?