I'm new to server development, and I'm trying to figure out where the division of responsibilities is for Data intensive tasks, I'm using Node.js.
As an example say I'm making a Single Page Application:
- The Client renders a map with population per area
- The Server handles requests for the population data
However, assume that I'm also not the owner of the data, but that I collect it from a third party via an API. That API simply gives me a list of people and their location. I will have to scrub through each person, group them based on location, total them up and then store that data so it can be accessed for the purpose of the application. This process doesn't need to be done all the time, since population won't be changing all the time, but say I refresh this dataset once a day.
My question is putting this process on the Server seems like it would be process intensive enough that it would slow down requests. I see a few options but I'm not sure which one would be used in a professional development setting/ be most efficient.
- Send the client the raw data, and they can do all the parsing themselves - but then the user experience slows down across the board
- Have the Server do the processing on the same thread, and once a day the server slows down
- Spawn worker threads on the same server to handle it (discouraged?)
- Have an entirely separate third server, which my main server pings once a day to process the data and return it.
Is there another way of doing this that I'm not aware of, or is any one of these methods better practice?