I have an application that returns projects. A
Project has many
Task has a field called
hours. A project returns the total
hours for all the
Tasks that belong to it.
My application processes hundreds of thousands of tasks updating and synchronizing. In the past, I have done no calculating when a task gets saved or updated to the database. Currently, when someone requests the Project, at the time of the request I do a calculation to see what the current total is at the time of the user's request.
Unfortunately, this is not efficient. The overhead cost of calculating the total on the fly is enough to make the response times too slow and I want faster response times.
When my app synchronizes itself with the source of where it is scraping data from it gets each task as HTTP requests in groups of between 10-400 individual POST/PUT requests each for the same Project.
I am concerned that if I trigger a background task to run for each task, recalculating the total for each one that I might cause too much stress on my app/database host CPU. I will essentially be going from calculating the total a few times on a project whenever needed, to potentially 10-400 times in under a minute. Are there any industry-standard ways to recalculate totals?
I want to send the recalculations to a background task, but I don't want it to run 399 times when it only needs to run once. I have no way of knowing when my app receives the last request.