I'm writing a public web app to get stock data from a magento store. I've accessed the data, but it turns out that i have to query each product individually for stock data. With thousands of items this is a resource hog and takes lots of time. This cannot be avoided. It's just how their API works and I have to do it this way.
How can I code something in php that runs in the background, harvesting the api data by nibbling it in a calmer more friendly manner, and not hammer their shops with SOAP api calls?
Am I right in thinking that I need to use a background process like this: $pid = shell_exec(sprintf('%s > /dev/null 2>&1 & echo $!', $command));
...and then query that $pid? Store them in a database? Are there any examples of background process managers?