1

I'm writing a public web app to get stock data from a magento store. I've accessed the data, but it turns out that i have to query each product individually for stock data. With thousands of items this is a resource hog and takes lots of time. This cannot be avoided. It's just how their API works and I have to do it this way.

How can I code something in php that runs in the background, harvesting the api data by nibbling it in a calmer more friendly manner, and not hammer their shops with SOAP api calls?

Am I right in thinking that I need to use a background process like this: $pid = shell_exec(sprintf('%s > /dev/null 2>&1 & echo $!', $command));

...and then query that $pid? Store them in a database? Are there any examples of background process managers?

  • 1
    The traditional route has often been to create a cron job (or the equivalent scheduled task in the OS you're using) which executes a PHP script. – Mike Mar 20 '14 at 12:49
  • Did you contact the owner/developer of this site? They may be able to change the API, since it would be in their own interest to have fewer requests. Otherwise you may even run the risk that they block you when they notice the traffic you generate. – thorsten müller Mar 20 '14 at 12:54
1

Taking Mike and Thorsten's comment - if this data is not time sensitive - you can create a cron job that will get these data and put it in your own SQL table and then have your App pull from it. Majority of good API calls typically have a pull all API request handler with a delimiter so that you can retrieve bulk data as they don't want you to hammer them because depending on their firewall configuration it might end up being a brute force spam program that wants to poll if you are polling too often. So in conclusion - it might not be you but them.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.