1

I want to create an api based on Node.js and Express.js. This api should return an array of products on /products. To get these products on my node server, I need to call a third party api. However, I don't want to call the third-party API every time a client calls the API I provide, because these products only need to be updated once a day. So I want clients to call the API route frequently and internally call the third party API only once a day.

My assumption is to use a package like node-cron to call the API once a day and then update the product array I send on an API call. This would basically look like this.

let products = []

cron.schedule('0 0 * * *', () => {
  let response = await fetch(/* third party api */)
  products = response
});

app.get('/products', function (req, res) {
  res.send(products)
})

I'm not sure if this is a good way to solve this use case, or if there are more sophisticated ways to solve this?

2
  • 3
    What if the dataset is huge? Will the server be able to keep the data? Or if there is any crash, the data will be lost. You may have to consider sqlite or a cached database if you need the data after crash. But for normal operations, this is fine. Nov 17, 2021 at 17:37
  • Nice one mentioning sqlite. I am always amazed what people do with it Nov 17, 2021 at 18:20

2 Answers 2

5

This does the trick. Depending on your business requirements this is a viable way to go.

But in general you would want a cache, i.e. some kind of database which is prefilled and where you have your own backups for when your service is up and running. In general your customers need data. And they want it fast.

There are cases, that the source is unreachable or your service is restarted and you couldn't retrieve data again. Or for other reasons why you want some kind of (old) data, which in many cases would be better than having no data at all.

You could do it this way. But perhaps it bites you later.

1
  • 3
    Could probably mention some concrete tools that they could look into: Redis, memcached, a reverse proxy like Nginx even (depending on use case).
    – gntskn
    Nov 17, 2021 at 19:34
4

This solution is clean and easily understandable, and I would probably do it just like that. Remember to get the lists of products when you start up the service, as you don't want to return an empty list until the list has been refreshed at midnight.

As an alternative, you might implement a kind of cache that is only refreshed when a request is made and the old cache contents is either not present (because service was restarted) or it's from the previous day, i.e. known to be stale. The downside of that approach is that the first client requesting the product list is slightly penalized and has to wait until it has been retrieved from the external service. Whether this is acceptable depends on the time required to retrieve the list, of course, and on the responsiveness required of your application.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.