How to create an API on Node.js that needs to be updated once a day?

Question

I want to create an api based on Node.js and Express.js. This api should return an array of products on /products. To get these products on my node server, I need to call a third party api. However, I don't want to call the third-party API every time a client calls the API I provide, because these products only need to be updated once a day. So I want clients to call the API route frequently and internally call the third party API only once a day.

My assumption is to use a package like node-cron to call the API once a day and then update the product array I send on an API call. This would basically look like this.

let products = []

cron.schedule('0 0 * * *', () => {
  let response = await fetch(/* third party api */)
  products = response
});

app.get('/products', function (req, res) {
  res.send(products)
})

I'm not sure if this is a good way to solve this use case, or if there are more sophisticated ways to solve this?

Answer 1

This does the trick. Depending on your business requirements this is a viable way to go.

But in general you would want a cache, i.e. some kind of database which is prefilled and where you have your own backups for when your service is up and running. In general your customers need data. And they want it fast.

There are cases, that the source is unreachable or your service is restarted and you couldn't retrieve data again. Or for other reasons why you want some kind of (old) data, which in many cases would be better than having no data at all.

You could do it this way. But perhaps it bites you later.

Answer 2

This solution is clean and easily understandable, and I would probably do it just like that. Remember to get the lists of products when you start up the service, as you don't want to return an empty list until the list has been refreshed at midnight.

As an alternative, you might implement a kind of cache that is only refreshed when a request is made and the old cache contents is either not present (because service was restarted) or it's from the previous day, i.e. known to be stale. The downside of that approach is that the first client requesting the product list is slightly penalized and has to wait until it has been retrieved from the external service. Whether this is acceptable depends on the time required to retrieve the list, of course, and on the responsiveness required of your application.