1

I have a service which provides an endpoint that fetches from another service, saves, and returns an record. Here are the details:

  • The caller calls the endpoint using an identifier (say a customer's student ID number)
  • The service checks DB for that student ID. If no record that has not expired is found, the server calls an outside service to fetch the record. (which costs money per call)
  • The record is saved in DB with timestamp, and returned
  • If a non-expired record with the same ID is found in the DB, that saved record is returned instead.

So basically it's caching the records with an expiration date. The problem is that there are cases where the same ID is getting called for multiple times within a split second. They happen too close to each other so the fetch and save operation by the first call has not finished before the second call comes. As a result, duplicate calls are fired to the external service and duplicate records are in the DB.

So is there any way to cache this type of duplicate calls. Would memory cache be a good candidate for solving this problem?

1
  • most dbs will implement their own cache
    – Ewan
    Sep 28, 2020 at 8:18

2 Answers 2

2

Instead of caching the results your cache could store Promiseobjects (or whatever your language calls it) whose value is the result. If a request for a value is in flight when a second request is made, both requestors will wait on the same promise.

0

Your problem is not so much the caching the response of duplicate queries, but preventing duplicate calls to the 3rd party service.

Here you are forced to implement a lock which will span instances of your api but at the same time not be a single point of failure.

The simplest solution I can think of would be to add an extra "locks" table to your db which is checked/updated before making the 3rd party request.

This has the benefits of:

  • no extra dependency, presumably if you database breaks your whole api breaks anyway
  • databases already have fail over and clustering options
  • database have atomic transactions

The downside is extra load on your database.

I think I might opt for a separate api wrapping the 3rd party a You could either simply have a single instance (with failover) or send the incoming request via a message queue and split them over several workers via the id. Say odd numbers goto worker 1 and even to worker 2.

This would enable each worker to have a local lock without risk of both sending the same request

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.