I'm running a Twisted web server and trying to figure out a good way to handle retries from the client. In normal operation the client makes a request of the server with some JSON data and receives a JSON response. Every request includes a header with a unique identifier.

I'm trying to handle the case where the client doesn't receive the response (might time out) and needs to resubmit the request. Since the request can change the server state I don't want it to reprocess if it's already been processed.

Possible approaches:

  • What I'm thinking of is caching every JSON response and if the client submits a request with a duplicate request ID then just return the cached data. The thing is I'm not sure how often this situation will actually come up in the wild and caching every response and inspecting every request might be a lot of overhead (some responses are quite large JSON structures).

  • I could minimize much of the caching by just returning a HTTP response that indicates it was a duplicate request, but then the client won't know what the original response was.

Is there a best-practice for handling duplicate requests from a server?

1 Answer 1


A lot depends on how much traffic you expect to see and how often this situation will occur.

To avoid premature optimization, I'd first go with a variation of your 2nd choice - treat the duplicate request as an error (and log it as such).

If you see it happening often enough to warrant action, then your 1st choice - cache the output - seems reasonable. You can set rules on how long the data is to be cached - it doesn't need to be that long.

For that matter, I think there are HTTP headers you can include with your output to indicate that the various servers & firewalls are allowed to cache the response for a time. That might be easier than cooking up your own cache mechanism.

  • Makes sense. I still want to avoid the chance of duplicate calls being processed so I will still inspect the request ID to check for duplicates but I'll leave cached responses to the rest of the pipeline. Thanks.
    – Nick Gotch
    Sep 29, 2015 at 19:00

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.