I have an API where every time is called there are multiple calls to multiple webservices in the background, sometimes it takes up to 20 seconds to process the request calling up to 10 different web services depending the specific use case.
The API is developed using .NET Core hosted in IIS behind a web balancer, the web balancer uses a round robin logic to distribute the requests to the web server cluster.
The API has been in production for several years, but just recently there is an API client that is making two request almost simultaneously. These requests are sent each to different IIS, so there is a race condition between the two web servers handling a request associated to the same customer, and since there isn't any synchronization between the servers the behavior is somewhat unpredictable, the API was designed to process requests associated to the same customer sequentially.
Our first approach, was to ask the API client to just wait until the first request is fulfilled before sending the second one. For reasons out of the scope of this question, the API client cannot be changed.
Our second approach, was to ask the web balancer responsible to send requests associated to the same customers to the same web server, if they were able to do this, we can easily check if a request for a customer is currently being processed process by the server, and wait the first request to be completed before processing the next one. But also, for reasons out of the scope of this questions, the web balancer logic cannot be changed.
So the question would be, what do you think could be a good solution to this issue?, almost every solution that I think is not that simple and/or could add a lot of overhead to the API since this is just a very specific use case, normally, requests associated to the same customer are received with a difference of days, even weeks.
Maybe synchronization to "lock" the customer, via a shared database, or via communication between every web server of the cluster with all the corner cases and overhead that we would need to take into account. Or I don't know, maybe there is already a product designed to be used with this type of cases in mind or maybe just add another layer of web balancers were we have the control to change the logic and therefore be able to send requests associated to the same customer always to the same web servers.
Any help would be greatly appreciated.
Best Regards, Mario
