8

We are currently maintaining a homegrown python "web server" where generating the response for some requests can take a very long time mostly due to heavy computations - these requests are basically posts with very long timeouts (think minutes to tens of minutes).

One problem of this architecture is that there is sometimes the need to cancel such a request - e.g. the user noticed a mistake while configuring the request. Currently, the cancellation is another request, which cancels the long running request - but there are lots of gaps, e. g. what happens if the client simply closes the website?

Currently, we are planning to retire the homegrown abomination of a web-server and switch to something sensible - e. g. Flask running inside of an IIS using wfastcgi. Due to political reasons, IIS is set, so switching to something like gunicorn is out of the window.

All development has stalled on that because nobody has an idea, how to kill the processes run by (w) fastcgi - that concern is simply not part of the fastcgi spec.

My feeling is, that an attempt to build something that incorporates that is a mistake - I would prefer a solution where the server simply offloads such computation intensive tasks to some background server (flask + celery?) and the front-end polls for that.

Unfortunately, the old solution was in place for so long that some devs want to retain the behavior at all costs.

Being not a web server guy I would like some tips / patterns what sensible solutions for such a problem could look like.

8

What you are suggesting is quite right.

It should be asynchronous.

You can post a request and the request gives you a unique Id. It posts a Done on this unique ID in some store where you can poll.

If you need to cancel a request, a cancel request using same Id should be posted.

Also on the back end where this computations are running, eg in a run interface of a Thread (java style), you can check after every few second if the process is cancelled by checking if a cancel post has been requested. If it is, then the thread must exit. This is a clean exit. You could also use an interrupt to the thread.

  • IMHO this solution is the one that fits best with the 'stateless' approach that suits Http so well -- you separate the act of doing the calculation from the request/response communication. In this pattern the request/response communication becomes a conversation like this: 'please start .. okay I will .. how you getting on? .. still working .. please cancel .. okay I am cancelling .. how you getting on? .. calculation was cancelled .. please start ... (waits) .. how you getting on? .. calculation completed here are the results.'. – Robert Nov 23 '16 at 9:00
2

I suggest you look into wsgi and use multithreading. You can manage each request in a thread, and implement timeouts in the thread. You should also be able to manage the threads and cancel requests more easily.

1

The problem you currently face with a simple HTTP post simply cannot be solved in that protocol. HTTP works asynchronously in strict client/server so there is no server-side notification and no native cancel. Also, you will face the problem of answers lost because of internet network timeouts. I will suggest a different and modern approach. Disclaimer : a bit less than 90% browser support

I'd suggest you reverse the information flow and use websockets. What you need to do once a task is done is to notify your user and push a notification from server to client.

When a task is posted, or, more accurately, as a replacement for the post, open a websocket (use the server technology you like, tornado, gevent, python-websocket etc.). Associate the task thread on server websocket open event. If the client quits, the client will send a close event to your websocket handler, so you can terminate the thread associated with it. Else, if your task ends normally, the server can send the data to the client and close the websocket. Also, you have to make the server ping once every 30s because websockets closes if inactive for over a minute.

If you apply this, this will be the faster and cleaner than a client-side polling, while solving gap issues since you have a real bidirectionnal channel. Note that you can implement additionnal services over the task schema such as sessions, progression pings etc.

0

If your task needs that much time to be done, don't do that on that request. Queue it up and make some another process that will do the work.

After work is done, simply throw a notification to user that "operation x" is done, in meantime show message "your work will be done in approx. x minutes".

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.