7

We are currently developing an image processing restful api. Server performs some CPU-heavy computations image processing upon request and return the image to the client.

We want to make this a single http request (all computation should be done).

My questions.

  • What is the best workflow for this. With speed has an high factor.

  • Other alternatives suggest using a Master/Worker Server workflow. But will mean multiple requests (first to put job in queue, others to poll if job is done). We want to make it a one trip request. How can we achieve this?

  • The single request would most definitely take some time how do we handle timeouts

One more thing, we are using NodeJS.

Improve this question
6
  • A Restful API that has only one http call is not a restul API. What's the execution time of the processing? Just to get an idea. Does it run in a millisecond? a second? or a minute? Are you using GPUs or fancy hardware?
    – cauchi
    Dec 4, 2015 at 12:52
  • Lets say little seconds. AWS Architecture, can always scale up
    – Tom Peach
    Dec 4, 2015 at 13:25
  • 1
    If I understand nodejs correctly, it is not the ideal server for blocking CPU heavy calls as it will cause the queue to pile up under heavy load.
    – user40980
    Dec 4, 2015 at 16:02
  • Why do you want it all in one request? The time for making a request between machines will be nothing compared to the time the CPU will take to process the image, so you are better sharing the jobs out amoung multiple machines.
    – Cormac Mulhall
    Dec 11, 2015 at 14:05
  • Thanks @CormacMulhall, Already implemented a web-worker model for it. Had two separate machines (will definitely scale them up). One for receiving the request, the other for processing like TMN suggested
    – Tom Peach
    Dec 13, 2015 at 5:05

1 Answer 1

Reset to default
9

You should submit your requests using a POST, and your service should return a URL that will retrieve the image once it's been processed. If the URL is accessed before the processing is complete, you should return a 202 (ACCEPTED) response. Once processing is complete, you can serve the processed image.

Improve this answer
1
  • Thanks TMN. I will add Worker Server to the workflow, and queue up image processing jobs there.
    – Tom Peach
    Dec 4, 2015 at 14:11

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.