This might seem as a very basic and maybe a vague question but I'd really appreciate advice and experience sharing on this:
Given you have a service, which processes tasks in a queue, e.g. because they might be time-consuming, or there might be too much of them coming in, or maybe because they require fifo order, if a client (e.g. web browser or another service, but let's assume web browser for simplicity) initiates such task, what are the best practices to give feedback and update it on the progress in simple cases?
E.g. consider a case when you resize pictures - you upload one, press "compress" and after that you can see a meter, reporting the progress, and eventually a link to download the resized version rendered. Consider that after submitting the picture the service doesn't hold the request, but just enqueues the job and responds with e.g. job id immediately. Given we don't use websockets or another streaming solution, what are the best practices to achieve seamless progress reporting (e.g. 5% completed, 50%, etc) and getting the result as soon as the job is done?
First thing which comes to my mind is polling, but that looks a bit workaround-ish (at least to me and at least right now) and can increase the load drastically, just because of those check requests.
Are there any better solutions? What are the best practices and patterns for that?
If this could be achieved only with a streaming or pubsub solution, pls suggest, I really wanna improve my understanding and get some ideas from experienced people.
Sorry if it's kinda vague, but it seems to me that it still raises some good design questions...