-1

I have a situation on the front end where a user clicks on “Download” button and the query behind the scenes is going to take over a day. I have a Spring boot web app running as far as web services are concerned.

What I want:

When the download button is clicked on the front end, I would like to put this process in some sort of queue. I believe putting in the queue is the only option. And when the download is completed, it should dump the file at some server location.

In order to do this, I have been suggested by someone that I should use JMS. So basically, I need to integrate the JMS feature with the RESTful web service. I haven’t used JMS before and hence wondering :

1) Is JMS a good approach to use in this scenario? 2) If yes, then do I need to put this JMS thing talk to specifically one web service, which in my above case would be the web service responsible for downloading the stuff when download button is clicked? 3) Any other approach if possible or recommended over this?

Please let me know if I can answer more questions.

  • If you have a db you could just create a table there with "jobs" or "queue items". Would work fine. – Esben Skov Pedersen Nov 1 '18 at 5:30
  • Could you please elaborate on this? – John Nov 1 '18 at 5:33
  • Is the download running in the browser of the client, somehow connected to the running process? More specifically: 1.) Does the download take the same amount of time as the process running on the server? 2.) If the user cancels the download, does it cancel the server process? – kiltek Nov 1 '18 at 10:35
  • @kiltek Actually, this thing hasn't been implemented yet. I am still trying to get some idea on this. Could you please explain your approach (if any) that would be beneficial for me? Thanks – John Nov 1 '18 at 13:26
  • What problem is the JMS queue solving for you? The main reason for using distributed queues given the current state of hardware is to coordinate work across hosts. For most anything else, there are simpler solutions. Introducing JMS is going to add a lot of administrative overhead. It's not clear that it's worth it for this. – JimmyJames Nov 1 '18 at 16:57
0

As you already have a RESTful API, I would design this feature as described below. I am assuming that you are creating some kind of report, which takes a lot of time to prepare, but is not really that large after it has been created.

  1. The API service provides a resource "FooReport" that represents both the status of the report and its contents. These resources are served under the endpoint /foo/reports/
  2. When the user presses the 'Download' button, the frontend sends a POST request to /foo/reports/ to trigger the creation of a new report.
  3. The API service creates a new FooReport resource, triggers a background service to start creating the report (this is where JMS could come into play, or just adding an entry to a specific table in the database) and reports the URL (e.g. /foo/reports/42) to the newly created resource back to the front-end.
  4. The front-end can check the status of the FooReport resource periodically or on request of the user by sending a GET request to /foo/reports/42.
  5. When the report has been created, several possibilities exist to inform the user that the content is now available, depending on how critical it is that the user gets a timely notification.
    1. The background service causes a email with the report and/or a download link to be sent to the requester
    2. If the user can receive push messages (e.g. through websockets in a web application), then that channel can be used to notify the user that the report is ready
    3. When the front-end next asks for the status of the resource, the status indicates that the content is available, either inline in the resource representation or as a download link.

In this design, there are three parts: - The front-end that provides the UI to the user - The API service that provides a REST API - The background service that does the heavy lifting

Depending on the number of concurrent requests for creating a FooReport, multiple instances of the background service might be needed and you have to think about ensuring that each service instance works on a different report.
For a low number of concurrent requests, just recording the status in the database as 'pending', 'processing', 'created' should be sufficient and that also makes it easy for the API service to report the correct status.

0

Your instinct to save some state regarding this download is a sound approach. Generally queues and databases have some overlap. E.g it is easy enough to implement some parts of a message queue using a database. Where queues really shines is when we need to get a high number of messages to a consumer with low latency(few milliseconds). Using a database like this often involves polling which reduces the latency to maybe a minute.

If you a more comfortable using a database and you can live with a little extra latency(sounds like you can when we are talking days) then by all means use a database for this.

  • Thank you for elaborating. So if I have to bring database into play here with queue, I am trying to understand how would it work. I mean, is it like some part of data from the queue will get inserted into the database at regular interval of time? Also, if this is the case, then would retrieving the data be quick and easy by the user once the whole process is finished? – John Nov 1 '18 at 13:23
  • You basicly use your existing database as a queue. You could make a table called DownloadJobs in which you insert one row when user clicks "Download" Then from a console application you poll the table at regular intervals to see if something needs to be done. – Esben Skov Pedersen Nov 2 '18 at 6:21

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.