3

I have a project that I'm working on where a user uploads a big file which gets parsed by the backend, and then returns the data back in a friendlier format. Now I'm wondering whether I actually need a database to store it, or should I just send all the data back to the user in JSON and let JavaScript to show the data properly on button presses?

I'd like to add a database and have all project organised MVC, but I feel like storing the data and then extracting it to serve it to the user and querying it at each button press is unnecessary and would slow down the communication.

I am aware that using the no-database method would mean other users can't access the data that the user who uploads the file can see, but that's not a big issue. Any other issues that I should consider?

1
  • 1
    The same user would not be able to access the data later, or if there is a temporary network issue after the file is loaded but before all the parsed data is served. You need to work with your users to know how important a requirement that is. Can use Kafta or other middle ware to async db write so front end can get the data asap. – tgkprog Apr 24 '15 at 17:11
1

It mostly depends on the size of the data you return and whether the user is expected to use all the data at once.

For instance, if it's a list containing hundreds of thousands of complex entries:

  • The response served as a single JSON will be rather large, and:

  • It is unlikely that the user will actually need to see all the data at once.

Instead, the user will prefer accessing the data either with paginated table or a table limited by criteria or based on a search, or in a form of a report which can eventually be configured on the fly. In those cases, there would be not one huge JSON response, but multiple JSON requests, serving a tiny part of the overall data which was collected or generated.

On the other hand, it may be that there is no much to return and that the user will expect to see everything at once. In this case, multiple JSON requests are not really useful, and may even cause perceived slowness.

If this is an important project, make sure you work with qualified interaction designers who can advise you according to the actual situation.

Example

Take an example of an application which let photographers submit a JPEG image and shows them the EXIF data. You determined, together with your users, that there are two sets of EXIF information: one set contains the data the users absolutely need: such as focal length or ISO—the ordinary photographers' stuff they manipulate on daily basis; another set contains the data that might eventually be useful: interoperability index, bits per pixel—technical stuff many photographers will ignore during their whole career.

Let's assume you did a prototype which returns all EXIF data at once, and it appears that it is too slow. Indeed, some photographs contain a lot of data which may be problematic to return in a single JSON response.

So you decide to return the first set immediately and put a button "See more" which loads the remaining EXIF information. The second prototype appears a failure as well: users who actually need the remaining information find your application very unresponsive: they click on "See more" expecting the data to pop out immediately, and instead find themselves watching an animated "loading" GIF for a second.

What's next? An alternative would be to:

  • Flush the essential EXIF data immediately with a "See more" button,

  • Once the page is loaded, do another JSON request which loads the remaining data without the user noticing something is happening in the background. Once the data is loaded, either "See more" can be replaced by the actual result, or may act as before, just without the 1 second delay.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.