1

I have a page where I list some products based on search data. The products shown are not saved in local database, but are taken live using API (and this is how it has to stay).

When page loads, it first lists products from one provider (first API call). After page finishes loading, then with Ajax the 2nd call to 2nd provider is made, product list comes and is appended to the page.

Considering each api call takes 2+ seconds to return data, I can't leave more than 1 call on page load.

My question is: How to do pagination in such scenario?

Idea 1

Doing static pagination: Pagination is done in browser. This is really simple, but on large result sets it might slow down the device, or even crash the page.

Idea 2

On first load, cache all results and do pagination using cache file (dificulty: medium, also it will not be ok to keep that cached file for too long, since data can change frequently)

Idea 3

No cache, pagination doen on server. Here I have no idea how to do it.

Things to consider

After figuring out the idea behind pagination, I'll also have to add filters and sorting.

Using php, mysql, js.

  • It is unclear what path are taking the ajax calls, do they call your backend which call the API, or are they done to an external API directly ? Also, what size is roughly an api call, at maximum ? If this is just an json list of say 1.000 text elements it's not the same than a huge html list with embedded media. – Arthur Havlicek Nov 20 '18 at 21:33
  • @ArthurHavlicek I did not give many details so that there would be less constraints to the solution. Ajax calls backend which calls API. Long list of small html elements. – DonJoe Nov 21 '18 at 2:02
1

Let's review your options :

  • Option 1 is unpracticable if the result set is very large, the page might become unresponsive and the user prompted to stop the script. This is only true above a few thousands elements. But if on some search result this is the case, it cannot be considered.

  • Option 2 - caching solves the problem of latency, but not the problem of volume, a huge dataset in cache is still a huge dataset. I don't really understand how caching simplifies the problem of pagination.

  • Option 3 don't have the same drawbacks and only require server processing power. This is why it should be considered.

When working with a database the pagination is handled directly by sending a dynamic limit parameter to the database. Here you only have the option to post-process, so when calling the api in the backend, crop anything above page * pagesize and below (page + 1) * pagesize results. You will probably need to parse the html to do this.

You can later add option 2 which would enable you not to request the API with the same search whenever asked for page 2. You would get the cached request for page 1 instead. Libraries should make this job quite easy.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.