7

I have the following scenario:

I have a user that needs to see some news feeds from different sites and I want to paginate them and sort them by date. The problem is I don't understand how to do that schemantically.

Fetch top-N from each site and then limit the results? That could result in inconsistent results if for example:

I ask the first 100 results from each site and then select in total the newest 100 from all sites. The second time I do this if site A had all of its first 100 results newer than the other sites , it is a problem. I will ask from the sites results from 100-200 where site B's results should come from 0-100 because they were not displayed the last time.

How to do this kind of aggreggation without fetching all available (or an obsene amount of results) from each site?

4

2 Answers 2

Reset to default
0

You have to remember the start point when paginating, so if you retrieve the latest results for all sites, you have to be able to repeat the query for the same result set - that will depend on the result data you get back, for example, I will use an ID value for my SQL data results, but it could be based on date for other systems.

Once you have this, you can repeat the query asking for next (or previous) pages of data. If you ever want to start again, you'll have to include a reload or refresh option that resets the start point. You only need the start point data for paginated queries, never the first one (though it might help if you have data that doesn't return something you can use, you might have to store the datetime you requested the data before calling, and pass that through to prevent a new entry appearing between the time requested and results returned).

1
  • of course, I need to keep a timestamp pointer AND paginate the results and always ask for the max number of elements I want from each site because they may come only from one source.
    – arisalexis
    Commented Apr 16, 2015 at 8:45
0

There can be no general rule.

If all of A's articles are newer than B why would you expect any B articles in your list?

You need to nail down your expected behaviour and make a decision based on that. Also think about what happens if one resource is unavailable at query time etc.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.