Data backend for a 'Nearby' places application

Question

Context

I am building an application for searching places 'Nearby' akin to Yelp/Google.

Objective for this question

Recommend a strategy to do paginated searches based on location and ask for advice on the same. Further, ask for frameworks and tooling that support the design.

Please note: I do not want to complicate this by doing any form of ranking, etc. This is simply 'Nearby' search.

Architecture

Requirement: User opens the app and requests restaurants in a 5 mile radius. The server may send 20 results back. User may exhaust these 20 results and request for another set of restaurants. The server may send another 20 results (that are not duplicate).

Solution

I propose a strategy that is based on 2 simple data stores - Redis to enable Geospatial Search & a generic data store for storing Restaurants (MongoDB/PostgreSQL).

Client (1) -> Server (2) -> Redis (3) -> Mongo (4) -> Client (5)

1. Client sends a request (REST/GraphQL) to search for restaurants nearby; params: location, distance, skip: 0.
2. Server handles the request and fires up a request to Redis to get 'locations' that match the query.
3. Redis stores data like "location -> restaurantID". Redis does a spatial search and finds all locations that satisfy the query.
4. Server then takes the first 20 in this list and batch queries Mongo to get a list of restaurants and sends it back to the client.
5. Clients handle any deduplication as I am unable to understand how to enable cursor based pagination for search.

For the next 20 results:

• Server skips the first 20 from Redis and handles pagination accordingly.

Considerations

• Simple indexing built without cost for exhaustive search indexing.
• 2 roundtrips to datastores per query.
  • Same query for redis for pagination
  • Don't know if possible but short-term redis caching may help.
• Current backend:
  • NodeJS + GraphQL + Prisma + MongoDB

Requires Advice On

• Is this conceivable for a medium sized production solution?
• Do these roundtrip costs make this service 'slow'?
• How to achieve pagination generally for geospatial searches? Is there a form of cursor based pagination when it comes to searches?
• What are other ways to accomplish the same? Happy to read through more complex topics and build a slightly more complicated solution.

Answer 1

Thoughts

Consider caching the full result set from the geo-spatial search.

You have a redis, so... it could go there. Now when the person requests the next page, there is a list tied to that location. You can page from that. You would still need de-dupping in the client for when redis drops the list.

Even better you could cache to the (location,page index) the document containing the 20 odd restaurants. This would speed things up even more.

If you reduced the location resolution, many users in the same N km square can be served from the same cached list. This not only makes caching more viable but also reduces your mean response time.

Consider having a batch process hunt for and expire cached data that is no longer correct. Be that because a restaurant was added/deleted or its details updated. This will allow you to maintain the cache as both fresh, and with the fewest number of updates needed. You may wish to balance this against a simple timeout policy per key. Your data will tell you what works.

Finally consider pre-computing all of that for at least your hotspot locations. Even better if you can do this for all locations with restaurants known to be nearby. This allows you to present your quickest response times for everybody.

Roughly:

1. Reduce Location resolution
2. Check redis for a restaurant page document at (location, page index)
3. If that fails, look for a previous Restaurant listing at (location)
   • if that resolves, request the page of restaurants, cache it at (location, page index) and return it.
   • perhaps pre-emptively cache the next 20 too...
4. If that fails, perform a geo-location search.
   • cache the list at (location)
   • request the page of restaurants, cache it at (location, page index) and return it.
   • perhaps pre-emptively cache the next 20 too...

In the background:

• Have a batch process expiring listings/documents with old data.
• Have a program looking for regions without a cached geo-search, near to restaurants. (Of course skew this to check for hot-spots).