Auto-completion :- When we start typing, does system suggest the product from cache or hit the DB every time or hit the DB only when no result found in cache ?

Search:- When enter any product, does system straight away hit the DB or tries to fetch it from cache first ?

Orders :- I believe here order must be straight away cached in DB. No cache must be involved here.

Can someone provide insights here ?

  • 3
    How can we answer this question without being a backend engineer working for Amazon? Aug 7, 2018 at 5:54
  • @ChrisCirefice I understand your point. My intent was how to design these features for scalable site like amazon. May be Amazon does not use that design but that design can stand for that scale. I have modified the question to reflect the same Aug 7, 2018 at 7:18
  • So I think you have an X/Y problem here. I don't think any highly-scalable system (like Amazon) does most (or even any) of their search in the database. Most, if not all, use a highly-scalable solution like ElasticSearch or Solr. I answered your question based on the assumption that what you're looking for is actually not cache vs db, but how to scale search functionality. If this is what you were really thinking of, you may want to edit your question again to avoid downvotes! Aug 7, 2018 at 8:28

1 Answer 1


In every system I've ever designed, search and autocomplete are run through a secondary system such as ElasticSearch, Apache Solr, Sphinx, etc. Generally your application accepts the request with search parameters and passes those off to the secondary system. The reason is because databases have their limits for search-related things. For instance, PostgreSQL native text search doesn't support stemming in all languages, i.e. Japanese or Chinese, which makes text search in those languages less accurate than it could be. Dedicated search systems like those I mentioned have very specific dedication to perfecting search. This concerns not only language-specific issues, but performance optimization as well.

For example, my Ruby on Rails application accepts a number of parameters such as query and collection. If it is an autocomplete request, the server passes the request onto ElasticSearch and only returns results where query matches word_start in a particular collection (filtered by the collection's unique id). You can think of this as searching film titles for the drama genre. The results are then sent back to my server as an array of ids - at this point, my server performs a SQL query like:


The ids ElasticSearch returns are the ids of the records in the table that is being queried.

For a full search, ElasticSearch does the exact same thing, but with more parameters available in the search (think the filters you have available on Amazon after you do a text search).

The point is, the only thing that the application server is doing is:

  1. Accepting an HTTP request with parameters
  2. Passing that request along to an ElasticSearch server
  3. Accepting the HTTP ElasticSearch response
  4. Doing a very simple SQL with the condition WHERE id IN (LIST_OF_IDS)

So the actual search functionality is passed off to a secondary system with its own scalability, monitoring, statistics, configuration, etc. If my application server can handle 5x the web requests compared to searches based on response time, I can have 1 application server instance and 5 ElasticSearch instances for proper load balancing.

I believe that this is how most web applications like Amazon scale - they know what the throughput is for their web servers, and know the throughput of the search, and balance the resources accordingly.

Additionally, I have seen a few systems where search is pulled out into a microservice as a REST API. Basically, all search-related queries go through a standard REST API which connects to the provisioned "search engine" (ElasticSearch, Solr, etc.). Then, the system is more logically separated and can be more finely tuned to meet whatever throughput requirements are in play.

Now, I'm not sure what best practice is here, as I've never designed something as complex or high capacity as a site like Amazon's, but I think there are a lot more things to think about than separation of concerns and request throughput... i.e. if you have multiple applications using the search functionality or you want to expose it as a public API, having search as a microservice might make a lot of sense. But, at that point it's a business concern and not necessarily a technical one.

  • But elastic search also stores the data in its internal db. So for application it will come from elastic search but elastic search will get it from DB internally using it's internally maintained index. Is n't it ? Aug 7, 2018 at 13:07
  • @user3198603 I have no idea how ES works internally to build and use a search index, but it is open source if you want to dig around to find out. In the end though, does it really matter what ES is doing internally? For the record, I’m 99.9% sure that it doesn’t use any sort of internal database for the actual text search functionality. Aug 7, 2018 at 13:24
  • Per my understanding after reading ES is it stores all indexes in internal DB and search uses those index only Aug 8, 2018 at 7:24
  • You must set up a way to sync both DBs. Updates to products must be propogated to the search index. It's not easy work, but not terribly difficult either. Sep 6, 2018 at 9:58
  • @GregBurghardt Depending on the tools you have, it’s either incredibly complex or insanely simple. For example, I use Seachkick ElasticSearch with Ruby on Rails and it’s literally a few lines of boilerplate code to keep the search index updated after an insert, update or delete. Sep 6, 2018 at 14:45

Not the answer you're looking for? Browse other questions tagged or ask your own question.