I need to implement product search using Elastic search similar to it is done on any scalable ecommerce site.I am planning below algorithm for this
- Whenever product is added to system, First add it in DB and then in ES server. ES will create the index on document provided to it and then keep both index and document in memory .So it will kind of write through cache(where with Cache here I meant indexes and document in memory. Not to be confused with key value cache like redis/memcache).
- Say,my document in DB has 30 fields, but I need to search only on three fields(name,description and type) . So ES will have 4(3 + 1) fields . 3 are searchable fields and fourth field is primary key id representing unique key in DB
- ES will create the index on 3 fields internally while adding and keep it in memory.
- While search, search query will go the ES server for multimatch and fetch the relevant result with order according to boosting factor in query.
- Search result will contain the field id representing the DB primary key with which we can further fetch the product details from DB based on primary key id.
- We need to reindex the 3 fields again while ES server startup. For this I will fetch the data from DB during startup and give it to ES.
With this design all the searchable field will be in cached in memory under system at any point of time and there won't be cache miss and system does have to hit DB . Does this design looks good or I am missing anything here ?
Auto-complete Same design we can use for auto complete when user starts typing. For example when user starts typing say 3 letters, query will go to back end and above design will work there too. The only difference will be it will return only limited result say top 10 results instead of returning all results.
Will post a separate question about my question/thoughts how to make it scalable.