I have gone through Elastic working and understands how inverted index(faster look up) and index( storing actual document disk in ES). My understanding is that inverted index has most of the required stuff when search query is executed. For example : word to document id mapping/location/TF-IDF/boost score etc

Say If I search for Java developer new york, Inverted index has all the stuff score/document id/primary key of record in DB to return as response etc.

So my question is should not we just store inverted index only but not actual documents on disk as query search is done on inverted index only not on documents ?

  • Both are used because they support different types of search. Inverted index is optimized for full-text search results while the table view is optimized for data filtering (i.e. discreet value matching or range queries) Feb 24 '20 at 14:28
  • @BerinLoritsch My question is Elastic has two parts i.e inverted index (for mapping the words to actual documents stored in elastic index) and index storing actual documents. Now when search query is executed, as all data is available in inverted index only(like location/TF-IDF/boost score), does it really need to access the actual documents stored in index ? Feb 25 '20 at 1:15
  • I have also edited my question to make it clearer . Feb 25 '20 at 1:30
  • If all you want is a full text search of raw content, then no, you do not need to store the document in Elastic. However, Elastic is your data store then you do need to store the document. However, outside of the inverted index, you also have a more traditional table index so that range queries (from date to date) and other discrete value filtering can be more efficient. Feb 25 '20 at 14:11

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.