Why would I use ElasticSearch if I already use a graph database?

Question

I don't find any deep explanation on the web about a comparison between ElasticSearch and the graph databases.

Both are optimized to traverse data.
ElasticSearch seems to be optimized for analytics.
However Neo4j is also based on Lucene to manage indexes and some fulltext features.

Why would I use ElasticSearch if I already use a graph database ?

In my case, I'm using Neo4j to build a social network.
What real benefit may ElasticSearch bring?

UPDATE ----------

I've just found this paragraph:

There are myriad cases in which elasticsearch is useful. Some use cases more clearly call for it than others. Listed below are some tasks which for which elasticsearch is particularly well suited.

• Searching a large number of product descriptions for the best match for a specific phrase (say “chef’s knife”) and returning the best results
• Given the previous example, breaking down the various departments where “chef’s knife” appears (see Faceting later in this book)
• Searching text for words that sound like “season”
• Auto-completing a search box based on partially typed words based on previously issued searches while accounting for mis-spellings
• Storing a large quantity of semi-structured (JSON) data in a distributed fashion, with a specified level of redundancy across a cluster of machines

It should be noted, however, that while elasticsearch is great at solving the aforementioned problems, it’s not the best choice for others. It’s especially bad at solving problems for which relational databases are optimized. Problems such as those listed below.

• Calculating how many items are left in the inventory
• Figuring out the sum of all line-items on all the invoices sent out in a given month
• Executing two operations transactionally with rollback support
• Creating records that are guaranteed to be unique across multiple given terms, for instance a phone number and extension
• Elasticsearch is generally fantastic at providing approximate answers from data, such as scoring the results by quality. While elasticsearch can perform exact matching and statistical calculations, its primary task of search is an inherently approximate task.
• Finding approximate answers is a property that separates elasticsearch from more traditional databases. That being said, traditional relational databases excel at precision and data integrity, for which elasticsearch and Lucene have few provisions.

Can I assert that if I don't need approximate answers, then ElasticSearch would be useless compared to an already used graph database?

Answer 1

I hesitate to call ElasticSearch a database. It is not a replacement for a database, but it makes a good addition to add functionality, specifically advanced text searching, along side your existing database.

I see where you can get them confused. They can actually fit the same need, but not always. ElasticSearch does exactly what it sounds like, searches. A graph database doesn't specify relations or indexes, where as ElasticSearch does. So fundamentally they work quite differently. ElasticSearch analyzes documents with, for example, English analyzer. What this does it will take words and analyze different variations of that word or even synonyms. For example, dig, would be anaylzed as dig,digs,dug,digging,digger .... When you run a query on elasticsearch your queries can also be analyzed, then those words are queried for and can be scored by relevance.

ElasticSearch is a great tool, because it's really flexible. You can find a wide range of relative content, or you can find a needle in the hay stack, and its relatively easy.

Graph Databases have their advantage too. Finding relevance/relations between things like hash tags for example, or things with many mutable relations. They're great and interesting pieces of technology, however I'd have to say that its not as powerful as ElasticSearch. Mostly because ElasticSearch is geared towards this sort of thing, and it handles analysis for you so you can do full-text search. However if you're looking to use a system more so like twitter's search that's based on predefined tagging/keywords, then you'd be better off using the Graph Database your already using.

The question is how robust do you want your searching to be? If you have a need to do really fine grain(full text) searches I'd use elasticsearch. Otherwise you can always implement a search relatively easily on a graph database. Once you have search implemented its not impossible to migrate to elasticsearch if you find yourself later needing a more robust search engine, just implement your search with that in mind.

Answer 2

Both of these database has their specific need to solve specific problem at certain level of application requirement. Although we have not used Graph Database. But we are using elasticsearch with MySQL in one of our project from last 5 years. That project has a massive data to be searched through 6m documents and has massive relationships between those entities (10m relationship documents).

Use Case: Like search through hotels which have been liked by my friends and sort all hotels with the number of likes they have. And if you see it closely. this case has involved 2 relations (Friend, Like). So i need to search through Like relation ship between Hotels and My Friends and then hotels should be sorted by total number of likes they have. So for such searches, graph database is good.

Elasticsearch is doing great job for full test search in documents but when it comes to search through relationships like above it is not that good. List document(entities) who are my fans and sort them by their number of fans. But these are one level deep and when it comes to search more deeper. Elasticsearch is not good enough.

So understand your application requirement and then go for the database. You may need to have both.

Answer 3

I think this question is valuable and a familiar question is here. And for your use cases Elasticsearch has many extra tools. See the list.

In my case; Elasticsearch + Kibana stack is great to store logs. You can visualize and monitor your logs with Kibana.

And secondly utilizing it, as a search engine. Elasticsearch is a full-text search engine and it's easy to combine with relational databases or other graph databases.

Elasticsearch introduction says:

- Add a search box to an app or website
- Store and analyze logs, metrics, and security event data
- Use machine learning to automatically model the behavior of your data in real time
- Automate business workflows using Elasticsearch as a storage engine
- Manage, integrate, and analyze spatial information using Elasticsearch as a geographic information system (GIS)
- Store and process genetic data using Elasticsearch as a bioinformatics research tool

In other words, a database is for retrieving and managing data. But Elastic stack is used for searching and analytics.

This competitive chart may help you to identify the roles in a project.

https://db-engines.com/en/system/Elasticsearch%3BGraph+Engine%3BNeo4j