Questions tagged [elasticsearch]
Elasticsearch is an Open Source (Apache 2), Distributed, RESTful, Search Engine built on top of Lucene.
44
questions
0
votes
2answers
88 views
What is the advantage of Kafka connector to MongoDB over using MongoDB change streams?
I need to stream changes from my MongoDB instance to Elasticsearch. I also need to perform complex transformations on changed documents. After doing some research I narrowed my options down to 2:
...
0
votes
0answers
5 views
How to set up elasticsearch index from data with similar fields?
I'd like to set up Elasticsearch index for documents which have several fields which are somewhat similar. The fields are:
shortDescription
longDescription
alsoKnownAsInOtherLanguages
mongoId (the ...
0
votes
1answer
30 views
How to handle efficiently notifications for query searches results changes in elasticsearch?
Each user could save preferred queries. They want to be notified when one item that is the result of these queries is created or updated.
A naive solution would be to execute each queries and check if ...
0
votes
1answer
137 views
Do I have to use Filebeat to parse logs with logstash?
I was under the impression that Filebeat is only necessary if I wish to provide a constant, paced, fresh input to logstash.
But the tutorial here starts with configuring Filebeat to send log lines to ...
0
votes
0answers
21 views
Kibana Elasticsearch query autocomplete outside Kibana
Background:
I've started dumping several hunderds of GBs to Elastic Search in an attempt to make my query faster through denormalization. They are faster indeed!
At this point I would like to exposed ...
2
votes
1answer
63 views
eCommerce category path - names vs IDs
I am building an eCommerce website and there is a Category table in my MongoDB database which has a field called path. This field is used to store category hierarchy e.g Boots category will have /...
0
votes
0answers
29 views
Filtering data from elastic search based on mongodb
I've a list of items in my ElasticSearch.
User enters a query and I fetch the results from elastic search. Now, I've a requirement where I don't want to show the user the items which he has already ...
2
votes
1answer
73 views
Elasticsearch and RDMBS combination
I read both this thread Elasticsearch and PostgreSQL combination and this one Elasticsearch and relational database combination, but I could not come to an answer.
Let's say I have to re-create a ...
1
vote
0answers
99 views
Combine/Sync Amazon S3 and MongoDB Atlas with Elastic Search
I have many continuously growing (through scrapping) collections in MongoDB Atlas. The documents in each collection follows the following schema:
{
"source_url": "<some url on the web>",
"html":...
4
votes
1answer
89 views
ReactJS with Elasticsearch App Architecture
I want some advice regarding my architecture and hosting options.
I'm attempting to build an e-commerce site for e-books. It will use nestJS for the backend and ReactJS+Typescript for the frontend.
...
0
votes
0answers
96 views
Elastic Search inverted index and index containing actual documents?
I have gone through Elastic working and understands how inverted index(faster look up) and index(
storing actual document disk in ES). My understanding is that inverted index has most of the required ...
1
vote
1answer
763 views
Elasticsearch and PostgreSQL combination
I have a marketplace application and I stored data in a PostgreSQL. I have performance problem for product search, I know I can improve search performance if I use Elasticsearch instead of PostgreSQL ...
0
votes
0answers
126 views
Sorting and filtering: ElasticSearch vs MongoDB
This is a problem related to a typical e-commerce requirement.
I am using ElasticSearch for all the below use cases.
I am confused about whether or not to use MongoDB for the sorting part.
I have the ...
1
vote
0answers
108 views
Log data anonymization with ELK
I'm considering using the ELK stack for logging within our company. To comply with our data retention policy, we would like to anonymize data after one year so it cannot be traced back to an ...
0
votes
1answer
120 views
Is AWS Elasticsearch is completely managed service or is it just a Elasticsearch installation on set of ec2 instance
I want to understand the difference between AWS Elasticsearch service vs on-premise version. Does AWS provides a fully managed service for Elasticsearch (like DynamoDB, SQS etc) Do we have to manage ...
4
votes
6answers
7k views
Is it worth logging HTTP requests when they enter an API server?
I'm designing an API and have reached the topic of logging. I'm going to store my logs in Elasticsearch.
I'm certainly going to do some logging at the time the HTTP response is sent back to the ...
1
vote
1answer
300 views
Designing a Social Network with Graph Databases
I am prototyping a social network as part of a project I am working on and want to learn the right way to do it as if it were for an industry leading company.
I've decided on implementing the social ...
-2
votes
1answer
328 views
Elastic search vs chatbot?
I have an in-house built job scheduling application that accepts Job configurations in form of Apache Config files (something similar to XML).
There are tons of configurations (for example a way to ...
1
vote
1answer
267 views
Some Guidance on Parent-Child Relationships in Elasticsearch
I would like some guidance in setting up my document structure Elasticsearch. The company I work for has an app that stores around 20,000 new phone records each day in a SQL database. And we feel we ...
1
vote
1answer
1k views
Logging architecture in microservices
We are going to develop an enterprise level application which will be run by a dozen microservices. Every microservice will be inside docker and all this will be orchestrated by Kubernetes.
Obviously, ...
1
vote
0answers
78 views
Java dashboard for machine data
I am currently working on a project for a production facility. The machines in the production are saving so called 'Telemetry Data', measurements like water pressure, plastic volume or temperature ...
5
votes
1answer
253 views
Design point for product search and auto-complete using Elastic search?
I need to implement product search using Elastic search similar to it is done on any scalable ecommerce site.I am planning below algorithm
for this
Whenever product is added to system, First add it ...
1
vote
0answers
46 views
Giving search on customizable fields
I have some user defined custom fields corresponding to a ticket (A ticket is an entity which has subject, status and custom fields). I need to search tickets based on its custom field values.
A ...
0
votes
1answer
91 views
Problems with evaluating results of search engine by comparison
We're building a search engine at a client's place. To evaluate the results, the client is comparing top N results of our search engine to top N results of a competitor. And they want me to get at ...
-3
votes
1answer
370 views
Design the cache and search feature like hotel booking?
I need to design the feature like in encommerce(consider hotel booking site) we can enter the keyword/partial-keyword and need to return the relevant results.
For Example :- If site admin or hotel ...
1
vote
1answer
376 views
Data layer Abstraction from Business layer for search
This is regarding designing a search api. We have 3 tier architecture now using Spring MVC. All communications are via REST api.
UI <-> Business <-> Data(sql).
Now to implement faster ...
3
votes
3answers
1k views
Architecting a universal search for a product with microservices
We are building a new product in real estate space and the end users of this product are not so tech savvy. To have better user experience with our product, we want our users to find relevant things ...
2
votes
1answer
118 views
Decouple service complex query building from datastore
We have a very large search service (written in golang if it matters) that runs on top of ElasticSearch, receive requests, builds the correspondent query, and returns the results (with some post-...
2
votes
1answer
1k views
How to handle UI updates dependent on slow API responses
A side project I'm working on with some friends currently deals with an API interface that, for many reasons, is slow and cannot be changed to improve speed. I'm talking API responses that take ...
3
votes
1answer
2k views
Why is elastic search popular? [closed]
Elastic search is basically about indexing of data.
In database world,
Multiple indexes can be created on a MongoDB collection
Collection in MongoDB can be schema-less.
In MongoDB, BSON encoding ...
7
votes
1answer
3k views
ElasticSearch vs SQL Server full text index for small datasets?
I've got a relatively high-traffic public-facing product-based website backed by SQL Server as the point of authority. It has some search capability on some of the columns of the Items table (year, ...
1
vote
1answer
130 views
Is it good to consider elastic search a datastorage?
I am very new to elastic search and i need to know it is even useful to store data?
because i was curious to try my hands on how elastic search works and i used postman>run elastic search then added ...
3
votes
0answers
1k views
How should I store user activities in ElasticSearch and figure out popular searches?
I've got java logging user activities to Fluentd, Fluentd is then writing these activities into an elasticsearch index.
Every user activity is recorded, some include:
User1 follows user2
User1 likes ...
0
votes
1answer
2k views
Creating a facet filtering using Elasticsearch
I have been asked to create a filtering functionality for e-commerce application written in PHP. Since the app is using MySQL database, I made the decision to use Elasticsearch with Logstash and ...
2
votes
1answer
842 views
How should I implement a full text search into my meteor web app?
I am making a web application using the javascript framework Meteor, which runs with mongodb. In the web app I have a db set up that holds individual video games in the documents. An example of this ...
3
votes
1answer
1k views
Best practices for dashboard of near real-time analytics
I’m currently building a dashboard to view some analytics about the data generated by my company's product.
We use MySQL as our database. The SQL queries to generate the analytics from the raw live ...
0
votes
2answers
4k views
How to properly index MongoDB with ElasticSearch?
we are working on a JAVA EE project which handles huge amount of data, but has to provide full-text-search option (in hungarian language).
So we started to think about what kind of architecture could ...
2
votes
1answer
2k views
Feeding events to Elasticsearch, do I really need Logstash?
I'm designing an analytics system that feeds all events to Elasticsearch. The event lifecycle is as follows:
Visitor does something.
Custom analytics server gathers data, makes an event out of it and ...
6
votes
1answer
2k views
Elasticsearch and relational database combination
Imagine I have a marketplace application - where users can search for products (we concentrate on clothes). Every product has an ID, name (text), description (text), price (numeric), size (numeric), ...
0
votes
0answers
163 views
Using Google for full text database search
I have a lot of text and I am storing it in Elastic search. Using Lucene, NLP and Wordnet filters the search is good but not as good as Google's because none of these methods use AI for the search so ...
0
votes
0answers
505 views
Google-like search solution for an enterprise application stack using Elasticsearch
I know, title is a little confusing but the problem is, too. I am working on a company, it has several different applications and wants these applications have search functionality. We have developed ...
3
votes
4answers
2k views
Are NoSQL databases the best choice for more efficiently querying large amounts of data?
We are planning to build a travel website in which we will be integrating multiple APIs (eg. DOTW, GTA, Expedia) for Hotels. I have initially tried to use MySQL but since there are huge amounts of ...
18
votes
3answers
29k views
Why would I use ElasticSearch if I already use a graph database?
I don't find any deep explanation on the web about a comparison between ElasticSearch and the graph databases.
Both are optimized to traverse data.
ElasticSearch seems to be optimized for analytics.
...
7
votes
2answers
2k views
How does Elastic Search approach the problem of distributed membership and consensus?
I already understand that Elastic Search is supposed to be deployed in a distributed topology, in that you can have multiple nodes for a cluster of ES instances.
I like the API, and it looks promising....
