I would like some guidance in setting up my document structure Elasticsearch. The company I work for has an app that stores around 20,000 new phone records each day in a SQL database. And we feel we could benefit from the features that Elasticsearch provides (fast searching). So we are going to start storing these records in an Elasticsearch database, and provide some search tools that will leverage this as well.
I'm new to NoSQL and Elasticsearch (my background is SQL). So I've read some tutorials on Elasticsearch (and NoSQL in general) to get me started on this project. And I think that I understand (from a general perspective) how to go about this. The basic structure I want to set up is this:
--Owner (ownerId) --Call (callId, to, from, location) --Call (callId, to, from, location) --Call (callId, to, from, location) --Owner (ownerId) --Call (callId, to, from, location) --Call (callId, to, from, location) --Call (callId, to, from, location)
It's really straightforward. Each business (business=owner) has many calls that we store. And each call also has some additional info (that will be search-able), such as caller names, phone numbers, locations. Also, each owner has a unique identifier, and each call has a unique identifier. Also, calls from different owners never cross-pollinate. We never have a reason to join them.
Upon reading some docs and watching some tutorials, I see 3 possible ways I could implement this structure. I would like some guidance (from seasoned Elasticsearch developers) on these 3 methods. I feel confident that each method would work. But I'm not sure what the long-term implications might be. I need to make sure the method I use supports a high volume of adds and updates (20,000 each day), and still provides fast searching. It also needs to support the initial import of our existing data (we have around 4 million call records already in the database).
Here are the 3 methods:
- Parent-Child Relationship (some info found here, among others).
- Utilize one
callstore for all owners, and store the
Calldocument. When creating the search interface, I would add the
ownerIdas a hidden search param to make sure the user only retrieves calls for his/her business. (Note: this resembles a one-to-many relationship created by a FK in traditional SQL)
- Create a separate document store for each owner. For this one, I would create a separate
callstore for each owner. I would append the
ownerIdto the name of the document store. So I would have a store called
calls_8j934jok83and another one called
call_98ged34h2, and so on (
Having no experience in NoSQL or Elasticsearch, I feel like I am playing with fire here. Like I said, I feel that any of these three would "word". But I want to make sure I use best practices, and that the structure I set up will be scale-able (we have around 20,000 calls coming in each day now, but we are growing).