The team i'm in is moving to use
couchbase rather than
elasticsearch DB. Now this is only an example but my question is more general. In
elasticsearch you have the notion of parent-child relationships . When we move to a standard key value store we don't have these. My Question:
Assuming I do want to enforce relationships such as:
- When parent is deleted all children should be deleted.
- When child is deleted parents which had a pointer to this child should be updated.
- Child is rather large doc so we have a pointer from parent to child.
Which design would yo go with?
Option 1: Build a general layer on top of the
KV store which knows to handle parent child relationships such that your programmers only need to specify relationships and your generic sophisticated-dao layer will know to handle these relationships.
Option 2: Relationships should not be managed by DAO layer so the above logical layer should manage it. No generic solution, each domain entity which has this relationships should write their own code in logical layer to manage this layer.
Option 3: Parents hold children in same doc - one doc (both parent and children) - not going to do that, children are too big, and there are other types of children as well. (more reasons I can specify if needed).
Option 4: Use specific db features - I want to avoid that so next time we migrate into a different db I won't need to face this problem yet again. A solution that will work for any KV.
Option 5: Don't move to KV store - move to a relational database, or keep current db(updated by thimothy comment), this is beyond my control, we will move to KV, so this option is out of question. There topmost preference is for speed above any db features.
Option 6: Have an async process which manages the relations, so if I delete a child (in our child-parent example), then if a client gets the parent, then fetches each of child by it's id it would see a child does not exist, only 30 min later an async process will see this inconsistency and fix it, in our case the client would need to decide if this inconsistency is a real error or a temporal inconsistency. This is a possible solution so named as option 6 however I fear that each developer would get who knows where errors and would need to decide whether this is an ok state and to ignore or not.
Option 7: Event sourcing, so that I keep replaying and rerunning at least or compensating in case of a failure where parent is updated and child is not, but I think this does not solve the whole thing child/parent, i would still have multiple commands and events and it's not like one transaction, right now although I would be happy to use event sourcing i'm not at this point maybe in future on a total refactor of codebase, no immediate remedy this can give me.
I know that this kind of questions tend not to have a definite answer I would like to know if anyone faced this general question before, and wether it's best to come up with a general framework a general DAO which knows to manage these relationships or let any developer manage his own relationships.
Please take into consideration that giving each developer the responsibility to manage his own relations is dubious because the more developers you have the more you want to have some base infrastructures and not reinventing the wheel or being inconsistent in between teams (think enterprise). However in my case i'm not sure I do want to build this whole architecture and layers above a KV, my tendency is currently, If I have a KV allow the end-user-developers manage by own the dependencies however this leaves too much room for each to come up with his own solution, which option would you choose?
UPDATE: I think for me the best route to go would be under the assumption of a KV store without relationships:
- Do not implement generic relationships layer on top of the KV store.
- Light DAO layer.
- Each logical/service layer which uses the light DAO layer would need to implement the relationships in it's business logic.
As an appendix to that: if a programmer faces an inconsistency (parent points to a child but child does not exist) then he would need to deal with that with his own custom code + Consider an async process to cleanup these inconsistencies.