Data Storage Architecture: Storing Hierarchical Data (JSON/BSON)

Question

Say you have the hierarchical storage concept of "Foo." "Foo" holds an id, various values, and zero to many "Foo." Obviously, in turn, the child elements can hold zero to many Foos. The data is purely hierarchical: a given Foo will belong to zero to one Foos, and thus there is never a concern of multiple parents.

The data is stored in some sort of data format (JSON, BSON, XML, etc), within a storage technology (such as MongoDB, flat file, etc). We'll assume that both of these are unknown, so a system agnostic architecture is desired.

What architecture principles/choices would dictate if the objects should be stored hierarchical (directly into the storage medium with children existing as actual children) or flatly, with all parents and children at the same level (and references to children via ids).

---

My inclination is that storing the objects in the natural hierarchical pattern is the ideal, however I'm uncertain if this is possibly a performance concern (if a specific child is needed to be found, which would require some sort of searching through the tree structure?).

I've seen Web APIs exposed both ways, and code repositories dealing with both forms. I can't, however, find the decision points/architecture concerns formally dealt with.

Answer 1

I think two issues to consider are:

1. (and you're referenced this above) - do you need to find child nodes ? In which case you perhaps want to avoid storing sizeable hierarchies and instead store the individual Foos such that they're immediately searchable. That may impact tree construction/retrieval times
2. Do you want to move children between trees? If you do, again, I would look to keep these separate otherwise you'll have to search for these and then detach and reassign to a new tree

I think the APIs for clients reflect what the clients want to do, rather than the underlying storage mechanism. Understanding what the clients want (as per the above, do they want to search, rebuild etc.) will inform your decision further.

I note also that you're completely undecided on the underlying storage mechanism. A document-oriented database will have very different characteristics from a flat-file solution. As such you should consider the client-facing API separately from the underlying technology and let the client requirements inform your non-functional requirements.