2

I am in the process of designing a database organizing real world items in different storage locations.

One main principle is that items can contain other items.

So the relevant table is:

  • An "items" table in which one column "insideOf" has a FOREIGN KEY restraint to the id of the item containing it.

Setup

the table could look like:

items:

id name insideOf
1 "graue box" 5
2 "rote box" 1
3 "kabel blau" 2
4 "keller"
5 "Regal metall" 4
6 "pappkarton" 5
7 "MacBook Pro" null

My goal is to create item records that include a string path looking like

4/5/1/2

which would be the example for the path of containers of item 3.

I learned so far that i could use a SELECT query with a common table expression like this:

WITH RECURSIVE cte_name(depth,itemid,itemname,itemcontainer) AS(
    SELECT 0, items.id, items.name, items.insideOf -- non-recursive term
        FROM items
        WHERE items.id = '3' -- start item
    UNION
    SELECT depth+1,items.id, items.name,items.insideOf -- recursive term
     FROM cte_name, items
    WHERE items.id=itemcontainer
    AND depth < 5 -- limits depth on request
) SELECT * FROM cte_name;

this works, but delivers me more infos than i need.

So I built a database function getContainerPath that returns only the required path string, that can be then easily queries like

SELECT * FROM getcontainerPath('3')

and looks like:

DECLARE 
 ret TEXT;
BEGIN
    -- QUERY: get all parent items of startitem (reverse tree) and return one value string in path form
    WITH RECURSIVE cte_name(depth,itemid,itemcontainer,path) AS(
        SELECT 0, items.id, items.insideOf, '' -- non-recursive term
            FROM items
            WHERE items.id = startitem -- start item
        UNION
        SELECT depth+1,items.id, items.insideOf, CONCAT_WS('/',itemcontainer,path) -- recursive term
         FROM cte_name, items
        WHERE items.id=itemcontainer
        AND depth < 5 -- limits depth on request 
    ) 
    SELECT path FROM cte_name -- only path
    INTO ret
    WHERE itemcontainer IS NULL -- only last path
    ;RETURN ret;

END;

the actual issue

Works fine as a single query, BUT:

  • items will be queried a LOT. Most item queries will request the parent-child (container-item) tree of this item.
  • Using this function on every standard item query and particular while retrieving lists of items, will quickly get performance heavy
  • I can limit the tree depth to 4 or 5 steps.

Possible Solutions

possible solutions I came up with but am terribly uncertain of:

  1. item CREATES, UPDATES are less common than pure READ queries, yet still fairly common. So I could add the getContainerPath function to CREATE and UPDATE CHECK Event and add an extra column containerPath to the item table containing each items full string path. But then I would have to CHECK the whole old tree as well as the new tree and UPDATE every item accordingly on every UPDATE in the path.
  2. I think I could follow the Nested Set Model described in this article but this also describes a tedious amount of extra work on every item UPDATE, DELETE or CREATE for the left and right neighbour nodes have to be updated. Because items will move a fair amount. I am uncertain if this could ever result in less performance costs than 1. Plus: As a SQL noob and human being I find it extremly hard to read and that will result in faulty queries and functions. And i could only find a tree traversal from the container/parent downwards and am unsure how it would work the other way around in my case.
  3. I could imagine leaving the querying of getContainerPath() to the clients/scripts connecting with the database, to split an item READ query up into 2 distinct queries (one to retrieve basic item info, a second one to retrieve the containerPath if necessary).

I am absolutely unsure if any of those ways would be better or worse for my use case. Maybe I am missing the correct idea or search term to help with this decision?

Or maybe there is a well weathered solution/best practise for the item-container relationship I try to model?

9
  • 2
    I have seen a structure very much like yours in a production database with millions of items work fairly well. Though, the concrete path was not queried very often there.
    – marstato
    Commented Feb 3, 2021 at 12:10
  • 1
    Also, this question is too broad to answer as is. What have you tried? Have you tested this solution and measured its performance? How performant does it need to be?
    – marstato
    Commented Feb 3, 2021 at 12:11
  • 3
    Fundamentally, a relational database is not a good fit to model hierarchical relationships. There are (actually older) databases based on a hierarchical model, and newer variations like graph databases that can also directly model hierarchies. Read en.wikipedia.org/wiki/Hierarchical_database_model and en.wikipedia.org/wiki/Nested_set_model Commented Feb 3, 2021 at 14:47
  • 2
    Setting up a table with ~10 million rows, 4-5 levels of nesting and 10 (??) items per level should be pretty quick. That would allow you to measure performance. My guess is that, at least for the start, performance will not be an issue. I agree with Michael that, squeezing out more performance from a relational storage will be very hard. However, to a graph database, this task is a piece of cake.
    – marstato
    Commented Feb 3, 2021 at 14:53
  • 1
    Performance is not the only reason to go one way or another. It's a matter of adequacy. That's why I have suggested testing other DB systems. See if other DB systems provide you with features closer to your needs and requirements. Relational databases are not silver-bullets, despite they can back many of the solutions out there. Otherwise, there wouldn't exist other types of databases.
    – Laiv
    Commented Feb 3, 2021 at 15:34

2 Answers 2

1

What is the best practice for hierarchical container?

You have already described it with suggested schema and query. This is a standard setup, and you have it exactly right.

actual issue

... while retrieving lists of items, will quickly get performance heavy.

No.

That's what indexes are for.

The backend will retrieve the parent rows and traverse your shallow hierarchy with comparable performance to simple requests for single rows. What you are doing is bog standard, and DB vendors have examined, benchmarked, tuned such queries so your application will be performant.

Pick up any of Oracle's "dead tree" documentation from the 1980's and you'll see this as a CONNECT BY query against the emp employee table, navigating an HR org chart to find total headcount under a VP. Or consult the docs for postgres, MariaDB, t-sql or other major RDBMS to find self joins, recursive CTEs and the like.

You didn't specify a performance target in your requirements document, and you didn't show any measured timings that fail to meet such target, so there's no performance issue here. You could choose to denorm, trading storage cost for query performance, perhaps by adding an UPDATE trigger. But there's zero justification for it at this point. Populate the table, index it appropriately, and worry about such tweaks down the road if a problem arises. I predict you'll have bigger fish to fry.

0

Entity–attribute–value database model could accommodate variable depth hierarchies. To sketch a solution an items table with 4 columns id, name, inside_of, linked_to where inside_of and linked_to are foreign keys referring id column:

id name inside_of linked_to
1 1 1
2 1 1
3 1 2
4 4 4
5 4 4
6 4 5
7 4 6
8 4 7

and the PostgreSQL query to retrieve the hierarchy path with / separator would be:

SELECT string_agg(id, ‘/’) FROM item GROUP BY inside_of ORDER BY inside_of, linked_to

Inserts require information just from parent record.

A sketched idea with the overhead of the memory to store the information for linked_to column.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.