I have a mail_queue table that is written to by web requests then worked on by a worker to send the mails. I've now had a moment of enlightenment re. what I was not comfortable with in the design.

Before: One database table called mail_queue, which the worker processes, & adds a sent_at field when sent. That means to get messages to send, it has to do a select from where sent_at is null. To be parsimonious about database tables I don't really need another table, because a null sent_at can entirely well indicate a message still needs to be sent.

Yet, there was something I was not comfortable about having a queue that was not purged. What if the sent_at field got nulled by accident? I can't likely see this happening, but even so, I was uneasy with this design.

Reading up on queues a bit more I encountered the idea that some queues may be worked on by multiple types of process. I can see that what my worker could do is take the record off mail_queue, process it, and if successful, delete it from the queue, plus add pretty much the same record back to a new table mail_sent, along with the new field sent_at.

What is the justification for having an extra table, when the first method outlined above seems entirely ok in terms of being parsimonious with database representations?

The answer I've come upon is that I can now see that, say, I wanted to listen for incoming webhooks such as delivered, the process which receives that webhook no longer needs access to the mail_queue, just the mail_sent.

Is that a kind of "decomposition"? It seems to me it is. It's not about splitting up operations; rather about splitting up data storage based on usage; "decomposition by role" "by usage", "by access". Are those a thing?

Just being a bit discursive here, it helps a lot to write things down and I really appreciate the opportunity to share here and discuss.

(I was just watching a video by Rich Hickey talking about Clojure which got me thinking about decomposition :))

  • What you are describing isn't really a queue. It's a database with the queue in the application. The database is just recording the state of the queue. There are proper queue servers that have proper queue semantics built in. This solution will always feel a little off because it is a little off. If you really just need queuing semantics, you don't need a database at all. However, if you need to read the data in the table as a table, and simply keep track if something has been sent, this works. – Berin Loritsch Oct 5 '18 at 19:59

What is the justification for having an extra table, when the first method outlined above seems entirely ok in terms of being parsimonious with database representations?

I don't see a justification for that. Maybe there were some special circumstances that led the author to prefer that design. Likely, he was just wrong about it.

The single-table design is better because it is simpler and does the same thing.

Is that a kind of "decomposition"?

Yes, it's a pointless decomposition. Decomposition does not magically make architecture better. Sometimes you gain nothing but introduce management overhead. This is the case here.

"decomposition by role" "by usage", "by access"

You cannot come to a good architecture by applying such generic "decompositions"/patterns. Good programmers have a mental toolbox of 1000 patterns. They tastefully chose which of them apply to a given solution. You cannot decide by any hard rule. Rather, you have to consider the whole of the system you are building and find the best possible architecture.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.