4

I've been using MongoDb (mainly with Meteor) and for me, it has been a great way to mock up prototypes and proof of concepts quickly.

I'm just wondering, after building the initial product, if your product is likely to grow and change, is it a good idea to stick with mongoDb?

Cause it seems like with MongoDb, you need to know a lot about how you're going to use the data upfront. From my experience only, I seemed to constantly be trying to decide whether a datatype should be a document in its own right or a sub-document.

Making it a document gives it flexibility, but if you have all your datatypes as documents, well then you pretty much just have an SQL database with foreign keys that have no foreign key constaints.

Making it a subdocument makes things perform better whilst being convenient - you don't need to join tables (which you seem to have to do manually) - but the problem seems to be if you need to query that datatype outside the context of the parent, it's very hard.

So for example, if I'm building a website with chatrooms, I might have two datatypes - chatrooms and textEntries.

Chatroom will be its own document. But textEntries can either be a document in its own right or a subdocument within Chatroom.

If I make TextEntries a document, then I would need to manually keep track of the relationship between chatroom and textEntries which defeats the purpose of using mongoDB at all.

If I make TextEntries a subdocument, this will make things easier but what if later on, I have a new requirement where I want to search across all TextEntries in all chatrooms for a particular word?

With a prototype, I know exactly what I'm building so these decisions tend to be based on just getting the product working. But longer term, I wouldn't know how or in what way I'm going to want to query the data so in light of that, would I be better off just moving to an SQL database?

  • I don't understand your comment "If I make TextEntries a document, then I would need to manually keep track of the relationship between chatroom and textEntries which defeats the purpose of using mongoDB at all.". Couldn't your Chatroom have an array of IDs of it's TextEntries? I'm missing something in your design - please elaborate. – user949300 Aug 26 '14 at 5:53
  • @user949300 : you could do that, but you'd still have to manually maintain the integrity of this list. Every time you added or removed a text entry, you'd have to apply it to that list. And you'd never be absolutely certain like you would with a foreign key constraint that the text entry exists. – RoboShop Aug 26 '14 at 7:36
  • Since all are likely to change, you are practically asking whether MongoDB is appropriate for any system at all. – Tulains Córdova Aug 26 '14 at 15:59
3

MongoDB is a NoSQL database that focuses on documents.

Document searching in general relies on arbitrary matches in the middle of large amounts of text, fuzzy logic/matching, and other tasks that are not cut and dry boolean logic comparisons that exist in SQL.

I personally know multiple developers at Hyland Software, the company behind OnBase which is a proprietary NoSQL database that efficiently searches millions of documents. To get data into the software, one must scan it in: OCR capabilities latch on to textual data which can later be searched. Rather than making an appeal to authority, I will say that this has enlightened me to the use of NoSQL databases in the enterprise and their proper role.

In a SQL database, you often must know the context of the data you want. Is it a name? Form number? Some other field? With a document-based NoSQL solution, you can search everything using logic that is less than ideal (for SQL) with minimal performance hit. You are not saying "find me records where field X contains Y" you are saying "find me documents containing Y" and the database is optimized for those types of queries. Documents often do not contain clear fields. While a letter (dear grandma) may contain an address, it is not annotated as such. In a RDBMS, the address fields would be split out and queryable. Want the street or city? It is distinct from other elements. In a document database, it is all jammed together. Searching for a specific postal code is not as clear of a query, but document databases are built for it.

Other times, relational databases perform better. Do you need records where you know the field is a specific value, or links to a specific record? SQL is the winner.

Martin Fowler states that each database type has its place, and often are used together. Based on his expert advice, I would not choose one database or the other: I would use both together, leveraging each where its strengths are greater than its weaknesses.

In the specific example of chat on a web site, it makes sense that a text entry would be a subdocument to a chat room (document). A text entry might be quoted out of context, but its identity is, in part, the chat room that contains it. Without the chat room, text entries are a jumbled, meaningless mess.

  • RoboShop also gives a usecase with the chatrooms. How would you handle this? – velop Dec 24 '15 at 13:37
  • @velop good point, I added a note about this. – user22815 Dec 26 '15 at 23:03
2

My general answer is that use any databases that you think would improve your service.

You wrote:

I'm just wondering, after building the initial product, if your product is likely to grow and change, is it a good idea to stick with mongoDb?

Yes, especially if the schema is going to change, MongoDB is easier to maintain in this case. In a way, it is one of the reasons why a lot of people actually prefer it. One can say it is one of its main use-case.

Cause it seems like with MongoDb, you need to know a lot about how you're going to use the data upfront. From my experience only, I seemed to constantly be trying to decide whether a datatype should be a document in its own right or a sub-document.

On the contrary, a lot of people prefer to just dump the documents in, and worry about it later. Obviously I am exaggerating, but it has its merits when you have clients who change their minds too often, or the project advances in a sporadic fashion due to entering uncharted waters. Obviously it makes it hard to use indices efficiently, but would be helpful when changing parts of your logic and you wouldn't have to migrate your data.

Making it a document gives it flexibility, but if you have all your datatypes as documents, well then you pretty much just have an SQL database with foreign keys that have no foreign key constaints.

Making it a subdocument makes things perform better whilst being convenient - you don't need to join tables (which you seem to have to do manually) - but the problem seems to be if you need to query that datatype outside the context of the parent, it's very hard.

So for example, if I'm building a website with chatrooms, I might have two datatypes - chatrooms and textEntries.

Chatroom will be its own document. But textEntries can either be a document in its own right or a subdocument within Chatroom.

You are very correct. You could experiment by denormalizing your data, or fetch your data in 2 steps like in below (using a imperative logic for demo). This will be quite fast, most likely faster than a sql solution. (make sure to index the queried fields)

var user = db.user.query({_id: user_id});
var chatrooms_id = user.current.chatrooms_id;
var messages = db.textEntries({
  chatrooms_id: chatrooms_id, 
  created_at: { $gt: since_last_checked }
});

If I make TextEntries a document, then I would need to manually keep track of the relationship between chatroom and textEntries which defeats the purpose of using mongoDB at all.

You also have to make sure you don't surpass the 16 MB document limit, if you decide to jam a lot of data into on document. For that reason, storing all those textEntries within a chatroom document is overkill and would not work with large rooms. If you missed this link on data modelling examples in the documentation, it would help to have a look.

If I make TextEntries a subdocument, this will make things easier but what if later on, I have a new requirement where I want to search across all TextEntries in all chatrooms for a particular word?

I would just use ElasticSearch for that. I am unsure if a sql solution would be much faster than Mongodb anyway. Their full-text search systems are generally not very performant, compared to a dedicated solution like ES, Sphinx, Lucene, Solr, just pick your favorite.

With a prototype, I know exactly what I'm building so these decisions tend to be based on just getting the product working. But longer term, I wouldn't know how or in what way I'm going to want to query the data so in light of that, would I be better off just moving to an SQL database?

I would use all three: sql, nosql, text-search engines. Further, I would also add in-memory, columnar and graph databases to this, if your needs require them. Why limit yourself? It is not one or the other, it is which ones will get me where I want to go the fastest within the time I have.

For example, you could use MongoDB for client-facing writes, as it is faster to write. You could sync those results to a sql db, and do analytics and certain queries on that. You could keep last 15 minutes of text entries in Redis, so you would not even hit the disk for most cases.

1

Since MongoDB has no schema many people would argue that it is more appropriate to systems which are likely to change, although I completely agree with you that you need to carefully structure the data appropriately for your specific use case.

Regarding the use case you mention, it may be worth keeping in mind the maximum size for an individual document - 16MB I think, though admittedly that would be one busy chat room -

It may be my advanced age but I feel that the current shift to NoSql first is probably mistaken. It can be a lot simpler mapping object graphs to Document Databases but there are some significant compromises which come with NoSql and personally I feel that using an RDBMS until you start hitting the performance limits that it imposes (or have some specialised need an RDBMS cannot fill) is probably a safer bet.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.