2

How do the blogging sites, Q&A sites (or any other sites host prose like content) store their data? That is, how do you store blogs and Q&A content in the database?

I hope it is not good to store this huge text data inside RDBMS table columns.

closed as too broad by Simon Bergot, gnat, user40980, ChrisF Jan 6 '14 at 21:07

Please edit the question to limit it to a specific problem with enough detail to identify an adequate answer. Avoid asking multiple distinct questions at once. See the How to Ask page for help clarifying this question. If this question can be reworded to fit the rules in the help center, please edit the question.

  • 1
    Sharing your research helps everyone. Tell us what you've tried and why it didn’t meet your needs. This demonstrates that you’ve taken the time to try to help yourself, it saves us from reiterating obvious answers, and most of all it helps you get a more specific and relevant answer. Also see How to Ask – gnat Oct 10 '13 at 10:07
  • For stackexchange, many posts are < 1 KB large (assuming UTF-8). This is smaller than the block size on my filesystem! “Huge” is the completely wrong word to describe this. A PDF or an image is large, with sizes in the megabytes. – amon Oct 10 '13 at 11:48
7

It's perfectly good to store huge text data inside RDBMS table columns. Provided appropriate type (CLOB, TEXT or BLOB, depending on specific database) is used. Many RDBMS even have an extension for creating full text index.

In contrast images are usually stored in filesystem, because then they can be served statically by the web server which can than often use the sendfile(2) optimization. Since the text has to be embedded into the HTML output, no similar optimization is possible for it. The text is also not really huge.

  • Actually, it's a lousy idea to store text and images that you're not going to search in a database. But you're absolutely right, all the major blogging platforms do exactly that. The benefit they gain is that backups are consistent, without having to coordinate DB and filesystem backups. – Ross Patterson Oct 10 '13 at 10:29
5

Depends on the size and functionality of the web site. As a first step they will in fact store this information in a relational database in many cases. Though you won't use a normal index on large text fields. But it is still a very good model to keep all the relations (which answer goes to which question, which question or answer was posted by which user).

For the large text parts you will need full text search. Here you have two options. Some RDBMS have the ability to create such a type of index. Otherwise (or if you want very specific full text search abilities that your DB doesn't offer) you will copy this part of your data into some full text search engine like Solr, ElasticSearch, Sphinx (or any other, there are lots of them).

There are alternative options with NoSQL databases too of course and some may decide to use those for some advantages those may offer for their project.

5

Large texts stored in RDBMS aren't unusual, in fact, most Q&A and blogging sites make use of RDBMS for storing their content, normally as TEXT or CBLOBs or BLOBs

i.e.

  1. StackOverflow uses SQL Server 2008, which an RDBMS (info may be out of date since article dates back to 2008)
  2. Wordpress requires MySQL to work

Given that these examples use RDMBS, and still manage to work well, should be a testament that RDBMS are more than capable of storing large walls of text for blogs and Q&A sites. Though depending on your use case, you may refer Thorsten's answer for unstructured databases like NoSQL

Not the answer you're looking for? Browse other questions tagged or ask your own question.