In an upcoming project I need to create a Twitter like web application where the basic idea is to read / write short messages and status updates (similar to tweets) and group these messages (just like hashtags)

Also users can have followers and can see their status updates.

I'm estimating between 10 - 50 million message writes in a span of 1 - 2 months, and for reads it could be anything in the millions. Reads will surely be higher than writes.

I already decided that some kind of NoSQL is probably the best approach to storing this data, but I don't know which one would be the best fit for my needs.

I looked at Hadoop / Cassandra / HBase but it seems a little bit of overkill. I'm also considering MongoDB.

The whole web app will be setup on Nginx, PHP with a RESTful architecture (front-end only on mobile phones)

  • For this app, what do you see as the limits for a RDBMS? What makes the current list of NoSQL options over-kill? – JeffO Feb 10 '12 at 22:17

Why not use what Twitter itself uses? It looks like Twitter uses a few different approaches to get what you are looking for. If it needs to work exactly like Twitter as you describe then use the tools that they have proven work together. This is a great article that goes over where to start from:

How Twitter Uses NoSQL

Websites that have the kind of volume that you describe are not built in a day and may need several iterations to really work perfectly. I suggest getting in there and seeing what really works for your problems after real world testing.

Not the answer you're looking for? Browse other questions tagged or ask your own question.