-1

I'm developing a social network with Django (Python) + Postgres SQL. My site will have a chat feature so that users can communicate to each other in real-time, and the communication will be only from user-to-user (so there won't be chatrooms with more than two people).

Let's say that in the future my social network has ten millions of registered users (I know, I know, but for the sake of my question let's assume that this happens) and an average of 20,000 chats open between users at the same time 24/7.

Assuming that I run my app on the Cloud (Digital Ocean, AWS or whatever) with a traffic balancer, can I expect my Django + SQL app to run seamlessly or should I use Node JS + noSQL to scale my app without so much pain as it grows?

I heard that the ME*N stack is meant for these kind of use cases (real-time applications with thousands of concurrent connections), but I already developed around 25% of my app in Django + Postgres and I get discouraged to think that I will probably have to re-do everything again from scratch. But on the other hand, I heard that some other big websites such as Instagram have been developed using Django, so I don't know what to think.

I'm aware that it's possible to connect Django with MongoDB, but I still have the problem with managing the big amount of concurrent real-time connections... Plus, I will use React heavily on the front-end and it might be easier to couple it with Node than with Django.

What is the best decision here?

3
  • You don't need NoSQL or special technology to support many connections / websockets, but you do need an async backend. NodeJS has robust async support, but Django 3 has recently added async support as well (Python had async/await since version 3.5). It could still make sense to outsource some parts of your software to a separate chat service, and e.g. write just that service with Node. – amon May 24 '20 at 10:18
  • @amon Thanks. So even if I don't use NoSQL, will I still be able to store React components created by users in a relational database? – Fran May 24 '20 at 12:09
  • I don't know how you'd store user-created React components, but assuming you can represent it as a string or a binary blob, you can put it into any database. And in case you like JSON, even most SQL databases now have native support for JSON columns. It's probably best to stick to PostgreSQL by default, and only look at other DBs if you have special feature/replication/performance/integrity requirements (e.g. MongoDB has a better replication and sharding story, but laughable integrity guarantees and a more limiting query language). – amon May 25 '20 at 17:04
4

Always be prepared to do things over, sometimes you have to.

But also don’t let perfect be the enemy of good enough.

Since this is a first attempt, program it using the languages and libraries you know. Once you have a good feel for things and have actual data, you can refactor/rewrite parts of to meet your performance expectations.

3

So I did some googling, Django has its own set of stuff for websockets, Django channels and behind that Daphne servers.

There doesn't seem to be any documentation about how scaleable Daphe is. There are some post complaining about this lack of documentation.

But it is specifically written for your use case, so there is no reason to expect it to be less good than other open source solutions.

If you manage to get millions of users you will no doubt be bombarded by sales people wanting to sell you enterprise versions of everything under the sun who will be happy to tell you how much faster and more scaleable their version is.

1

"Fran, don't borrow trouble. You'll never get a good night's sleep if you do!"

(import Robert Baron's above response in its entirety!)

Okay, "20,000 open chats." In any given clock-millisecond, how many requests will we need to process, and how long will it take to process each one, and how delay-tolerant might they actually be before they begin to complain, "too slow?"

Never underestimate the actual firepower of even a single "cloud computer node."

And if they do, "can you 'throw silicon at it?'" (Check out services such as "pythonanywhere" or the venerable "rackspace." You have plenty of options ... and, you have the luxury of expanding into them.)

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.