Break up django monolith into microservices

Question

We currently have a big Django web application: all data is centrally managed by the webapp, via models backed by Postgres.

We want to offer access to the data in a more decentralized way, and the solution that we are aiming for is to provide small uServies dealing with the different application domains. That is, from this architecture:

• Django + single DB

We want to move to the following architecture:

• Django (without data)
• uService 1 + DB1
• uService 2 + DB2
• ...

Thus we will have a smooth transition path: the webapp will keep working as before, but we will now have the option of offering access to the data in a fine-grained manner, with different requirements (for scalability, ...) for each of our application domains.

The refactoring must be done in the following way:

• The Django webapp must remain unchaged (except for the models). In particular, views and templates must remain unchanged.
• Each uService will deal with a problem subdomain, and will be completely autonomous, offering access to the data via REST. The technology for each uService is not defined: it could be DRF, or Flask, or Scala, or Java, it does not really matter; this is actually one of the selling points of the new architecture: each problem can be addressed with the right tool for the job.
• Access via the django webapp will ultimately GET/POST/DELETE/PUT from the uServices.

How can django be used in this way? Ideally we would still have models in the webapp, but instead of being backed by the database, they should access the REST endpoints to get / modify the data.

Are there any example applications using this kind of architecture in the Django world?

Answer 1

Yes, Django can certainly be used in that way: views could call your new services (uService1 and uService2) to fetch the data they need to render templates, or to mediate user interactions to PUT/POST/DELETE to your new services.

But I see lots of potential friction against Django's fundamental design: models are meant to be backed by a SQL datastore, not your own upstream business services. User interactions are meant to be mediated through forms, which are closely integrated with models...

It really depends on your exact scalability requirements, your team's size, and the nature of the problems you are trying to solve. However, on the surface with the details you provided, it seems like you should consider the following: Keep one large database, even if there are huge number of tables. Unless there are clear boundaries, splitting tables into multiple data stores is more trouble than it's worth, as you lose referential integrity, joins, and all the power and flexibility of running SQL queries against a single dataset.

I would advise the following: Use Django REST Framework to expose your data and operations, decoupled from your HTML views. DRF can sit easily alongside your vanilla Django views and models. Plus, it's conceptual similarity to Django means you can harness your team's existing knowledge. You can create an "api" app, or "api" subfolders in each of your apps. Pay attention to the structure and organization of your Django project's code, and you may be surprised at how flexible and productive a "majestic monolith" can be. You can now offer data and business operations (POST /cancel) in a fine-grained way to mobile apps or experiment with new, fancy JS frontends. If you truly have scalability requirements that simple horizontal scaling can't solve, you can always migrate pieces to a Spring or Go service. Maybe by then, your presentation layer will be React or Vue, and you can simply update your UI to use your new service endpoints instead of the DRF ones.