I am designing a web application with Django as the backend. Overview of the system is as follows

  1. Data is collected from user related to different entities. Example : Plant, Project, Machine, Test, etc.

  2. Configuration Blocks (consolidated, fully self contained, dated & versioned JSON files) are created by user by selecting different instances of above entities into a package.

  3. The config block is sent to an automated API server, which generates an API for the said block on the fly.

I am trying to model the database structure the step 1.

Known details

  1. Different classes of entities. Ex. Plant, Project, Machine, etc are known before hand and are fixed.

Everything else is unknown. I don't know the data in the entities, the links between them, etc.

I plan to user Postgres for a searchable relational data and a document database, CouchDB to handle the unknown data schema part.

Envisaged Architecture

I plan to have template tables for different entities, in a Postgres database, where, the administrator defines a set of templates for data required to be filled by the user. The template is a JSON document, with a protocol for defining different fields the user will fill out.

My UI will read the JSON document and will generate the required UI components on the fly. Currently the plan is to make a react component.

I have split the user data into the following categories.

Validated Manual Input
Check Box
Selection from a list
Selection from a set of linked lists (Ex. State selection, then City selection, etc.)
Link to other documents (Ex. Location of a machine in a particular plant)

I plan to have a protocol in the JSON template document that the UI processor will understand, and appropriately flesh out the UI, doing necessary REST API calls to the back-end based on pointers in the JSON document.

So, the administrators preset a list of templates. The UI converts the templates into user input forms. The system takes data from the forms and generates the relevant data in the Postgres and Document database (CouchDB).

The idea is to have a linked set of documents in the document database, with administrator defined schema, and a list of statically linked tables in postgres that encapsulate known relations between the entities.

Is the above architecture a good enough design, as in, is it a good method is solve the issue of data handling without knowing the schema before hand??


An architecture critically depends on the non-functional requirements, e.g.

  • how many documents do you need to be able to process will be critical input into which document store to use
  • or if the back-end needs to support event based processing or not
  • manageability requirements will determine, if using and mixing storage tech is a good idea
  • cost and price will determine, if building from scratch is possible.

In your question you only give functional requirements.

A good architecture is simple, supports hitting targets for non-functional requirements and provides a clean separation of concerns. Have a look at architecture styles, for examples of architecture patterns. Some of that might be given by your choice of Django.

Which non-functional requirements lead you to choosing two persistence layers? In my opinion, this will make it more difficult to support and design.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.