I am designing a web application with Django as the backend. Overview of the system is as follows
Data is collected from user related to different entities. Example : Plant, Project, Machine, Test, etc.
Configuration Blocks (consolidated, fully self contained, dated & versioned JSON files) are created by user by selecting different instances of above entities into a package.
The config block is sent to an automated API server, which generates an API for the said block on the fly.
I am trying to model the database structure the step 1.
- Different classes of entities. Ex. Plant, Project, Machine, etc are known before hand and are fixed.
Everything else is unknown. I don't know the data in the entities, the links between them, etc.
I plan to user Postgres for a searchable relational data and a document database, CouchDB to handle the unknown data schema part.
I plan to have template tables for different entities, in a Postgres database, where, the administrator defines a set of templates for data required to be filled by the user. The template is a JSON document, with a protocol for defining different fields the user will fill out.
My UI will read the JSON document and will generate the required UI components on the fly. Currently the plan is to make a react component.
I have split the user data into the following categories.
Validated Manual Input Check Box Selection from a list Selection from a set of linked lists (Ex. State selection, then City selection, etc.) Link to other documents (Ex. Location of a machine in a particular plant)
I plan to have a protocol in the JSON template document that the UI processor will understand, and appropriately flesh out the UI, doing necessary REST API calls to the back-end based on pointers in the JSON document.
So, the administrators preset a list of templates. The UI converts the templates into user input forms. The system takes data from the forms and generates the relevant data in the Postgres and Document database (CouchDB).
The idea is to have a linked set of documents in the document database, with administrator defined schema, and a list of statically linked tables in postgres that encapsulate known relations between the entities.
Is the above architecture a good enough design, as in, is it a good method is solve the issue of data handling without knowing the schema before hand??