Let me explain my thoughts about architecture of the project I'm working on. The project code repository consist of:
- Scrapy component - of course it serves to scrape data, process it and calculate relations between data. It populates MySQL database.
- Django visualization component - it simply displays data stored in database using many filters.
Right now they are deployed as two separated docker containers which works fine. The idea of former collegues was to go further with split and split also code repositories.
I can see potential ability to create CI/CD per repository, so it will only run tests/lintes/checks and eventually will only deploy container which actually was modified. It won't run everything for other container which is ok(logical separation).
But because they are actually working on same database tables(Scrapy populates them, Django reads them) it looks like overkill for me. I would need to maintain two separate DB model specifications in sync in both repositories. Right now Scrapy uses Django ORM for interaction with DB.
What do you think? Do you think it's worth splitting code repository to two separated ones and keep in sync models in both of them? Or maybe not? Is there a way to trigger/run Gitlab CI/CD process for only affected container in single repository?
Thank you