We have a large monolithic database. As part of a new requirement, I am proposing to move our financial transactional system into a new separate database via microservice-esque services. These financial transactions are very important for our customers and performance is key. It’s a very large module with a lot of business complexities. Expectation is that our system should be able to handle 1000 TPS minimum.
There are two schools of thought in our development team.
Side A is to keep it as a monolithic DB but improve some bad designs and improve code quality in general. Advantage is less steep learning curve, less amount of surprises in later stage.
Side B (me included) is to use this opportunity as a challenge and implement micro service-based architecture-based solution with a smaller scalable database. Challenge is fear of unknown, stepping out of comfort zone and since project is time bound, chances of failure.
Following are the challenges while designing this system as an independent system.
- A lot of our data will reside in a monolithic badly designed database. A lot of tables expect reference of transaction ID generated as result of a financial transaction. For example, players can get offers. If they redeem offers, their account will be credited with X amount of money. Now to support various reports, we need to maintain which player redeemed which offer and what was the corresponding ID of that transaction.
- Reports will be tricky. They will need to refer to both databases. Only option is to use linked server or open query but their performance will surely be degraded.
- Right now, most of our logic is in the database. So now C# simply need to make a database call to invoke a HUGE SP which will call a lot of functions and further SPs. But yes, there will be only one call. Now if we move to service bus driven design, there will be a lot of calls over network. A typical flow could easily have 8 to 10 calls. These calls will have a lot of back and forth. It's not easy to do a POC to guarantee that multiple API calls will perform better than single procedure call.
I am looking for your opinion on microservice architecture and smaller database. In your experience, is it really worth to emphasize smaller databases? Separating databases will definitely trigger rewrite of many components since we will need to break complex SP logic to API calls.