Moving a module out of monolithic database using microservice patterns - challenges

Question

We have a large monolithic database. As part of a new requirement, I am proposing to move our financial transactional system into a new separate database via microservice-esque services. These financial transactions are very important for our customers and performance is key. It’s a very large module with a lot of business complexities. Expectation is that our system should be able to handle 1000 TPS minimum.

There are two schools of thought in our development team.

Side A is to keep it as a monolithic DB but improve some bad designs and improve code quality in general. Advantage is less steep learning curve, less amount of surprises in later stage.

Side B (me included) is to use this opportunity as a challenge and implement micro service-based architecture-based solution with a smaller scalable database. Challenge is fear of unknown, stepping out of comfort zone and since project is time bound, chances of failure.

Following are the challenges while designing this system as an independent system.

• A lot of our data will reside in a monolithic badly designed database. A lot of tables expect reference of transaction ID generated as result of a financial transaction. For example, players can get offers. If they redeem offers, their account will be credited with X amount of money. Now to support various reports, we need to maintain which player redeemed which offer and what was the corresponding ID of that transaction.
• Reports will be tricky. They will need to refer to both databases. Only option is to use linked server or open query but their performance will surely be degraded.
• Right now, most of our logic is in the database. So now C# simply need to make a database call to invoke a HUGE SP which will call a lot of functions and further SPs. But yes, there will be only one call. Now if we move to service bus driven design, there will be a lot of calls over network. A typical flow could easily have 8 to 10 calls. These calls will have a lot of back and forth. It's not easy to do a POC to guarantee that multiple API calls will perform better than single procedure call.

I am looking for your opinion on microservice architecture and smaller database. In your experience, is it really worth to emphasize smaller databases? Separating databases will definitely trigger rewrite of many components since we will need to break complex SP logic to API calls.

Answer 1

While I dont have anything against it in the first place, I think few of the problems you highlight would not be solved by moving to microservices.

You have bad table design (wont fix that with MS), performance guarantees on a central data store (REALLY wont fix that with MS), and then new issues that would be created with MS that are unknowns - the network latency you point out is super killer, if you have just 10ms latency per network call, you get 80-100ms per transaction overhead for every single thing you do.

I am a bit biased as I do DBA work and most performance concerns are something that can be addressed, especially when you have enough political will to change core structures.

I would also consider that a MS based arch is usually something you do to enable teams to split off and work on their own problems while maintaining the various contracts the service guarantees. If you don't have that setup, it may not even provide that much benefit, and in many cases gives you a monolith with one microservice, which now has the downsides of both.

A smaller database doesnt matter except in terms of RPO/RTO - the table and index size, the cardinality of the queries, and overarching considerations that have little to do with MS arch are really what drive query performance.