2

TLDR;

what if you are tasked with taking out a feature and making it app 2 to reduce memory/compute foot print from app 1? Making an API contract between app 1 & 2 would defeat the purpose because APIs run on app 1.

How then could we split a feature from a monolith that relies on the same data of app 1?

I need to review our architecture and wonder if any of you want to give it a go, I would appreciate learning from the insights.

We have a main spring java web app - Central. We had an export feature that exported our data via a background job. As the export component began to have more features added to it - send live events instead of daily exports etc, our architect decided to make it a separate app called integration(ims). His motivation? If the feature is separate, it can scale and not increase load (JVM) of Central as Central is user-facing and IMS is used for all kinds of requests from clients (exports, events etc). On paper, sounds nice. Here are the concerns below.

so IMS now queries Central because it needs data to export. All those queries were already present in Central, but they had to be copied and pasted - along with utils, ORM Entity classes - et al.

I shared with my product owner that IMS directly querying Central database as opposed to using API contracts - is an anti-pattern because any change in Central database is unknown to IMS app. If the contract is done via API and not queries, any change in DB structure is invisible to the API and would work for Central and IMS. My boss agreed. However, wasn't the whole idea of splitting the app because we didn't want to increase load on Central? Making API contracts between IMS and Central would be back to square one (as those APIs would run on Central)

Apart from DB changes, the other issue I am finding is the repition of code. As IMS needs to query Central for everything, it requires the same kind of service classes, utils, security improvements, S3 implemetations etc. So we're having a

problem no. 2: "have you remembered to make this change on IMS too?" - i.e Inablility to share code

We're not the first ones to split an app to reduce JVM load. And IMS is merely querying (read-only) Central. How do other teams tackle DB change impacts on shared java apps & inability to share util, classes etc? Or what have we done wrong and what could we have done better?

I don't think this question is limited to Java, its an architectural design questions that applies to all technologies.

The split has already occured, so any thoughts are appreciated.

Improve this question
6
  • 1
    I'd probably opt for an own data layer in that case which can be used by both. Here you could partition the db-layer further with replicas i.e. to divide concerns further. The current master would then receive all writes while any replica is only used for read operations. If you look at microservice architecture in particular then data duplication isn't a big deal there. It is more important that the respective applications are master of their own data and know how to get data they need. In term of traceability most DB provide a log feature of some form
    – Roman Vottner
    Commented Dec 15, 2023 at 2:24
  • When you say "data layer" do you mean a separate web app or a jar? Can you share any reference ? I can't find any one doing something similar
    – veritas
    Commented Dec 15, 2023 at 2:44
  • Depending on your environment you can either directly expose the DB as data layer to your applications or create simple Spring backend applications that are able to use various technologies (i.e. Web API, Redis or the like) to exchange data with other applications. Such applications can be scaled to multiple instances or containers easily usually. It fully depends on which architecture model you prefer. I.e. with microservices you try to keep the db with the data of that service as close to the application as possible and thus usually deploy the DB to the same container the app is running in
    – Roman Vottner
    Commented Dec 15, 2023 at 2:58
  • @veritas I don't see any problems as long as both cheap API calls and long DB queries are used in the IMS. Please clarify why you think they are mutually exclusive.
    – Basilevs
    Commented Dec 15, 2023 at 7:58
  • 1
    Security concerns are irrelevant as IMS is not an user-facing service. Service classes are not shared, because most will not be needed in the new service. In other words all of @veritas concerns are easily dispelled. The talk to product owner was an error.
    – Basilevs
    Commented Dec 15, 2023 at 8:01

1 Answer 1

Reset to default
1

It sounds like Central has interactive OLTP latency requirements, while IMS has batch reporting throughput requirements. And you want them served from different containers, to alleviate observed resource shortages.

Pursue these two improvements in parallel:

  1. Replicate your database so writes go to a leader and are then propagated to follower(s). Now you have more read bandwidth available for readonly reporting tasks.
  2. Write a common data layer library which both Central and IMS use to make database queries. If for example we delete a column, we update the data library to know about it. If it turns out to be a breaking change, javac will tell us about callers that depended on that column as part of the signature they used.

Now you can deploy M instances of Central, each hitting the database leader node, plus N instances of IMS which can directly query as many as N near-realtime readonly database replicas. And the people did rejoice.

Improve this answer
9
  • how do I make a common data layer? use a common jar for both Central & IMS? I'm afraid I dont know how a common data layer looks like. Are there any references you can share?
    – veritas
    Commented Dec 25, 2023 at 11:32
  • Yes, that's exactly right. Author a common .jar that both Central & IMS will both import. The data layer makes database / spring calls, while Central & IMS lack such imports, they only call routines in this new layer. Start small -- invent a brand new table which the three modules access directly or indirectly. Be sure to use JUnit to write an automated integration test. Once you have that working, follow the pattern to convert existing table access.
    – J_H
    Commented Dec 25, 2023 at 17:49
  • The common jar (data layer that has ORM classes and perhaps JPA API) solves issue of losing visibility on IMS due to change on Central. The concern left unaddressed is that if the two apps were supposed to be independent, their deyployments now seem coupled i.e whenever a column is dropped on Central and if Central (app + common Central DB) goes live, though the changes are visible to IMS, IMS would still break because those changes have not gone live on IMS even if they are available on git pull. That is, this is great for CI but does not address CD (continuous deployment). Any thoughts?
    – veritas
    Commented Dec 28, 2023 at 3:16
  • Yes, when dropping a column there is definitely coupling ("versionitis") among {DB, data_layer, Central, IMS}. It is a good goal to allow deploying new DB in week 1, new data_layer in week 2, new Central in week 3, and new IMS in week 4. To accomplish that, you need to version the database, perhaps at granularity of "whole database". Personally, I tend to do it at table level, and I will "fix it with a VIEW". That is, version 1 of data layer uses SELECT x, y, z FROM foo_v1. Construct a projection that drops column Z with: CREATE VIEW foo_v2 AS SELECT x, y FROM foo_v1. Now "SELECT *" works.
    – J_H
    Commented Dec 28, 2023 at 5:50
  • versioning the database assumes that the read-only instance of Central IMS is calling is static. The read-only central DB is an AWS managed replica so structure changes will be instantly passed (thereby breaking IMS that is unbeknowst to the changes). So week1-week4 strategy is not going to work.
    – veritas
    Commented Dec 28, 2023 at 6:11

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.