How to share a code base among 10+ projects and minimize pain?

Question

I have a number of applications that share the same data in the same database. To try and minimize redundancy of code the Data Access Layer is a shared project. This prevents each project from needing to re-code it's own data access, but this also creates a big pain point. When one team needs to update the data layer, all other teams need to pull in and test the changes to make sure that they haven't broken anything and this is a slow and painful process.

I have thought about the idea of removing the shared data layer and just having each team manage their own data layer but the problem is all teams still access the same database so if there are table changes the pain point is still there because every team needs to update their relevant code.

So my question is, how could I design our data and access layer such that many projects be driven off of the same data source, and minimize the pain of making changes to the database or the access layer?

Answer 1

The shared data layer looks a pretty good idea. The solution to minimize the pain is:

1. make test modules that throws exception (like: everything stops working not just a message) when some test fails.
2. Put ALL the test modules in shared folder. I mean all 10+ projects' test modules must be in the same folder (Or a easy to understand path).
3. If a programmer wants to edit the critical layer he must execute all the tests in that folder, if something doesn't work he can't commit the code.

In this way if i make a piece of code i'm motivated to write a good test class, like a real test class because if my test class explodes with some modification means that those modification will break my code. So every programmer is forced to write good test (if i don't my code could break from time to time). Executing all projects' tests without no errors means: "ok you can commit", when something goes wrong it means "no, those modification would break the system at some line x"

This is a good point because the test stuff is too often left behind in a project... of course maintain the test code requires some additional effort.

Answer 2

I have experienced this problem first hand. It is difficult, and I was only integrating five applications. I can offer a few tips:

1. Decouple the database by using RESTful services for as many things as possible. This allows you to easily create multiple interface versions if needed, and it makes database migrations much easier.
2. If database access for some of the applications are read-only, create database views of the needed data. If you have to do a batch process to calculate values, do it in a materialized view.
3. Think carefully about design. Any database change will most likely break other applications. Do all the apps really need full database access? If not, break them out to use a service. Decouple any database access from any logic.

Answer 3

The short answer is that you need to put a LOT of time into your shared code to make it as robust/solid/reliable as possible.

As you're discovering, you can't afford to have your library code change very often.

It may be necessary to split your data layer into two parts - a low level part that does very simple things but never changes (i.e., the shared code) and a high level part that may have to be unique for each application.

For the low level shared portion, you need to think like a library developer & not an application developer. This means a lot of unit tests, a very clearly and thoroughly defined application interface, correct, complete and useful documentation, and a lot of defensive code - all inputs are invalid until proven otherwise.

The shared code should be written to serve the needs of the applications you have today that will use the code, but you'll want to put in the extra time to consider how this code will work for the next dozen applications that haven't been proposed yet. Put another way, the shared code solves the general problem of exposing data in a meaningful way to its consumers. If done well, new consumers can use this library without change.