How to maintain different, customized versions of the same software for multiple clients

Question

we have multiple clients with different needs. Although our software is modularized to a degree, it's almost certain that we need to adjust every module's business logic here and there a little for each customer. The changes are probably too tiny to justify splitting the module into a distinct (physical) module for each client, I fear problems with the build, a linking chaos. Yet, those changes are too large and too many to configure them by switches in some configuration file, as this would lead to problems during deployment and probably a lot of support troubles, especially with tinker-type admins.

I'd like to have the build system create multiple builds, one for each client, where changes are contained in a specialized version of the single physical module in question. So I have some questions:

Would you advise letting the build system create multiple builds? How should I store the different customizations in source control, svn in particular?

Answer 1

What you need is feature branching-like code organisation. In your specific scenario this should be called Client-specific trunk branching because you'll likely use feature branching as well while you develop new stuff (or resolve bugs).

http://svnbook.red-bean.com/en/1.5/svn.branchmerge.commonpatterns.html

The idea is to have a code trunk with new features' branches merged into. I mean all non-client-specific features.

Then you also have your client-specific branches where you also merge same features' branches as required (cherry-picking if you will).

These client branches do look similar to feature branches although they're not temporary or short-lived. They are maintained over a longer period of time and they're mainly only merged into. There should be as little client-specific feature branch development as possible.

Client-specific branches are parallel branches to trunk and are being active as long as trunk itself and don't even get merged as whole anywhere.

            feature1
            ———————————.
                        \
trunk                    \
================================================== · · ·
      \ client1            \
       `========================================== · · ·
        \ client2            \
         `======================================== · · ·
              \ client2-specific feature   /
               `——————————————————————————´

Answer 2

Don't do this with SCM branches. Make the common code a separate project that produces a library or project skeleton artifact. Each customer project is a separate project which then depends on the common one as a dependency.

This is easiest if your common base app uses a dependency injection framework like Spring, so that you can easily inject different replacement variants of objects in each customer project as custom features are required. Even if you don't already have a DI framework, adding one and doing it this way may be your least painful choice.

Answer 3

Store software for all clients in a single branch. There's no need to diverge the changes made for different clients. Most lilkely, you'll want your software to be of best quality for all the clients, and the bugfix to your core infrastructure should affect everyone without unnecessary merging overhead, which also may introduce more bugs.

Modularize the common code, and implement the code that differs in different files or guard it with different defines. Make your build system have specific targets for each client, each target compiling a version with only the code related to one client. If your code is C, for example, you might want to guard features for different clients with "#ifdef" or whatever mechanism your language has for build configuration management, and prepare a set of defines corresponding to the amount of features a client has paid for.

If you don't like ifdefs, use "interfaces and implementations", "functors", "object files" or whatever tools your language provides for storing different things in one place.

If you distribute sources to your clients, it's best to make your build scripts have special "source distribution targets". Once you invoke such a target, it creates a special version of sources of your software, copies them to a separate folder so you can ship them, and does not compile them.

Answer 4

As so many have stated: factor your code properly and customize based on the common code—this will improve maintainability significantly. Whether you're using an object-oriented language/system or not, this is possible (though it is quite a bit more difficult to do in C than something that is truly object oriented). This is precisely the type of problem that inheritance and encapsulation help to solve!

Answer 5

The changes are probably too tiny to justify splitting the module into a distinct (physical) module for each client, I fear problems with the build, a linking chaos.

IMO, they can't be too tiny. If possible, I would factor out the client-specific code using the strategy pattern pretty much everywhere. This will reduce the amount of code that must be branched, and reduce the merging required to keep the general code in sync for all clients. It will also simplify testing... you can test the general code using default strategies, and test the client-specific classes separately.

If your modules are coded such that using policy X1 in module A requires using policy X2 in module B, think about refactoring so that X1 and X2 can be combined into a single policy class.

Answer 6

Very Carefully

Feature Branching is an option but I find it to be somewhat heavy. Also it makes deep modifications easy which can lead to a forking outright of your application if not kept under control. Ideally you want to push up as much as possible the customizations in an attempt to keep your core code base as common and generic as possible.

Here is how I would do it though I do not know if it is applicable to your code base without heavy modifications and re-factorings. I had a similar project where the basic functionality was the same but each customer required a very specific set of features. I created a set of modules and containers that I then assemble through configuration (à la IoC).

then for each customer I created a project that basically contains the configurations and the build script to create a fully configured installation for their site. Occasionally I place there also some components custom made for this client. But this is rare and whenever possible I try to make it in a more generic form and push it down so other projects can use them.

The end result is I got the level of customization I needed, I got customised installation scripts so that when I get to the customer site I do not look like i'm tweeking the system all the time and as an added VERY significant bonus I get to be able to create regression tests hooked directly on the build. This way any-time I get a bug that is customer specific I can write a test that will assert the system as it is deployed and thus can do TDD even at that level.

so in short :

1. Heavily modularised system with a flat project structure.
2. Create a project for each configuration profile (customer, though more than one can share a profile)
3. Assemble the required functionality sets as a different product and treat it as such.

If done properly your product assembly should contain all but a few configuration files.

After using this for a while I ended up creating meta-packages that assemble the mostly used or essential systems as a core unit and use this meta-package for customer assemblies. After a few years I ended up having a large toolbox that I could assemble very quickly to create customer solutions. I'm currently looking into Spring Roo and see if I cannot push the idea a bit further hoping one day I can create a first draft of the system right with the customer in our first interview... I guess you could call it User Driven Development ;-).

Hope this helped

Answer 7

If you are writing in plain C, here is a rather ugly way to do it.

• Common code (eg unit "frangulator.c")

• client-specific code, pieces that are small and used only for each client.

• in the main unit code, use #ifdef and #include to do something like

#ifdef CLIENT=CLIENTA
#include "frangulator_client_a.c"
#endif

Use this as a pattern over and over in all code units that require client-specific customisation.

This is VERY ugly, and leads to some other troubles but it is also simple, and you can cross-compare client-specific files one against the other quite easily.

It also means that all of the client-specific pieces are clearly visible (each in own file) at all times, and there is a clear relationship between the main code file and the client-specific part of the file.

If you get really clever you can set up makefiles to create the correct client definition so something like:

make clienta

will build for client_a, and "make clientb" will make for client_b and so on.

(and "make" with no provided target can issue a warning or usage description.)

I have used a similar idea before, it takes a while to set up but it can be very effective. In my case one source tree built about 120 different products.

Answer 8

You could use your SCM to maintain branches. Keep the master branch pristine/clean from client custom code. Do main development on this branch. For every customized version of the application maintain separate branches. Any good SCM tool will do really well with merging branches (Git comes to mind). Any updates in the master branch should be merged into the customized branches, but client specific code can stay in its own branch.

---

Though, if at all possible, try to design the system in a modular and configurable way. The downside of these custom branches can be that it moves too far away from the core/master.

Answer 9

In git, the way I would do it is to have a master branch with all of the common code, and branches for each client. Whenever a change to the core code is made, just rebase all of the client-specific branches on top of master, so that you have a set of moving patches for the clients that are applied on top of the current baseline.

Whenever you are making a change for a client, and notice a bug that should be included in other branches, you can cherry-pick that into either master, or into the other branches that need the fix (although of different client branches are sharing code, you should probably have them both branching off a common branch from master).