Unity injection with too many constructor parameters

Question

The question is about choosing the appropriate design for the scenario described below. This is a repost from https://stackoverflow.com/questions/51940180/unity-injection-with-too-many-constructor-parameters where it was suggested to put the question here.

The question seems to cause a severe controversy even among some of the best-known C# gurus. In fact, it is far beyond C# and it falls more into pure computer science. The question is based on the well-known "battle" between a service locator pattern and pure dependency injection pattern: https://martinfowler.com/articles/injection.html vs http://blog.ploeh.dk/2010/02/03/ServiceLocatorisanAnti-Pattern/ and a subsequent update to remedy the situation when the dependency injection becomes too complicated: http://blog.ploeh.dk/2010/02/02/RefactoringtoAggregateServices/

I read everything that I could find on the subject and even contacted several well-known C# gurus directly, but yet it is still unclear what would be the best choice.

Development setup and requirements

1. We use Unity for DI / IOC. As such we never create classes directly and inject only interfaces through constructors.
2. We use Moq with MockBehavior.Strict for all unit tests to ensure that we always get expected behavior and no hidden surprises.
3. We use partial mock for integration tests where all external services are mocked in some way and all (or almost all) internal services are real.
4. We use a single composition root where all interfaces are registered.
5. We want to simplify the design and decrease the maintenance time / cost.

Business setup

We have a large (50 – 100+) collection of "rules", which I call micro services, because each rule is wrapped into a single service. If you have a better name, please "apply" it when reading. Each of them operates on a single object, let's call it quote. However, a tuple (context + quote) seems more appropriate. Quote is a business object, which gets processed and serialized into a database and context is some supporting information, which is necessary while quote is being processed, but is not saved into the database. Some of that supporting information may actually come from database or from some third-party services. This is irrelevant. Assembly line comes to mind as a real-world example: an assembly worker (micro service) receives some input (instruction (context) + parts (quote)), processes it (does something with parts according to instruction and / or modifies instruction) and passes it further if successful OR discards it (raises exception) in case of issues. The micro services eventually get bundled up into a small number (about 5) of high-level services. This approach linearizes processing of some very complex business object and allows testing each micro service separately from all others: just give it an input state and test that it produces expected output.

Here is where it gets interesting. Because of the number of steps involved, high-level services start to depend on many micro services: 10-15+ and more. This dependency is natural, and it just reflects the complexity of the underlying business object. On top of that micro services can be added / removed nearly on a constant basis: basically, they are some business rules, which are almost as fluid as water. Somebody may state that such high-level services violate Single Responsibility Principle. Well, if I have to apply, let’s say, 15+ "rules" in order to create a quote, then the implementation of IQuoteCreateService.CreateQuote has to apply all of them, while performing only a single task, creating quote!

That severely clashes with Mark's recommendation above: if I have 15+ effectively independent rules applied to a quote in some high-level service, then, according to the third blog, I should aggregate them into some logical groups of, let's say no more than 3-4 instead of injecting all 15+ via constructor. But there are no logical groups! While some of the rules are loosely dependent, most of them are not and so artificially bundling them together will do more harm than good.

Throw in that the rules change frequently, and it becomes a maintenance nightmare: all real / mocked calls must be updated every time the rules change.

And I have not even mentioned that the rules are US state dependent and so, in theory, there are about 50 collections of rules with one collection per each state and per each workflow. And while some of the rules are shared among all states (like "save quote to the database"), there are a lot of state specific rules.

Here is a very simplified example.

Quote - business object, which gets saved into database.

public class Quote
{
    public string SomeQuoteData { get; set; }
    // ...
}

Micro services. Each of them performs some small update(s) to quote. Higher level services can be also built from some lower level micro services as well.

public interface IService_1
{
    Quote DoSomething_1(Quote quote);
}
// ...

public interface IService_N
{
    Quote DoSomething_N(Quote quote);
}

All high-level and micro services inherit from this interface. It is convenient because the low-level implementation: QuoteProcessor provides some common tasks, like call for data validation, perform finalizer tasks (if necessary), etc... This is irrelevant to the question but explains why micro services also inherit from this interface.

public interface IQuoteProcessor
{
    List<Func<Quote, Quote>> QuotePipeline { get; }
    Quote ProcessQuote(Quote quote = null);
}

// Low level quote processor. It does all workflow related work.
public abstract class QuoteProcessor : IQuoteProcessor
{
    public abstract List<Func<Quote, Quote>> QuotePipeline { get; }

    public Quote ProcessQuote(Quote quote = null)
    {
        // The real code performs Aggregate over QuotePipeline.
        // That applies each step from workflow to a quote.
        return quote;
    }
}

One of high level "workflow" services:

public interface IQuoteCreateService
{
    Quote CreateQuote(Quote quote = null);
}

and its actual implementation where we use many of low level micro services.

public class QuoteCreateService : QuoteProcessor, IQuoteCreateService
{
    protected IService_1 Service_1;
    // ...
    protected IService_N Service_N;

    public override List<Func<Quote, Quote>> QuotePipeline =>
        new List<Func<Quote, Quote>>
        {
            Service_1.DoSomething_1,
            // ...
            Service_N.DoSomething_N
        };

    public Quote CreateQuote(Quote quote = null) => 
        ProcessQuote(quote);
}

Issues

There are two main ways to achieve DI:

Standard approach is to inject all dependencies through constructor:

    public QuoteCreateService(
        IService_1 service_1,
        // ...
        IService_N service_N
        )
    {
        Service_1 = service_1;
        // ...
        Service_N = service_N;
    }

And then register all types with Unity:

public static class UnityHelper
{
    public static void RegisterTypes(this IUnityContainer container)
    {
        container.RegisterType<IService_1, Service_1>(
            new ContainerControlledLifetimeManager());
        // ...
        container.RegisterType<IService_N, Service_N>(
            new ContainerControlledLifetimeManager());

        container.RegisterType<IQuoteCreateService, QuoteCreateService>(
            new ContainerControlledLifetimeManager());
    }
}

Then Unity will do its "magic" and resolve all services at run time. The problem is that currently we have about 50-100 such micro services and the number is expected to increase. Subsequently some of the constructors are already getting 10-15+ services injected. This is inconvenient to maintain, mock, etc...

Sure, it is possible to use the idea from here: http://blog.ploeh.dk/2010/02/02/RefactoringtoAggregateServices/ However, the micro services are not really related to each other and so bundling them together is an artificial process without any justification. In addition, it will also defeat the purpose of making the whole workflow linear and independent (a micro service takes a current "state", then preforms some action with quote, and then just moves on). None of them cares about any other micro services before or after them.

An alternative idea seems to create a single "service repository" or service locator:

public interface IServiceRepository
{
    IService_1 Service_1 { get; set; }
    // ...
    IService_N Service_N { get; set; }

    IQuoteCreateService QuoteCreateService { get; set; }
}

public class ServiceRepository : IServiceRepository
{
    protected IUnityContainer Container { get; }

    public ServiceRepository(IUnityContainer container)
    {
        Container = container;
    }

    private IService_1 _service_1;

    public IService_1 Service_1
    {
        get => _service_1 ?? (_service_1 = Container.Resolve<IService_1>());
        set => _service_1 = value;
    }
    // ...
}

Then register it with Unity and change the constructor of all relevant services to something like this:

    public QuoteCreateService(IServiceRepository repo)
    {
        Service_1 = repo.Service_1;
        // ...
        Service_N = repo.Service_N;
    }

The benefits of this approach (in comparison to the previous one) are as follows:

All micro services and higher-level services can be created in a unified form: new micro services can be easily added / removed without the need to fix constructor call for the services and all unit tests. Subsequently, maintenance and complexity decreases.

Due to interface IServiceRepository, it is easy to create an automated unit test, which will iterate over all properties and validate that all services can be instantiated, which means that there will be no nasty run time surprises.

The problem with this approach is that it starts looking a lot like a service locator, which some people consider as an anti-pattern: http://blog.ploeh.dk/2010/02/03/ServiceLocatorisanAnti-Pattern/ and then people start to argue that that all dependencies must be made explicit and not hidden as in ServiceRepository.

Yet a few more ideas were provided in the answers to the original question (in the link above). They all centered around the idea of using IEnumerable to pass as parameters array or as the collection of rules to be applied. I personally feel that using parameter array to pass services into constructor will do more harm than good due to the need for maintaining each constructor call in sync and without compiler to the rescue.

Question

I want to decrease complexity associated with injecting too many parameters through constructor. The rules (or parameters) are fairly independent, so I don't have a logical reasoning to bundle them up to decrease the number of parameters for the top class(es). From the other side, the business setup does require that many nearly independent rules must be applied at the level of the top business object(s) (quote, for example).

Answer 1

This piece of code caught my attention:

public override List<Func<Quote, Quote>> QuotePipeline =>
    new List<Func<Quote, Quote>>
    {
        Service_1.DoSomething_1,
        // ...
        Service_N.DoSomething_N
    };

The fact that you're taking a list of discrete elements and immediately putting them in a list suggests that there is an implicit business concept here that isn't embodied in a class. If it were me, I'd have a class to represent the pipline and inject either the pipeline itself or a factory that will allow the caller to create one.

If you're writing a factory, it is "OK" to pass in the Unity Container itself, since dependency and lifespan management is part of a factory's single responsibility. This way you are getting the benefits of the service locator pattern without breaking the rules too badly for IoC.

When you need to unit test, you can just substitute a different pipeline factory that returns mocks.

So, first we define the pipeline:

public class QuotePipeline : List<Func<Quote,Quote>>
{
    public Quote Execute(Quote quote)
    {
        foreach (var f in this) quote = f(quote);
        return quote;
    }
}

Now write a factory. Unity will always inject itself automatically if your class has IUnityContainer as a constructor argument.

class QuotePipelineFactory : IQuotePipelineFactory
{
    protected readonly IUnityContainer _container;

    public QuotePipelineFactory(IUnityContainer container)
    {
        _container = container;
    }

    public QuotePipeline GetPipeline()
    {
        var p = new QuotePipeline();

        var d1 = _container.Resolve<Service_1>();
        p.Add( q => d1.DoSomething_1(q) );

        var d2 = _container.Resolve<Service_2>();
        p.Add( q => d2.DoSomething_2(q) );

        return p;
    }
}

Then inject the factory and retrieve the pipeline itself:

public class QuoteCreateService : IQuoteProcessor
{
    protected readonly IQuotePipelineFactory _quotePipelineFactory;

    public QuoteCreateService(IQuotePipelineFactory quotePipelineFactory)
    {
        _quotePipelineFactory = quotePipelineFactory;
    }

    public Quote CreateQuote(Quote quote)
    {
        var p = _quotePipelineFactory.GetPipeline();
        return p.Execute(quote);
    }
}

Notice now there is only one dependency, and it can support any number of quote processors.

If your quote pipelines are always the same, you can of course store it as a single instance within the factory, which will be appropriately lifetime-scoped by Unity.

If the pipelines differ, you can always add input arguments to GetPipeline() and then possibly use a strategy pattern to pick and choose which processors to include in the pipeline.

Answer 2

I personally don't like injection through constructors at all. That's because classes will need injected classes that only they and nobody else cares about, but now anyone creating an instance of the class has to perform these injections, so they need them injected to their constructors as well, and so on and so on.

Much better if each class looks after itself. Have some factory, or have singletons that return a replacement if needed, and everyone collects whatever injected objects they need. Seriously, why would you want an injected class in your constructor that is used by some other instance seven levels away?

Answer 3

I never thought that I would answer my own question, though a substantial part of the credit should go to https://softwareengineering.stackexchange.com/users/115084/john-wu - he was the one who had my mind set in a proper direction.

Nevertheless, nearly two years have passed since the time when I asked the question and while I built the solution to the question slightly after asking it (and thanks for everyone who replied), it took more than a year for most of the developers in the company that I work for to actually understand how does it work and what does it do (and yes, they all are well above average developers and yes, the code is written in pure C# with no external libraries). So, I think that it could be important for others who might have similar business scenarios.

As mentioned in the question, the root of our problem is that the parameter space that we are dealing with is too large. We have about 6-8 values of what we call workflow (call it W), about 30-40 values of what we call a state config (call it S) – this is a combination of US state code and two other parameters, though not all triples are possible (the actual content of what is that state config is irrelevant), and about 30-50 values of what we call a risk rule (call it R) - that value depends on the product but this is also irrelevant as different products are treated differently.

So, the total dimension of parameter space is N = W * S * R and it is around 10K (and I am not much concerned about a precise value). Which means that when the code runs, we need approximately the following: for each workflow (obviously only one is running at a time but all of them do run at some time) and each state config (again only one is running at a time but any of them could run at some time) we need to evaluate all risk rules, which are relevant for that workflow and that state config.

Well, if the dimension of parameter space is around some N, then the number of tests needed to cover the whole space is at least on the order of that N. And this is exactly what the legacy code and tests were trying to do and what resulted in the question. The answer turned out to be in a pure math, rather than in a pure computer science and it is based on what is called separable spaces: https://en.wikipedia.org/wiki/Separable_space and what in the group theory terms is called irreducible representation: https://en.wikipedia.org/wiki/Irreducible_representation . Though I have to admit that the latter one was more like an inspiration rather than the actual application of the group theory.

If you already lost me, that’s fine. Just, please, read the math mentioned above before proceeding further.

The space separability here means that we can choose such a space N so that subspaces W, S, and R become independent (or separable). To the best of my understanding, this can always be done for finite spaces that we are dealing with in CS.

This means that we can describe N space as e.g. S lists (or sets) of some rules whereas each rule is applicable in some of W workflows by assigning a set of applicable workflows to each rule. And yes, if we have some bad rules that originally should be applied in some weird combinations of workflows and state configs then we can split them into more than one rule, which would then allow maintaining separability.

This, of course, can be generalized, but I will skip the details as they are irrelevant.

At this point, someone may wonder, what’s the point. Well, if we can split N dimensional space (and N is about 10K in our case) into independent subspaces, then the magic happens and instead of writing on the order of N = W *S *R tests to cover the whole parameter space we only need to write on the order of W + S + R tests to cover the whole parameter space. In our case the difference is about 100X.

But that’s still not all. As we can describe the subspaces in the notions of sets or lists (depending on the needs) that naturally brings us to the notion of useless tests.

Wait, did I just say useless tests? Yes, I did. Let me explain. A typical TDD paradigm is that if the code failed, then the first thing that we need to do is to create a test, which would’ve caught that bug. Well, if the code is described by a static list or set (== list or set that was hard coded in the code) and the test would be described by an identity transformation from that list/set, then this make such a test useless as it would have to repeat the original list/set…

The state configs form a historical pattern, e.g., let say, that we had some set of rules for the state of CA some time in 2018. That set of rules might be slightly changed to some other set of rules in 2019 and into some set of rules in 2020. These changes are small: a set of rule might pick up or lose a few rules and/or the rule might be tweaked a little bit, e.g. if we are comparing some value to be above some threshold, then the value of that threshold might be changed at some point for some state config. And once the rule or collection of rules is changed, then it should stay as it is until it changed again. Meanwhile some other rules could be changed, and every such change requires introduction of what we call state config. So, for each US state we have ordered collection (list) of these state configs and for each state config we have a collection of rules. Most of the rules don’t change but some of them do sporadically change as described. A natural IOC approach is to register each rule collection and each rule for each state config with IOC container, e.g. Unity using a combination of unique “name” of the state config and name of rule / collection (we actually run more than one collection of rules during workflow), whereas each rule already has a collection of workflows where it should be applicable. Then when the code runs for a given state config and a given workflow we can pull the collection out of Unity. A collection then contains the names of the rules that should be run. Then combining the name of the rule with the name of state config we can pull the actual rule out of Unity, filter the collection to leave only the rules that are applicable for a given workflow and then apply all the rules.

What happens here is that rule names / collection names form some closed sets and they benefit greatly by describing them that way. We obviously don’t want to register each rule / collection for each state config by hands as that would be tedious and error prone. So we use what we call “normalizers”. Let’s say that we have a general rule – that’s a rule that is the same for all state config. Then we register it by name only and the normalizer will “automatically” register it for all state configs. The same goes with the historic versioning. Once we register a rule / collection with Unity by rule / collection name + state config, then the normalizer will fill in the gap until we change the rule at some later state config.

As a result, each rule becomes extremely simple. Most of them have either zero or one injected constructor parameter, a few of them have two, and I know only one rule that has three injected parameters. As rules are independent and very simple, the tests for rules become very simple as well.

We do have some ideas to make the core of whatever I wrote above open source, provided that it could bring some value to the community...