Dealing with Feature Intersections

Question

I have recently witnessed more and more problems similar to the ones explained in this article on feature intersections. Another term for it would be product lines, though I tend to attribute these to actually different products, whereas I usually encounter these problems in the form of possible product configurations.

The basic idea of this type of problem is simple: You add a feature to a product, but somehow things get complicated due to a combination of other existing features. Eventually, QA finds a problem with a rare combination of features that no one thought of before and what should have been a simple bugfix may even turn into requiring major design changes.

The dimensions of this feature intersection problem are of a mind-blowing complexity. Let's say the current software version has N features and you add one new feature. Let's also simplify things by saying that each of the features can turned on or off only, then you already have 2^(N+1) possible feature combinations to consider. Due to a lack of better wording / search terms, I refer to the existence of these combinations as feature intersection problem. (Bonus points for an answer including reference(s) for a more established term.)

Now the question I struggle with is how to deal with this complexity problem on each level of the development process. For obvious cost reasons, it is impractical up to the point of being utopian, to want to address each combination individually. After all, we try to stay away from exponential complexity algorithms for a good reason, but to turn the very development process itself into an exponentially sized monster is bound to lead to utter failure.

So how do you get the best result in a systematic fashion that does not explode any budgets and is complete in a decent, useful, and professionally acceptable way.

• Specification: When you specify a new feature - how do you ensure that it plays well with all the other children?

  I can see that one could systematically examine each existing feature in combination with the new feature - but that would be in isolation of the other features. Given the complex nature of some features, this isolated view is often already so involved that it needs a structured approach all in itself, let alone the 2^(N-1) factor caused by the other features that one willingly ignored.

• Implementation: When you implement a feature - how do you ensure your code interacts / intersects properly in all cases.

  Again, I am wondering about the sheer complexity. I know of various techniques to reduce the error potential of two intersecting features, but none that would scale in any reasonable fashion. I do assume though, that a good strategy during the specification should keep the problem at bay during implementation.

• Verification: When you test a feature - how do you deal with the fact, that you can only test a fraction of this feature intersection space?

  It is tough enough to know that testing a single feature in isolation guarantees nothing anywhere near error-free code, but when you reduce that to a fraction of 2^-N it seems like hundreds of tests are not even covering a single drop of water in all oceans combined. Even worse, the most problematic errors are those that stem from the intersection of features, which one might not expect to lead to any problems - but how do you test for these if you don't expect such a strong intersection?

While I would like to hear how others deal with this problem, I am primarily interested in literature or articles analyzing the topic in greater depth. So if you personally follow a certain strategy it would be nice to include corresponding sources in your answer.

Answer 1

We already knew mathematically that verification of a program is impossible in finite time in the most general case, due to the halting problem. So this kind of problem is not new.

In practice, good design can provide decoupling such that the number of intersecting features is far less than 2^N, though it certainly seems to be above N even in well designed systems.

As far as sources, it seems to me that almost every book or blog about software design is effectively trying to reduce that 2^N as much as possible, though I don't know of any that cast the problem in the same terms as you do.

For an example of how design might help with this, in the article mentioned some of the feature intersection happened because replication and indexing were both triggered of the eTag. If they had available another communication channel to signal the need for each of those separately then possibly they could have controlled the order of events more easily and had fewer issues.

Or, maybe not. I don't know anything about RavenDB. Architecture can't prevent feature intersection issues if the features really are inexplicably intertwined, and we can never know in advance we won't want a feature that really does have the worst case of 2^N intersection. But architecture can at least limit intersections due to implementation issues.

Even if I'm wrong about RavenDB and eTags (and I'm just using it for the sake of argument - they're smart people and probably got it right), it should be clear how architecture can help. Most patterns people talk about are designed explicitly with the goal of reducing the number of code changes required by new or changing features. This goes way back - for example "Design Patterns, Elements of Reusable Object-Oriented Software", the introduction states "Each design pattern lets some aspect of the architecture vary independently of other aspects, thereby making a system more robust to a particular kind of change".

My point is, one can get some sense of the Big O of feature intersections in practice by, well, looking at what happens in practice. In researching this answer, I found that most analysis of function points/development effort (i.e. - productivity) found either less than linear growth of project effort per function point, or very slightly above linear growth. Which I found a bit surprising. This had a pretty readable example.

This (and similar studies, some of which use function points instead of lines of code) doesn't prove feature intersection doesn't occur and cause problems, but it seems like reasonable evidence that it's not devastating in practice.

Answer 2

This won't be the best answer by any means, but I've been thinking on some things which intersect with points in your question so I thought I'd mention them:

Structural Support

From the little I've seen, when features are buggy and/or don't mesh well with others it is largely due to poor support provided by the core structure/framework of the program for managing/coordinating them. Spending more time fleshing out and rounding out the core, I think, should ease the addition of new features.

One thing I've found to be common in the applications where I work is that the structure of a program was set up to handle one of a kind of object or process but a lot of the extensions we've done or want to do have to do with handling many of a kind. If this were taken into account more at the outset of the application's design, then it would have helped adding these features later.

This becomes quite critical when adding support for multiple X's which involve threaded/asynchronous/event-driven code because that stuff can go bad quite quickly -- I've had the joy of debugging a number of issues related to this.

It's probably difficult to justify this kind of effort up front though, especially for prototypes or one-off projects -- even though some of those prototypes or one-offs go on to be used again or as (the basis of) the final system, meaning the expenditure would have been worth it in the long run.

Design

When designing a program's core, starting out with a top-down approach can help turn things into manageable chunks and lets you wrap your head around the problem domain; after that I think a bottom-up approach should be used -- this will help make things smaller, more flexible, and better for adding on to later. (As mentioned in the link, doing things this way makes for smaller implementations of the features, which means fewer conflicts/bugs.)

If you focus on the core building blocks of the system and make sure they all interact well, then anything built using them will also probably behave well and should integrate better with the rest of the system.

When a new feature is added, I think a similar route could be taken in designing it as was taken designing the rest of the framework: decomposing it then going bottom-up. If you can reuse any of the original blocks from the framework in implementing the feature then that would definitely be helpful; once you're done you can add any new blocks you get from the feature to those already in the core framework, testing them with the original set of blocks -- that way they'll compatible with the rest of the system and usable by future features as well.

Simplify!

I've been taking a minimalist stance on design of late, starting with simplifying the problem, then simplifying the solution. If time can be made for a second, simplifying design iteration on a project then I could see that being very helpful when adding things later on.

Anyway, that's my 2c.