How to work out a complex, untested scope?

Question

The scope: it is part of a big ERP system which provides specific calculations. One can think of it as a salary calculation module in a CRM. Yet this is way beyond that in both complexity and integration points. Thus, there are a lot of technological debt, but not unmaintanable one. There some some unit/integration tests written. It is testable, but it wasn't done much. It is an old part of that ERP system (3 years old), over the years it had mutated into what it is now, and there are willingness to try to complete it even if it costs some major efforts.

The problem: the scope which provides a lot of various calculations based on a specific list of objects and the outcomes are not right sometimes.

There are a lot of interesting (not so common) features about this scope:

• Dynamic C# formullas (loaded to context and then included in calculcations)
• A lot of reports doing the same calculations but on the database
• The scope is unstable and period-intense (new features and bugfixes are shipped every month)

This scope has a lot of input parameters on all of the dependant objects. By multiplying all the possible parameter values we have 1,62*10⁷ cases of possible outputs. Not all the parameters warry that much, but most of them do. So slicing half of that number is something around the practical possible otucomes.

This scope is eating up a lot of time and money and is far from done in its current form.

I would like to know what is the silver bullet to the most viable way to stabilize and finish the scope to the state where only minor bugs and nice to haves will occur?

First of all I was thinking, that establishing a DoD (Definition of Done) would be a relevant thing to do. Then I came up to the fact that having that huge number of outputs working is half of that - everything must also be flexible on the UI scope as well.

Will it be enoguh to establish a test plan, maybe do some bruteforcing tests? Should I consider any kind of design patterns? Validation options (like a solid third party validation service, etc.)? Will it be enough or maybe I have gaps in my modest knowledge that I should fill?

Note: the application is not what would one call monolithic. It is based on a lot of different services that can be easily extended, etc.

Answer 1

You need to find the "guiding principles" behind the scope. ax + by = c has an infinite number of solutions, but it's still a bounded equation. It's a matter of finding sufficient test cases where you and the customer are satisfied that ax + by = c is fulfilled. Obviously, you don't need a test for every possible output value. What you need are enough tests to achieve confidence that the software works as it should.

When you put a new condition into the software somewhere, a bifurcation is created (a point where the logic branches off in two different directions). To achieve adequate code coverage, create a test case for each branch in every bifurcation, and tests to cover any remaining edge cases (such as critical number precision boundaries). See here for an example of how this works.

Have the customer agree to a set of test cases that, once fulfilled, will allow you to declare success. Then your scope will be bounded by your test cases.

Answer 2

These two points

new features and bugfixes are shipped every month

and

I would like to know what is the most viable way to stabilize and finish the scope

are in conflict. If there are new features every month, there are new requirements every months, someone seems to be in need of that features, someone else is programming them and that is new software - which will typically contain some bugs at the beginning and will get better by time through tests and user feedback. Moreover, there is no "end" of this process, it is "living software" where changes seem to be part of the normal process.

It would be really helpful if you had lost some word how the current development of that "scope" looks like, is there one dev, a team, are there many people requesting new features etc.?

Best advice I can give you is to try separarating the parts where the requirements are stable from the parts with the unstable requirements. The parts where the requirements are stable should be made bullet-proof and well tested by automated regression tests. Those parts can be somewhat "finished".

The other parts - well I don't know who is defining that dynamic formula part, but assumed that is the part which changes every month. It is important to give the ones who are defining those formulas immediate feedback if they have done it right. Perhaps you have to provide some kind of "interactive testing environment" for those people, so they can try out the formulas before bringing them into the real system.

For example, think of a spreadsheet software like MS Excel - the program itself is a (more or less) stable software, (hopefully) well tested, where you don't get new versions every month. It can be used as platform for formulas and reports, and the user of those formulas and reports can enter those formulas by himself, test them, gets immediate feedback and change those things 10 times a day if he likes. If you can create a comparable "separation of concerns" in you "scope" (whatever that means), then you will be a big step further.