Return to Question

I've worked on a large financial transaction system for a bank that looked after Pensions and Investments. After 15 years of feature changes, the manual regression test cost had climbed to $200K per release. (10M LOC, $10M transacted per day). This system also interfaces with 19 other systems around the company moving lots of data around.

  This system was implemented in Java. One of the occasionally touted benefits of OO is 'reuse'. What we observe however, is that the more 'reuse' we do, the more regression test costs go up.

My question is is there a software engineering principle that describes the relationship between reuse and regression test costs?

The reason I'd ask this question is that arguably there is a cost benefit in decomposing the system into smaller parts to be tested.

Assumptions:

1. 'Regression test' means 'acceptance test' - ie another group spending time to write new and reuse old tests against the system on behalf of the business, including environment and data setups.

2. I know the knee-jerk reaction to a big regression test cost is 'more automated tests'. This is a good principle. In this environment there are a couple of challenges.

(a) Automated tests are less useful across system boundaries, unless that system has a high automated test coverage as well. (Sphere of influence challenge).

(b) It is culturally difficult to get momentum on programmer time or capital investment on high automated test coverage when your system is already large and complex.

(c) The cost of maintaining automated tests is hidden on a project, and so they are easily discarded at a project level.

(d) This is just the cultural reality of working in a bank.

(e) I'm working to solve this problem in a different way (decomposition).

3. I'm aware that because of Alan Kay's quote about OO

Actually I made up the term "object-oriented", and I can tell you I did not have C++ in mind

there isn't an agreed definition of what OO means. For the purpose of this question I'm going to assume that 'OO' means 'Java OO' - so encapsulation, inheritance and polymorphism.

I've worked on a large financial transaction system for a bank that looked after Pensions and Investments. After 15 years of feature changes, the manual regression test cost had climbed to $200K per release. (10M LOC, $10M transacted per day). This system also interfaces with 19 other systems around the company moving lots of data around. This system was implemented in Java. 

What we observe however, is that the more 'reuse' we do, the more regression test costs go up. (The reason being that you need to "test the code you touch" - and reused/shared code impacts a multiplicity of places when it is touched. So despite 'DRY - Do Not Repeat Yourself' - ie don't copy and paste code - we observe a financial incentive to copy and paste code. This is to drive regression test costs down, because we don't want to modify code that could be shared, because that will cause a big regression test impact. )

My question is is there a software engineering principle that describes the relationship between reuse and regression test costs?

The reason I'd ask this question is that arguably there is a cost benefit in decomposing the system into smaller parts to be tested.

Assumptions:

1. 'Regression test' means 'acceptance test' - ie another group spending time to write new and reuse old tests against the system on behalf of the business, including environment and data setups.

2. I know the knee-jerk reaction to a big regression test cost is 'more automated tests'. This is a good principle. In this environment there are a couple of challenges.

(a) Automated tests are less useful across system boundaries, unless that system has a high automated test coverage as well. (Sphere of influence challenge).

(b) It is culturally difficult to get momentum on programmer time or capital investment on high automated test coverage when your system is already large and complex.

(c) The cost of maintaining automated tests is hidden on a project, and so they are easily discarded at a project level.

(d) This is just the cultural reality of working in a bank.

(e) I'm working to solve this problem in a different way (decomposition).