20

Preliminary notes

I'll not go into the distinction of the different kinds of test there are, there are already a few questions on these sites regarding that.

I'll take what's there and that says: unit testing in the sense of "testing the smallest isolatable unit of an application" from which this question actually derives

The isolation problem

What is the smallest isolatable unit of a program. Well, as I see it, it (highly?) depends on what language you are coding in.

Micheal Feathers talks about the concept of a seam: [WEwLC, p31]

A seam is a place where you can alter behavior in your program without editing in that place.

And without going into the details, I understand a seam -- in the context of unit testing -- to be a place in a program where your "test" can interface with your "unit".

Examples

Unit test -- especially in C++ -- require from the code under test to add more seams that would be strictly called for for a given problem.

Example:

  • Adding a virtual interface where non-virtual implementation would have been sufficient
  • Splitting -- generalizing(?) -- a (smallish) class further "just" to facilitate adding a test.
  • Splitting a single-executable project into seemingly "independent" libs, "just" to facilitate compiling them independently for the tests.

The question

I'll try a few versions that hopefully ask about the same point:

  • Is the way that Unit Tests require one to structure an application's code "only" beneficial for the unit tests or is it actually beneficial to the applications structure.
  • Is the code generalization that is needed to make it unit-testable useful for anything but the unit tests?
  • Does adding unit tests force one to generalize unnecessarily?
  • Is the shape unit tests force on code "always" also a good shape for the code in general as seen from the problem domain?

I remember a rule of thumb that said don't generalize until you need to / until there's a second place that uses the code. With Unit Tests, there's always a second place that uses the code -- namely the unit test. So is this reason enough to generalize?

2
  • 8
    A common meme is that any pattern can be overused into becoming an anti-pattern. The same is true with TDD. One can add testable interfaces past the point of diminishing returns, where the code tested is less than the generalized test interfaces added, as well as into the too low cost-benefit area. A casual game with added interfaces for testing like a deep space mission OS could completely miss its market window. Make sure that the added testing is before those inflection points.
    – hotpaw2
    Commented Nov 14, 2011 at 14:50
  • @hotpaw2 Blasphemy! :)
    – maple_shaft
    Commented Nov 14, 2011 at 14:53

5 Answers 5

23

Unit test -- especially in C++ -- require from the code under test to add more seams that would be strictly called for for a given problem.

Only if you don't consider testing an integral part of problem solving. For any nontrivial problem, it ought to be, not only in the software world.

In the hardware world, this has been learnt long ago - in the hard way. The manufacturers of various equipment have learnt through centuries from countless falling bridges, exploding cars, smoking CPUs etc. etc. what we are learning now in the software world. All of them build "extra seams" into their products in order to make them testable. Most new cars nowadays feature diagnostic ports for the repairmen to get data about what's going on inside the engine. A significant portion of the transistors on every CPU serve diagnostic purposes. In the hardware world, every bit of "extra" stuff costs, and when a product is manufactured by the millions, these costs surely add up to large sums of money. Still, the manufacturers are willing to spend all this money for testability. Probably they have figured (or learnt the hard way) that the risk of their product failing in the hands (or under the bums) of end users, and the ensuing loss of reputation, lawsuits, bad PR etc. etc. etc. is way higher than the cost of designing and making their product testable from the start.

Back to the software world, C++ is indeed more difficult to unit test than later languages featuring dynamic classloading, reflection etc. Still, most of the issues can be at least mitigated. In the one C++ project where I used unit tests so far, we didn't run the tests as often as we would in e.g. a Java project - but still they were part of our CI build, and we found them useful.

Is the way that Unit Tests require one to structure an application's code "only" beneficial for the unit tests or is it actually beneficial to the applications structure?

In my experience a testable design is beneficial overall, not "only" for the unit tests themselves. These benefits come on different levels:

  • Making your design testable enforces you to chunk your application into small, more or less independent parts which can only influence each other in limited and well defined ways - this is very important for the long term stability and maintainability of your program. Without this, the code tends to deteriorate into spaghetti code where any change made in any part of the codebase may cause unexpected effects in seemingly unrelated, distinct parts of the program. Which is, needless to say, is every programmer's nightmare.
  • Writing the tests themselves in TDD fashion actually exercises your APIs, classes and methods, and serves as a very effective test to detect whether your design makes sense - if writing tests against and interface feels awkward or difficult, you get valuable early feedback when it is still easy to shape the API. In other words, this defends you from publishing your APIs prematurely.
  • The pattern of development enforced by TDD helps you focus on the concrete task(s) to do, and keeps you on target, minimizing the chances of you wandering off solving other problems than the one you are supposed to, adding unnecessary extra features and complexity, etc.
  • The fast feedback of unit tests allows you to be bold in refactoring the code, enabling you to constantly adapt and evolve the design over the lifetime of the code, thus effectively preventing code entropy.

I remember a rule of thumb that said don't generalize until you need to / until there's a second place that uses the code. With Unit Tests, there's always a second place that uses the code -- namely the unit test. So is this reason enough to generalize?

If you can prove that your software does precisely what it's supposed to do - and prove it in a fast, repeatable, cheap and deterministic enough way to satisfy your customers - without the "extra" generalization or seams forced by unit tests, go for it (and let us know how you do it, because I am sure a lot of people on this forum would be as interested as me :-)

Btw I assume by "generalization" you mean things like introducing an interface (abstract class) and polymorphism instead of a single concrete class - if not, please clarify.

2
  • Sir, I salute you.
    – GordonM
    Commented Jul 25, 2012 at 21:18
  • A brief, but pedantic note: the "diagnostic port" is mostly there because governments have mandated them as part of an emission control scheme. Consequently, it has severe limitations; there are many things that could potentially be diagnosed with this port that are not (i.e. anything not having to do with emissions control). Commented Aug 28, 2012 at 23:22
4

I am going to throw The Way of Testivus at you, but to summarise:

If you are spending a great deal of time and energy making your code more complicated to test a single part of the system, it may be that your structure is wrong, or that your testing approach is wrong.

The simplest guide is this: What you are testing is the public interface of your code in the way it is intended to be used by other parts of the system.

If your tests are becoming long and complicated, it's an indication that using the public interface is going to be difficult.

If you have to use inheritance to enable your class to be used by anything other than the single instance that it's currently going to be used for, then there's a good chance your class is too heavily tied into its usage environment. Can you give an example of a situation where this is true?

However, beware of unit-testing dogma. Write the test that allows you to detect the issue that will cause the client to shout at you.

1
  • I was going to add the same: make an api, test the api, from the outside. Commented Nov 15, 2011 at 0:27
2

TDD and Unit Testing, is good for the program as a whole, and not just for the unit tests. The reason for this is because it is good for the brain.

This is a presentation about a specific ActionScript framework named RobotLegs. However if you flip through the first 10 slides or so, it starts getting to the good parts about the brain.

TDD and Unit testing, forces you to behave in a manner which is better for the brain to process and remember information. So while your exact task in front of you is just making a better unit test, or making code more unit testable... what it actually does is make your code more readable, and thus make your code more maintainable. This gets you coding in habbits faster, and gets you able to comprehend your code faster when you need to add/remove features, fix bugs, or in general open up the source file.

1

testing the smallest isolatable unit of an application

this is true, but if you take it too far it doesn't give you much, and it costs a lot, and I believe that it is this aspect that is promoting the use of the term BDD to be what TDD should have been all along - the smallest isolatable unit is what you want it to be.

For example, I once debugged a network class that had (among other bits) 2 methods: 1 to set the IP address, another to set the port number. Naturally, these were very simple methods and would pass the most trivial test easily, but if you set the port number and then set the ip address, it would fail to work - the ip setter was overwriting the port number with a default. So you had to test the class as a whole to ensure correct behaviour, something I think the concept of TDD is missing but BDD gives you. You don't really need to test each tiny method, when you can test the most sensible and smallest area of the overall application - in this case the networking class.

Ultimately there is no magic bullet to testing, you have to make sensible decisions about how much and at what granularity to apply your limited test resources to. The tool-based approach that auto--generates stubs for you doesn't do this, its a blunt force approach.

So given this, you do not need to structure your code in a certain way to achieve TDD, but the level of testing you do achieve will depend upon the structure of your code - if you have monolithic GUI that has all its logic tightly bound to the GUI structure, then you're going to find it harder to isolate those pieces, but you can still write a unit test where 'unit' refers to the GUI and all the back-end DB work is mocked. This is an extreme example, but it shows you can still do automated testing on it.

A side-effect of structuring your code to make it easier to test smaller units does help you define the application better, and that allows you to replace parts easier. It also helps when coding as it'll be less likely that 2 devs will be working on the same component at any given time - unlike a monolithic app that has intermingled dependencies that breaks everyone else's work come merge time.

0

You've reached a good realization about tradeoffs in language design. Some of the core design decisions in C++ (the virtual function mechanism mixed with the static function call mechanism) make TDD tough. The language doesn't really support what you need to make it easy. It is easy to write C++ that is nigh impossible to unit test.

We've had better luck doing our TDD C++ code from a quasi-functional mindset- write functions not procedures (a function that takes no arguments and returns void), and use composition wherever possible. Since its tough to substitute these member classes, we focus on testing those classes to build a trusted base, and then know that basic unit functions when we add it to something else.

The key is the quasi-functional approach. Think about it, if all your C++ code was free functions that accessed no globals, that would be a snap to unit test :)

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.