Implementing unit testing at a company that doesn't do it

Question

My company's head of software development just "resigned" (i.e. fired) and we are now looking into improving the development practices at our company. We want to implement unit testing in all software created from here on out.

Feedback from the developers is this:

• We know testing is valuable
• But, you are always changing the specs so it'd be a waste of time
• And, your deadlines are so tight we don't have enough time to test anyway

Feedback from the CEO is this:

• I would like our company to have automated testing, but I don't know how to make it happen
• We don't have time to write large specification documents

How do developers get the specs now? Word of mouth or PowerPoint slide. Obviously, that's a big problem. My suggestion is this:

• Let's also give the developers a set of test data and unit tests
• That's the spec. It's up to management to be clear and quantitative about what it wants.
• The developers can put it whatever other functionality they feel is needed and it need not be covered by tests

Well, if you've ever been in a company that was in this situation, how did you solve the problem? Does this approach seem reasonable?

Answer 1

It seems that you are mixing up two different kinds of tests: unit tests and system / acceptance tests. The former operate on a lower level, testing small pieces of code (typically individual methods), which usually reside deep inside the program, not directly visible to the users. The latter tests the whole program as seen by its users, on a much higher level of granularity. Thus, only the latter can be based on any form of system specification.

Separating the two issues makes it easier to start moving towards an improved development process. Start writing unit tests as soon as possible, regardless of how the software is (not) specified at high level. Whenever a developer creates or changes a method, it does something concrete, which can (and should) be unit tested. In my experience, even changing high level requirements do not typically affect these lower level code blocks dramatically: the code mostly needs to be rearranged, rather than thrown away or rewritten completely. Consequently, most of the existing unit tests will keep running fine.

One important condition to enable unit testing is: the deadlines should not be decided by management up front, but should instead be based on work estimates by the developers (who in turn should include in their estimation the time needed to write proper unit tests). Or, if the deadline is fixed, the scope of delivery should be negotiable. No amount of (mis)management can change the fundamental truth that a given number of developers can only deliver a certain amount of quality work in a given amount of time.

Parallel to this, start discussing the best way to clarify and document requirements and turn them into high level acceptance tests. This is a longer process of successive refinement, which can easily take years to get to a better, stable state across a whole organization. One thing seems fairly sure from your description: trying to fix constantly changing requirements by writing large specification documents up front is just not going to work. Instead, it is recommended to move towards a more agile approach, with frequent software releases and demonstrations to users, and lots of discussions about what they actually want. The user has the right to change his/her mind about the requirements any time - however, each change has its cost (in time and money). The developers can estimate the cost for each change request, which in turn enables the user / product owner to make informed decisions. "Surely, this feature change would be nice... but if it delays the release of this other crucial feature, and costs this much, let's put it into the backlog for now".

Getting users to define acceptance test cases and create test data is a great way to involve them more, and to build mutual confidence between users and developers. This forces both parties to focus on concrete, measureable, testable acceptance criteria, and to think use cases through in a lot more detail than typical. As a result, users get to check the current status of development first-hand with each release, and developers get more concrete, tangible measurement feedback about the status of the project. Note though that this requires greater commitment from users, and new ways of operation, which may be a tough thing to accept and learn.

Answer 2

My experience with making the transition

For many years I was under the misapprehension that I didn't have enough time to write unit tests for my code. When I did write tests, they were bloated, heavy things which only encouraged me to think that I should only ever write unit tests when I knew they were needed.

Recently I've been encouraged to use Test Driven Development and I found it to be a complete revelation. I'm now firmly convinced that I don't have the time not to write unit-tests.

In my experience, by developing with testing in mind you end up with cleaner interfaces, more focussed classes & modules and generally more SOLID, testable code.

Every time I work with legacy code which doesn't have unit tests and have to manually test something, I keep thinking "this would be so much quicker if this code already had unit tests". Every time I have to try and add unit test functionality to code with high coupling, I keep thinking "this would be so much easier if it had been written in a de-coupled way".

Comparing and contrasting the two experimental stations that I support. One has been around for a while and has a great deal of legacy code, while the other is relatively new.

When adding functionality to the old lab, it is often a case of getting down to the lab and spending many hours working through the implications of the functionality they need and how I can add that functionality without affecting any of the other functionality. The code is simply not set up to allow off-line testing, so pretty much everything has to be developed on-line. If I did try to develop off-line then I would end up with more mock objects than would be reasonable.

In the newer lab, I can usually add functionality by developing it off-line at my desk, mocking out only those things which are immediately required, and then only spending a short time in the lab, ironing out any remaining problems not picked up off-line.

My advice

It seems like you have started off well, any time you are going to make big changes to your development workflow, you have to make sure that everyone is involved in making that decision, and ideally that most people have bought into it. From your question, it looks like you've got this right. If people don't have an enthusiasm for the idea, it is doomed to either fail or generate bad will.

Unless you can present a compelling business case, I would not recommend a ground up implementation of unit tests and specifications for your whole system. As I mention above, if a system isn't designed with testing in mind, it can be very difficult to write automated tests for it.

Instead I would recommend starting small and using the the Boy Scout Rule:

Always leave the campground cleaner than you found it.

If while you are implementing something on this codebase, you can identify the specific tests required to test the existing behaviour and transition from the old behaviour to the new, then you have both documented the change in spec and made a start on implementing units tests for your system.

Modules which you don't touch don't get unit tests, but if you aren't touching them then it is probably because they are already thoroughly tested in use and need no changes, or they are never used.

What you want to avoid is wasting a whole load of developer effort writing tests which are never going to be needed (YAGNI works just as well for test code as for production code *8'), never going to be used again and demoralise people into thinking that tests are useless after all.

Summary

Start small, build trust in the tests incrementally and gain business value from developing tests when and where they benefit your team the most.

Answer 3

The first thing to do is concentrate not on testing, but on getting the overall process right. No point testing anything if you don't fully understand what it's supposed to do!

So.. specs first, and documented specs that do not change (well, not immediately). You must look at getting those done. I'd recommend a web site where the users can upload the specs, or type them in directly. You can also link that to a bug tracker and a guide on progress of the project.

Chances are, that's all you really need. You can add unit testing to that internally and management need never know that the developers are doing unit tests. In a way, that's the way its supposed to be.

You will still need to do system testing, but that too can be linked to the project management website, once released, the test team (even if that is another lucky developer on rotation) can update it with tests they've used to see if the whole thing hangs together.

I honestly don't think this is going to change overnight, if you're used to getting 'word of mouth' specs, then the battle is almost entirely going to change this behaviour - and you're going to get resistance to this. A user (or BA or PM or whoever) who is used to saying "just make it do x" and now needs to write it all down is not going to respond well, chances are they'll write vague spec documents and then clarify them with word-of-mouth updates. So forget unit testing, and start on the big problem you have with the development lifecycle.

Answer 4

First issue: in order to "give the developers a set of test data and unit tests", you first must write those unit tests, which is a job of developers. Unit tests doesn't replace the specification neither: the spec is intended to have an abstraction level higher to the unit tests.

Second issue: it seems that you want the maximum unit test coverage. In this case, yes, it will cost too much time and money to write both tests and code in a context where the requirements constantly change. Instead, decide what parts of the code are critical, and unit test only those parts. In many cases, the requirements which change are not affecting the critical parts of the product. A customer (or CEO, or whatsoever) usually asks to move this panel to the right, or change the color of this title from red to green: things nobody cares about and which doesn't require intensive testing. On the other hand, a customer will never ask to change the hash algorithm from SHA512 to SHA256 or to change the way you store sessions, while it's those parts which will require the most testing.