Kent Beck famously said that

Software features that can't be demonstrated by automated tests simply don't exist

But is this too extreme? I mean, I can't demonstrate that my UI really works, the screen pixel is correct in automated tests. So my features don't exist?

  • 1
    Is this a question, or are you just looking for people to agree with you about an opinion? – Marcie Jan 5 '11 at 13:53
  • 2
    @Marcie, this is a question. The question is: is the statement too extreme? – Graviton Jan 5 '11 at 14:29
  • Soon Hui, so the answer you're looking for is "yes"? :) – Marcie Jan 5 '11 at 14:44
  • @Marcie, actually I don't know. – Graviton Jan 6 '11 at 1:23

Any time I hear an extreme statement like that, I consider the problem that they are addressing and how they propose to solve it. After that, I use my brain to judge when the statement doesn't apply.

12 years ago, Unit Testing (at least as we know it today) was a relatively unknown phenomenon except for the magi that were promoting it. Kent Beck was one of those magi. Any time you want to convince someone to try something new, it seems that a hyperbole is a requirement.

The intent of the statement was that if you test all the internal aspects of your code such as the model and controller portions of your app, the application is functionally correct. I quickly learned that the only way to test user interaction is with (gasp, cough, choke) actual users.

That said, don't throw the baby out with the bath water, but do use common sense when you hear statements like that.

  • 1
    +1. Well said. "Good advice comes with a rationale so you can tell when it becomes bad advice." - Raymond Chen – Steven Evers Jan 5 '11 at 16:44
  • 2
    +1: Think. Blanket, extreme, absolutist statements always exist to stimulate a response. Thinking is that response. – S.Lott Jan 5 '11 at 17:01

Software features that can't be demonstrated by automated tests simply don't exist

It is one of the most arrogant and stupidest things I have ever heard.

Forget that nonsense. (I suppose the expression art deserves the same kind of response)

With what automated tests can you measure the user experience? And it's one of the key features in modern software products. Intangible, I'll give you that, but nonetheless crucial to the market success. You can't write a machine test for that, but the human brain will test this characteristic in a matter of minutes with an all-penetrating, cruel and not-forgiving test.

  • 3
    Is user experience a 'software feature' though? – Mchl Jan 5 '11 at 13:34
  • 5
    A highly communicative user interface is often not just a software feature, but one of the best selling ones. Try to sell business intelligence products which can't communicate efficiently their findings... – em70 Jan 5 '11 at 13:51
  • 1
    Just because you may need manual user testing doesn't mean you don't also need automated unit testing. If the only way to test a feature is via manual testing, your code needs to be decomposed further. – Steve Mar 15 '12 at 1:59

He wrote that 12 years ago.

This did not exist at that time, or was very limited.

When I read what I wrote 12 years ago, I want to hide myself under my bed, and cry.

  • 2
    @Machado: I don't read my 10 years old code in order to avoid suicide. – user2567 Jan 5 '11 at 12:16
  • 1
    I already want to hit myself with something if I read my code from 2 years ago... – Bobby Jan 5 '11 at 12:19
  • 1
    +1, but if you don't want to cry when you read what you wrote 12 years ago, you need to find a new line of work because that means you're not learning anything. – Larry Coleman Jan 5 '11 at 13:04
  • 1
    If you had done premature optimization on your 12 year code, you would have had to hide and cry a LOT more and earlier. Procrastination is the thing! – user1249 Jan 5 '11 at 13:56
  • 1
    I'm not going to watch a video, but if that's about UI testing it did exist 12 years ago. Apple was very determined to get UI right, and ran a great many tests to improve MacOS. One thing they struggled with was teaching testers to report if anything felt bad to them, which isn't something you're getting with automated tests. – David Thornley Jan 5 '11 at 15:30

Software features that can't be demonstrated by automated tests simply don't exist

It is one of the most useful and helpful things I have ever heard.

Knowing that you can't measure the user experience, you still have to create software that works. Whether the software works or doesn't work is independent of how it looks.

Good-looking software can do nothing.

Bad-looking software can totally solve a problem elegantly and effectively.

I mean, I can't demonstrate ... the screen pixel is correct in automated tests. So my features don't exist?

"my features don't exist" is taking the comment too far out of context.

You have two unrelated things.

  1. Works.

  2. Looks Correct.

One must be tested. If you can't test it, then it doesn't exist.

The other must be experienced. Note that you can make UI changes that users can't experience. They don't exist either.

The color-blind users, for example, can't tell that you fixed the "red" background to be a different "red". If you've chosen a color that a user cannot see, than the color does not exist.


You ought to read this article, in which Joel Spolsky tears that notion apart and explains that taking it to its logical conclusion is basically "the software equivalent of a perpetual motion machine."

  • Isn't that article more about proof? Not testing? There's a small bit about visual inspection of a GUI (something not testable). But it doesn't seem to totally refute all testing. What did I miss? Can you provide a quote that would help clarify your point? – S.Lott Jan 5 '11 at 16:16
  • 2
    @S.Lott: It starts out talking about proofs, but later on it gets into tests and user experience, which is more than just visual/GUI stuff. The point is that the stuff that really matters, that makes the users really care about a product and think it's well-made, tends to be precisely the stuff that "can't be demonstrated by automated tests." – Mason Wheeler Jan 5 '11 at 17:53

What about complex random processes? You can verify that the resulting data object is legal for the rules in effect but you can't make it stable over code changes. Mocking the random generator won't help if you change the sequence of calls to the generator.

Such a thing is unlikely in the business world but randomly generated game worlds are another matter.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.