-2

I am getting started with software testing in Java and was wondering if test-driven dev and white-box testing could be combined directly... If not, what ways can we extend TDD to combine white box testing?

4
  • 1
    Nothing prevents you from supplementing your black box TDD tests with white-box tests after the code for the TDD tests has been written and all of the black box tests are green. – Robert Harvey Dec 11 '20 at 22:08
  • 1
    @RobertHarvey but why would you? You've already got tests for all of the behaviour, adding more tests is likely going to couple you more to the implementation and make it harder to refactor. – jonrsharpe Dec 11 '20 at 22:37
  • @jonrsharpe: The most likely scenario is a discovered bug. Yes, you can treat that like a black box test, but it's going to be white box in practice because you've got to go into the code and fix the bug, and because the test wouldn't exist absent the bug's discovery. – Robert Harvey Dec 11 '20 at 22:44
  • @RobertHarvey but that's not going to be adding new green tests on green tests, it's not outside the TDD process. That's going to be adding a new failing test that demonstrates the bug, then fixing it. You (almost) always have to "go into the code" to get the test passing, that doesn't make it white box. TDD is about testing behaviour through public APIs, it's far more closed then open box, even once you've drilled down from E2E to unit level. – jonrsharpe Dec 11 '20 at 22:55
2

TDD and white-box testing are fundamentally incompatible with each other.

White-box testing means that you write the test based on knowledge of the internals of the code.

TDD means that you write the tests before the code.

It is fundamentally impossible to write tests based on knowledge of code that does not exist.

Note that you of course can write additional white-box tests after you have implemented the code. But that has nothing to do with TDD.

It is of course also possible to do some sort of "fake" TDD, where you write the code in your head, and then write a white-box test based on your mental image of what the code will be. But again, that is not TDD. In TDD, the tests drive the code. If you imagine the code in your head before you write the test, then you are not doing TDD. It doesn't matter that you didn't write the code down in your IDE, you still did write it in your head.

4
  • 1
    This answer is not completely wrong, but it ignores the fact that TDD is some kind of white-box technique, it only works well when you know which parts of the code are already covered by the former tests, and which not, so you can leave out tests which don't increase any coverage. And the resulting tests are usually undistiguishable from classic white-box tests which are written with the existing code in mind. – Doc Brown Dec 11 '20 at 20:28
  • 1
    @DocBrown it's really not. The idea that you'd "leave out tests which don't increase any coverage" suggests what you're describing is not TDD. You write tests because you need new behaviour from the code, then you write the simplest code that provides that new behaviour and all previous requirements; high coverage is an outcome. – jonrsharpe Dec 11 '20 at 22:19
  • I agree with Doc Brown here. The red part of red/green/refactor will ensure that a TDD test exercises some new behaviour, thus effectively excluding behaviour that is already covered. TDD could theoretically work as a black box approach, but in practice the developer knows very well what's going on. The outcome is indistinguishable, except that TDD leads to a more testable design. – amon Dec 12 '20 at 9:28
  • 2
    Knowing what’s going on is different from white box. A product owner could be aware of a bug and how to reproduce it without knowing the internals of the code. Implementing this behaviour in a test to reproduce such bug using only the public api is black box testing. You don’t need to know the internals. White box testing means you know for sure the implementation details of the code. TDD is always at least one unit higher than the code, since it’s only using it. It does not know how it works, it knows what it does. – Steve Chamaillard Dec 12 '20 at 10:58
0

Some definitions (my emphasis):

Test-driven development (TDD) is a software development process relying on software requirements being converted to test cases before software is fully developed, […]. This is opposed to software being developed first and test cases created later.

Basically, TDD means writing the tests before changing any of the underlying code. Crucially, the TDD process includes changing the underlying code, not just writing the tests.

White-box testing […] is a method of software testing that tests internal structures or workings of an application, as opposed to its functionality (i.e. black-box testing).

In other words, white-box testing verifies the internals of an application with full visibility of the code, while black-box testing verifies the externally visible functionality, with (ideally) no knowledge of the underlying code. Contrary to TDD, white-box testing is only concerned with the test writing phase, not with changing the underlying code.

TDD therefore requires white-box testing: it is not possible to write the underlying code to satisfy the tests without having knowledge of the existing underlying code. The only exception would be when the new code simply wraps the underlying system, and so requires only knowledge of the existing inputs and outputs.

5
  • 4
    TDD is not white-box testing; if anything, the box is empty! One of the advantages of TDD is that, because you're writing code prior to implementation, you can't couple your tests too closely to the implementation, and have to think about behaviour and functionality instead. – jonrsharpe Dec 11 '20 at 9:50
  • @jonrsharpe The box is only empty when you write the very first test. And without extensive knowledge of the existing code you could not write a decent test which involves any existing functionality - you wouldn't know where to begin. – l0b0 Dec 11 '20 at 19:10
  • 3
    It's just a metaphor, but if you'd prefer think of it as writing each test for something that's not yet in the box, whether or not there's anything in there already. And you can absolutely write a new test without extensive knowledge of the existing code, because you're thinking at the level of the new behaviour you want; where exactly that fails tells you where to start looking. These are the advantages of writing tests that aren't coupled to the implementation details. – jonrsharpe Dec 11 '20 at 19:25
  • 2
    Your last paragraph isn't true, and doesn't really make sense. TDD does not require white-box testing and the fact that you have to write an implementation at some point, as opposed to just writing the tests, is not an argument at all; it's "test-driven development", not just "test". The point is that you're writing tests before the implementation exists, therefore cannot be writing them based on its internal structures. Compare this to writing tests after implementation, where you could (but shouldn't) choose to use the implementation details to drive your testing. – jonrsharpe Dec 11 '20 at 23:41
  • 2
    I agree with @jonrsharpe. To put it another way, you can do pair programming with someone that doesn’t know anything about the insides of the code and that dev could write the next failing test just by looking at requirements and existing tests. This is purely black box testing. – Steve Chamaillard Dec 12 '20 at 11:08
-1

The questions, comments, and answers contain so many problems that I felt compelled to write an answer to this heavily downvoted question.

There are, at least, two different scenarios in which TDD is successfully applied.

There are, of course, infinitely many scenarios in which it was futile. But that's not a point. Not every rocket company ends up launching a rocket.

Scenario 1: (Macroscopic) Test-Driven Design (TDD Design)

In this approach, TDD is used to help the architect think through how the software's architecture and interfaces should look like, without being tied up with details such as implementation specifics.

TDD Design tests tend to be high-level, covering the API level, sometimes at the end-to-end level. TDD Design may entail creating and then throwing away a lot of tests as well as mock implementations.

To the extent that "very high level tests tend to be white box tests", TDD Design tend to be white box.

Scenario 2: (Microscopic) Incremental Test-Driven Development (ITDD)

In this approach, the software framework is already well-established, and code changes tend to be incremental, in that existing functionality are seldom affected. ITDD ensures that new code changes have high coverage to begin with, and that none of the existing functionality were adversely affected.

To the extent that "incremental tests tend to be unit tests", ITDD tend to result in additions to an existing unit test suite.

7
  • In scenario 1 you start by saying "without being tied up with details" and end with ""very high level tests tend to be white box tests"". Which is it? Also does it entail throwing away a lot of tests? In my experience you generally remove tests only when you don't need that behaviour any more. – jonrsharpe Dec 11 '20 at 23:44
  • @jonrsharpe Test-Driven API Design's goal is to design an API that users will feel "right". They only test that the API design "appears to be logically sound", typically without checking the result of computation (because, at that early stage, few of the computations would have been implemented). If Test-Driven API Design sounds completely foreign to you, it is possible that you have not had experience with it. – rwong Dec 11 '20 at 23:54
  • Test-driven API design as you describe is not TDD, and more like consumer-driven prototyping. Also by definition that can't be white box, there isn't a real implementation to couple to, so the third sentence of your first scenario still doesn't make sense. – jonrsharpe Dec 11 '20 at 23:57
  • I said "details such as implementation specifics". It might be more clear if I had said "without being tied up with implementation details". What I mean is, TDD API Design is typically done when implementation details do not exist yet. – rwong Dec 11 '20 at 23:57
  • @jonrsharpe "Test-driven API design as you describe is not TDD, " to which I reply, "that's what you said." – rwong Dec 11 '20 at 23:58

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.