A tester does not care about HOW the black box works, they only care THAT it works.
This is the core issue with your notion on TDD. At every point, you show a deep interest in the internal workings, but that is something you (as a tester) should not care about. Similarly, as the driver of a car, you should only care whether the car works or not. When the car doesn't work, you shouldn't actually care why it doesn't work. What you do then is go to a mechanic. It is the mechanic's job to figure out why the car doesn't work.
You need to observe the two very different roles of tester/developer, just like how in my example the driver/mechanic are two very different roles.
My question is: If you write the most straightforward code possible to get the light green, and if you don't test internal methods, does it follow that from time to time you allow yourself to start off by writing a class that does far more than it should with a bunch of member variables you know will not survive refactoring? And wind up after refactoring with a bunch of methods that aren't explicitly tested?
As I get familiar with TDD, it feels like anyone would at least be tempted to test internal methods. And many practitioners flatly say you shouldn't. E.g. (of about 23 million results)
Never lost track of what your test aims to confirm.
The purpose of your tests is to confirm that the application is behaving as intended. Behavior is externally observable, and thus limited to the public (as opposed to internal/private) realm.
Example: you want an application that calculates the average of two numbers. I give you a compiled application and claim that this is what you want. You do not know the source code and I refuse to tell you.
You can still test if this application is correct simply by using it and confirming that its output is what you expect it to you. You don't need to know HOW the application achieves the correct outcome. Whether my application uses mathematical operations, googles it, or simply guesses the result; does not matter. As long as the application consistently renders the correct output, it should pass all of your tests.
Pedantically, my example is a UAT and not a unit test or integration test, but the principle remains the same. There is always some degree of "black box guts" that your test should not care about.
1) non-trivial logic winds up getting tested only indirectly, AND tested in the same calls as other non trivial logic.
Trivially readably logic does not warrant a test. Logic which trivially calls non-trivial logic very much needs to be tested.
2) you make methods public that you don't expect clients other than the SUT and the tests to call.
A public method which is only used by a test and not by production code should not exist. It makes no sense and it defeats the purpose of testing.
When you're testing, you want to test the actual code path. But if you have your test use a method that isn't actually used by your production code, the test result is no longer fully indicative of whether the application works as intended.
- If there is an issue in this test-only public method, then your test will fail. But your production code would have passed.
- If there is an issue in the "production counterpart" of what this test-only method provides, then your test will pass (as it's still using the test-only methods which does not contain this issue) but your production code will fail.
- If there is no "production counterpart" to the test-only method, then what is the point of including this method in a test? You're testing it just to test it; without producing any actually meaningful results from the fact that it passes/fails the test.
But even though it sounds reasonable -- if I don't test internals I feel like I lose part of what helps keep the tests DRIVING the coding.
I hear this argument often, and I always think back to the same quote by Antoine de Saint-Exupéry:
“If you want to build a ship, don’t drum up the men to gather wood, divide the work, and give orders. Instead, teach them to yearn for the vast and endless sea.”
What you're advocating is that while the ship is being built, we are testing if people are hammering the right nails in the right planks. But that is excessive micromanagement. What you should instead test is:
- Are these men intending to build a ship?
- Do these men have the particular skills (hammering, sawing) to build a ship?
Bring it back to your case, what you're trying to do is use tests to confirm whether your developers are working. But tests are not a timetracking tool!
Look at it this way: you've defined the public interface (in a sense, that is what the analysis defines for you). The developers are tasked with implementing this interface. How they choose to achieve the adherence to the public interface is not yours to decide, and should not be policed using unit tests. This is the developers' personal resonsibility: they are the code smiths, they retain the autonomy to create the code that they deem to be the best solution to the problem.
If there's some internal logic that has to turn a String to a valid int32, int64 or decimal value depending on what's in the string and the type of some other object, I want to test that little bit of logic, not just find out that the whole method failed.
You are conflating two very different levels of detail. Those who write tests should not care that a string needs to be converted. That is an implementation detail that no one (other than the developers) should care about.
What you're doing is the developer's equivalent of a manager who micromanages. The manager should leave the micro tasks to their employees, and they should only focus on the macro tasks of managing the employees. The drawbacks to micromanaging are twofold:
- By taking on additional micro tasks, they are taking away valuable time they could be spending on the macro tasks
- They are acting as an obstacle for the employees who are responsible for the micro tasks (because 9 times out of 10 the micro employees have more applicable experience with the micro).
Instead, stick to your own role. When you test, you test if it works as expected. Only if it doesn't do you then assign this task to the developers. It is then up to the developers to figure out why it isn't passing the test.
While the tester and the developer can be the same physical person, they are two separate roles, each with their own responsibilities, and trying to blend the two only leads to a lack of accountability that is clearly tied to either role (if you want an off-topic analogy, this is exactly why there is such a strong focus on the separation of church and state, or of different governmental branches. Blending the separate roles muddies the water for both roles).
Or do they just recognize that as a failure point and write a test with input that will cause a failure pinpointing those few lines of code? If they do that it's testing the implementation though. It seems so natural to want to test that code in isolation, and verify it fails where it should, even though it's an implementation detail.
What you're calling "testing that code in isolation" is really just debugging. Part of the approach to debugging is to run the code and see what happens, which you can call a sort of "test" in semantical English. But it is not a test in the TDD sense.
When you say "write a test with input that will cause a failure pinpointing those few lines of code", this implies that the test writer has accurate knowledge of the internal code, which they should not have (or at least not base their tests on).
This is the key part of where you're wrong: the tester should not care about HOW the black box works, they only care THAT it works.
Say you want to put data from an ISAM file into a database. There is one public method to the envisioned consumer of the code: public void LoadIt(SqlConnection c). "Loadit" has a bunch of other dependencies that will be resolved using configuration files and environment variables: some code has to determine that it can find the name of the folder where the ISAMs live; to find, load and parse the schema file for the isams using the ISAM vendor's DDL library; so forth+so on. But none of that is of any interest to any envisioned calling code.
Okay, so let's san there is an IsamToDatabaseLoader
class which has the Loadit(SqlConnection c)
method.
When you are (unit) testing the IsamToDatabaseLoader
, you are testing its behavior regardless of what the exact dependencies are. If anything, you should be mocking any dependency that's anything more than a simple primitive value or data class.
The key thing you need to understand here is that IsamToDatabaseLoader
is not responsible for the correctness of the dependencies it has been provided with. And if it's not responsible for it, then it shouldn't be blamed for any dependency being incorrect. And if it can't be blamed for it, then you shouldn't be testing it!
However, what the IsamToDatabaseLoader
class is responsible for, is how it chooses to behave in case something goes awry. Suppose you give it a bad connectionstring. What is it you want to happen, what do you consider correct behavior?
- Should the class throw an exception? (any exception will do)
- Should the class throw a specific exception? (e.g.
BadConnectionstringException
)
- Should the class log an error?
- ...
Notice that the things are listed are all public behavior. We don't care what happens inside of the IsamToDatabaseLoader
class, we only care what comes out of it.
Depending on what you want to happen, you will have to write the tests that confirm that this happens. Let's say that you want a specific exception (BadConnectionstringException
).
You could then write a unit test which gives IsamToDatabaseLoader
a bad connection string, and catches any potential exception. The test's assertions seem simple enough:
- If no exception is encountered, the test fails.
- If the encountered exception is a
BadConnectionstringException
, the test passes.
- If the encountered exception is not a
BadConnectionstringException
, the test fails.
I think you'll agree that this is a simple and clean test.
But now I have one question for you: how does IsamToDatabaseLoader
decide to throw a BadConnectionstringException
? You don't actually know.
That's the key point. You understood the requirements (throwing the needed exception), you understood how you needed to confirm whether this requirement was met, you were able to agree that the proposed tests is a valid way to confirm that this requirement is met, and at no point did you care about HOW this requirement was going to be met.