123

Recently, a person here asked a basic question about how to compute in Python all permutations of elements from a list. As for most questions asked by students, I haven't provided the actual source code in my answer, but rather explained how to approach the problem, which essentially ended up in a basic presentation of test driven development.

Thinking about the problem itself, I remembered all the similar problems I did have to solve when I was myself studying programming in university. We were never taught about test driven development, nor any comparable method, but nevertheless were constantly asked to write algorithms which sort elements from a list, or solve Towers of Hanoi puzzle, etc. Nor do all the students who come on SoftwareEngineering.SE seem to know how TDD (Test Driven Development) would help them solving the problems they are given: if they were, they would have formulated their questions very differently.

The fact is, without TDD, all those problems are indeed quite challenging. I consider myself a professional developer, but I would still spend some time writing a list-ordering algorithm without TDD, if asked, and I would be mostly clueless if I were asked to compute all the permutations of the elements in a list, still without using TDD. With TDD, on the other hand, I solved the permutations problem in a matter of minutes when writing the answer to the student.

What is the reason TDD is either not taught at all or at least very late in universities? In other words, would it be problematic to explain the TDD approach before asking students to write list sorting algorithms and similar stuff?


Following the remarks in two of the questions:

In your question, you appear to present TDD as a "problem solving device" [...] In the past, TDD has been presented as a "solution discovery mechanism," but even Bob Martin (the principal advocate of TDD) concedes that you must bring a significant amount of prior knowledge to the technique.

and especially:

I'm curious why you think TDD makes a tricky algorithm problem with a well-defined spec easier.

I find it necessary to explain a bit more what is, in my opinion, so magical about TDD when it comes to solving problems.

In high school and in the university, I didn't have any specific techniques to solve problems, and this applied to both programming and mathematics. In retrospective, I suppose that one of such techniques is to review the current/last lesson/lecture and seek relation with the exercice. If the lesson is about integrals, there are chances the problem the teacher asked to solve requires to use integrals. If the lecture was about recursion, there are chances that the puzzle given to the students could be solved using recursion. I'm also sure there are well formalized approaches to solving problems in mathematics, and those approaches can be applied to programming as well; however, I never learned any.

This means that in practice, my approach was simply to poke the problem around, trying to guess how should it be solved. Back then, if I was given the challenge of generating permutations of elements from a list, I wouldn't start with an empty list as input, but rather an illustrative example, such as [4, 9, 2], trying to figure out why are there six possible permutations, and how can I generate them through code. From there, I need a lot of thinking to find a possible way to solve the problem. This is essentially what the author in the original question did, ending up using random. Similarly, when I was a student, no other students of my age would start with []: all would immediately rush to the case with two or three elements, and then remain stuck for half an hour, sometimes ending up with code which doesn't compile or doesn't run.

The TDD approach, for me, appears to be counter-intuitive. I mean, it works very well, but I would have never figured out myself, before reading a few articles about TDD, that (1) I should start with the simplest case, (2) write the test before writing code, and (3) never rush, trying to fulfil several scenarios in code. Looking at how beginner programmers think, I have an impression that I'm not the only one finding it counter-intuitive. It may be, I believe, more intuitive for programmers who have a good understanding of a functional language. I suppose that in Haskell, for example, it would be natural to handle the permutations problem by considering first a case of an empty list, then a case of a list with one element, and then a case of a list of multiple elements. In languages where recursive algorithms are possible, but not as natural as in Haskell, such approach is however much less natural, unless, of course, one practices TDD.

  • 11
    I believe, this question is on-topic on both sites, and given the different audiences may generate interesting answers on both. There are some related questions already: cseducators.stackexchange.com/q/961/945, cseducators.stackexchange.com/q/908/945, cseducators.stackexchange.com/q/872/945 . – Jörg W Mittag Dec 31 '19 at 22:42
  • 13
    Either I am entirely misunderstanding what you mean by TDD, or there is a problem with your examples, because they are... well, I suppose unitary would be the word. Either you understand the problem, and can write code that works, or you don't. There are really no intermediate steps or sub-modules that you can test. – jamesqf Jan 1 at 18:05
  • 8
    Are you very confident in this question that you mean "TDD" rather than "making a list of expected inputs and outputs"? – Dewi Morgan Jan 2 at 0:45
  • 17
    If I had to make a list of "vital concepts that CS students aren't taught, but should be", version control would be at #1, and TDD would be very far down. – Ian Kemp Jan 2 at 20:38
  • 6
    The problematic assumption is the fact that TDD is good and has no negatives. It's also not the only solution discover mechanism out there. There are many other mechanism (I prefer top-down programming) since I think unit tests are counterproductive in many programs, especially in frontend applications. – Sulthan Jan 2 at 22:41

14 Answers 14

135

I am a part-time programming teacher at a local community college.

The first course that is taught at this college is Java Programming and Algorithms. This is a course that starts with basic loops and conditions, and ends with inheritance, polymorphism and an introduction to collections. All in one semester, to students who have never written a line of code before, an activity that is completely exotic to most of them.

I was invited once to a curriculum review board. The board identified a number of problems with the college's CS curriculum:

  1. Too many programming languages taught.
  2. No courses about the Software Development Life Cycle.
  3. No database courses.
  4. Difficulty in getting credits to transfer to state and local universities, partly because these schools can't agree on a uniform definition for the terms "Computer Science," "Information Technology," and "Software Engineering."

My advice to them? Add a two-semester capstone class to the curriculum where students can write a full-stack application, a course that would cover the entire software development life cycle from gathering requirements to deployment. This would make them hireable at local employers (at the apprentice level).

So where does TDD fit into all of this? I honestly don't know.

In your question, you appear to present TDD as a "problem solving device;" I see TDD mostly as a way to improve the design and testability of code. In the past, TDD has been presented as a "solution discovery mechanism," but even Bob Martin (the principal advocate of TDD) concedes that you must bring a significant amount of prior knowledge to the technique.

In other words, you still have to know how to solve problems in code first. TDD just nudges you in the right general direction relative to software design specifics. That makes it an upper-level course, not a lower-level one.

| improve this answer | |
  • 11
    Considering all the other subjects students need to learn as part of the CS and SE degrees, it would be very difficult to fit it in. TDD has enough depth to warrant at least 2-3 credits worth of work. Particularly when you attempt to cover mocks, stubs, theories, parameterized tests, and even more fundamentally what should and should not be tested. – Berin Loritsch Dec 31 '19 at 19:46
  • 6
    Agreed with the fact that TDD is an upper-level course. But TDD does not just "nudge you in the right general direction". Writing tests first is like setting up a line and ruler before mounting a brick wall. It helps you do a better job in less time by transforming after-the-fact repetitive alignment check into a continuous alignment jig. It's better than checking alignment after laying out each brick, which in turn is better than checking alignment after laying out bigger and bigger lumps of bricks. I don't remember my CS degree insist that much that I automate my tests, just that I test. – Laurent LA RIZZA Jan 2 at 13:47
  • 12
    Devil's advocate: Many usages of TDD I have seen actually crippled the architecture, creating tons of unnecessary abstraction layers just for the sake of testability. And the tests themselves were actually a duplication of tested code just for the sake of having tests. Testability is good but often some code is just impossible to test effectively, usually when it heavily uses some external framework (e.g. UI code) and to test it you have to mock the entire framework. Other form of tests (e.g. e2e tests) are then better than unit tests. – Sulthan Jan 2 at 22:51
  • 6
    Writing tests first feels more like first moving people in, and then building the house around them... – memo Jan 3 at 17:19
  • 4
    @memo: More like writing intelligent blueprints that can automatically inspect the house for correctness as construction progresses. – Robert Harvey Jan 3 at 21:17
97

First of all, we have to fundamentally distinguish between Computer Science and Software Engineering. (And maybe to a lesser extent between Software Engineering and Programming or "Coding".)

As one of my CS professors put it: if you need a keyboard, you are not doing CS.

TDD is very much a Software Engineering practice. It doesn't really have much relation to Computer Science. So, if you are asking why CS students don't learn TDD, it's because TDD doesn't really have much to do with CS.

Now, Software Engineering, that is a whole different kettle of fish. I very much agree that TDD should be taught there. I had one Software Engineering class (90 minutes per week for one semester) and in this class we learned Waterfall, V-Model, RUP, CMM, CASE, RAD, Spiral, Iterative, and probably some other stuff that I forgot. Surely, there would have been space to squeeze in TDD, especially if you integrate it with the rest of the courses and apply TDD to all homework and classwork in all courses where programming is required.

To be fair, though, in my case, the Agile Manifesto hadn't been written yet, and TDD was an obscure niche technique.

If you take a look at my personal favorite for teaching programming, How To Design Programs, you will find that they teach very early on that functions should come with usage examples and only slightly later, the book introduces automated unit testing and suggests that those usage examples should be written in the form of tests. They also teach that usage examples and what they call "purpose statements" (you can think of it as the summary line of a JavaDoc comment) should be written before the code.

They don't explicitly teach TDD (they have their own methodology), but this is at least somewhat close. HtDP is designed for high-school students to be taught in less than one year, so I think it is more than feasible to teach it in a single semester in university.

To be honest, though, it would already be a great win of students were simply taught that they can actually run the code they have written with different inputs. (In other words, a crude form of manual testing.) I am amazed at how often students are not able to take this mental step.

For example, I have seen multiple questions on Stack Overflow that essentially amount to "which method do I need to implement to make this code work". Note that this was not about the code of the method (the asker knew how to implement the method), it was purely about the name. However, not one of the askers came up with the idea of simply running the code and looking at the name of the missing method in the NoMethodError exception that will invariably be raised.

| improve this answer | |
  • 45
    I agree with First of all, we have to fundamentally distinguish between Computer Science and Software Engineering. but (at least in the US) many schools still don't differentiate between the two (or among other disciplines in computing) and graduates of a Computer Science program often end up working in industry with little knowledge of or experience in the practices that one needs to be effective working in teams on complex software systems. – Thomas Owens Dec 31 '19 at 23:53
  • 18
    @tgm1024--Monicawasmistreated A CS curriculum typically includes classes on computability theory. While it might be useful for software engineering, it's definitely distinct from SE. Just look at CS(theory).SE vs se.SE. – Daniel M. Jan 1 at 22:53
  • 17
    Sorry, @tgm, but Daniel is correct. There is absolutely a distinction between the two things, no matter how many times you claim there isn't. – Lightness Races in Orbit Jan 2 at 2:16
  • 13
    I think what's harmful is not recognizing and accounting for the distinction between CS and SE, and pretending it doesn't exist. The most harm is to those students who are told that getting a CS degree will prepare them to be a software engineer. It most certainly does not. – Eric King Jan 2 at 15:16
  • 15
    @tgm1024--Monicawasmistreated Saying CS and SE are fundamentially identical is like saying so for physics and mechanical engineering. One deals in theory and science, the other deals in practice and application. Studying one helps with the other, but they are by no means identical disciplines. – Abion47 Jan 2 at 21:19
53

TDD is a nice process in "real-world" programming because problems often arrive on our desks underspecified. "Add a feature that does X", "fix the bug where it shows the wrong thing if you do Y", etc. TDD forces us to create a spec before we begin programming.

In CS/programming classes, the problems usually come to us extremely well-specified. "Write a function that takes an array of integers and returns an array with the same integers arranged in non-decreasing order". The spec is handed to us.

I'm curious why you think TDD makes a tricky algorithm problem with a well-defined spec easier. Specifically, I want you to explain this:

I would be mostly clueless if I were asked to compute all the permutations of the elements in a list, still without using TDD. With TDD, on the other hand, I solved the permutations problem in a matter of minutes when writing the answer to the student.

I suspect you'll say because you wrote test cases that asserted P([]) -> [[]], P([1]) -> [[1]], P([1, 2]) -> [[1, 2], [2, 1]], etc, and that caused you to see how the recursion should work. But that's not TDD, that's just... thinking about the problem. You can think about the problem without the process of writing actual unit tests that fail.

| improve this answer | |
  • 6
    What you described in your last paragraph is TDD. And it being identical to "just thinking about the problem" is the whole point of TDD - it forces you to think through the problem before you actually write it out. It also enables you to employ YAGNI by only writing code you actually need rather than a bunch of other stuff to make it unnecessarily fancy or "scalable". Plus, the non-TDD approach would be to write print or equality statements to verify the output, so the TDD approach, in exchange for only slightly increased verbosity, automatically includes these checks in an organized way. – Abion47 Jan 2 at 21:25
  • 14
    TDD is just the specific process of writing unit tests before writing code. TDD doesn't have a monopoly on thinking. – MattPutnam Jan 3 at 14:33
  • 1
    @MattPutnam, precisely, with one "however" from me. TDD as I've seen touted IRL is an absurd overkill. For instance, back when Java incremental test-harnesses were roughly limited to public static void main()s sprinkled here and there, I was an advocate of having many such methods. But right now, its turned into a belief system that every last thing needs to have a unit test first which is simply forgetting that they've likely now exceeded the expense of fixing bugs later. Holy moley, everyone should see the extent some folks go through. – tgm1024--Monica was mistreated Jan 3 at 14:39
  • 2
    @MattPutnam That is the "what" behind TDD while ignoring the "why". TDD isn't just about writing tests just for the sake of writing tests. It's the mindset that writing tests before code makes that code safer and more secure, but above all else, it makes it more intentful. The act of writing unit tests forces the developer to write code that has purpose rather than just throwing code around to see what finally sticks, and as such it drastically reduces the need to develop iteratively. Ending up with code that has full test coverage isn't the end goal, it's just the icing on the cake. – Abion47 Jan 3 at 18:35
  • 1
    In most all cases, the app is the test. Contrived tests do the exact same thing as the application. It is ridiculous to suggest otherwise. If I have a price calculator that takes in 8 parameters to calculate a price, and the application generates correct prices, test passed. You're not going to generate a test that covers a scenario outside the bounds of the application or else you are not developing to the application spec at all. You're just coding for the sake of coding. The first mars rover crashed and burned because someone used English units instead of metric. TDD is not a panacea. – Newclique Jan 9 at 17:27
33

I suspect it's mostly because writing automated tests is more difficult than writing the code being tested. When you're still struggling with the basic mechanics, it's difficult to add another layer. Also, as other answers have pointed out, TDD is perceived as a software engineering process, even though its practitioners think of it more as a didactic device. Also, knowing what tests to write first is a skill in itself.

If I were teaching an introductory course, I would provide some tests in TDD order, and instruct the students to get the tests passing one at a time in that order. That's a very similar experience to pair programming with an experienced mentor. Personally, I think it would make classes like algorithms easier to teach, which would offset the time spent on the technique, because of the step-by-step progression. It would get students thinking about questions like, "I have some proven working code that will find all the permutations of a two-element list. How do I reuse as much of that code as possible to make it work for 3+ elements?"

| improve this answer | |
  • 12
    As a "didactic device," unit tests have a pretty high barrier to entry. It's hard to get that right, and it presumes a lot of prior knowledge. I like your idea of providing pre-written unit tests to students. – Robert Harvey Dec 31 '19 at 23:04
  • 4
    But the students need to know that just testing each specific requirement is insufficient. Users are going to do things the spec writer did not anticipate. Someone once claimed my bug report must be a mistake because no one had ever reported it in the already long length of the project. My retort: “then I guess no one has ever made a right turn with the type of vehicle.” Just one of the many times I found a fault by a random deviation from a test script. I’ve also cleaned up after a coder who implemented a requirement to do X by writing code to make sure it wouldn’t do anything else! – WGroleau Jan 1 at 17:50
  • @WGroleau then you introduce the didactic device of the EnterpriseDeveloperFromHell, who provides programs that pass all the tests but are as useless as possible. – Caleth Jan 2 at 12:14
  • 1
    How exactly is writing automated tests harder than writing the code being tested? IMO it's not harder, it's a different, complementary skill. Writing automated tests forces you to take decisions about what results you actually want, which to me is the harder part. – Laurent LA RIZZA Jan 2 at 12:27
  • "Automated tests" is ambiguous. Running tests can be automated, or creating tests can be automated. More than once I have reviewed tools that were advertised as automatically creating test drivers. Every one of them analyzed source code and created test cases that could only fail if a compiler bug existed. – WGroleau Jan 2 at 16:45
9

The basic issue is that inventing the tests in the TDD process has nothing to do with computer science or software engineering. It requires knowledge of the application domain.

Of course that is the strength of TDD in real world applications: you can't escape from thinking about what the system you are building is actually going to be used for. But in typical CS and SE activities, the tasks the student is given have minimal context, and exist only to test or illustrate some learning objective.

For example, if the learning objective is to demonstrate understanding of the "case" construct in a programming language, TDD is irrelevant: you can write software that passes all the TDD tests without using "case" at all. If the example was artificial, not using "case" might in fact be a better solution to the problem in the real world, but though not in terms of the learning objective that was intended.

By all means encourage students to invent their own test cases and run them, but trying to use a formal TDD process is beside the point.

A second issue might be more to do with the way CE and SE are taught, but it is still a practical one: how to you grade the student's collection of tests when using TDD? There is no obvious way to automate that process.

Even if you produce a model solution which says "you should have tested for the following 15 or 20 situations" you still have to manually identify which of the student's tests corresponds (wholly or partially) to each part of your model answer. And except in trivial cases, the student might have produced a good set of tests that are logically structured in a different way from the model answer.

| improve this answer | |
  • Maybe I misunderstand your second point. It seems to me that evaluating the rigor of a set of tests is quite simple to automate - by running them against each of a canned set of implementations each with a specific known bug, and determining if there are test failures (as would be expected). Just like one might run a canned set of tests against a student's implementation to evaluate the accuracy of that implementation. – starchild Jan 2 at 8:56
  • 4
    @starchild for same reason that this is hard to do in reality. Your still only testing some corner cases. So what if the student has found 7 valid corner cases whereas you have found 10 but only 5 of the corner cases between you and the student match. So your automated grading now gives student 5/10 whereas it should be giving 7/10. Or 7/12 and all others scores should be lowered? – joojaa Jan 2 at 10:13
  • The basic issue is that inventing the tests in the TDD process has nothing to do with computer science or software engineering. It requires knowledge of the application domain. I believe you're confusing Functional Requirements and Design Requirements. TDD is about the implementation of a particular design. When TDD proponents (I'm not one) refer to functionality, they're talking at the implementation phase, not application level functionality. "Does this sort routine work?---Make a test for it first, have it fail, and then implement it.", (etc.) – tgm1024--Monica was mistreated Jan 3 at 16:59
8

The average student really has a bad overview of what they are expected to know and do. To teach TDD they would need to understand:

  1. How to troubleshoot technical problems. In the beginning they think code is written in one go. Its only later when they realize basic debugging strategies that they move over to more incremental writing. Even if you tell them this up front they will still do it this way because they think that's the way things are done (this is by the way also why people don't know how to draw, play piano, do math, etc etc.)

  2. How to introspect your own actions so that you can find repeatable patterns in your own behavior. Once you can do this you can automate your own work. But this is utterly alien to most people until they reach a certain level.

  3. How to write requirements.

  4. Understanding corner cases of your problem space.

  5. Understanding engineering philosophy on management on risks. (ha but you don't even know you're an engineer right)

  6. Understanding refactoring and maintenance of code.

  7. Understanding how programming works in a real company.

In fact I have had colleagues who were introduced to TDD way too soon, they didn't really benefit all that much. However, it is still possible to take baby steps towards this effect. Its just that people need quite some experience behind them to benefit.

So you can teach it but not something you start with. Especially since there is code that does not lend itself to TDD very easily.

| improve this answer | |
  • That is a great list of things to teach students who want to work as software develpers. It would be nice of courses had a real life software module that had experiences from former students. – Paul McCarthy Jan 14 at 22:39
6

TDD is not more popular in universities, because (generally speaking) universities do a very poor job in ensuring that the knowledge that is being given to the students can be transpiled to real-world contexts.

Arguably, that is not the role of university in the first place, and, it's more about teaching the fundamentals and core concepts of CS. That is instructive, important, and, essential, if the students would be interested in pursuing a career in academia, but, it falls short when the students will finish their degrees and enroll in the industry, working integrated in a team, that follows development best practices, CI/CD, etc.

Obviously, there are intersections and both can benefit from each other, but, ultimately, university is simply not yet up to speed with what needs to be done to graduate more capable, relevant, and up-to-date software engineers.

| improve this answer | |
  • Great answer, Bruno. E Feliz Ano Novo, homem. – Trunk Jan 3 at 15:25
  • Which is why code camps could be a better preparation for a career in programming. And many can self-teach and land a job based on actual apps they have developed and published. Degrees aren't always the answer. – AlanKley Jan 8 at 17:46
4

I never considered TDD helpful as a way to actually solve a problem and after reading your question I still don't.

TDD merely forces you to look at a problem from one end: the client's end. This can help prevent you to come up with a solution that does not match the client's question. This is generally a good thing, although it could also limit you, it keeps you from looking at the problem from a wider angle, there may be a better, more general solution to the problem your client did not consider.

Another way to look at it is that it is the ultimate top down approach. TDD could stand for Top Down Development. You start development at the most course level and gradually zoom in.

Either way, it is an abstract concept. You can tell a student about it, provide a definition, but that will be it. You cannot ask that many questions about it in an exam. So in any SE or CS class, although useful in a SE context, it could never be more than a side note.

| improve this answer | |
4

To answer the question directly:

As someone suggested, there's enough learning for a whole semester. A separate course? Maybe. I could see TDD being combined with software design topics. IMO TDD's greatest value is locking down existing behaviors so we can add new behaviors later, and change the design to be appropriate for those new behaviors. So a software design class?

It would be nice to see the thinking process behind TDD (poorly named, and I once asked Kent Beck if he'd reconsider; he said "that train has left the station") be introduced as early as possible. I've been slinging code since 1976, and I know that TDD felt very unnatural at first. But now it feels quite natural. My time isn't recouped by writing test code, it's recouped later, when I only have to fix a serious defect once per year (The UofM Transplant Center's OTIS2 software, written entirely test-driven, had its last "software emergency" in 2004).

I encourage fresh graduates to try it out for a month. But it would be so much easier on them if they were exposed to TDD much earlier. I find using the unit-test framework useful when I'm learning a new language (True story: "Now how do I get this test to pass in Ruby? MAYBE this will work...HEY IT WORKED!"), or exploring a new library ("The docs are vague about the order these will be returned...I'll write a test...").

I find TDD valuable when implementing an algorithm (NOT inventing one...I'll get to that), or building any mildly complicated business rule from the ground up, as it were. Each test that is passing is "locked in" for all eternity. If the test is discrete enough, it won't ever have to change (but that takes a lot longer than a month to get good at).

So it would be interesting to incorporate TDD into earlier programming classes. Three points of improvement:

  1. We could focus less on debugging and break-fix, which most developers think is just the nature of their job. It ain't. It doesn't have to be that painful or time-consuming.
  2. Unit test frameworks are a simple replacement for #ifdef DEBUG / printf("...") / #endif - so we've been doing something similar (but more risky) since the beginning of time.
  3. When we hear "you MUST write the test first" we're being mislead. I want to write down my expectations for the code I'm about to write. I want to leave that running while I go focus on the next behavior, or on cleaning up the design.

Another thing that might be useful is a textbook or, at the very least, a condensed no-nonsense curriculum with labs designed to convey various aspects of the TDD practice. I have a suggestion for that (see experience/disclaimer later).

To answer the critics:

I'm new here, but I note that some "answers" don't answer the question, but are fair critiques of TDD. So I'm guessing it's okay to address some of those concerns?

  1. It's true, studies that have been done comparing TDD to unit-test-after (UTA?) show that TDD was initially slower. However, the UTA group had much lower test-coverage, and more defects. I think it was this study: Longitudinal Study of the Use of a Test-Driven Development Practice in Industry

If developers working on a product for more than a few months were really spending the majority of their time writing new features, then this would be a negative. But the six TDD teams that I worked on as a developer did spend their time that way, whereas the decade I spent slinging code prior to TDD were spent mostly in the debugger, or cautiously copying/pasting/modifying so I wouldn't break older/someone else's code. The benefits of TDD aren't all instantaneous, but they've been remarkably reliable and amazingly valuable as measured by lower developer overtime (and stress) and increased revenues. Bottom line: I always looked forward to going to work with those TDD teams. (Once I'd had some tea.)

  1. It's also true that TDD cannot invent a new algorithm. I think it was Ron Jeffries who tried to create a Sudoku solver using TDD, and tried not to let his own Sudoku skills interfere with the implementation. He failed. I'm not surprised. TDD helps break down, simplify, and implement business rules; helps confirm we got an algorithm right (e.g., a complex encryption algorithm that may not be reducible); helps us reshape code later so that we can safely add new features; and does this by pinning down the investment we've already made in the behaviors of the product.

When you write a "test" (aka specification, aka scenario) you have to know the answer! Also, you want to write a test that is a small enough bite out of the solution that you already have some notion (prior knowledge) of how you are going to implement it.

What is often a surprise is how clear and simple the design can become if we pay attention to code-smells and we refactor as needed whenever all the tests related to that code are passing.

No one is suggesting we throw out our good knowledge of design patterns, or language idioms, or SOLID principles, or how to play Sudoku, or how to write a heap-sort.

TDD is about creating a safety-net that gives us confidence to rapidly add new features and alter the design to fit those new features without breaking anything else.

UTA is patching the safety net after someone falls through a hole to their death. (Interesting side note: The OTIS2 project is a life-critical system, so the risk of patient death wasn't a joke to us. Over 20,000 tests all running in 10 minutes, now that is a safetynet!)

Experience/disclaimers:

I did TDD pretty much full time from 1998 to 2004 and with ~6 different teams. All were at least 6 months of development. Before that, I had spent 13 years writing software the other way, and came to hate debuggers and manual testing with printf().

I started teaching software development practices in 2001, now including TDD, BDD, and SA-CSD courses. Sure, it's a good living, but I also know that TDD is a straightforward way to make team-centric software development sane and enjoyable again. So...

I'm writing what I hope will become a college or high-school textbook (with online repos, labs, videos, etc, of course): Essential Test-Driven Development

| improve this answer | |
4

I was taught computing science in the 1970s, long before the acronym TDD was invented, and of course we were never explicitly taught the technique. But for one of the programming exercises we were given, we were supplied with a set of test cases and were told that our solution had to pass the tests. So I learnt the value of the approach without ever being taught it explicitly, and have used it ever since.

| improve this answer | |
3

While writing tests is an essential skill for software development, the scientific evidence does not indicate that TDD ends up producing any better software than iterative test-last (ITL) development (OTOH, it also isn't worse).

For evidence, you can see Davide Fucci et al. "An External Replication on the Effects of Test-driven Development Using a Multi-site Blind Analysis Approach" (link) and Turhan et al's meta-analysis in Making Software (link).

So, unless we start seeing evidence to the contrary that TDD does, in fact, provide measurable gains it's less important that TDD as a specific practice is taught as opposed to simply instilling the good habit of test writing at some point in the development process.

| improve this answer | |
  • 1
    Well, some would say that in theory there is no difference between theory and practice... . It's a whole different think to look at the theoretical performance of a few (unexperienced) students on projects that will never be maintained, and to look at practical results in real CI/CD projects with teams twice as large as the number of students involved in that experiment. Being understood that the main difference between both approach will be observed in the maintenance and evolution and not in the first release. – Christophe Jan 9 at 22:59
  • 1
    There are numerous studies in Turhan's meta-analysis that work with postgraduate and professional workers, and the evidence for the effectiveness of TDD is still inconclusive. If TDD practitioners wish to make their case, they will need to up their level of evidence to show a real effect. The plural of anecdote is not data. – Anzel Jan 10 at 15:03
2

but I would still spend some time writing a list-ordering algorithm without TDD, if asked, and I would be mostly clueless if I were asked to compute all the permutations of the elements in a list, still without using TDD. With TDD, on the other hand, I solved the permutations problem in a matter of minutes

This is confusing me seriously.
To me test driven development means to think about tests early and take the time to implement tests and keep them up to date with code.
Of course thinking about what to test is a good chance to elaborate the problem to solve and in some cases get a good picture of what the pitfalls are.
But not doing TDD, not concentrating on tests first doesn't forbid to think about the underlying problem anyway. You always should think about what you do and make the situation clear, no matter if you want to go one more step and implement tests right now or not.

would it be problematic to explain the TDD approach before asking students to write list sorting algorithms and similar stuff?

It depends to which extent you would explain TDD.
It's helpful to hear about something. But students hear a lot of stuff they can't yet interprete because other basics are still missing. So you shouldn't go too deep into a concept that without knowledge of basics would be a useless and alienating mess of unknown terms to them.
Besides this probably many programmers know what it feels like if you are looking for a how-to on some problem and find examples that indeed show a solution but first of all you have to sort out all kind of other irrelevant stuff the author added.

So to answer this question, if I was a student I'd like to have things separated. If you announce to explain us how to sort a list then please do tell us how to do that but don't leave the path to fill in testing stuff. If you want to start explaining tests then don't announce to implement sorting because this still doesn't happen for a long time.
Again there is some thinking required before we start to write the sorting or tests. What do lists look before and after sorting? Make examples, think about pitfalls, what must be considered to avoid failing when a list is empty and so on. All this must be considered but no line of tests is written yet.

Generally I'd say you are mixing up two things that should be kept seperately.

  • Thinking about the nature of the problem you want to solve.
  • Thinking about input to your code and what output should be given.

Thinking about the problem is very different from focusing on writing test cases.

How to do it bad

Once I came across an example of TDD by someone who was sort of obsessed with TDD.
Unfortunately it appeared to me the author was too fond of his tutorial so he didn't realize what crap it actually was.

The author only concentrated on test cases, not on the problem their code should handle. As you never can find all input permutations, you must have an overview of what you are actually doing, you must see the entirety of the problem. You can't always start with empty input, then subsequently add some more input and always append code to handle new input correctly.
You must have an idea what you're actually dealing with in general. But the author didn't.

If I try to translate it to sorting a list, we'd start with an empty list, then one element that can be returned as it is, a list with two elements that perhaps need to be swapped and perhaps end up in a recursion because three elements is like two (we already solved that) plus one more with one extra step...
A horrible szenario for a sorting algorithm- but noone will realize it because we only concentrate on out test cases.

Conclusion

The comments make me write a little more about my opinion.

I think the term "test driven development" is wrong. It should be "test-supported development", meaning we don't just code and hope but we think about testing too and we know it's always good to know early when something goes wrong.

If development is named driven by tests this could imply everything depends only on tests and we are done as soon as a couple of test cases are satisfied. This requiremend would be met even by very insufficient code that never tried to see a problem as a whole but got hacked back and forth until all test cases acidentally turn to green - then fails in real operation.

| improve this answer | |
  • Yeah, I was always doubtful about the idea of just adding enough code to satisfy the test. This is fine for boundary value tests. But useless if you're testing a continuous range. Can't just put in a statement dealing with a single mid-range value and then another and another, etc. Seemed to me that TDD is only effective where a separate test team write the tests and the sequence in which they are achieved by the code. – Trunk Jan 3 at 15:31
  • 1
    @Trunk tests can help the developer too, the one who writes code can think more detailed about his code and perhaps finds pitfalls that should pass a test. Furthermore tests can help avoid that code changes open another pitfall. What I want to emphasize is, don't rely on tests only. Tests are not the one and only salvation. Don't stop thinking about code, absolutely don't produce bulls..t code that is "good enough" if only it passes some test cases and ignores the whole rest. To me tests should be a supplement to coding, but not an entire coding method that allows leaving everything else out. – puck Jan 3 at 16:15
  • 1
    @Trunk the distinction really is that you do just add enough code to satisfy the test, but you have to do so in good faith. You write the test to represent the test case and then you implement the code to satisfy the test case, not some contrived attempt to make the test green but undermine the process. What gives you the confidence is then the consistency between your intuitive understanding of the test case and the automated verification of the test - not the test alone, which can always be fooled. – Ant P Jan 7 at 8:43
  • 1
    The mistake that the "purist" crowd makes is trying to treat TDD as a rigorously formal process, which it isn't - it's a technique, which has to be applied with intuition. – Ant P Jan 7 at 8:44
  • 1
    @Trunk I think the original idea of TDD has been somewhat skewed and misrepresented over the years into something of a dark art. TDD doesn't seek to replace manual testers, the two things serve very different purposes. Ultimately TDD exists to help me write code that I'm confident in, which is a much narrower scope than what a QA person is interested in. – Ant P Jan 7 at 13:26
1

You've updated your question to be more "Why isn't TDD not taught as a core learning tool?". Other answers already explain well enough why TDD isn't a good topic for coding 101, but the main answer is really just that TDD, at its core, is not a problem solving tool. It can be used for that purpose, but like any tool you have to first understand when and how to use it.

TDD is a testing process, and thus most naturally will be taught either as part of a Development Processes course, or as part of a Software Testing course. In coding 101 courses, the goal isn't for students to solve problems, it's to teach them how to use various programming concepts. Generally, most coding 101 and 102 projects will be very explicit about how to solve the problem, the students just need to understand what they have learned to do the task in a non copy-paste kinda way.

Every student learns in different ways. Some students need to read about it, others need it verbally explained to them, and others will just never get it unless they get neck deep in code. Teaching TDD to aid in the learning process will not actually help most students, and the ones it does help? Teachers will have to decide if the time to teach TDD was worth the extra learning speed. On a net whole, teaching any learning method will not be worth the class time that could be spent on actual course specific topics. (In general, learning and problem solving skills are usually left to students to learn themselves, since only the student can identify what works best for them)

TL:RD; Different people have different effective processes. Universities don't prescribe how you should do anything; Just give you the tools so you can do what works best for you.

| improve this answer | |
  • TDD is a testing process, and thus most naturally will be taught either as part of a Development Processes course, or as part of a Software Testing course. The very start and end of your statement are incorrect. TDD is not a "testing process". It's a developement process that involves testing. The distinction isn't mundane: A testing process is something that has something first and then establishes a process to test it. TDD is about creating that thing in the first place (by employing testing). It would never be taught in a Testing course. – tgm1024--Monica was mistreated Mar 9 at 15:48
  • @tgm1024--Monicawasmistreated a testing process doesn't need something to exist yet. For example you can develop user stories that the final product can pass before development even starts. TDD just changes the order things are done, but the key part is still the Testing. The process part of it would be covered at a high level in a Dev-Proc class, and the low level actual automated test writing for TDD would be covered in a Testing class. (and that's how it was taught at my college) I would also argue to try not to focus on the exact words used too much, because they aren't standardized. YMMV – Tezra Mar 9 at 16:21
  • No, again, that's not a testing process, and I know what the ordering in TDD is. You're still missing the fact that TDD is not a testing process, but merely puts testing as a component of a development process. The steps in TDD are not to test something. They are to create something: The way that they do this is to establish a test first. A testing process would be akin to listing out functional specifications and verifying that each works, and then regression testing to make sure they didn't walk you backward. THAT'S a testing process. You don't even need developers for it at all. – tgm1024--Monica was mistreated Mar 11 at 0:12
  • @tgm1024--Monicawasmistreated You seem to be missing missing the fact that the words you are using are not standardized. The general consensus of what they mean has been changing constantly since software development became a thing. The point of this answer is that TDD is a tool, and like any other tool it has a time and a place but is not automatically good for everyone. Don't worry about the semantics of the words used. Focus on the general point. – Tezra Mar 11 at 12:10
  • It's by definition a semantic concern when you're using the wrong term, and yes the terms are standardized; I have no idea why you're stubbornly holding onto this misinformation. Here, I'll spell it out differently for you: A QA engineer is trained in testing techniques and testing processes. A QA engineer would never venture anywhere near TDD, because they don't deal in development processes. That's for software engineers. – tgm1024--Monica was mistreated Mar 15 at 14:26
1

TDD is a fantastic implementation tool, and I think you are correct about it being beneficial to those who wish to write software.

What is the reason TDD is either not taught at all or at least very late in universities? In other words, would it be problematic to explain the TDD approach before asking students to write list sorting algorithms and similar stuff?

Probably the biggest reason is that the professors teaching these programs rarely know how to develop software, since that isn't their field of expertise. As other answers have mentioned, computer science and software engineering are different disciplines, and I would compare the expectation that computer science students learn software engineering to physics students learning how to design cars. TDD is a skill that takes a decent amount of practice to really be able to teach effectively, and computer science professors spend the bulk of their career working on computer science, so expecting the computer science faculty to really be able to teach TDD in a way that won't merely confuse their students is pretty unrealistic in my opinion.

We need to treat computer science and professional software development as the distinctly separate fields that they are. If your goal is to learn computer science, you should not be burdened with paying thousands of dollars learning how to make websites in React incorrectly from someone who has spent the last 30 years of their career doing graph theory on a chalk board. Likewise, if your goal is to become a software engineer, I don't know why you would want to spend 4 years and tens of thousands of dollars learning what is essentially just a specific field of mathematics. It's good to have a foundational understanding in the field, just like how someone designing an exhaust manifold needs to understand some amount of physics, however the person designing the exhaust manifold doesn't need too in-depth of an understanding in quantum mechanics, special relativity, and electromagnetism to do their job.

If you want to be an accountant, you can go get a degree in accounting and your professors will likely all have been CPAs at one point or another. If you want to be a mechanical engineer, you can get a degree in mechanical engineering and your professors will likely all have been licensed engineers at one point or another. But if you want to be a software engineer, any degree in software engineering will actually just be a computer science degree with some of your electives already chosen for you, and almost none of your professors will ever have been professional software developers. The accounting degree isn't part of the math department, and the mechanical engineering degree isn't part of the physics department, but the software engineering degree will almost always be part of the computer science department. Until academia as a whole completely separates these two fields into different departments run by different staff, I think there will always be a long list of things like TDD that are not taught to students aspiring to be software engineers.

| improve this answer | |
  • It may be true that professors "rarely" know how to develop software, but you can't pin any observed pushback on TDD on that. I've been a professional programmer for 40 years. I'm also not a fan of it, for the reasons I elaborate on in comments. It's yet another trend (similar to extreme programming) that showed up supposedly as a "tada!" moment, that ultimately shot itself in the foot by forgetting the bottom line: Are you spending more time in component-to-component developement now than you would otherwise be spending in unit testing prior to release? – tgm1024--Monica was mistreated Mar 9 at 15:53
  • IOW, if you're good at TDD, you're already good at the fundamentals of unit testing. If you weren't, any of your pre-tests you came up with during component level development would be similarly weak. Not necessarily specific to you, but if you've only been in the game a couple decades or so, it's particularly difficult to see how trends come and go, and more so: Why they do. – tgm1024--Monica was mistreated Mar 9 at 15:55

Not the answer you're looking for? Browse other questions tagged or ask your own question.