2

I'm in a funny situation. We have a development model that works for an in-house software suite which is focused on responsiveness to the customer. To give you a sense of the environment, when we were using SCRUM, we would often intentionally commit to only 50% of our maximum workload, with a full expectation that the remaining 50% would be filled with quick response work.

The development community at my workplace as a whole is moving more towards TDD, so I have an interest in figuring out how it might be adapted to fit my workflow. That will make it easier to poach other employees, I mean, facilitate the smooth re balancing of resources.

However, I find myself regularly faced with situations that do not have testable requirements until the majority of the work is completed. Because of the rapid response nature of the job, finding the solution that can be done is more important than holding to an arbitrary API picked at the start of the task. It's of no use to my customer if I can build their dream API in a month, when they need to have a working product in a week. The API only really gets finalized after we understand how the code is actually going to do the task.

In situations where testable requirements are hard to come by, has anyone had success leveraging TDD in a meaningful way, even if it's not the "traditional" way?

  • 7
    TDD isn't about testable requirements. BDD is. – Robert Harvey Apr 27 '18 at 21:05
  • 4
    "I find myself regularly faced with situations that do not have testable requirements" - because the requirements are so unclear, because they are not testable, or because they are not automatically testable? Please clarify - a typical example would probably help a lot to understand what you have in mind. – Doc Brown Apr 27 '18 at 22:21
  • 1
    That sounds like a clear requirement which is perfectly suited for automated testing and TDD. Can you explain why you think it is not? – Doc Brown Apr 28 '18 at 5:37
  • 1
    @DocBrown I wont know what "correctly handling the data" means until I can analyze what the code can be made to do within their timeframe. They won't be able to tell me it's "correct" enough until they can look at some of the results. Thus, the code is already written at that point. – Cort Ammon - Reinstate Monica Apr 28 '18 at 14:40
  • 2
    Scrum is about how a team works, tdd is about how an individual works. Saying your switching from one to the other doesn't make much sense, since you can do both at the same time. – Bryan Oakley Apr 29 '18 at 0:49
5

If you think you can't test without requirements then you don't understand the purpose of tests.

I'm in a funny situation.

No you aren't. This is typical of many shops that use TDD.

To give you a sense of the environment, when we were using SCRUM, we would often intentionally commit to only 50% of our maximum workload, with a full expectation that the remaining 50% would be filled with quick response work.

I've been in shops where that was generous. It's a different world when you have operational systems with up-time requirements. Scrum and TDD still work fine if you do them right.

However, I find myself regularly faced with situations that do not have testable requirements until the majority of the work is completed. Because of the rapid response nature of the job, finding the solution that can be done is more important than holding to an arbitrary API picked at the start of the task. It's of no use to my customer if I can build their dream API in a month, when they need to have a working product in a week. The API only really gets finalized after we understand how the code is actually going to do the task.

Who convinced you keeping a death grip on an API regardless of your needs was traditional?

In situations where testable requirements are hard to come by, has anyone had success leveraging TDD in a meaningful way, even if it's not the "traditional" way?

I have had success with TDD when the only requirement I had was, "I wonder what this thing does".

Tests are not requirements. You might have a requirement. You might write a test that proves that requirement is met. You might even mention that requirement number in the comments of the test (or, God help us, in its name). Fine. But if that's all you're doing with tests you don't know what you're doing. Tests are for more than that. Much more.

I write tons of tests. More than I ever even check in. Why? Because tests help me read code. I write up test after test and watch my understanding of what I can do grow and grow. I change code and I change tests, rapidly. Because I'm learning. The tests lock down what I comprehend now. They remind me what I was doing when I come back from the bathroom.

So yeah, sometimes I write tests without ever looking at or thinking about a requirements document.

I dump a bunch of tests as my understanding grows. I find better ways to express what I'm trying to say. Just like I come back to this answer and rewrite it and move things around I do the same to my code and my tests trying to find the best way to express an idea. Not because the CPU needs me to. Not because requirements demand it. Not because TDD told me to. Hell I was doing this before TDD was a thing. I do it for the humans. I'm looking for the best way to present an idea so that the clueless newbie that comes after me can easily find the one line they need to change because the world changed after I left. I do that by using code and tests to tell the story of the idea well.

The purpose of tests is to help people read code.

They show what it does, what it needs, how much code you have to read to understand what is happening, but they don't tell you what you need. Your needs will change when the tests haven't been touched. It's up to you to decide if the code does what you need. Not the tests.

When you do figure out some requirements you can write other tests for them. You can show them off to your product owner. You can even put them in a special place so you can tell them apart from the ones you just write to help yourself read the code. That's BDD.

So don't complain that you can't write tests because your requirements are up in the air. That describes over half of everything I've done using TDD. Any code you write can be put under test. That test is "I think the code will do this, let's see if it does". That tells future coders (including me) what I was thinking. But deciding if that is needed isn't the tests job. That's yours. If you change your mind it's time to make changes. Change the test and then change the code. That's TDD.

  • The purpose of tests is to help people read code -- You wouldn't know that, judging from some of the tests I've seen. Often, the code is more readable (and shorter) than the test. – Robert Harvey Apr 30 '18 at 14:58
  • @RobertHarvey shops that write tests because "were supposed to write tests" often produce such abominations. If you're not doing it because it helps you're just adding noise. – candied_orange Apr 30 '18 at 15:04
  • 1
    I see the problem a lot with code that requires extensive mocking. Unfortunately, it's not always possible to write code solely from pure functions. Increasingly, I'm finding that the more interesting that code is from a testing perspective (i.e. it could benefit from unit tests), the less likely it is to be easily testable. – Robert Harvey Apr 30 '18 at 15:08
  • @RobertHarvey Interesting and easy seldom go together. Mocking is a tad easier if you just roll up your sleeves and write your own anonymous classes and lambdas rather then assume framework magic will solve everything. Design counting on magic and you quickly create nightmares. Always assume you might end up doing it all yourself. – candied_orange Apr 30 '18 at 16:22
  • An assumption that I've learned the hard way to trust. Moq seems kinda cool (it being a lightweight toolkit of the sort you're describing) but I never taught myself to use it. – Robert Harvey Apr 30 '18 at 16:25
0

It's hard to determine exactly what you mean, but I've worked in a place who insisted that they couldn't deliver testable requirements, I'll assume you mean similar.

In this environment, my problem was more "I don't really know exactly what they're asking me to build", which was why it was very hard to write tests for it. Requirements were vague, and everyone was just sortof muddling towards an ever-changing target each day.

They also called this "responsive", but I now know it was just a manifestation of amateurism and confusion.

In practically any object-orientated programming team, you should know what you're building in advance. You should always be starting with a requirement against which you can at least write a unit test, even if you can't write the implementation.

If someone asks you to do something, you can write a test. It's that simple. If you cannot imagine a way to write the unit test you're not really being given a requirement. If all you've known is one of those "muddling through" employers (which isn't uncommon) this might be hard to grasp, but requirements in a well-built, object-orientated system should almost always equate to a particular behaviour exhibit by a particular object given a particular set of inputs (the S of the SOLID principle), and it is against that object that you can write your test.

If doing this seems completely impossible, that's a process smell--a huge one.

0

Can you change the culture? It sounds like you will need to in order to iteratively develop the UAT and refine the requirements while developing the code.

It depends on the organization and how the organization determines the definition of done. Point out that if they want to increase their software quality, there needs to be changes in how developers and customers interact.

In the most functioning companies that I have worked for, refining the requirements was up to the the senior technical staff and the business analysts.

  • 1
    There have been several attempts to change the culture, from high up the management chain. The results have always been that nobody can figure out how to accomplish what we need with a more standard approach. And of course, any rational business will choose a model which works over one which doesn't. – Cort Ammon - Reinstate Monica Apr 30 '18 at 16:56
0

There are decent pieces of information in these answers but I'm not seeing much really actionable information. I'm not a testing expert but the way the way I understand TDD in a nutshell is that you follow this process:

  • Write a test
  • Make sure it fails for the right reason
  • Write code to make the test pass

'Testing' is a fairly broad topic but I think you can apply the above easily to unit tests and also acceptance tests.

In the case of unit tests, requirements don't really matter IMO because unit tests don't exist to prove the code meets one or more requirements. Unit tests are written to prove your code works the way you meant it to work. That's it. If I have an add() method, I should test that it adds correctly. It might be the case that I really needed a multiply method. That's not something you worry about in unit testing. Therefore, a lack or requirements should not be a barrier to unit testing. It's part of the coding process.

As far as acceptance testing goes, one approach that is pretty 'test-driven' is writing the requirements as executable tests. The delivery of requirements is then "make these tests work." This is very cool if you can get the process going but the time to write these tests is significant and can become an development effort in it's own right. I'm not saying it's not worth it. If you can get it working, the benefits are tremendous. It might be tough though if you are playing 'find a rock':

[you] here's a rock

[them] not that rock.

You will be spending a lot of time writing and rewriting the test scripts. You can still do it, but it could be costly. Essentially, you make a guess at what you are supposed to deliver and put that into the tests. You write the code to make that test work and when the 'deciders' tell you they wan't something else, you modify the tests, rinse and repeat.

If you aren't doing TDD unit testing, it's a lot easier to get things going that way but I have a sense you are talking about something more like acceptance testing here.

  • In my experience, rapid prototypes can be developed much easier and faster if you're not saddled with the additional burden of writing unit tests. – Robert Harvey Apr 30 '18 at 15:47
  • @RobertHarvey I'm inclined to agree but the question is about how to incorporate TDD, not about whether it's a good idea. I have to force myself to write unit tests but when I have done it, I've been surprised at how quickly I found subtle errors in my code. I'm not really a zealot for TDD but I can see value in it. – JimmyJames Apr 30 '18 at 15:56
-1

There are always requirements, otherwise you would have no reason to write code.

Some times those requirements are given to you, sometimes as a programmer you have to make up the requirements yourself using your understanding of the aims of the organisation you work for. In practice it's generally a mixture of the two.

Wherever the requirements come from, if they are specific enough to enable you to write some code, then you should also be able to use them to write tests if you want to.

In some ways tests are particularly important if you have made up some of the requirements and are likely to need to change them, since the tests may then be the only documentation of those requirements, and are a place to show any changes to requirements that might happen after you get feedback on a version of the code.

  • I'm looking to deal with situations where there's a lack of testable requirements and how I might integrate TDD. – Cort Ammon - Reinstate Monica Apr 30 '18 at 14:54
  • What distinction are you making between a requirement and a testable requirement? Can you give an example? – bdsl Apr 30 '18 at 15:10
  • I can edit them into the body of the question, but the comments on the question discuss an example. A testable requirement might be "The software should properly model the WSG84 Earth geodedic system." A nontestable requirement might be "we need to operate on data that doesn't seem to be in any standard system. Make the software handle this well enough that we can convince our external customer that we understand the data, even if it isn't a standard." – Cort Ammon - Reinstate Monica Apr 30 '18 at 15:15
  • 1
    If you can't decide what you are to do with the data I don't get how you can write the production code. Don't you have the same problem as you do with writing the test? – bdsl Apr 30 '18 at 16:04
  • 1
    If you do TDD as Uncle Bob describes you switch between writing tests and production code every few minutes, if not more often. You wouldn't stop to write an estimate before writing production code. – bdsl Apr 30 '18 at 18:46
0

If you write tests because you have requirements it seems wrong o me. During my application developing process I also write a lot of tests as they help me to test my code in a small and defined environment. Without this approach I would have to run the entire application which is very time consuming.

2

If you need to do some experiments and prototyping to find out the correct requirements and a working implementation, a "test first" approach to automated testing may not be working well.

However, writing unit tests and other kind of automated tests afterwards will still work. Sure, this is not "TDD by the book", but the design impact on your code will typically stay almost the same as if you had written the tests beforehand.

3

I find myself regularly faced with situations that do not have testable requirements until the majority of the work is completed.

I would argue that it is impossible to finish any portion of a piece of work if you do not have testable requirements for it, because there is a literally zero percent chance that it will be what the customer wants. That said, the requirements may not be unit testable, but that is a good thing - in any high level piece of work you want the requirements to be high level so that you have flexibility during implementation. If the requirements were unit testable you would be right back to a waterfall workflow, where everything is specced up front and you end up with something hard to develop and terrible to use (because up front design does not account for any surprises or learning more about the problem as you work on it).

Now, once you have your high level requirement it's your job to figure out what the next step is. If any part of the work is risky you'll want to do that first. Figure out a high level test (acceptance or even journey level) that, when passing, proves to you that the risk is resolved (for example, your API returns an encrypted version of the input in less than 5 ms). To implement that testable goal you'll discover integration and then unit level testable pieces of work, write tests for them, and continue in standard TDD fashion. Once all of the unit and integration level tests pass your higher level test should also pass, and you have a requirement done and well tested.

  • 1
    Highly-iterative methodologies were created for customers who don't know what they want. – Robert Harvey Apr 27 '18 at 22:22

Not the answer you're looking for? Browse other questions tagged or ask your own question.