Is there a need to keep tests for simple (self-contained) functions?

Question

Consider this:

public function polynominal($a, $b, $c, $d)
{
    return  $a * pow($x, 3) + $b * pow($x, 2) + $c * $x + $d;
}

Suppose you write various tests for the above function and prove to yourself and others that "it works".

Why not then remove those tests, and live happily ever after? My points is that some functions do not need to be tested continuously after they have been proven to work. I am looking for counter-points that state, yes these functions still need to be tested, because: ... Or that yes, these do not need to be tested...

Answer 1

Regression testing

It's all about regression testing.

Imagine the next developer looking at your method and noticing that you are using magical numbers. He was told that magical numbers are evil, so he creates two constants, one for the number two, the other one for the number three—there is nothing wrong in doing this change; it's not like he was modifying your already correct implementation.

Being distracted, he inverts two constants.

He commits the code, and everything seems to work fine, because there are no regression testing running after each commit.

One day (could be weeks later), something breaks elsewhere. And by elsewhere, I mean in the completely opposite location of the code base, which seems to have nothing to do with polynominal function. Hours of painful debugging lead to the culprit. During this time, the application continues to fail in production, causing a lot of issues to your customers.

Keeping the original tests you wrote could prevent such pain. The distracted developer would commit the code, and nearly immediately see that he broke something; such code won't even reach the production. Unit tests will additionally be very precise about the location of the error. Solving it wouldn't be difficult.

A side effect...

Actually, most refactoring is heavily based on regression testing. Make a small change. Test. If it passes, everything is fine.

The side effect is that if you don't have tests, then practically any refactoring becomes a huge risk of breaking the code. Given that is many cases, it's already difficult to explain to the management that refactoring should be done, it would be even harder to do so after your previous refactoring attempts introduce multiple bugs.

By having a complete suite of tests, you are encouraging refactoring, and so better, cleaner code. Risk-free, it becomes very tempting to refactor more, on regular basis.

Changes in requirements

Another essential aspect is that requirements change. You may be asked to handle complex numbers, and suddenly, you need to search your version control log to find the previous tests, restore them, and start adding new tests.

Why all this hassle? Why removing tests in order to add them later? You could have kept them in the first place.

Answer 2

Because nothing is so simple that there can't be bugs.

Your code, while on the face of it looks to be bug free. It is in fact a simple programmatic representation of a polynomial function.

Except it has a bug...

public function polynominal($a, $b, $c, $d)
{
    return  $a * pow($x, 3) + $b * pow($x, 2) + $c * $x + $d;
}

$x is not defined as an input to your code, and depending on the language or the runtime, or the scoping, your function may not work, may cause invalid results, or may crash a spaceship.

---

Addendum:

While you may consider your code bug free for now, how long that remains the case is hard to say. While it might be argued that writing a test for such a trivial piece of code isn't worth it, having already written the test the work is done and deleting it is deleting a tangible safe guard.

Of additional note is code coverage services (like coveralls.io) that give a good indication of the coverage a test suite provides. By covering every line of code you give a decent metric of the quantity (if not the quality) of testing you perform. In combination with a lot of small tests, these services at least tell you where not to look for a bug when it happens.

Ultimately, if you already have a test written, keep it. Because the space or time saving from deleting it will likely be much less than the technical debt later on if a bug does arise.

Answer 3

Yes. If we could say with 100% confidence, with certainty: this function will never be edited and will never run in a context which could cause it to fail - if we could say that, we could drop the tests and save a few milliseconds on every CI build.

But we can't. Or, we can't with many functions. And it's simpler to have a rule of running all the tests all the time than to put effort in determining exactly what confidence threshold we're satisfied with, and exactly how much confidence we have in the immutability and infallibility of any given function.

And processing time is cheap. Those milliseconds saved, even multiplied many times, don't add up to nearly enough to justify taking the time with every function to ask: do we have sufficient confidence that we need never test it again?

Answer 4

Everything said in the other answers is correct, but I will add one more.

Documentation

Unit tests, if well written, can explain to a developer exactly what a function does, what its input/output expectations are, and more importantly, what behavior can be expected of it.

It can make spotting a bug easier and lower confusion.

Not everybody remembers polynomials, geometry, or even algebra :) But a good unit test checked in to version control will remember for me.

For an example of how useful it can be as documentation, look at the Jasmine introduction: http://jasmine.github.io/edge/introduction.html Give it a few seconds to load, then scroll to the bottom. You'll see the entire Jasmine API documented as unit test output.

[Update based on feedback from @Warbo] Tests are guaranteed to be up-to-date as well since if they are not, they will fail, which generally will cause a build failure if CI is used. External documentation changes independently of the code, and therefore isn't necessarily up to date.

Answer 5

Reality Check

I have been in challenging environments where testing is "a waste of time" during budgeting and schedule, and then "a fundamental part of quality assurance" once the customer is dealing with bugs, so my opinion is more fluid than others might be.

You have a budget. Your job is to get the best product you can on that budget, for whatever definition of "best" you can scrape together (it's not an easy word to define). End of story.

Testing is a tool in your inventory. You should use it, because it's a good tool, with a long history of saving millions, or perhaps even billions of dollars. If given a chance, you should add tests to these simple functions. It may save your skin some day.

But in the real world, with budget and schedule constraints, it may not happen. Don't make yourself a slave to procedure. Test functions are nice, but at some point, your man-hours may be better spent writing developer documentation in words, rather than code, so the next developer doesn't need tests as much. Or it might be better spent refactoring the code base so you don't have to maintain as difficult of a beast. Or perhaps that time is better spent talking to your boss about the budget and schedule so that he or she better understands what they're bidding for when the next round of financing comes down the pipe.

Software development is a balance. Always count the opportunity cost of anything you are doing to ensure there wasn't a better way to spend your time.

Answer 6

Yes, keep the tests, keep them running and keep them passing.

Unit tests are there to protect you (and others) from yourself (and themselves).

Why is keeping the tests a good idea;

• Validate the previous requirements' functionality in the face of new requirements and additional functionality
• Verify that refactoring exercises are correct
• Internal documentation - this is how the code is expected to be used
• Regression testing, things change
  • Do the changes break the old code?
  • Do the changes require further change requests or updates to current functions or code?
• Given that the tests are already written, keep them; it is time and money already spent that will reduce maintenance costs further down the line

Answer 7

Developer Documentation

• How do I (as another developer) know that this has been tested?
• If I want to fix a bug in the self contained function, how do I know that I am not introducing a bug that you had already considered?
• Complexity indicator: # of tests can be a good measure of how complex something is. This may indicate that you shouldn't touch it because it is mature and stable or that it is bloated and coupled.

User Documentation

• Pending poor doc, I can look and see if {zero, negative values, empty sets, etc} are accepted and what the expected return value is.
• It also gives a good example of how I should use the object/function