36

Consider this:

public function polynominal($a, $b, $c, $d)
{
    return  $a * pow($x, 3) + $b * pow($x, 2) + $c * $x + $d;
}

Suppose you write various tests for the above function and prove to yourself and others that "it works".

Why not then remove those tests, and live happily ever after? My points is that some functions do not need to be tested continuously after they have been proven to work. I am looking for counter-points that state, yes these functions still need to be tested, because: ... Or that yes, these do not need to be tested...

  • 32
    Isn't it true for every function that unless you change your code, if it worked yesterday, it will work tomorrow, too? The point is that software is changed. – 5gon12eder Aug 17 '15 at 22:18
  • 3
    Because no one will ever write override_function('pow', '$a,$b', 'return $a * $b;'); in their right mind... or try to rewrite it to handle complex numbers. – user40980 Aug 17 '15 at 22:19
  • 8
    So... um... that "proven to be bug free code"... it has a bug. – user69037 Aug 18 '15 at 5:42
  • For small functions like these you may want to consider property based testing. Property based testing automatically generates testcases, and tests for pre-determined invariants. They provide documentation through the invariants, which make them useful to keep around. For simple functions like this, they are perfect. – Martijn Aug 18 '15 at 16:20
  • 6
    Also that function is a great candidate for a change to Horner's form (($a*$x + $b)*$x + $c)*$x + $d which is easy to get wrong, but is often considerably faster. Just because you think it wont change doesn't mean it wont. – Michael Anderson Aug 19 '15 at 3:55
78

Regression testing

It's all about regression testing.

Imagine the next developer looking at your method and noticing that you are using magical numbers. He was told that magical numbers are evil, so he creates two constants, one for the number two, the other one for the number three—there is nothing wrong in doing this change; it's not like he was modifying your already correct implementation.

Being distracted, he inverts two constants.

He commits the code, and everything seems to work fine, because there are no regression testing running after each commit.

One day (could be weeks later), something breaks elsewhere. And by elsewhere, I mean in the completely opposite location of the code base, which seems to have nothing to do with polynominal function. Hours of painful debugging lead to the culprit. During this time, the application continues to fail in production, causing a lot of issues to your customers.

Keeping the original tests you wrote could prevent such pain. The distracted developer would commit the code, and nearly immediately see that he broke something; such code won't even reach the production. Unit tests will additionally be very precise about the location of the error. Solving it wouldn't be difficult.

A side effect...

Actually, most refactoring is heavily based on regression testing. Make a small change. Test. If it passes, everything is fine.

The side effect is that if you don't have tests, then practically any refactoring becomes a huge risk of breaking the code. Given that is many cases, it's already difficult to explain to the management that refactoring should be done, it would be even harder to do so after your previous refactoring attempts introduce multiple bugs.

By having a complete suite of tests, you are encouraging refactoring, and so better, cleaner code. Risk-free, it becomes very tempting to refactor more, on regular basis.

Changes in requirements

Another essential aspect is that requirements change. You may be asked to handle complex numbers, and suddenly, you need to search your version control log to find the previous tests, restore them, and start adding new tests.

Why all this hassle? Why removing tests in order to add them later? You could have kept them in the first place.

  • Pedantic note: nothing is risk free. ;) – jpmc26 Aug 18 '15 at 1:33
  • 2
    Regression is the number one reason for tests for me - even if your code is clear about responsibilities and only uses whatever is passed to it (e.g. no globals), it's all too easy to break something by accident. Tests aren't perfect, but they're still great (and almost always prevent the same bug from appearing again, something most customers are quite unhappy about). If your tests work, there's no maintenance cost. If they stop working, you likely found a bug. I'm working on a legacy application with thousands of globals - without tests, I wouldn't dare make many of the necessary changes. – Luaan Aug 18 '15 at 7:15
  • Another pedantic note: Some "magic" numbers are actually ok, and replacing them with constants is a bad idea. – Deduplicator Aug 18 '15 at 18:06
  • 3
    @Deduplicator we know that, but a lot of especially freshly university trained junior programmers are extremely zealous about blindly following mantras. And "magic numbers are evil" is one of those. Another is "anything used more than once must be refactored into a method" leading in extreme cases to a plethora of methods containing but a single statement (and then they wonder why performance suddenly dropped, they never learned about callstacks). – jwenting Aug 18 '15 at 18:11
  • @jwenting: Just added that note because MainMa wrote "there is nothing wrong in doing this change". – Deduplicator Aug 18 '15 at 18:13
45

Because nothing is so simple that there can't be bugs.

Your code, while on the face of it looks to be bug free. It is in fact a simple programmatic representation of a polynomial function.

Except it has a bug...

public function polynominal($a, $b, $c, $d)
{
    return  $a * pow($x, 3) + $b * pow($x, 2) + $c * $x + $d;
}

$x is not defined as an input to your code, and depending on the language or the runtime, or the scoping, your function may not work, may cause invalid results, or may crash a spaceship.


Addendum:

While you may consider your code bug free for now, how long that remains the case is hard to say. While it might be argued that writing a test for such a trivial piece of code isn't worth it, having already written the test the work is done and deleting it is deleting a tangible safe guard.

Of additional note is code coverage services (like coveralls.io) that give a good indication of the coverage a test suite provides. By covering every line of code you give a decent metric of the quantity (if not the quality) of testing you perform. In combination with a lot of small tests, these services at least tell you where not to look for a bug when it happens.

Ultimately, if you already have a test written, keep it. Because the space or time saving from deleting it will likely be much less than the technical debt later on if a bug does arise.

  • There is an alternative solution: switching to non-dynamic non-weak languages. Unit tests are usually a workaround for insufficiently strong compile-time verification and some languages are kind of good at this en.wikipedia.org/wiki/Agda_(programming_language) – Den Sep 3 '15 at 9:21
21

Yes. If we could say with 100% confidence, with certainty: this function will never be edited and will never run in a context which could cause it to fail - if we could say that, we could drop the tests and save a few milliseconds on every CI build.

But we can't. Or, we can't with many functions. And it's simpler to have a rule of running all the tests all the time than to put effort in determining exactly what confidence threshold we're satisfied with, and exactly how much confidence we have in the immutability and infallibility of any given function.

And processing time is cheap. Those milliseconds saved, even multiplied many times, don't add up to nearly enough to justify taking the time with every function to ask: do we have sufficient confidence that we need never test it again?

  • I think that's a very good point you're bringing up here. I can already see two guys spending hours arguing about keeping tests for some small functionality. – Kapol Aug 18 '15 at 18:21
  • @Kapol: not an argument, a meeting to determine which can go and which can stay, and what criteria should be used to decide, as well as where to document the criteria, and who signs off on the final decision... – jmoreno Aug 25 '15 at 1:36
12

Everything said in the other answers is correct, but I will add one more.

Documentation

Unit tests, if well written, can explain to a developer exactly what a function does, what its input/output expectations are, and more importantly, what behavior can be expected of it.

It can make spotting a bug easier and lower confusion.

Not everybody remembers polynomials, geometry, or even algebra :) But a good unit test checked in to version control will remember for me.

For an example of how useful it can be as documentation, look at the Jasmine introduction: http://jasmine.github.io/edge/introduction.html Give it a few seconds to load, then scroll to the bottom. You'll see the entire Jasmine API documented as unit test output.

[Update based on feedback from @Warbo] Tests are guaranteed to be up-to-date as well since if they are not, they will fail, which generally will cause a build failure if CI is used. External documentation changes independently of the code, and therefore isn't necessarily up to date.

  • 1
    There's an big advantage to using tests as (one part of) documentation that you left implicit: they're automatically checked. Other forms of documentation, eg. explanatory comments or snippets of code on a Web page, can get out of date as the code evolves. Unit tests will trigger a failure if they're no longer accurate. That jasmine page is an extreme example of this. – Warbo Aug 18 '15 at 12:50
  • I'd like to add that some languages, like D and Rust, integrate their documentation generation with their unit testing, so you can have the same piece of code both compile into a unit test and get inserted to the HTML documentation – Idan Arye Aug 18 '15 at 16:16
2

Reality Check

I have been in challenging environments where testing is "a waste of time" during budgeting and schedule, and then "a fundamental part of quality assurance" once the customer is dealing with bugs, so my opinion is more fluid than others might be.

You have a budget. Your job is to get the best product you can on that budget, for whatever definition of "best" you can scrape together (it's not an easy word to define). End of story.

Testing is a tool in your inventory. You should use it, because it's a good tool, with a long history of saving millions, or perhaps even billions of dollars. If given a chance, you should add tests to these simple functions. It may save your skin some day.

But in the real world, with budget and schedule constraints, it may not happen. Don't make yourself a slave to procedure. Test functions are nice, but at some point, your man-hours may be better spent writing developer documentation in words, rather than code, so the next developer doesn't need tests as much. Or it might be better spent refactoring the code base so you don't have to maintain as difficult of a beast. Or perhaps that time is better spent talking to your boss about the budget and schedule so that he or she better understands what they're bidding for when the next round of financing comes down the pipe.

Software development is a balance. Always count the opportunity cost of anything you are doing to ensure there wasn't a better way to spend your time.

  • 4
    In this case, there already was a test, so the question is not whether to write them, but whether to commit them and run them with each build or release or whatever schedule there is for running all the tests. – Paŭlo Ebermann Aug 18 '15 at 17:53
0

Yes, keep the tests, keep them running and keep them passing.

Unit tests are there to protect you (and others) from yourself (and themselves).

Why is keeping the tests a good idea;

  • Validate the previous requirements' functionality in the face of new requirements and additional functionality
  • Verify that refactoring exercises are correct
  • Internal documentation - this is how the code is expected to be used
  • Regression testing, things change
    • Do the changes break the old code?
    • Do the changes require further change requests or updates to current functions or code?
  • Given that the tests are already written, keep them; it is time and money already spent that will reduce maintenance costs further down the line
  • 2
    This doesn't appear to offer anything substantial over the other answers that have been provided. – user53019 Aug 18 '15 at 15:23
  • 1
    There was a problem posting it earlier, but in retrospect, it is a nice summary. I'll remove it. – Niall Aug 18 '15 at 17:41
-1

Developer Documentation

  • How do I (as another developer) know that this has been tested?
  • If I want to fix a bug in the self contained function, how do I know that I am not introducing a bug that you had already considered?
  • Complexity indicator: # of tests can be a good measure of how complex something is. This may indicate that you shouldn't touch it because it is mature and stable or that it is bloated and coupled.

User Documentation

  • Pending poor doc, I can look and see if {zero, negative values, empty sets, etc} are accepted and what the expected return value is.
  • It also gives a good example of how I should use the object/function
  • 1
    this doesn't seem to offer anything substantial over points made and explained in prior answers. Even "nice summary" has been posted already (to my taste it's not so nice but oh well) – gnat Aug 18 '15 at 21:26

protected by gnat Aug 18 '15 at 22:10

Thank you for your interest in this question. Because it has attracted low-quality or spam answers that had to be removed, posting an answer now requires 10 reputation on this site (the association bonus does not count).

Would you like to answer one of these unanswered questions instead?

Not the answer you're looking for? Browse other questions tagged or ask your own question.