1

tl;dr: Libraries and classes I use as client are well tested. How I can use that knowledge to reduce amount of testing?

Elaboration: Let's assume for theoretical purposes that we're implementing function which does something fairly simple, say, creating an array of sin(x) of consecutive integers. (That's Ruby, but choice of the language was arbitrary)

def arr_of_sins(num)
  Array.new(num){|i| Math.sin(i) }
end

We have here the use of 2 well-tested methods, of which we're perfectly confident: Array.new and Math.sin.

Question: how would you test our #arr_of_sins to minimize retesting of dependencies, but to be still confident enough to use it somewhere else?

Bonus question: which tests of those you came out with answering previous question you(as developers) would leave as automated ones?

  • 1
    why would you do unit tests at all? your compiler, CPU and OS are thoroughly tested, and you're just using them... – AK_ May 15 '15 at 13:40
  • @AK_, if you're serious: because I would like to make sure I haven't mess something up when composing a new thing with those well-tested. But when I do it, a lot of effort is wasted on restesting. I understand, that this overhead is inevitable. Hence there's the question, how to keep it as small as possible. If you're being sarcastic: what's wrong with what I've asked? – Nikolay Rys May 15 '15 at 13:51
3

I would make a single unit test that provides some input and checks that the output is correct. Maybe 1-3 others for bad (negative, null, zero) input. And I would leave them all automated since this sort of this is tailor made for unit testing.

I don't particularly care (in this case) that I'm retesting sin and Array.new. For one, I shouldn't know/care about the unit of work's implementation details. For another, everything builds on something else; to exclude all dependencies sets yourself up for insanity.

3

I would test this as a black box. That means, I would only test the "interface" of a function, like I've never seen it's implementation, which I consider a subject to change. For a given example, I may check:

  • Resulting array length
  • ... bounds (I assume they are 0..num-1)
  • Behavior in case of argument is zero (empty array as a result)
  • ... is negative
  • ... is null
  • If contents are actually results of Math.sin, and not some other random function
  • Types of argument/result, if it is significant part of API or otherwise "exposed"

Some of those tests definitely may overlap with test from the standard library (e.g. bounds checking), but if someone would change the implementation later, then tests would show the difference, preventing regression, which is a desired property of a test suite.

  • Good. But it bothers me that this approach is insanely expensive. I will use my arr_of_sins on higher levels of architecture. For example, I may print it to console. Should I use black-box appoach here too? Repeat all tests with the smallest differences? – Nikolay Rys May 15 '15 at 14:06
  • In case if you need to test some printing function, it should be as abstract as "print some arbitrary array of numbers", so you wouldn't need to check anything related to array_of_sins function whatsoever. So, that printing function should be print_array and not print_array_of_sins. That is not directly related to testing, but rather it is about separation of concerns and code reuse. – scriptin May 15 '15 at 14:18
  • 1
    Yep, black box testing for the win! There is this religious insistence on unit testing, which is quite prevalent, and quite misguided. Unit testing is just one of the many different kinds of testing. It is useful some times, but that does not mean that it is the only way to test. – Mike Nakis May 15 '15 at 15:17
  • Agree with Mike. Testing should be layered. Unit testing is one of those layers. However, quality comes from the LAYERS, not from perfection at any layer. So let's say you don't unit test this function for some reason (not enough time, whatever), the next layer (some type of functional testing) should catch it. Let's say for some reason you can't do that functional testing, then lastly some layer of integration testing (pre-production) should catch it. As long as you have somewhat healthy layers with ongoing care and feeding, THAT is where your quality comes from, not perfect unit testing. – Calphool May 15 '15 at 18:03
1

for simple scenarios

In your example of arr_of_sins, it has two dependencies: Math.sin and Array.new, however, both of these are used statically and thus to be confident in your tests you must treat it as you wrote the Math.sin and Array.new methods yourself in the class your testing.

This sounds completely ridiculous, but this is the problem of static dependencies that you cannot determine how your logic uses them.

For your particular case, sin can be safely and easily used in any context, so for your test you could call sin itself to see if the result of your class matches up with the expected outcome, however, you will soon find that your unit test will look exactly like your production code, which is pointless.

I would follow @Telastyn's answer and do the cases to get a good approximation of the range of outputs you can expect.

for complex scenarios

The problem arises when you get into more complex scenarios where you're testing code which relies on other systems / sub-systems. This is where patterns like dependency injection and mockism can help with making your unit tests robust.

For example, if you're testing some code which relies on getting data from another system, say you need to build a report in CSV format from a query on a database.

If a CSV formatter and a repository are dependencies of your report generator, then you can mock both those dependencies out and all you end up testing is that your class uses the dependencies in a way you expect and not that they go to the database to execute query xyz.

to summarize

You write unit tests to ensure that the content of the methods your testing are correct, if you assume that sin behaves as expected that's fine, but you still need to ensure that the unit you're testing is in fact calling sin, and in the example above, the only way to do that is to verify the output.

This is fine for smaller units where the static dependencies have no dependencies of their own, but for more complex cases you need to have a different strategy to ensure the unit your testing is correct.

  • Hopefully the static parts are building blocks of your code, not some huge and complicated processors (think: Math.sin not DataProvider.GetDataFromTenWebServicesAndThreeDatabases ;) ). While you wouldn't create mock of Math.sin, you really, really want a mock of DataProvider.Get.... – Gerino May 15 '15 at 14:17
  • 1
    Most definitely, the point I'm trying to make is there is no one-size-fits-all for unit testing. – Matthew May 15 '15 at 14:28
  • I've heard this argument about static dependencies before, but how is a static method any different than an idempotent method on an object? Surely there's a way to test a static method and, once that's covered by tests, you're good, right? To put it another way, how does one write unit tests in a functional language, where you don't have objects at all? – Robert Harvey May 15 '15 at 16:47
  • @RobertHarvey I don't understand the question as methods being static and methods being idempotent are orthogonal. In my answer I used examples where the complex scenario has side effects where the simple scenario is pure and idempotent. Quite often it's the side effects you want to avoid when testing a unit, instead you would test that the dependency that would causes a side effect is invoked with the correct parameters, and that the caller handles the mocked result correctly. – Matthew May 15 '15 at 19:01
1

Ask yourself what the purpose of this function is. Write your tests based on that.

That you are using a well tested library is an implementation detail.

Should you decide to reduce the number of tests because that implementation detail allows you to do so, then you have to make sure that there is a test that would warn you about the implementation changing in such a way that your decision to reduce the number of test is no longer appropriate and should be re-evaluated.

In other words: I am with @Telastyn on this.

If you really want to reduce the number of tests given that you are calling a library you trust, pick a couple of essential tests to keep as part of your own automated test set. In addition to that I would add one or more tests to show that the proper library is used.

1

You say the sin function is well tested an you trust it. You're probably right that it is well tested. So sin (0) is 0, sin (90) = 1, sin (180) = 0, sin (270) = -1 and so on, right? You see, even if the function does 100% correctly what it is supposed to do, it doesn't mean it does what you think it should do.

But in your unit test, do you care whether the array new and the sine function work? You don't, not in your unit test. What you care about is that the function that you wrote returns the right values, and that's what you should test. What do you gain by having a unit test for the sine function? If the unit tests for your function work, the sine function unit test doesn't gain you anything.

If the unit test for your function breaks, you have to decide whether your code was wrong, or whether the sine or the array code is bad. So you won't have a colleague telling you "guess what, the sine function is broken" and then you say "well, that explains why my unit test fails". But that wouldn't happen anyway, because the sine function isn't broken. So just good old fashioned debugging.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.