Term for unit testing that separates test logic from test result data [closed]

Question

So I'm not doing any unit testing. But I've had an idea to make it more appropriate for my field of use. Yet it's not clear if something like this exists, and if, how it would possibly be called.

---

Ordinary unit tests combine the test logic and the expected outcome. In essence the testing framework only checks for booleans (did this match, did the expected result result). To generalize, the test code itself references the audited functions, and also explicites the result values like so:

unit::assert(  test_me() == 17  )

What I'm looking for is a separation of concerns. The test itself should only contain the tested logic. The outcome and result data should be handled by the unit testing or assertion framework. As example:

unit::probe(   test_me()   )

Here the probe actually doubles as collector in the first run, and afterwards as verification method. The expected 17 is not mentioned in the test code, but stored or managed elsewhere.

How is this scheme called? Or how would you call it? I hope I can find some actual implementations with the proper terminology.

---

Obviously such a pattern is unfit for TDD. It's strictly for regression testing. Also obviously, it cannot be used for all cases. Only the simpler test subjects can be analyzed that way, for anything else the ordinary unit test setup and assertion steps are required. And yes, this could be manually accomplished by crafting a ResultWhateverObject, but that would still require hardwiring that to the test logic.
Also keep in mind that I'm inquiring for use with scripting languages, and not about Java. I'm aware that the xUnit pattern originates there, and why it's hence as elaborate as it is.

Btw, I've discovered one test execution framework which allows for shortening simple test notations to:

test_me();   // 17

While thus the result data is no longer coded in (it's a comment), that's still not a complete separation and of course would work only for scalar results.

Answer 1

There is such a thing as data-driven tests. These aren't the same thing you are asking for, but they may help with some of what you want.

Basically, the idea is that we define data structures and perform tests based on them.

An example:

translations = {
    1 : 'I',
    2 : 'II',
    4 : 'IV',
    125 : 'CXXV'
}

def test_roman():
    for number, roman in translations.items():
        assert to_roman(number) == roman
        assert from_roman(roman) == number

This makes it much easier to add additional test cases. In frameworks with support for it it can easily be made so that this would actually get recorded as many tests.

I'm not sure what exactly you are attempted to get from your technique. You have to deal with all the setup of unit tests but skipping over what would seem to be a relatively minor part: specifying the output. It seems such a small savings and would only rarely be useful.

Answer 2

Hey.
If you really want to separate logic, in the end you want to build test automation framework for your app, and then write unit tests that will be using this new api, instead of just writing unit tests. Most of programmers may say that are not unit tests anymore... and maybe they don't. But that is not important. You can still do it nicely. Below are links, that should get you going. Although they talk about creating functional tests that operate on whole application (gui level), I think the same approach you can apply to achieve your goal. I short it presents how to write 3-layer tests, so you will have separation of logic from implementation.

• Quick overview
• Bigger explanation
• Inspiration for all above and few pdf describing it