Unit Testing For Different Environments

Question

How does one do unit testing for multiple environments that behave differently?

Here is my problem. I have a PHP class for which I'd like to create unit tests. The class provides shortcuts for manipulating folders. On Linux, you can set the folder permissions without a problem, on Windows, you can't at all.

Knowing this, what is the best practice for creating a unit test for this class? The test suite could be run on either Windows or Linux, but if you test the folder settings on Windows, the test will fail. Do you just live with the failing test, make it conditional on the OS, omit testing it altogether, or something else?

Answer 1

In this case, I would make it conditional on the OS and make it clear in the test's name that it's only intended to run on Linux.

I'm not familiar with PHP's unit testing frameworks, but if I were doing this in Python, I'd set up the test to be skipped when running in Windows, since unittest supports conditionals in the "skip this test" markup.

If all you need to do is set up folders with particular permissions and then see how your script reacts, see if you can use another language (PowerShell? Python? I'm not sure.) to set up your environment first. Then you could test your PHP code on pre-created folders.

Failing that, set up interactive test "scripts" on Windows that will have to be run manually. By "scripts" here I mean steps to be followed in each test to verify that your PHP code works as expected. It's not a unit test at this point, but it's definitely better than not having any testing at all.

Answer 2

Create unit tests for both! You need to know that it is working code!

PHP has a great unit testing tool known as PHPUnit, which offers a few ways of doing this. My recommendation would be to group the tests that are OS-specific with that OS name.

/**
 * @group linux
 */
class LinuxTest extends PHPUnit_Framework_TestCase {
    // all tests within this class will be in the group, linux 
}


class MixedTests extends PHPUnit_Framework_TestCase
{
    public function testSomeTestThatAlwaysRuns()
    {
        // this test is not in a group
    }

    /**
     * @group windows
     */
    public function testWindowsOnlyCase()
    {
        // this test is in the group, windows
    }
}

Then, when in linux, run all tests except for tests in "windows" and vice-versa. This will run all tests that should run on both platforms, and will only exclude those for the given group.

phpunit --exclude-group windows
phpunit --exclude-group linux

Also, you can exclude multiple groups:

phpunit --exclude-group windows,mac

Answer 3

Move the algorithm of creating folders to separated method, and in this method create definitions, what to in each OS.
While your script is supposed only to work in Linux (with linux-specific code), you should test this script only in Linux. I doubt that you can "simulate" linux-environment in unuit-test.
If your script is supposed to work in Windows too, then it should be coded inside the script, not in unit-test.