How should the code for a program outputting to command line be tested/designed?

Question

Imagine a program similar to this in Python:

import subprocess

class Example():

    _cmd_args = (['ls', '-a', '/usr/bin/'], ['ls', '-al', '/usr/local/bin/'])
    _default_args = 0

    def init(self):
        pass

    def run_ls_command(self):
        p = subprocess.Popen(self.get_command(), stdout=subprocess.PIPE)
        out, err = p.communicate()
        return err

    def get_command(self):
        return self._cmd_args[self._default_args]

    def set_args(self, args):
        self._default_args = args

    def do_magical_stuff(self):
        #do some stuff
        self._default_args = 0
        self.run_ls_command()

This is a simple example - but imagine a program where the primary "interactions" will be through executing commands through subprocess against the host operating system.

The specific commands and arguments will be constructed as the result of a variety of other configurations and the state of the program at the given time.

The above code is horribly not testable.

It seems like it might be best to design it in such a way to have objects like:

class MyArgs():

    def __init__(self, _cmd, _cmd_args, _cmd_path):
        self.cmd = _cmd
        self.cmd_args = _cmd_args
        self.cmd_path = _cmd_path

    #more manipulation methods

    def get_command(self):
        return ([self.cmd, self.cmd_args, self.cmd_path])

and

class Example():

    args = MyArgs()

    def __init__(self):
        pass

    def run_ls_command(self):
        print self.get_command()
        p = subprocess.Popen(self.get_command(), stdout=subprocess.PIPE)
        out, err = p.communicate()
        return err

    def set_args(self, c, a, p):
        #this would be more complicated logic in future and likely not just one method
        self.args.set_args(c,a,p)

    def get_command(self):
        return self.args.get_command()

I would then be able to test that MyArgs.get_command() works correctly, directly on the class itself, for its given inputs (especially since I doubt I would be constructing a single "set all variables" actual command).

That would allow me to modify my tests against Example to primarily test what Example.get_command() returns whatever other manipulations would be happening to Example.

This is still not great to me, since I'm testing what should otherwise be private members - other than for testing, get_command should be private.

I am primarily intending to test that the right commands are triggered - not whether they execute successfully. So for example, if a user passes in ['ls', '-a', '/a/dir/that/does/not/exist'] I do not care at this point that the ls will fail - I want to validate the resulting command (actually executing the commands will take longer than I want to do for the unit test suite). Knowing in this case that the Popen attempted ls -a /a/dir/that/does/not/exist is sufficient.

Is there a better way to design this architecture to facilitate testability?

Answer 1

In the end, what you want to verify is that subprocess.Popen gets called with the right arguments. To do that, you need a testing seam at the boundary between Example and the call to subprocess.Popen, so that you can replace Popen in your tests with a mock.

I am not familiar enough with python to know if you can directly mock Popen, but if you can't then it is easy enough to write a small wrapper class that you inject into Example. The production version of this wrapper just blindly passes everything through to Popen, while in your tests you can use a mock version that allows you to verify that the right arguments are passed.

The wrapper class should be simple/stupid enough that a simple code review can prove that there are no bugs in the wrapper itself, so that you don't need to write tests for that.