-2

I am writing a class that has one function that I want to test.

The function, textToPdf, takes a list of text files and creates and saves a PDF at the file path argument.

In order to test that this function works, it seems like I would need a list of "example" text files and an "example" pdf which was made from those files.

It also seems like, in order to test this function, the PDF file would need to be saved to the user's disk.

Answers like this one always recommend "mocking the file system", but I am not sure what that means.

The last line in the code essentially calls fs.writefile(path) where path is a parameter. The file writing code does not have a "use interface" option or any file-like object that can be passed in, all it takes is a filepath (on disk).

This sounds like an integration test but, since textToPdf is just one function, I think it classifies more as a Unit test (where the unit of code under test is that single function).

Is there a way I can share this code with my teammates, and have them run tests without the need for test input text files, a corresponding test output pdf file and the potential that the finished PDF is saved on their local disk when they run the unit tests?

OR, if I do need to provide example input and output files, how can I write a test in a way that other developers can run the code and verify that the output is correct - that the function does what it is supposed to (i.e. combine text files and save a PDF on disk)?

5
  • I think you need to think a little more about what your goals are. "Mocking the file system" sounds like a fine idea, until you actually have to do it. And "without the need for test files" sounds like an arbitrary condition without purpose. Is your question really the one you ask in the last paragraph? There are a number of things you can do to provide a spot check: see if the target file is the size it is supposed to be, etc. You can also check the output file against a test candidate output file and see if they contain the same content. Aug 20 '21 at 3:55
  • 3
    You could rewrite the class so it doesn’t work with paths, but with some sort of file objects like memory streams or byte arrays. That should make it a lot easier to test.
    – Rik D
    Aug 20 '21 at 5:17
  • 1
    Does your function actually create the whole PDF file (which could be a pretty complex thing) or does it just call an external process or library? In the former case, it's either much too big for a single function or calls a lot of smaller functions that should be unit tested, and in the latter case we're not talking about unit testing anymore but about integration testing. Aug 20 '21 at 6:38
  • Does this answer your question? How can I write unit test case of file format converter utility?
    – gnat
    Aug 20 '21 at 6:47
  • It might interest. Golden Master Testing
    – Laiv
    Aug 20 '21 at 7:49
4

Sometimes code needs to change to be easier to test.

Many standard libraries have abstractions over files that can be used instead. For example, Java has an InputStream and OutputStream with both file and byte array implementations. If you have to make your own, it is a lot simpler, because you don't have to make it flexible enough for every single standard library user, you can make it application specific.

So a common approach is to make a textToPdf(inputs: List[InputStream], output: OutputStream) and do most of your unit testing on that, then make your original textToPdf do nothing but create the InputStreams and OutputStreams from the files, then call the other function.

That still takes an integration test to verify, but the scope of what you have to verify is much smaller. You're just verifying you're grabbing the correct files, and a few additional file-specific error conditions, instead of validating every complex bit of logic in an integration test.

1
  • "do most of your unit testing on that" - so how would you test that? Ultimately it does seem like the aforementioned "file converter test" but the answers to that question was not great. Even if file streams are the inputs and outputs, how to I verify that the combination / conversion step is correct - that the output bytes are what they should be?
    – Startec
    Aug 20 '21 at 22:47
3

This sounds like an integration test but, since textToPdf is just one function, I think it classifies more as a Unit test (where the unit of code under test is that single function).

The mark of a unit test is not "just one function". The mark of a unit test that there is only one component under test (i.e. real, not mocked, and the focus of the test), but it's perfectly possible for a single public behavior to call several functions.

This is an integration test because you're using more than one real component: your class and the file system.

It also seems like, in order to test this function, the PDF file would need to be saved to the user's disk.
Answers like this one always recommend "mocking the file system", but I am not sure what that means.

We can write a lot of layers in our logic. There is no practical upper limit on that. However, no matter how many layers we write, there's always one layer on the outside, which has to interact with the real world (e.g. a monitor, network, OS, file system, IO device, ...)
It is the nature of the beast that the last layer can never truly be unit tested, because it inherently relies on an external resource. All of the inside layers can be tested, because their neighboring layer can be mocked.

You want to unit test this class. This implies that there is business logic in the class which you want to run your test on. But the class also contains direct dependency on an external resource, in this case the file system, making it impossible to unit test the class.

Let's use this example:

public class TextToPdf
{
    public JobResult DoJob()
    {
        // Read from disk
        var input = File.ReadAllLines(@"C:\input.txt");

        // Complicated business logic that you want to test
        var result = ...;

        // Write to disk
        File.Write(@"C:\output.pdf", result);

        return new SuccessJobResult(result);
    }
}

The solution is to separate the business logic from the external resource handling logic. In essence, you're separating this layer into two layers: the business logic and the file system interaction.

public interface IFileReader
{
    IEnumerable<string> Read(string path);
}

public class FileReader : IFileReader
{
    public IEnumerable<string> Read(string path)
    {
        return File.ReadAllLines(path);
    }
}

public interface IFileWriter
{
    void Write(string path, byte[] data);
}

public class FileWriter : IFileWriter
{
    public void Write(string path, byte[] data)
    {
        File.Write(path, data);
    }
}

This is now the outer layer. Now we can change our original class to depend on this outer layer:

public class TextToPdf
{
    public TextToPdf(IFileReader reader, IFileWriter writer)
    {
        this.reader = reader;
        this.writer = writer;
    }

    private readonly IFileReader reader;
    private readonly IFileWriter writer;

    public JobResult DoJob()
    {
        // Read from disk
        var input = this.reader.Read(@"C:\input.txt");

        // Complicated business logic that you want to test
        var result = ...;

        // Write to disk
        this.writer.Write(@"C:\output.pdf", result);

        return new SuccessJobResult(result);
    }
}

Now, the file system handling is the outer layer, and the business logic is another one of those inside layers, which we can now test by mocking its neighbor (the file system handling).

Notice that this class takes in a IFileWriter and IFileReader interface. We could pass in the real classes that actually use the file system, but we could also design classes that implement the same interface but don't actually use the file system.

In a unit test, we're going to use fake classes so that we don't actually use the file system.

public class FakeFileReader : IFileReader
{
    public IEnumerable<string> Read(string path)
    {
        return new List<string>()
        {
            "This is just some text",
            "which doesn't come from a file",
            "but the caller doesn't know that!"
        };
    }
}

public class FakeFileWriter : IFileWriter
{
    public void Write(string path, byte[] data)
    {
        // Let's just not do anything
    }
}

Now you can unit test your TextToPdf class without touching the actual file system:

var mockedReader = new FakeFileReader();
var mockedWriter = new FakeFileWriter();

var unit = new TextToPdf(mockedReader, mockedWriter);

var result = unit.DoJob();

// Assert that result matches the expected output

And that's it.

This is a very simplified example. In general, your mocked classes should be set up in a way that the test can define the return values (e.g. the text that would allegedly be read from a file), so that the test can decide what fake values should be used.


You might wonder how you could now write unit tests for FileReader and FileWriter. The short answer is you can't. You can only integration test them.

If these classes still contain some business logic that you want to unit test, then you have to repeat the process. Split the business logic from the interaction with the real external resource. This kind of abstraction ensures that you can mock the connection between the business logic and the external resource.

5
  • 1
    Here is a lot of standard testing wisdom in this answer, how to mock the file system. However, to my experience, this does seldom pay off - it often simpler (and very often fast enough) to use real input and output files and write the tests around it. The real challenge here is to create a robust validation logic that some PDF output matches the expected output. This challenge stays the same, regardless of using real files or intercepting the output stream with a FakeFileWriter.
    – Doc Brown
    Aug 20 '21 at 8:43
  • ... this SO post might be of help here: Reliable way to (programmatically) compare PDFs
    – Doc Brown
    Aug 20 '21 at 8:47
  • @DocBrown: I don't disagree, in the sense that file system access isn't generally speaking such a failure-prone component, but it still would be an integration test, not a unit test, and this clashes with OP's primary question of "without the need for test input text files, a corresponding test output pdf file and the potential that the finished PDF is saved on their local disk when they run the unit tests"
    – Flater
    Aug 20 '21 at 12:41
  • @Flater what is SuccessJobResult ? If this function writes a large binary object, returning the result (in addition to writing it) may not be a good idea - right?
    – Startec
    Aug 20 '21 at 19:35
  • @Startec: I just invented a basic example of a method since you posted no reference code snippet. There is no specification implementation details here for the sake of example. What you return, and if you return anything, is up to you to implement as you need it.
    – Flater
    Aug 21 '21 at 2:51

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.