how and should I 'unit test' an entire package?

Question

I'm still learning to be good about doing unit level testing, as I've always been a little sloppy about only doing functional testing in the past, so I want to double check I'm doing this 'right'.

I have a package which is responsible for auto-magically updating a particular configuration which is expected to change regularly during runtime. Provide a list of objects to monitor to the constructor of my main package class. Then at some point call update it will go to all the multiple places that define configuration, detect change, update the appropriate state for the objects it's monitoring, and provide a report of what was changed. This is all automatic and otherwise transparent to all other packages.

My inclination is to test this as a unit, but it is a pretty 'big' unit, 5-6 classes and fetching from multiple files and a restful interface. It feels like testing any smaller units would take much longer, be less flexible to refactoring and only provide a slightly higher chance of detecting a defect, but that could just be my laziness talking. Would it be considered 'better' to test at a lower level?

Assuming it is 'right' to test the package as a unit what is considered appropriate for mocking something like this? There are methods of classes which are instantiated by my main class (ie, not something I pass in directly so I can't just pass my own mock) that have methods I want to control. I believe I can use powerMock in the two cases I can think of (or could, after I research PowerMock some more), but I'm not sure if doing so is transparent enough. Is configuring PowerMock to detect and return a mock file object any time my package tries to open a configuration file, even if it's buried deep in my package logic, acceptable, or is this considered to be abusing my knowledge of implementation specific details? Would it be 'cleaner' to actually modify configuration files on the file system and let them be read normally without further mocking? I would need to modify the files regularly for testing...

Edit: To clarify the question of coupling asked in one of the questions, I don't think that my classes are overly coupled. Effectively I have three classes that fetch state from A, B, and C, and then a 'main' class which takes the state from the three and decides how to combine it correctly (ie, if A and B don't match use A unless C). I could easily test A B and C separately and then mock them to test my 'main' class. However, it seems like the testing of my 'main' class would effectively test the comparatively simple interfaces for A, B, and C anyways if I didn't mock them. Thus it feels like duplicate work to test them individually. I may get some slightly better code coverage, maybe, with individual testing, and it's always nice to have one test test only one thing. But I don't know if it's worth all the overhead for minor benefits.

Answer 1

Unit testing is the lowest level of testing, but that means only that it is the lowest level you do, not the lowest level you could theoretically do. If you test at the class level, you could test at the method level, using an appropriate test tool to get at private members. But if you test at the method level, you could test at the statement level, using an appropriate test tool to get the statement in isolation so you test it. And if you did decide to test at statement level, that's still a decision not to test at instruction or byte code level. And that process of decomposition only stops when someone decides 'a smaller unit would be stupid'.

So the question 'how should I unit test this' is exactly equivalent to 'how should I first test this'. If the best way of testing a package is as a whole, because it is of a size and complexity where testing it as a unit is the most effective way of meeting your reliability and maintainability goals, then do so.

If it isn't, then don't.

Anyone trying to give more specific advice without either seeing your code base or knowing your project context (nuclear power plant in basement of an orphanage, or email spam generator?) is very likely not merely wrong, but also unwise.

Answer 2

This is a pretty common question to people starting to learn TDD. I don't know if that's your case, but the logic you can follow is the same.

First, forget the word "unit" for a moment. Think about how you would approach the development of the same package from ground zero and test-first. You can't start by testing "a class only" or "a method only" because you have nothing yet, and you don't know all of the classes and methods that will be necessary.

So you start by specifying the first piece of expected behavior in a test, to then make the test pass, all with the minimum amount of code required for the very specific part of the behavior. You repeat this cycle until all behavior is implemented, while refactoring in between cycles.

During refactor, you may split the logic between different classes, and try out design options. The tests will still be running on the "main" class, or the "public API" of your functionality. This is considered the best practice for the same reason you mentioned, to avoid hindering the freedom to refactor your code.

With this cycle in mind, in your specific case, it would hardly be necessary to mock any of your own collaborators. However, you should indeed mock the external dependencies of your code, such as the configuration files and restful services. Mainly because you should be able to test your code without depending on environment factors such as a file existing in a directory, and also you can increase the flexibility of your functionality if you provide an injection point for such dependencies.

PS: I know you're not going to write the code from scratch, but this was an outline of the process you'd have followed with TDD, just to give you an idea of how the tests would look like.

PS2: By "mock" I mean any kind of test double, such as fakes, dummies, stubs, spies and real mocks.

Answer 3

Unit testing is like inductive reasoning. You ensure that the small piece n is well-behaved, and then can trust that as you build up the system, each n + 1 piece is also well-behaved.

In your case, I would isolate as many of the 5-6 classes in individual testing to make sure that each one is doing what it should. (That has an extra benefit of speeding up your coding, because you're not always tripping over unexpected defects.)

If you then want to test those working together in isolation, that's good -- just consider what you need to mock or stub to kick off specific logic paths. At some point, something is calling into your system to trigger the update -- so mock that out.

A good rule of thumb is, don't test something you didn't write -- mock it.

On a more philosophical level ;) the definition of "unit" isn't a bright line. A unit can be a method, or a class, but even within a method, there's probably references to multiple objects. As long as you can draw a reasonably small boundary around the "unit" and run it in isolation, it's all good.

...

Another consideration is that if your 5-6 small classes are easy to unit test individually, then they are mostly likely well designed. If those classes are tightly coupled enough that you can't exercise each in isolation, it means you probably need to step back and look at your design.