I have recently become responsible for a legacy tool that analyses code and provides a log as output. As part of the JUnit suite for this, there are ~100 tests that rely on successfully matching the produced log with a static
Now this means that if the format of the log changes at all, it will break the unit tests. But I cannot decide if this is a good or a bad thing.
- Quickly failing unit tests let me know that something has changed
- Tests have to be updated every time the log style is changed
- A straight text compare is bad for this case, because the log file involves times. So if a run doesn't take exactly as long as was expected, the test fails.
I can't help but feel that these tests shouldn't be run. But at the same time I don't want to leave the log file process uncovered.
What is the solution here? Better unit tests? No unit tests? How can I do this without my tests being so brittle?
The log is being written to a file, and is not part of any logging framework (a file is generated after the tool runs, telling the user which files failed validation and why).