1

Simple question, what's more common / standard / won't make future developers look at me cross-eyed for unit testing:

Displaying just the errors, or reporting every passed test?

closed as too broad by GlenH7, user40980, durron597, user22815, Ixrec Oct 3 '15 at 9:57

Please edit the question to limit it to a specific problem with enough detail to identify an adequate answer. Avoid asking multiple distinct questions at once. See the How to Ask page for help clarifying this question. If this question can be reworded to fit the rules in the help center, please edit the question.

  • 2
    Most unit test frameworks will display the results of each test, whether it passes, failed, was inconclusive or was not run. – Oded Aug 14 '12 at 15:44
  • @Oded Just to help me wrap my head around this, could you describe a unit test which was inconclusive? – Philip Aug 14 '12 at 18:41
  • See here: nunit.org/index.php?p=utilityAsserts&r=2.5.10 – Oded Aug 14 '12 at 18:42
  • 1
    @Philip: "not implemented yet" is probably the most common cause for inconclusive unit test results - either the feature hasn't been implemented yet, or the unit test is just an auto-generated stub. – tdammers Aug 14 '12 at 19:06
  • Also, if the setup for your test fails (and not the test itself), then that could warrant calling the result inconclusive instead of failed. – MetaFight Jul 12 '13 at 7:46
3

I would say that showing every test and its pass/fail/skipped status is the best, because it gives positive feedback that each test was successfully run or purposefully skipped.

However, if you are building your own tools, I recommend giving the user an option to choose what they want to see.

1

Depends what you want to see. It's always nice to have at least the option of seeing all the results (as in, a complete list of all tests that have been run, and their status), and if you run the tests as a developer, you are probably also interested in error details for the failed ones. You do want a way to boil it all down to one single exit status though: either everything passes (which means global pass) or one or more tests do not pass (which means global failure). This is especially important if you want to hook your test suite into a Continuous Integration framework or similar automated checking mechanism.

So, long story short: If you can, make it configurable. Most off-the-shelf solutions have this option, and if you're rolling your own, it's not hard to do either.

1

If you're using a unit testing framework:

  • Use defaults. They're usually good, succinct and convey exactly the information you need. Look at nose in Python (find command line examples!) or the Test tools in Visual Studio 2010. In my experience, they're quite good.
  • Less is more. Don't clutter results with lots of long-winded error messages or needless warnings. Again, see default results for tests in the previous frameworks to get a good idea of what looks good.

If you're building a unit testing framework:

  • Above all make things intuitive and easy to understand. You can get pretty far just presenting pass/fail/skipped results for each test can be extremely helpful. Again, less is more: gussied-up presentations usually won't help things out too much. Even simple colour text-coding can help immensely, as can basic print-outs to a command window.
-2

I definitely vote for showing any test result, both passed and failed, maybe summarizing the failed tests at first.

IMHO when unit testing it's fundamental to get all details from test run.

Not the answer you're looking for? Browse other questions tagged or ask your own question.