11 added 192 characters in body
source | link

Yes, there is such a thing as too many unit tests.

While testing is good every unit test is:

  • A potential maintenance burden that is tightly coupled to the API

  • Time that could be spent on something else

  • A slice of time in the Unit Test suite
  • May be adding no real value because it is in effect a duplicate of some other test having minuscule chance that some other test will pass and this test will fail.

It's wise to aim for 100% code coverage but that far from means a suite of tests each of which independently provides 100% code coverage on some specified entry point (function/method/call etc.).

Though given how difficult it can to achieve good coverage and drive bugs out the truth is probably that there is such a thing as 'the wrong unit tests' as much as 'too many unit tests'.

Pragmatics for most code indicates:

  1. Make sure you have 100% coverage of entry-points (everything gets tested somehow) and aim to be close to 100% code coverage of 'non-errors' paths.

  2. Test any relevant min/max values or sizes

  3. Test anything you think is a funny special case particularly 'odd' values.

  4. When you find a bug add a unit test that would have revealed that bug and think about whether any similar cases should be added.

For more complex algorithms consider also:

  1. Doing some bulk testing of more cases.
  2. Comparing result to a 'brute-force' implementation and checking the invariants.
  3. Using some method of producing random test cases and checking against brute-force and post-conditions including invariants.

For example check a sorting algorithm with some randomized input and validating the data is sorted at the end by scanning it.

I'd say your tech lead is proposing 'minimal bare ass' testing. I'm offering 'highest value quality testing' and there's a spectrum in between.

Maybe your senior knows the component you're building will be embedded in some larger piece and unit tested more thoroughly when integrated.

The key lesson is to add tests when bugs are found. Which leads me my best lesson about developing unit tests:

Focus on units not sub-units. If you're building a unit out of sub-units write very basic tests for the sub-units until they're plausible and achieve better coverage by testing sub-units through their controlling units.

So if you're writing a compiler and need to write a symbol table (say). Get the symbol table up and running with a basic test and then work on (say) the declaration parser that fills the table. Only add further tests to the symbol table 'stand-alone' unit if you find bugs in it. Otherwise increase coverage by unit tests on the declaration parser and later the whole compiler.

That gets best bang for buck (one test of the whole is testing multiple components) and leaves more capacity for re-design and refinement because only the 'outer' interface is used in tests which tends to be more stable.

Coupled with debug code testing pre-conditions, post-conditions including invariants at all levels you get maximum test coverage from minimal test implementation.

Yes, there is such a thing as too many unit tests.

While testing is good every unit test is:

  • A potential maintenance burden that is tightly coupled to the API

  • Time that could be spent on something else

  • A slice of time in the Unit Test suite
  • May be adding no real value because it is in effect a duplicate of some other test having minuscule chance that some other test will pass and this test will fail.

It's wise to aim for 100% code coverage but that far from means a suite of tests each of which independently provides 100% code coverage on some specified entry point (function/method/call etc.).

Pragmatics for most code indicates:

  1. Make sure you have 100% coverage of entry-points (everything gets tested somehow) and aim to be close to 100% code coverage of 'non-errors' paths.

  2. Test any relevant min/max values or sizes

  3. Test anything you think is a funny special case particularly 'odd' values.

  4. When you find a bug add a unit test that would have revealed that bug and think about whether any similar cases should be added.

For more complex algorithms consider also:

  1. Doing some bulk testing of more cases.
  2. Comparing result to a 'brute-force' implementation and checking the invariants.
  3. Using some method of producing random test cases and checking against brute-force and post-conditions including invariants.

For example check a sorting algorithm with some randomized input and validating the data is sorted at the end by scanning it.

I'd say your tech lead is proposing 'minimal bare ass' testing. I'm offering 'highest value quality testing' and there's a spectrum in between.

Maybe your senior knows the component you're building will be embedded in some larger piece and unit tested more thoroughly when integrated.

The key lesson is to add tests when bugs are found. Which leads me my best lesson about developing unit tests:

Focus on units not sub-units. If you're building a unit out of sub-units write very basic tests for the sub-units until they're plausible and achieve better coverage by testing sub-units through their controlling units.

So if you're writing a compiler and need to write a symbol table (say). Get the symbol table up and running with a basic test and then work on (say) the declaration parser that fills the table. Only add further tests to the symbol table 'stand-alone' unit if you find bugs in it. Otherwise increase coverage by unit tests on the declaration parser and later the whole compiler.

That gets best bang for buck (one test of the whole is testing multiple components) and leaves more capacity for re-design and refinement because only the 'outer' interface is used in tests which tends to be more stable.

Coupled with debug code testing pre-conditions, post-conditions including invariants at all levels you get maximum test coverage from minimal test implementation.

Yes, there is such a thing as too many unit tests.

While testing is good every unit test is:

  • A potential maintenance burden that is tightly coupled to the API

  • Time that could be spent on something else

  • A slice of time in the Unit Test suite
  • May be adding no real value because it is in effect a duplicate of some other test having minuscule chance that some other test will pass and this test will fail.

It's wise to aim for 100% code coverage but that far from means a suite of tests each of which independently provides 100% code coverage on some specified entry point (function/method/call etc.).

Though given how difficult it can to achieve good coverage and drive bugs out the truth is probably that there is such a thing as 'the wrong unit tests' as much as 'too many unit tests'.

Pragmatics for most code indicates:

  1. Make sure you have 100% coverage of entry-points (everything gets tested somehow) and aim to be close to 100% code coverage of 'non-errors' paths.

  2. Test any relevant min/max values or sizes

  3. Test anything you think is a funny special case particularly 'odd' values.

  4. When you find a bug add a unit test that would have revealed that bug and think about whether any similar cases should be added.

For more complex algorithms consider also:

  1. Doing some bulk testing of more cases.
  2. Comparing result to a 'brute-force' implementation and checking the invariants.
  3. Using some method of producing random test cases and checking against brute-force and post-conditions including invariants.

For example check a sorting algorithm with some randomized input and validating the data is sorted at the end by scanning it.

I'd say your tech lead is proposing 'minimal bare ass' testing. I'm offering 'highest value quality testing' and there's a spectrum in between.

Maybe your senior knows the component you're building will be embedded in some larger piece and unit tested more thoroughly when integrated.

The key lesson is to add tests when bugs are found. Which leads me my best lesson about developing unit tests:

Focus on units not sub-units. If you're building a unit out of sub-units write very basic tests for the sub-units until they're plausible and achieve better coverage by testing sub-units through their controlling units.

So if you're writing a compiler and need to write a symbol table (say). Get the symbol table up and running with a basic test and then work on (say) the declaration parser that fills the table. Only add further tests to the symbol table 'stand-alone' unit if you find bugs in it. Otherwise increase coverage by unit tests on the declaration parser and later the whole compiler.

That gets best bang for buck (one test of the whole is testing multiple components) and leaves more capacity for re-design and refinement because only the 'outer' interface is used in tests which tends to be more stable.

Coupled with debug code testing pre-conditions, post-conditions including invariants at all levels you get maximum test coverage from minimal test implementation.

10 added 145 characters in body
source | link

Yes, there is such a thing as too many unit tests.

While testing is good every unit test is:

  • A potential maintenance burden that is tightly coupled to the API

  • Time that could be spent on something else

  • A slice of time in the Unit Test suite
  • May be adding no real value because it is in effect a duplicate of some other test having minuscule chance that some other test will pass and this test will fail.

It's wise to aim for 100% code coverage but that far from means a suite of tests each of which independently provides 100% code coverage on some specified entry point (function/method/call etc.).

Pragmatics for most code indicates:

  1. Make sure you have 100% coverage of entry-points (everything gets tested somehow) and aim to be close to 100% code coverage of 'non-errors' paths.

  2. Test any relevant min/max values or sizes

  3. Test anything you think is a funny special case particularly 'odd' values.

  4. When you find a bug add a unit test that would have revealed that bug and think about whether any similar cases should be added.

For more complex algorithms consider also:

  1. Doing some bulk testing of more cases.
  2. Comparing result to a 'brute-force' implementation and checking the invariants.
  3. Using some method of producing random test cases and checking against brute-force and post-conditions including invariants.

For example check a sorting algorithm with some randomized input and validating the data is sorted at the end by scanning it.

I'd say your tech lead is proposing 'minimal bare ass' testing. I'm offering 'highest value quality testing' and there's a spectrum in between.

Maybe your senior knows the component you're building will be embedded in some larger piece and unit tested more thoroughly when integrated.

The key lesson is to add tests when bugs are found. Which leads me my best lesson about developing unit tests:

Focus on units not sub-units. If you're building a unit out of sub-units write very basic tests for the sub-units until they're plausible and achieve better coverage by testing sub-units through their controlling units.

So if you're writing a compiler and need to write a symbol table (say). Get the symbol table up and running with a basic test and then work on (say) the declaration parser that fills the table. Only add further tests to the symbol table 'stand-alone' unit if you find bugs in it. Otherwise increase coverage by unit tests on the declaration parser and later the whole compiler.

That gets best bang for buck (one test of the whole is testing multiple components) and leaves more capacity for re-design and refinement because only the 'outer' interface is used in tests which tends to be more stable.

Coupled with debug code testing pre-conditions, post-conditions including invariants at all levels you get maximum test coverage from minimal test implementation.

Yes, there is such a thing as too many unit tests.

While testing is good every unit test is:

  • A potential maintenance burden that is tightly coupled to the API

  • Time that could be spent on something else

  • A slice of time in the Unit Test suite
  • May be adding no real value because it is in effect a duplicate of some other test having minuscule chance that some other test will pass and this test will fail.

It's wise to aim for 100% code coverage but that far from means a suite of tests each of which independently provides 100% code coverage on some specified entry point (function/method/call etc.).

Pragmatics for most code indicates:

  1. Make sure you have 100% coverage of entry-points (everything gets tested somehow) and aim to be close to 100% code coverage of 'non-errors' paths.

  2. Test any relevant min/max values or sizes

  3. Test anything you think is a funny special case particularly 'odd' values.

  4. When you find a bug add a unit test that would have revealed that bug and think about whether any similar cases should be added.

For more complex algorithms consider also:

  1. Doing some bulk testing of more cases.
  2. Comparing result to a 'brute-force' implementation and checking the invariants.
  3. Using some method of producing random test cases and checking against brute-force and post-conditions including invariants.

For example check a sorting algorithm with some randomized input and validating the data is sorted at the end by scanning it.

I'd say your tech lead is proposing 'minimal bare ass' testing. I'm offering 'highest value quality testing' and there's a spectrum in between.

The key lesson is to add tests when bugs are found. Which leads me my best lesson about developing unit tests:

Focus on units not sub-units. If you're building a unit out of sub-units write very basic tests for the sub-units until they're plausible and achieve better coverage by testing sub-units through their controlling units.

So if you're writing a compiler and need to write a symbol table (say). Get the symbol table up and running with a basic test and then work on (say) the declaration parser that fills the table. Only add further tests to the symbol table 'stand-alone' unit if you find bugs in it. Otherwise increase coverage by unit tests on the declaration parser and later the whole compiler.

That gets best bang for buck (one test of the whole is testing multiple components) and leaves more capacity for re-design and refinement because only the 'outer' interface is used in tests which tends to be more stable.

Coupled with debug code testing pre-conditions, post-conditions including invariants at all levels you get maximum test coverage from minimal test implementation.

Yes, there is such a thing as too many unit tests.

While testing is good every unit test is:

  • A potential maintenance burden that is tightly coupled to the API

  • Time that could be spent on something else

  • A slice of time in the Unit Test suite
  • May be adding no real value because it is in effect a duplicate of some other test having minuscule chance that some other test will pass and this test will fail.

It's wise to aim for 100% code coverage but that far from means a suite of tests each of which independently provides 100% code coverage on some specified entry point (function/method/call etc.).

Pragmatics for most code indicates:

  1. Make sure you have 100% coverage of entry-points (everything gets tested somehow) and aim to be close to 100% code coverage of 'non-errors' paths.

  2. Test any relevant min/max values or sizes

  3. Test anything you think is a funny special case particularly 'odd' values.

  4. When you find a bug add a unit test that would have revealed that bug and think about whether any similar cases should be added.

For more complex algorithms consider also:

  1. Doing some bulk testing of more cases.
  2. Comparing result to a 'brute-force' implementation and checking the invariants.
  3. Using some method of producing random test cases and checking against brute-force and post-conditions including invariants.

For example check a sorting algorithm with some randomized input and validating the data is sorted at the end by scanning it.

I'd say your tech lead is proposing 'minimal bare ass' testing. I'm offering 'highest value quality testing' and there's a spectrum in between.

Maybe your senior knows the component you're building will be embedded in some larger piece and unit tested more thoroughly when integrated.

The key lesson is to add tests when bugs are found. Which leads me my best lesson about developing unit tests:

Focus on units not sub-units. If you're building a unit out of sub-units write very basic tests for the sub-units until they're plausible and achieve better coverage by testing sub-units through their controlling units.

So if you're writing a compiler and need to write a symbol table (say). Get the symbol table up and running with a basic test and then work on (say) the declaration parser that fills the table. Only add further tests to the symbol table 'stand-alone' unit if you find bugs in it. Otherwise increase coverage by unit tests on the declaration parser and later the whole compiler.

That gets best bang for buck (one test of the whole is testing multiple components) and leaves more capacity for re-design and refinement because only the 'outer' interface is used in tests which tends to be more stable.

Coupled with debug code testing pre-conditions, post-conditions including invariants at all levels you get maximum test coverage from minimal test implementation.

9 added 76 characters in body
source | link

Yes, there is such a thing as too many unit tests.

While testing is good every unit test is:

  • A potential maintenance burden that is tightly coupled to the API

  • Time that could be spent on something else

  • A slice of time in the Unit Test suite
  • May be adding no real value because it is in effect a duplicate of some other test having minuscule chance that some other test will pass and this test will fail.

It's wise to aim for 100% code coverage but that far from means a suite of tests each of which independently provides 100% code coverage on some specified entry point (function/method/call etc.).

Pragmatics for most code indicates:

  1. Make sure you have 100% coverage of entry-points (everything gets tested somehow) and aim to be close to 100% code coverage of 'non-errors' paths.

  2. Test any relevant min/max values or sizes

  3. Test anything you think is a funny special case particularly 'odd' values.

  4. When you find a bug add a unit test that would have revealed that bug and think about whether any similar cases should be added.

For more complex algorithms consider also:

  1. Doing some bulk testing of more cases.
  2. Comparing result to a 'brute-force' implementation and checking the invariants.
  3. Using some method of producing random test cases and checking against brute-force and post-conditions including invariants.

For example check a sorting algorithm with some randomized input and validating the data is sorted at the end by scanning it.

I'd say your tech lead is proposing 'minimal bare ass' testing. I'm offering 'highest value quality testing' and there's a spectrum in between.

The key lesson is to add tests when bugs are found. Which leads me my best lesson about developing unit tests:

Focus on units not sub-units. If you're building a unit out of sub-units write very basic tests for the sub-units until they're plausible and achieve better coverage by testing sub-units through their controlling units.

So if you're writing a compiler and need to write a symbol table (say). Get the symbol table up and running with a basic test and then work on (say) the declaration parser that fills the table. Only add further tests to the symbol table 'stand-alone' unit if you find bugs in it. Otherwise increase coverage by unit tests on the declaration parser and later the whole compiler.

That gets best bang for buck (one test of the whole is testing multiple components) and leaves more capacity for re-design and refinement because only the 'outer' interface is used in tests which tends to be more stable.

Coupled with debug code testing pre-conditions, post-conditions andincluding invariants at all levels you get maximum test coverage from minimal test implementation.

Yes, there is such a thing as too many unit tests.

While testing is good every unit test is:

  • A potential maintenance burden that is tightly coupled to the API

  • Time that could be spent on something else

  • A slice of time in the Unit Test suite
  • May be adding no real value because it is in effect a duplicate of some other test having minuscule chance that some other test will pass and this test will fail.

It's wise to aim for 100% code coverage but that far from means a suite of tests each of which independently provides 100% code coverage on some specified entry point (function/method/call etc.).

Pragmatics for most code indicates:

  1. Make sure you have 100% coverage of entry-points (everything gets tested somehow) and aim to be close to 100% code coverage of 'non-errors' paths.

  2. Test any relevant min/max values or sizes

  3. Test anything you think is a funny special case particularly 'odd' values.

  4. When you find a bug add a unit test that would have revealed that bug and think about whether any similar cases should be added.

For more complex algorithms consider also:

  1. Doing some bulk testing of more cases.
  2. Comparing result to a 'brute-force' implementation and checking the invariants.
  3. Using some method of producing random test cases and

For example check a sorting algorithm with some randomized input and validating the data is sorted at the end by scanning it.

I'd say your tech lead is proposing 'minimal bare ass' testing. I'm offering 'highest value quality testing' and there's a spectrum in between.

The key lesson is to add tests when bugs are found. Which leads me my best lesson about developing unit tests:

Focus on units not sub-units. If you're building a unit out of sub-units write very basic tests for the sub-units until they're plausible and achieve better coverage by testing sub-units through their controlling units.

So if you're writing a compiler and need to write a symbol table (say). Get the symbol table up and running with a basic test and then work on (say) the declaration parser that fills the table. Only add further tests to the symbol table 'stand-alone' unit if you find bugs in it. Otherwise increase coverage by unit tests on the declaration parser and later the whole compiler.

That gets best bang for buck (one test of the whole is testing multiple components) and leaves more capacity for re-design and refinement because only the 'outer' interface is used in tests which tends to be more stable.

Coupled with debug code testing pre-conditions, post-conditions and invariants at all levels you get maximum test coverage from minimal test implementation.

Yes, there is such a thing as too many unit tests.

While testing is good every unit test is:

  • A potential maintenance burden that is tightly coupled to the API

  • Time that could be spent on something else

  • A slice of time in the Unit Test suite
  • May be adding no real value because it is in effect a duplicate of some other test having minuscule chance that some other test will pass and this test will fail.

It's wise to aim for 100% code coverage but that far from means a suite of tests each of which independently provides 100% code coverage on some specified entry point (function/method/call etc.).

Pragmatics for most code indicates:

  1. Make sure you have 100% coverage of entry-points (everything gets tested somehow) and aim to be close to 100% code coverage of 'non-errors' paths.

  2. Test any relevant min/max values or sizes

  3. Test anything you think is a funny special case particularly 'odd' values.

  4. When you find a bug add a unit test that would have revealed that bug and think about whether any similar cases should be added.

For more complex algorithms consider also:

  1. Doing some bulk testing of more cases.
  2. Comparing result to a 'brute-force' implementation and checking the invariants.
  3. Using some method of producing random test cases and checking against brute-force and post-conditions including invariants.

For example check a sorting algorithm with some randomized input and validating the data is sorted at the end by scanning it.

I'd say your tech lead is proposing 'minimal bare ass' testing. I'm offering 'highest value quality testing' and there's a spectrum in between.

The key lesson is to add tests when bugs are found. Which leads me my best lesson about developing unit tests:

Focus on units not sub-units. If you're building a unit out of sub-units write very basic tests for the sub-units until they're plausible and achieve better coverage by testing sub-units through their controlling units.

So if you're writing a compiler and need to write a symbol table (say). Get the symbol table up and running with a basic test and then work on (say) the declaration parser that fills the table. Only add further tests to the symbol table 'stand-alone' unit if you find bugs in it. Otherwise increase coverage by unit tests on the declaration parser and later the whole compiler.

That gets best bang for buck (one test of the whole is testing multiple components) and leaves more capacity for re-design and refinement because only the 'outer' interface is used in tests which tends to be more stable.

Coupled with debug code testing pre-conditions, post-conditions including invariants at all levels you get maximum test coverage from minimal test implementation.

8 added 35 characters in body
source | link
7 added 1050 characters in body
source | link
6 added 48 characters in body
source | link
5 deleted 18 characters in body
source | link
4 deleted 18 characters in body
source | link
3 added 10 characters in body
source | link
2 added 73 characters in body
source | link
1
source | link