5

My organization is investing time in writing the wrong kinds of tests (unit tests that are tied far too heavily to the implementation, are very time consuming to create, and often make the system harder to change). Also, context is often thrown away and tests are skewed to improve coverage metrics.

The same assumptions that are baked into the code are baked into the unit tests.

I believe that code coverage is a misleading metric. One can have 100% code coverage and still have many bugs, many untested edge cases, etc. The costliness of how much work it takes to get the last 10% of coverage does not provide commensure value.

The tests do not serve well as "readable specs" (an ideal that sounds great on paper but often does not translate).

I found a test that, for example, asserted that a certain dropdown menu "contains 7 items." That doesn't map to any real-world expectation held by business users or even technical people. It seems like the author of that test was just looking for "something to assert."

I fear that we are measuring what is easy to measure over what is important (though perhaps less measurable).

Some of the tests are written at such a level that they set up mocks to verify every tiny step of the implementation (was Dispose called on the UnitOfWork)? Etc.

I would prefer outcome-based testing - verify that the outcome is right, the "what" and not the "how."

I think it would be great to do BDD or ATDD (acceptance test driven development) but organizationally we are not ready for a big paradigm shift to Cucumber or SpecFlow or a similar tool (although maybe it is a good long term goal).

Here's the problem. If you express a desire to write less tests, the testing zealous will look at you as though you have revealed yourself as deeply ignorant/immoral. Maybe I'm projecting a little bit? But no one wants to sound like they're suggesting we "cut corners" for speed. Nonetheless, I think our process is dragged down by blind adherence to protocols that aren't always contextually appropriate.

Is there a "post-zealotry" stance on testing that I can start shifting us toward?

  • see How do I explain ${something} to ${someone}? – gnat Jun 21 '17 at 14:54
  • Interpret it as "I'm looking for resources to help clarify my own development on this issue," then? – blaster Jun 21 '17 at 14:56
  • 1
    I think this question should be edited instead of closed. It is not so much off topic, as phrased poorly. – TheCatWhisperer Jun 21 '17 at 15:04
  • 3
    As a matter of principle, the primary purpose of unit tests is to make sure your code works as expected, not to provide documentation, mirror the requirements, or provide anything else. Your question appears to suggest that it is not the way the tests are being written, but the way the code is being written, that is the actual problem, since, if you were writing testable code, you wouldn't need elaborate mocks, slavish adherence to code coverage metrics, or even Cucumber. – Robert Harvey Jun 21 '17 at 15:20
  • 3
    @blaster: In my view, code coverage (as in statement coverage) is one of the least important metrics. Coverage of boundary conditions and execution paths are way more important and automatically lead to a high statement coverage (but the reverse is pertinently not true). – Bart van Ingen Schenau Jun 21 '17 at 15:26
4

No, you are not crazy. TDD/unit tests are great, but it must be applied correctly.

There is a phenomenon called cargo-cult coding/architecture. That is when you apply an approach without knowing why you are doing so. This seems to have been the case with automated tests on your project.

To me,

  • Automated Integration Tests have uses, however, focus on unit tests should be preferred when ever possible.
  • Code based Automated Testing the UI is a waste of time. (I am not referring to a test framework like Selenium, which can be useful)
  • Any test that is more complex than the code it is testing is just another potential point of failure, and therefore, at best, a waste of time

The type of thing you are referring to happening on your project is a symptom of the following:

  • Tightly coupled code
  • Failure to isolate business logic from the infrastructure components of the code base

If you code was less coupled, only a minimum of mocking would be necessary.

If your business logic was properly isolated, it would be very obvious as to how to write effective and productive unit tests of your code base.

  • 3
    "Automated Integration Tests are largely a waste of time" -- can you explain further? Do you prefer manual integration tests? No integration tests? – mcknz Jun 21 '17 at 15:05
  • 2
    I prefer unit tests – TheCatWhisperer Jun 21 '17 at 15:05
  • Unit tests are great, but they don't serve the same purpose as integration tests. What do you do to cope with the lack of integration tests? How do you ensure your system actually works when you integrate all those units together? – Vincent Savard Jun 21 '17 at 16:47
  • @DocBrown this type of situation is exactly why I said, "with some exceptions" I feel like we are arguing over semantics instead if content – TheCatWhisperer Jun 21 '17 at 16:49
  • See how that edit strikes you @DocBrown ;) – TheCatWhisperer Jun 21 '17 at 17:15

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.