9

A project I am working on has a bunch of legacy tests that were not properly mocked out. Because of this the only dependency it has is EasyMock, which doesn't support statics, constructors with arguments, etc. The tests instead rely on database connections and such to "run" the tests. Adding powermock to handle these cases is being shot down as cost prohibitive due to the need to upgrade the existing project to support it (Another discussion).

My questions are, what are the REAL world tangible benifits of proper unit testing I can use to push back? Are there any? Am I just being a stickler by saying that bad unit tests (even if they work) are bad? Is code coverage just as effective?

Improve this question
3
  • 3
    Perhaps the code you are writing could benefit from some structural changes to make it more testable. We employ unit testing extensively at our facility without a mocking framework, and it seems to work pretty well for us.
    – Robert Harvey
    Dec 19, 2012 at 17:13
  • Wow I am impressed I would be interested in seeing how you manage this with third party JAR integration. Do you have any examples or links?
    – Jackie
    Dec 19, 2012 at 18:14
  • 1
    We don't use third-party JAR integration. :) We do, however, create stubs, as needed; mocking frameworks are just a convenience for achieving the same. If that process proves too arduous, however, we rethink our coding approach.
    – Robert Harvey
    Dec 19, 2012 at 18:41

5 Answers 5

Reset to default
8

  • Resources

    You need a test database to run the tests. The hardware to run the database is more expensive than using powermock. Releasing the resources used for unit testing against the database means the company doesn't need to upgrade the server as soon.

  • Reliability

    A test can fail because the database is down or in an inconsistent state. Guess what, your builds failed because the DBAs took the dev server down during the day to do some upgrades (and now the developers are trying to figure out what went wrong in the code - when its not the code).

  • Additional maintenance

    Tests can be run in any order. Adding or removing a test should not cause the test suite to fail. Running against a database means there is some state somewhere (in the database). Typically, these tests require extra maintenance to ensure that the database maintains the proper state for the next test to run.

  • Concurrency

    Two developers run the unit tests at the same time. Having a database means that the tests may collide on the database. The solution to this would be to clone the database for each test that is run, which is prohibitive on a real database. Which leads to another option to consider.

Consider also using HSQLDB with whatever dialect of database you are using. This way one can create an in memory database for each test. A unit test runs, constructs the necessary database, loads the data, the database connection from the code connects to HSQLDB and runs against the test data.

Improve this answer
4

My questions are, what are the REAL world tangible benifits of proper unit testing I can use to push back?

Good unit tests are (per Clean Code, and elsewhere):

  • Fast
  • Independent
  • Repeatable
  • Self-Validating
  • Timely

Not having true unit tests violates the first three (and usually the last one). This leads to some pretty significant problems:

  1. Your tests are slow. Slow tests get run infrequently. Tests that aren't run are almost useless.
  2. Your tests can fail due to reasons unrelated to the code you're testing. This makes it far harder to find out why tests have failed, wasting time and leading people to blame the environment rather than the code.
  3. As the database changes, your results change. Getting some database in a known good consistent, maintained state for tests is tedious, annoying and error-prone.

In general, a lack of isolation in unit tests leads to worse tests which take longer to write, longer to investiage, and provide less confidence in your codebase. That in turn leads to people writing less tests or ignoring the tests more which is the downward spiral to chaos.

Improve this answer
2

Unit tests are just a tool used to help a developer deliver reliable code. If you think like your boss thinks, you will be able to convince him if what you are proposing makes sense. However, if you present it as an evangelist on a band wagon, you won't get the resources allocated. You will need to explain to him how he benefits by spending the time and resources retrofitting unit tests to you legacy application.

You mentioned legacy tests - that implies legacy code. Unfortunately, fitting unit tests to code that was not designed to be unit tested is a difficult, time consuming and expensive process. The business case get even harder if you have tests that provide useful results, and all you are going to achieve is (from the business case POV) alternate tests providing the same results. You will need to focus on the cost(time) savings.....

My guess is that you will not be able to present your boss with a robust business case because there is not one.

Improve this answer
1
  • I can definitely see where you are coming from here, however, I was more looking for how to make this business case. Not just that worrying about quality at the margins is cost prohibitive.
    – Jackie
    Dec 20, 2012 at 18:12
1

From what you described it sounds like you don't like test framework A and want to switch to test framework B, both of which work. Suck it up and use what is existing and working, don't create more work reinventing the wheel so you can use your preferred test framework.

It takes more than a want to use the latest flavor of the month framework to justify throwing out existing working code and spending huge amounts of time and money for essentially zero benefit to users.

Improve this answer
1
  • Not exactly these are not a mater of framework and more a mater of allowing tests to access other resource. Thereby essentially making them functional tests instead of unit tests. The "frameworks" are both JUnit and EasyMock (Albeit older versions). I did suggest adding a NEW dependent framework.
    – Jackie
    Dec 19, 2012 at 18:16
1

Good unit tests provide localization. Functional tests might tell you "the add user function is broken", but a good unit test will tell you that it's broken because somebody changed the USER.LAST_NAME field in the database to be non-null. It's also possible to test small changes directly, instead of needing a complex test environment (which may need to be re-initialized or purged for some tests).

Improve this answer

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.