4

Brief History

I am new to Automated Unit Testing and Mocking objects world; previously we used to do Unit Testing (including Integration Testing and we mistakenly referred that as Unit Testing) manually, but now we have planned to change all of that to I am pushing our organization to do Automated Unit Testing.

Problem is: I have a component for which I have written unit Tests (note: I am not using any TDD here, code is written first and then Unit Tests are written) covering each and every branch(if/ else or loops) of code. If I run the Unit tests suite it says that every method works as expected. That's good; but when I try to traverse the entire flow, I see there are many bugs( missing method) that should have been added. Is there a way to check this automatically?

Update As per the below Answers from KeithS and Schleis, it seems that I need Functional tests (or integration tests). Are there some useful links on automated functional tests? I tried it on Google but they are all showing generic results talking about manual testing.

Improve this question
3
  • 3
    If you could automate identification of missing code, you could automate coding that missing code, and then we'd all be out of a job because our bosses could just tell the computer what the program should do, the computer would identify the code that's "missing" which would do it, and code the missing code.
    – KeithS
    Aug 2, 2013 at 17:40
  • @KeithS that is a nice joke :)
    – Devesh
    Aug 3, 2013 at 2:09
  • 1
    @KeithS it is possible to identify obviously-missing code (look for NotImplementedException, E_NOTIMPL, a function with a return signature but didn't actually had a return statement, or functions that had an empty body.) Still it would not imply that automating the coding is possible.
    – rwong
    Aug 3, 2013 at 2:40

2 Answers 2

Reset to default
12

Unit tests are just one level of the hierarchy of automated tests. Unit tests exist to verify that the code you (the developer) have actually written behaves the way you thought it should when you wrote it.

There are two caveats inherent in unit testing. First, coverage is not exercise. You may execute every line of code in your codebase via one or more unit tests, but if you do not assert, somewhere, that code which needed to do something actually did it, then as long as you don't get an exception, the test passes with or without the code doing this key thing.

Second, unit tests by definition exercise small, isolated pieces of your code (units), making sure each piece behaves the way the developer thinks it should. They don't test that these units play nicely with each other (that's an "integration" test), nor do they assert that the code, at any level, behaves the way the client thinks it should (that's an "acceptance" and/or an "end-to-end" test).

This second problem is your main issue in the case in point. You have 100% unit test coverage, but no integration testing, which would prove that the little pieces are put together the right way to do the larger job you expect. You also seem to have no automated acceptance testing, which approaches the entire program from the top down from the perspective of an end user. These are the levels of testing, which can be automated, that will identify your "missing" code based on failure to satisfy acceptance criteria.

Improve this answer
7
  • this means that there is lot of coding involved, even in Testing part. What about the timelines? Who writes these integration testing and acceptance testing? A developer who is working on the piece of code is always busy in writing unit tests.
    – Devesh
    Aug 3, 2013 at 2:06
  • So if I correctly understood the answer; as a thought process, functional tests( or integration tests) should be written first and then the entire unit testing suite should be written. Also in functional tests I should not be using any mock objects if the functionality takes input from database and writes output in database.
    – Devesh
    Aug 3, 2013 at 2:13
  • 1
    It's a recursive structure, and you drill down, then climb back out. Starting with one acceptance criteria, write an automated acceptance test that will prove the system meets the criteria. It will fail or not even compile because the feature doesn't exist. Then, you drill down. Identify the highest-level object that has to change to incorporate the feature, and write a test to prove it works. Then recursively identify the objects that one depends on that will have to do the job. As you fill out these branches you'll transiition from integration testing between multiple objects to unit tests.
    – KeithS
    Aug 5, 2013 at 14:44
  • 1
    Once you have the unit test(s) written and failing, write the lines of code that will make all the tests pass; unit first, then integration, then acceptance. When it's all green, clean up the code to meet conventions and best practices, then move on to the next acceptance criteria. Done this way, it really isn't as much testing code as it might look like, and when something doesn't pass first try, or breaks something else, you have this extensive, relatively fast-running suite to catch it and use it to debug.
    – KeithS
    Aug 5, 2013 at 14:47
  • 1
    More or less. Understand that these tests are at different levels. You write failing acceptance tests, failing integration tests, then failing unit tests, then make those tests pass in reverse order (unit->integration->acceptance). At the unit test level, you may write one test, pass it, then write the next test and pass it. That's perfectly acceptable. But, the whole point of TDD/BDD is that you know what you want to do before you do it, and you write a test that will tell you when you're there. That's as true at the functional level as for individual methods or lines of code.
    – KeithS
    Aug 7, 2013 at 14:29
3

You want functional tests that run through all components working together. A test where you give your application an input and have it work with all the actual objects and then check your output. These tests verify that the connection between the tests work (as well as mocked methods actually exist).

Even with writing tests, you want to have the entire application run either by yourself or a QA. You will likely have missed things or not considered certain scenarios.

Improve this answer

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.