# What should you test with unit tests?

I'm freshly out of college, and starting university somewhere next week. We've seen unit tests, but we kinda not used them much; and everyone talks about them, so I figured maybe I should do some.

The problem is, I don't know what to test. Should I test the common case? The edge case? How do I know that a function is adequately covered?

I always have the terrible feeling that while a test will prove that a function works for a certain case, it's utterly useless to prove that the function works, period.

• Take a look at Roy Osherove's Blog. There is plenty of information on unit testing there including videos. He has also written a book, "The art of Unit Testing" which is very good. – Piers Myers Sep 12 '10 at 0:43
• I wonder what do you think about it after almost 5 years? Because more and more I feel that people should better know "what to not unit-test" nowadays. Behaviour driven development has evolved from the questions like you have asked. – r.pankevicius Apr 18 '15 at 18:00

Among the plethora of answers thusfar no one has touched upon equivalence partitioning and boundary value analysis, vital considerations in the answer to the question at hand. All of the other answers, while useful, are qualitative but it is possible--and preferable--to be quantitative here. @fishtoaster provides some concrete guidelines, just peeking under the covers of test quantification, but equivalence partitioning and boundary value analysis allow us to do better.

In equivalence partitioning, you divide the set of all possible inputs into groups based on expected outcomes. Any input from one group will yield equivalent results, thus such groups are called equivalence classes. (Note that equivalent results does not mean identical results.)

As a simple example consider a program that should transform lowercase ASCII characters to uppercase characters. Other characters should undergo an identity transformation, i.e. remain unchanged. Here is one possible breakdown into equivalence classes:

| # |  Equivalence class    | Input        | Output       | # test cases |
+------------------------------------------------------------------------+
| 1 | Lowercase letter      | a - z        | A - Z        | 26           |
| 2 | Uppercase letter      | A - Z        | A - Z        | 26           |
| 3 | Non-alphabetic chars  | 0-9!@#,/"... | 0-9!@#,/"... | 42           |
| 4 | Non-printable chars   | ^C,^S,TAB... | ^C,^S,TAB... | 34           |

The last column reports the number of test cases if you enumerate all of them. Technically, by @fishtoaster's rule 1 you would include 52 test cases--all of those for the first two rows given above fall under the "common case". @fishtoaster's rule 2 would add some or all from rows 3 and 4 above as well. But with equivalence partitioning testing any one test case in each equivalence class is sufficient. If you pick "a" or "g" or "w" you are testing the same code path. Thus, you have a total of 4 test cases instead of 52+.

Boundary value analysis recommends a slight refinement: essentially it suggests that not every member of an equivalence class is, well, equivalent. That is, values at boundaries should also be considered worthy of a test case in their own right. (One easy justification for this is the infamous off-by-one error!) Thus, for each equivalence class you could have 3 test inputs. Looking at the input domain above--and with some knowledge of ASCII values--I might come up with these test case inputs:

| # | Input                | # test cases |
| 1 | a, w, z              | 3            |
| 2 | A, E, Z              | 3            |
| 3 | 0, 5, 9, !, @, *, ~  | 7            |
| 4 | nul, esc, space, del | 4            |

(As soon as you get more than 3 boundary values that suggests you might want to rethink your original equivalence class delineations, but this was simple enough that I did not go back to revise them.) Thus, boundary value analysis brings us up to just 17 test cases -- with a high confidence of complete coverage -- compared to 128 test cases to do exhaustive testing. (Not to mention that combinatorics dictate that exhaustive testing is simply infeasible for any real-world application!)

• +1 That's exactly how I intuitively write my tests. Now I can put a name on it :) Thanks for sharing that. – guillaume31 Nov 22 '13 at 14:59
• +1 for "qualitative answers are useful, but it is possible - and preferable - to be quantitative" – Jimmy Breck-McKye Jun 4 '15 at 14:05
• I think this is a good answer if the directive is "how can I get good coverage with my tests". I think it would be useful to find a pragmatic approach on top of this - is the goal that every branch of every piece of logic in every layer should be tested thoroughly in this way? – Kieren Johnstone Dec 6 '16 at 11:23

For unit tests, start with testing that it does what it is designed to do. That should be the very first case you write. If part of the design is "it should throw an exception if you pass in junk", test that too since that is part of the design.

Start with that. As you get experience with doing that most basic of testing you'll begin to learn whether or not that is sufficient, and start to see other aspects of your code that need testing.

Probably my opinion is not too popular. But I suggest that you to be economical with unit tests. If you have too many unit tests you can easily end up spending half of your time or more with maintaining tests rather than actual coding.

I suggest you to write tests for things you have a bad feeling in your gut or things that are very crucial and/or elementary. IMHO unit tests are not a replacement for good engineering and defensive coding. Currently I work on a project that is more or less unusuable. It is really stable but a pain to refactor. In fact nobody has touched this code in one year and the software stack it's based on is 4 years old. Why? Because it's cluttered with unit tests, to be precise: Unit tests and automatized integration tests. (Ever heard of cucumber and the like?) And here is the best part: This (yet) unusuable piece of software has been developed by a company whose employees are pioneers in the test-driven development scene. :D

So my suggestion is:

• Start writing tests after you developed the basic skeleton, otherwise refactoring can be painful. As a developer who develops for others you never get the requirements right at the start.

• Make sure your unit tests can be performed quickly. If you have integration tests (like cucumber) it's ok if they take a bit longer. But long running tests are no fun, believe me. (People forget all reasons why C++ has become less popular...)

• Leave this TDD stuff to the TDD-experts.

• And yes, sometimes you concentrate on the edge cases, sometimes on the common cases, depending where you expect the unexpected. Though if you always expect the unexpected, you should really rethink you workflow and discipline. ;-)

• Can you give more detail as to why the tests make this software a pain to refactor? – Mike Partridge Aug 21 '11 at 18:17
• Big +1. Having walls of unit tests that test the implementation instead of the rules makes any change require 2-3x as many – TheLQ Aug 22 '11 at 0:00
• Like poorly written production code, poorly written unit tests are hard to maintain. "Too many unit tests" sounds like a failure to stay DRY; each test should tackle/prove a specific part of the system. – Allan Nov 22 '13 at 6:20
• Each unit test should check one thing, so there are not too many unit tests but, missing tests. If your unit tests are complex, that's another issue. – graffic Apr 6 '14 at 13:41
• -1: I think this post is poorly written. There are multiple things mentioned, and I don't know how they all relate. If the point of the answer is "be economical", then how does your example relate at all? It sounds like your example situation (though real) has bad unit tests. Please explain what lessons I'm supposed to learn from that and how it helps me be economical. Also, I honestly, simply don't know what you mean when you say Leave this TDD stuff to the TDD-experts. – Alexander Bird Jul 18 '18 at 19:16

If you start following Test Driven Development practices, they will sort guide you through the process and knowing what to test will come naturally. Some places to start:

Tests come first

Never, ever write code before writing the tests. See Red-Green-Refactor-Repeat for an explanation.

Write regression tests

Whenever you encounter a bug, write a testcase, and make sure it fails. Unless you can reproduce a bug through a failing testcase, you haven't really found it.

Red-Green-Refactor-Repeat

Red: Start by writing a most basic test for the behavior that you are trying to implement. Think of this step as of writing some example code that uses the class or function that you are working on. Make sure it compiles/has no syntax errors and that it fails. This should be obvious: you haven't written any code, so it must fail, right? The important thing to learn here is that unless you see the test fail at least once, you can never be sure that if it passes, it does it because of something that you've done of because of some bogus reason.

Green: Write the most simple and stupid code that actually makes the test pass. Don't try to be smart. Even if you see that there's an obvious edge case but the test take in account, don't write code to handle it (but don't forget about the edge case: you'll need it later). The idea is that every piece of code yo write, every if, every try: ... except: ... should be justified by a test case. The code doesn't have to be elegant, fast or optimized. You just want the test to pass.

Refactor: Clean up your code, get the method names right. See if the test is still passing. Optimize. Run the test again.

Repeat: You remember the edge case that the test didn't cover, right? So, now it's its big moment. Write a testcase that covers that situation, watch it fail, write some code, see it pass, refactor.

You are working on some specific piece of code, and this is exactly what you want to test. This means that you should not be testing library functions, the standard library or your compiler. Also, try to avoid testing the "world". This includes: calling external web APIs, some database intensive stuff, etc. Whenever you can try to mock it up (make an object that follows the same interface, but returns static, predefined data).

• Assuming I already have an existing and (as far as I can see) working code base, what do I do? – zneak Oct 13 '10 at 1:25
• That may be slightly more difficult (depending on how the code is written). Start with regression tests (they always make sense), then you can try writing unit tests to prove to yourself that you understand what the code is doing. It is easy to get overwhelmed by the amount of work that is (seemingly) to do, but: some tests is always better than no tests at all. – Ryszard Szopa Oct 13 '10 at 8:01
• -1 I don't think it's a very good answer for this question. The question isn't about TDD, it is asking about what to test when writing unit tests. I think a good answer to the actual question should apply to a non-TDD methodology. – Bryan Oakley Aug 21 '11 at 22:22
• If you touch it, test it. And Clean Code (Robert C Martin) suggests you write "learning tests" for 3rd party code. That way you learn to use it, and you have tests in case a new version changes the behaviour you are using. – Roger Willcocks Feb 25 at 23:38

The stock answer is to "test everything that could possibly break".

What's too simple to break? Data fields, brain-dead property accessors, and similar boilerplate overhead. Anything else probably implements some identifiable portion of a requirement, and may benefit from being tested.

Of course, your mileage -- and your working environment's practices -- may vary.

• Okay. So which cases should I test? The "normal" case? The edge case? – zneak Sep 25 '10 at 18:35
• Rule of thumb? One or two right up the middle of the golden path, and just inside and just outside any edges. – Jeffrey Hantin Sep 27 '10 at 21:45
• @JeffreyHantin That's the "boundary value analysis" in a different answer. – Roger Willcocks Feb 25 at 23:40

If you are testing first with Test Driven Development, then your coverage is going to be up in the 90% range or higher, because you won't be adding functionality without first writing a failing unit test for it.

If you are adding tests after the fact, then I cannot recommend enough that you get a copy of Working Effectively With Legacy Code by Michael Feathers and take a look at some of the techniques for both adding tests to your code and ways of refactoring your code to make it more testable.

• How do you calculate that coverage percentage? What does it mean to cover 90% of your code anyways? – zneak Sep 3 '10 at 19:29
• @zneak: there are code coverage tools that will calculate them for you. A quick google for "code coverage" should bring up a number of them. The tool tracks the lines of code that are executed while running the tests, and bases that agianst the total lines of code in the assembly(ies) to come up with the coverage percentage. – Steven Evers Sep 24 '10 at 22:04
• -1. Doesn't answer question: The problem is, I don't know _what_ to test – Alexander Bird Jul 18 '18 at 19:20

My personal philosophy has thusfar been:

1. Test the common case of everything you can. This will tell you when that code breaks after you make some change (which is, in my opinion, the single greatest benefit of automated unit testing).
2. Test the edge cases of a few unusually complex code that you think will probably have errors.
3. Whenever you find a bug, write a test case to cover it before fixing it
4. Add edge-case tests to less critical code whenever someone has time to kill.
• Thanks for this, I was floundering over here with the same questions as the OP. – Stephen Nov 27 '10 at 2:13
• +1, though I would also test the edge cases of any library/utility-type functions to ensure you have a logical API. e.g. what happens when a null is passed? what about empty input? This will help ensure your design is logical and document the corner case behaviour. – mikera Jun 30 '12 at 10:49
• #3 seems like a very solid answer as it is a real life example of how a unit test could have helped. If it broke once, it can break again. – Ryan Griffith Oct 20 '15 at 1:04
• Having just started, I find I'm not very creative with devising tests. So I use them as #3 above, which ensures peace of mind that those bugs will never again go undetected. – dotslash Jun 7 '16 at 9:28
• Your answer was featured in this popular medium article: hackernoon.com/… – BugHunterUK Oct 7 '17 at 20:46

## protected by gnatNov 7 '16 at 14:43

Thank you for your interest in this question. Because it has attracted low-quality or spam answers that had to be removed, posting an answer now requires 10 reputation on this site (the association bonus does not count).