Software Engineering Stack Exchange is a question and answer site for professionals, academics, and students working within the systems development life cycle. It only takes a minute to sign up.
Sign up to join this community
I recently worked on a project where the team were writing lots of unit tests for hardcoded values, for example, unit tests that test the value of a string constant. The justification being that they protect against accidental change.
It feels like overkill to me, we had hundreds of tests and they never once caught an accidental change for me personally. What are your thoughts on this?
"Is it worth it" is generally not objectively answerable when talking about realistic testing expectations.
Generally speaking, we don't get as much time to write tests as we would like to have. If time were infinite, we'd write every possible test scenario we could think of.
But time is not infinite, so we make judgments based on time/effort taken to write and maintain certain tests, and how much value they add (i.e. by preventing uncaught bugs or regressions).
This judgment is very subjective. If you're creating a blog site, who cares if it happens to go down because of some remote bug? The cost of failure is so low that there's little reason to write very detailed test scenarios. However, if you're writing the driver for a medical device, or you're writing the core logic for a space rocket; the cost of failure is significantly higher. This means that there's more of an incentive to deeply test your work.
Tests should focus on behavior, not implementation. Whether a value is hardcoded or not is an implementation detail. On principle, you should not be making a decision on whether to write a test based on a particular value being hardcoded or not.
However, that's only part of the consideration. For example, let's say that this value is a key to an external API (and let's ignore that we should make this configurable for now).
There's still no value in a test that asserts that this hardcoded value has changed. However, there is value in a tests that asserts that you can connect and interact with that external API.
It might sound like I basically said the same thing twice, but context matters here. Being able to interact with an API is part of the application's behavior, which is what your tests should focus on. Whether or not this relies on a hardcoded value is an irrelevant consideration. Whether it does or doesn't, you still need to interact with that API.
Let's say that the external API changes tomorrow and no longer needs a key to work (e.g. because they've whitelisted your IP address). A test that confirms if the key value has changed would become redundant. A test that confirms that your application can interact with the external API would remain to be relevant.
This is why we test behavior, not implementation; because the implementation might change and evolve over time and it would be unproductive if this forced us to revisit our test suite.
const BLUE = 0x0000FF ?
You have a finite amount of time to complete a project, so it matters how much time you spend on each task. In general it is not an efficient use of time, simply to copy primary code into a unit test - since any bug in the primary code will now be duplicated in the unit test, hence the unit test is unlikely to catch said bug.
I would argue that a simple test/assert such as:
const THE_ANSWER = 42
...
assertEquals(THE_ANSWER, 42)
Falls in the category of copying production code to the test. Where as something like:
const RED = 0xFF0000
const BLUE = 0x0000FF
const PURPLE = 0xFF00FF
assertEquals(RED + BLUE, PURPLE)
"May" have some value, since it encodes a relationship between the constants.
TL;DR
There may be cases where it is valid to say, that such tests:
protect against accidental change.
However creating a "simple assert" test, is only really forcing the developer to enter the constant twice and update it twice if it changes.
A more flushed out version of this argument, would be that you could annotate every constant like this:
@TestValue(42)
const THE_ANSWER = 42
Then create an automated test suite to scan and test all the @TestValue annotations. IMO this is less valuable than a simple comment:
// Do not change this value without talking to [email protected]
PS: I quite like the idea of:
@TestValue(RED + BLUE)
const PURPLE = 0xFF00FF
I’m going to take a stance and say no, it’s not worth it to unit test hard coded values. Why? I wouldn’t test hard coded values simply because somebody might be subjected to change them and they very well might need to.
I would much rather unit test the actual calculations or results that those hard coded values produce. This is much more beneficial as hard coded values can change and should not exist outside of constants, as a great teacher once taught me hard coded values are magic and can vanish and change without rhyme or reason.
If someone tries to tell you that they have an important connection string or IP address coded then I would have some serious concerns about where they came from if they’re not from a config or constant. A constant can be changed and a unit test can be changed, behavior should be the bigger question of change.
You can do several things. One copy all your constants into a unit test, use your editor to change constant definitions into asserts, and then you are safe from some dummy changing the constants in your source code. Not very much effort, not very much benefit.
Or you have data from an external source that someone turned into code. And that data needs to be correct, like no typos. In that case, someone else than you can find a second source for the data, and writes a unit test that asserts that every constant matches that second source. That is boring, it may be over the top, but it is a useful unit test.
