1

I recently worked on a project where the team were writing lots of unit tests for hardcoded values, for example, unit tests that test the value of a string constant. The justification being that they protect against accidental change.

It feels like overkill to me, we had hundreds of tests and they never once caught an accidental change for me personally. What are your thoughts on this?

3
  • 1
    need examples if you want more than a waffle about how it might be ok sometimes
    – Ewan
    Commented Sep 4, 2023 at 17:03
  • I think it's really important to clarify what the constants do, exactly. For example, if they're a special key for looking up a value in a payload, it seems very reasonable to have a test that has a payload, and uses the constant as a key to retrieve some value. It has nothing to do with the concrete value of the constant, but the behaviour that it enables.
    – Alexander
    Commented Sep 5, 2023 at 3:23
  • It may be useful to review: stackoverflow.com/a/153565/54734 "I get paid for code that works, not for tests...." Commented Sep 5, 2023 at 12:08

4 Answers 4

7

Worth is subjective

"Is it worth it" is generally not objectively answerable when talking about realistic testing expectations.

Generally speaking, we don't get as much time to write tests as we would like to have. If time were infinite, we'd write every possible test scenario we could think of.
But time is not infinite, so we make judgments based on time/effort taken to write and maintain certain tests, and how much value they add (i.e. by preventing uncaught bugs or regressions).

This judgment is very subjective. If you're creating a blog site, who cares if it happens to go down because of some remote bug? The cost of failure is so low that there's little reason to write very detailed test scenarios. However, if you're writing the driver for a medical device, or you're writing the core logic for a space rocket; the cost of failure is significantly higher. This means that there's more of an incentive to deeply test your work.

Testing hardcoded values

Tests should focus on behavior, not implementation. Whether a value is hardcoded or not is an implementation detail. On principle, you should not be making a decision on whether to write a test based on a particular value being hardcoded or not.

However, that's only part of the consideration. For example, let's say that this value is a key to an external API (and let's ignore that we should make this configurable for now).

There's still no value in a test that asserts that this hardcoded value has changed. However, there is value in a tests that asserts that you can connect and interact with that external API.

It might sound like I basically said the same thing twice, but context matters here. Being able to interact with an API is part of the application's behavior, which is what your tests should focus on. Whether or not this relies on a hardcoded value is an irrelevant consideration. Whether it does or doesn't, you still need to interact with that API.

Let's say that the external API changes tomorrow and no longer needs a key to work (e.g. because they've whitelisted your IP address). A test that confirms if the key value has changed would become redundant. A test that confirms that your application can interact with the external API would remain to be relevant.

This is why we test behavior, not implementation; because the implementation might change and evolve over time and it would be unproductive if this forced us to revisit our test suite.

5
  • So what is the "behavior" we are testing (or are able to test) with a hardcoded value such as: const BLUE = 0x0000FF ?
    – DavidT
    Commented Sep 5, 2023 at 0:45
  • @DavidT: That's not behavior, that's state. Show me how this state affects the behavior, then test that behavior.
    – Flater
    Commented Sep 5, 2023 at 4:35
  • The point I was making is that it DOES matter if a value is hardcoded (which is opposite of your stated point), since in most cases hardcoded values have no implementation (or behavior) so don't require a test (the exception being when there is some implied relationship - per my answer). If the value is not hardcoded it is possible the lookup mechanism may fail, hence a test may be required.
    – DavidT
    Commented Sep 5, 2023 at 5:05
  • @DavidT State should not be tested directly. If the value does not impact any behavior, it has no function whatsoever and can be removed. Otherwise, the behavior that it impacts is what should be tested.
    – Flater
    Commented Sep 5, 2023 at 6:51
  • Just to be very clear here: state is commonly asserted, but it is done with the intent to prove that the behavior works as expected. You test the behavior by asserting the state (among other things), but you don't test the state.
    – Flater
    Commented Sep 5, 2023 at 11:51
0

You have a finite amount of time to complete a project, so it matters how much time you spend on each task. In general it is not an efficient use of time, simply to copy primary code into a unit test - since any bug in the primary code will now be duplicated in the unit test, hence the unit test is unlikely to catch said bug.

I would argue that a simple test/assert such as:

const THE_ANSWER = 42

...

assertEquals(THE_ANSWER, 42)

Falls in the category of copying production code to the test. Where as something like:

const RED    = 0xFF0000
const BLUE   = 0x0000FF
const PURPLE = 0xFF00FF

assertEquals(RED + BLUE, PURPLE)

"May" have some value, since it encodes a relationship between the constants.

TL;DR

There may be cases where it is valid to say, that such tests:

protect against accidental change.

However creating a "simple assert" test, is only really forcing the developer to enter the constant twice and update it twice if it changes.


A more flushed out version of this argument, would be that you could annotate every constant like this:

@TestValue(42)
const THE_ANSWER = 42

Then create an automated test suite to scan and test all the @TestValue annotations. IMO this is less valuable than a simple comment:

// Do not change this value without talking to [email protected]

PS: I quite like the idea of:

@TestValue(RED + BLUE)
const PURPLE = 0xFF00FF
0

I’m going to take a stance and say no, it’s not worth it to unit test hard coded values. Why? I wouldn’t test hard coded values simply because somebody might be subjected to change them and they very well might need to.

I would much rather unit test the actual calculations or results that those hard coded values produce. This is much more beneficial as hard coded values can change and should not exist outside of constants, as a great teacher once taught me hard coded values are magic and can vanish and change without rhyme or reason.

If someone tries to tell you that they have an important connection string or IP address coded then I would have some serious concerns about where they came from if they’re not from a config or constant. A constant can be changed and a unit test can be changed, behavior should be the bigger question of change.

0

You can do several things. One copy all your constants into a unit test, use your editor to change constant definitions into asserts, and then you are safe from some dummy changing the constants in your source code. Not very much effort, not very much benefit.

Or you have data from an external source that someone turned into code. And that data needs to be correct, like no typos. In that case, someone else than you can find a second source for the data, and writes a unit test that asserts that every constant matches that second source. That is boring, it may be over the top, but it is a useful unit test.

Not the answer you're looking for? Browse other questions tagged or ask your own question.