9

Given a class as follows:

class IntList {
public:
    IntList(std::vector<int> list) : m_list(list) {}

    std::vector<int> list() { return m_list; }

    std::string toString() {
        std::string repr;
        for (const auto integer: m_list) {
            repr += "int:" + std::to_string(integer) + ";";
        }
        return repr;
    }
    
private:
    std::vector<int> m_list;
};

Now I have an unrelated function initializeIntegers() whose I want to test the returned value. Is it acceptable during unit tests to use toString() to compare the expected values of the object?

void testCase() {
    const auto intList = initializeIntegers();
    assert((intList.toString() == "int:0;int:11;"));
}

It seems weird to me, I would rather compare the value returned by list() instead. However, I can't find any reference stating whether or not using the string representation is OK. Am I wrong if I suggest to not rely on string representation during unit tests? Do you know of any authoritative references explaining it?

Note that both list() and toString() methods exist and are used regardless of the unit tests (they weren't added just to be able to test the class).

5
  • 4
    "I would rather compare the value returned by list() instead." - then your test would depend on the correct implementation of list() instead of the correct implementation of toString(). And, not to forget, on the correct implementation of ==. Not much of a difference.
    – Bergi
    Jun 22 at 21:03
  • Do you have any sensible assertion framework to use instead? If you don't, I'd expect such case to be a workaround for the lack of proper tooling, like GoogleTest matchers. If the alternative is to write EXPECT_THAT(initializeIntegers(), ElementsAre(0, 11)), I'd say that using toString() is a horrible choice. If the alternative is to write a for loop, I'd say it's a good choice to pick a concise option, even if it's semantically inferior.
    – Frax
    Jun 23 at 9:56
  • 2
    if your goal is to test for equality, then use the == operator. In your example it is not implemented, but your unit test need already shows that there is a need for this operator. I would add the operator and use that in your test. Implementing the == operator forces you to define when you consider two objects to be the same, which is not necessarily thought about when making a toString() method.
    – slingeraap
    Jun 23 at 14:04
  • Yes, comparing string representations can let some errors squeak through. But it is far better to test that way than not to test at all. And better to have tests that are easy to write, that allow you to re-factor and change your implementation as required, then to have invested a lot of time of code used only for testing, not for the actual application… If that investment then becomes an obstacle to re-factoring and improving your design. If writing the non-string compare tests is fast And easy enough, sure do that from the start. We call it agile for a reason.
    – Krazy Glew
    Jun 24 at 3:20
  • string(x)=string(y) only implies x=y when string(x)=string(y) implies x=y. That's when string returns a canonical/normal form--by definition of canonical/normal form.
    – philipxy
    Jun 24 at 22:41

8 Answers 8

32

Let's say you check whether two values are equal. Your check can go wrong in two ways:

  • The values are equal, but the string representation is not (this is unlikely).
  • The values are different, but the string representation is the same (say the values are 0.048 and 0.052 and both are represented as the string "0.05").

The second misses a test failure, and worse, it misses a subtle test failure that you won't pick up otherwise. The first case is reasonably harmless - it's annoying because you have a test that fails but shouldn't so you fix the test.

Where it gets really bad is if your test should compare three values a, b, c and you totally missed that your text representation only contains a and b. Now c is completely untested.

So you need some very, very good unit tests for toString() first if you want to rely on it in a unit test.

8
  • 2
    I have, on occasion, been lazy enough to use JSON serialization as a shortcut, e.g., JObject.FromObject(a).ToString(), but I then usually, still compare to a hard-coded "expected" string rather than serializing another object.
    – Corrodias
    Jun 22 at 21:42
  • 16
    "The values are equal, but the string representation is not" could very well happen if you're implementing an unordered collection like a dictionary or set. Jun 23 at 4:21
  • "The values are different, but the string representation is the same" - Unless you allow scientific notation on integers (which you should not, imho), that's not possible on an integer list. Jun 23 at 18:38
  • Cmaster - says who? Apple broke lots of admittedly stupid code not long ago because a function meant to convert things to strings for debugging was changed to display only the beginning and end of large arrays.
    – gnasher729
    Jun 23 at 21:24
  • 1
    1024 could be 1,024 or 1’024 or 1_024 or 1K or $400.
    – gnasher729
    Jun 23 at 21:26
14

The main disadvantage of comparing string representations is that now there are two different reasons why that test might start failing:

  • the functionality of the class was broken by a bad commit
  • the toString() functionality was changed by an unrelated commit that is fine by itself (perhaps the new toString() is a lot more readable!)

...and it's not obvious to tell which it is at a glance.

How great that drawback is depends on how likely you are to evolve your string representations.

2
  • 3
    Third reason: there are elements of the object that are not included in the output of ToString().
    – bta
    Jun 22 at 23:12
  • A false test failure is no big deal, you just fix it. And it answers the question “is it bad practice…”. If it makes your tests fail, then yes, it is bad practice :-)
    – gnasher729
    9 hours ago
2

Is it bad practice to compare string representation on an object instead of its properties during unit tests?

It's perfectly ok.

When you compare string representations you are testing:

  • toString(),
  • everything toString() uses to create the string representation, and
  • not necessarily any properties at all

toString() might use properties. Might use something else. All we learn from the test is if whatever it uses works.

The way unit tests work (the good ones anyway) they don't force you to use a particular implementation. So a 'string representation' test is blissfully unaware of whether properties exist at all.

So it's perfectly ok. So long as you don't care whether properties exist.

1
  • 2
    In TDD, testing by comparing string representations is not that unusual, overall when you are in the early stages of the test. It's quite useful because the opposite might lead you to add useless getters or break encapsulation for "the sake of the test".
    – Laiv
    Jun 23 at 10:41
1

Assuming that you have already unit tested the IntList class, and know that all its public methods work correctly, then it's up to you how you test your new function. Using the toString() method of your class seems like perfectly good thing to do.

When it comes down to it, unit tests are written by programmers to convince themselves that the code they are writing works. Formal acceptance tests are a different thing.

1
  • 1
    I want to test the output of initializeIntegers(), I'm not testing the IntList class itself.
    – Delgan
    Jun 22 at 10:14
1

In many cases, objects may have aspects of their state which cannot be meaningfully captured within a string representation. Suppose there is a structure type foo which for simplicity simply contains an integer, and another structure type bar which for simplicity simply contains a foo*. There also exists an instance of foo, called boz, which happens to hold the number 12.

There is no way for code which produce a string representation of a bar, but which is unaware of the existence of boz, to meaningfully distinguish between a bar that contains a pointer to boz, and a bar that contains a pointer to some other foo that happens to contain the number 12. If, however, the value stored in boz were to change, the effect of that on a bar would depend upon whether that bar pointed to boz.

For some kinds of objects, such distinctions wouldn't matter, but for other types of object the relationships among pointers contained therein may be far more important than any information held in the targets of those pointers, but cannot be meaningfully be expressed as a string.

7
  • 1
    Why couldn't pointers be expressed as strings? Java's default toString method does basically that.
    – Bergi
    Jun 22 at 20:59
  • @Bergi: Java's default hashCode() method, used by toString(), is designed so that references to the same object will always yield the same value, and references to different objects will usually yield different value, but there is no guarantee that to completely unrelated objects won't have the same hash code, and the odds of that happening are not even terribly remote if many objects' hash codes are observed shortly after the objects are created. Further, there is no way code can know what the "correct" hash code (or default toString() representation) of an object should be.
    – supercat
    Jun 22 at 21:13
  • @Bergi: An object reference contains, as part of its implied encapsulated state, the whereabouts of every single other reference to that object that exists anywhere in the universe. If one were to simultaneously change every reference to an object so that they all identified a different object whose fields contained the same contents, that would not affect the state encapsulated by the object, but it's impossible to tell if running two different programs to the same point would leave an object in the same state without knowing the whereabouts of every reference to it.
    – supercat
    Jun 22 at 21:16
  • Sorry for mentioning Java, you're right there… But I don't get your other point. In unit testing, we do not care about the entire universe of objects. If we care about an object's identity, we can stringify its memory address, if we only are about structural equality, we can stringify its content values. (If we care about the object graph, its a bit more complicated, but even that can be serialised into a string). Either way, the string captures all the information that we care about, and can be used for comparisons in testing.
    – Bergi
    Jun 22 at 21:26
  • @Bergi: The fact that a particular object happens to be at address 0x08675309 on one particular program execution says nothing meaningful about whether address 0x08675309 will identify that same object on any future program execution.
    – supercat
    Jun 22 at 21:45
1

For completeness, I'm just adding a few thoughts I had trying to articulate why it didn't seem as good practice to me:

  • This question only exists for one reason: the lack of an == operator implemented in IntList. If such operator existed, I would not have to chose between toString() or list() comparison. If I had to define this operator, the implementation would more likely be based on the two objects properties rather than their string representation.
  • The string representation is mainly intended for humans. If I decide to improve the string representation to ease debugging, I don't want unrelated test cases to break because of this. The software internally uses the properties, those are the one I want to check to prevent regression in the code.
  • The toString() methods are often more complicated than properties lookup. It's transforming the object to another representation, which may loose information (like integer rounding) or miss internal bugs. I remember we used to rely on debugString() method of Protobuf, but null values weren't displayed which wasn't very convenient during tests. We should not rely on implementation details like so.
  • Looking at other languages, many Python classes provide __repr__ and __str__ methods out-of-the-box, yet they are rarely relied upon for unit tests.
  • Finally, the choice comes down to knowing what constitutes the identity of the object. If the string is a perfect representation of what the object is, then it is maybe acceptable to use it for comparison.
0

A unit test usually consists of 3 parts

// arrange (any required set up / initialization before the actual test )

//act (the action that is being tested)

// assert (where the expected result is compared to the actual result)

The fact or facts that you assert are entirely up to you as the developer, there is no best practice on whether you should test a single property, multiple properties, a string representation of an object, how many times a method is called, the type of exception thrown, the exception message, the state of multiple objects or even whether it is Tuesday when the test finishes.

That is all up to you as the developer. You have to decide what represents a passing test and what doesn’t.

Remember, a unit test is confirmation that the unit of code under test will behave in the expected manner — only you can decide what you expect it to do.

-2

One wrinkle I'd consider is that some languages will print out its string representation with some randomness included - for example, a dictionary or hash might print out its keys and values in random order, or in the order they happened to be inserted, or some other algorithm. Comparing these sorts of objects by looking at their string representation would result in random failures, which are the worst kind. I've been bitten by this before when trying to compare JSON strings.

This doesn't apply to the specific case quoted in the code, but is something to consider for the more general question posed.

New contributor
Cidolfas is a new contributor to this site. Take care in asking for clarification, commenting, and answering. Check out our Code of Conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.