3

Assume you have a class C. C defines a public method (member function) C::m1.

Calling c.m1() (c is an instance of C) can either

  1. Return after mutating the object c it was called on.
  2. Throw an exception of type E and leave c unchanged.

Because of (1), C::m1 cannot be marked const. However, if C::m1 throws it should (by the specification and by the caller's sanity) behave as if it were. I will assume no language has native support for a const-if-throws modifier.

A way of testing this is to make a copy of c (call it oldC), call m1 on c in a way in which it throws, and to test that c == oldC after the exception is caught. However, this requires an equality operator that would otherwise not exist (and, as a consequence, adds to the amount of code that needs to be written, documented, tested, and maintained).

Assume that adding a predicate C::willM1Fail that is const is impossible because m1 might fail because of external factors (other servers) and it is impossible to guarantee that the next call to m1 will succeed.

Also, assume that making m1 return a new object of type C and marking it const instead of mutating the object it was called on (which is an obvious solution) is undesired as m1 will virtually always take path (1) and copying C objects in production is bad, because of the performance penalty and the memory ownership complexity it will add.

The question then is: should the equality operator be defined, the exception forced (through mocking) and equality tested, or is there a better way to test that when an object of a class C throws an exception of type E it does not change the object (at least in the tested code paths)?

If you have a suggestion about a better design (that makes testing this property simpler), it would also be welcome.

Addressing comments

I thought this question was sufficiently language-agnostic, however, as it was pointed out in the comments by multiple individuals, the wording of this question makes it fairly clear that it was about a C++ case. I will be marking it as C++ even though it seems to be realizable to a good extent in Rust as well (&self/&mut self and not mutating the object when an error is returned).

There were also suggestions about changing some of the wording (const, copy, equals) into something more language-agnostic (immutable, clone, compare). Because the answers used the wording in the original question, I think it is better to leave it as it is.

Solution used

In the case that provoked this question, the solution adopted was adding another method tryResolvingForM1, marked as const, that throws or returns a new type D that can be passed in a call to m1. The main downside is that the caller now has to call both tryResolvingForM1 and m1. Testing was simplified as m1 can only be called if you have a D object and it will not fail. Trying to get a D object may fail but is guaranteed to not mutate the object it was called on.

  • 3
    Never thought of const as language agnostic before. – candied_orange Jun 11 at 2:37
  • "language agnostic", but the language must support const-ness, operator overloading, exceptions, mutable objects, the concept of ownership? Sorry, but I am pretty sure you have definitely C++ in mind, and I think the question should be tagged with C++. – Doc Brown Jun 11 at 8:46
  • Regardless of the cause, I think this question has been answered before in SE and, in a few words, we agreed with implementing production code for testing purpose was not a good idea, because it's unnecessary code in production we have to maintain. The same way we don't change access modifiers just for testing code which is not meant to be tested that way. There' are techniques to compare states, as for example comparing serializations. Let's say comparing toString() results or even comparing the logs they left. It comes to my mind the Golden Record Technique for legacy code. – Laiv Jun 11 at 12:53
2

Usually in such complex situations you aren't looking for the object to have strictly not changed, just that it has behaviourly returned to the state prior to the call.

Configuration is like a mini-language, controlling the function/object being used. Comparing the configuration prior/after a call is a good way to detect many state changes.

Another useful invariant is checking for behaviour despite an earlier failure. The idea being that if the function/object has shifted state then the next behaviour will be unexpected.

If you were in State A, and the error happened that should have left in in State A, then:

  • you make a call that requires state A it should pass
  • you make a call that requires state B it should fail

Beyond those invariants, you start to dictate implementation.

  • Which is fine for White Box testing.

  • But does make very brittle tests that are over fit to a specific implementation.

| improve this answer | |
  • 3
    If c is properly encapsulated it might not be possible to test it for a state change. It should be possible to test if its behavior changes. If not, who cares? – candied_orange Jun 11 at 2:42
  • @candied_orange Depends upon what you mean by properly encapsulated. To me properly encapsulated includes indicating the current overall state, so that other collaborators can tell where in the protocol things think they are. Though I do agree that there are many implementations which do make checking harder. The "If not, who cares?" is a bit confusing though. I would imagine that the library writer, and the library user care. I also care, as the end recipient of something that the software affects. – Kain0_0 Jun 11 at 6:31
  • 2
    If I'm an encapsulated object stop looking at my state. It's none of your business. I do different things for my own reasons. Don't you worry about why. You may test what I do but why is not for you to know. If you do know I break you when I'm redesigned. So stop peeking at my privates. Use the public interface like everyone else you peeping tom. – candied_orange Jun 11 at 6:35
  • @candied_orange Love the humour. Yes I agree with you. I was talking about checking the data returned via the public interface, to ensure that it conforms the protocol established by the interface. I was not discussing breaking through to the implementation. I thought I had made that clear by pointing out that beyond the external configuration/behaviour you start enforcing a specific implementation, and making brittle tests. – Kain0_0 Jun 11 at 6:44
  • Sorry, it's just you keep saying state. Keep in mind many 'objects' aren't encapsulated. Most that bother with equality operators are really data transfer objects or value objects, whatever you'd like to call them. These data structures share their state freely. My point is it'd be nice to know which kind you're talking about testing because it will impact your testing plan. Do you expect your advice to hold up equally with both kinds? If not please distinguish between them. – candied_orange Jun 11 at 6:55
2

Equality is all about semantics: what does it mean that two C objects are equal? That all values of all its attributes are the same? Or can we ignore some mutable attributes in the comparison? What about references to other objects: must they be identical or is it sufficient that the value of the objects referred to is the same? Or do we just look at the object’s own reference whatever its values are? Or is it sufficient that the two objects just have the same behavior (and how do you verify this)?

Equality semantics are furthermore related to copy semantics: if you copy an object, to compare two historical versions, the copy and the comparison operation must use the same semantics (some languages even have a rule for this). Terminological remark: Instead of copy and equality, maybe cloning and comparing could avoid language-specific traps (if the underlying language uses types having reference semantic)

Designing a type should care for its semantics: you have here a perfect example of why the design shall not be solely driven by requirements. Ok, there may be no explicit requirement to have an additional copy and a comparison operator, but when you think of it, you see already now that there could be a need for it.

Furthermore there are quality requirements: maybe there is no explicit requirement for having this equality operator. But your testing strategy was chosen to meet quality requirements. And this testing strategy leads you to this additional operator. So even if it is not explicitly required, you can justify its needs and trace them back to the overall requirements. (BTW implementing this equality operator self-documents this aspect of the semantic).

Conclusion: if you design a type, and think it is necessary to have a copy and a comparison operation, go for it. Your design, your choice.

Caution: not all objects can be copied: if copying a C is prohibited (or does not make sense because it corresponds to a unique resource, e.g. an OS semaphore), your testing approach is broken: if you can't copy, you can't compare. You'd then obliged to track changes (e.g. flags, version number) and this might become very tricky if some changes revert others) which might perhaps require a complete redesign the class. Cost-benefit? But that is food for other questions ;-)

| improve this answer | |
0

Let me first focus on the question from the title, which is indeed language agnostic: if you need an equality comparison for testing only, your choices are

  1. implementing it outside the class, in the testing code

  2. implementing it as a public method inside the class, as an overload of what the language provides for equality operators.

But approach #1 usually does not bring any benefit, you have to write the same (if not more) code than for approach #2. The comparison has to work correctly in both cases, so even the testing effort for this equality comparison does not become less. #2, however, gives you a better chance to reuse that code in the future, and it is the idiomatic choice for many languages.

A different question is if you can avoid the necessity for an equality comparison for your specific scenario completely, especially in C++. When it is possible to design the method m1 like this

void C::m1()
{
    if(!validate())   // validate is declared as a const function
        throw E();
    mutate();         // mutate must not throw
} 

it should be pretty clear that it follows the const-if-throws constract. You have to decide for yourself if it is worth the effort to write a test for m1 which makes sure it will follow that contract also in the future, maybe after a potential extension or refactoring. But in many contexts, it could be sufficient to rely on the comments, the simple structure of that method, and the const-ness of validate. If you think it is not safe enough, go forward, implement the equality check and write the test.

| improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.