2

I am writing tests TDD-style for a cryptography package containing encryption and decryption methods. I test my methods with various kinds of inputs, including multiline strings, mixed case, calling encrypt on decrypt, and mixed case of keys.

    @Test
    void testConversionLower() {
        String originalMessage = "how much wood can a woodchuck chuck until the woodchuck has chucked enough wood";
        String encryptedMessage = "xky cqex sqez eqj c mkqtyjkym sdwsg wdpkb pju sqezexqea dci yjkymuz gdkwwd yekf";

        assertEquals(encryptedMessage, Vigenere.encryptVariant("key", originalMessage));
        assertEquals(originalMessage, Vigenere.decryptVariant("key", encryptedMessage));
    }

    @Test
    void testConversionUpper() {
        String originalMessage = "how much wood can a woodchuck chuck until the woodchuck has chucked enough wood".toUpperCase(Locale.ROOT);
        String encryptedMessage = "xky cqex sqez eqj c mkqtyjkym sdwsg wdpkb pju sqezexqea dci yjkymuz gdkwwd yekf".toUpperCase(Locale.ROOT);

        assertEquals(encryptedMessage, Vigenere.encryptVariant("key", originalMessage));
        assertEquals(originalMessage, Vigenere.decryptVariant("key", encryptedMessage));
    }

    @Test
    void testConversion_MixedCase_Multiline() {
        String originalMessage = """
        Peter Piper picked a peck of pickled peppers
        A peck of pickled peppers Peter Piper picked
        If Peter Piper picked a peck of pickled peppers
        Where's the peck of pickled peppers Peter Piper picked?
        """;
        String encryptedMessage = """
        Favun Rylgh lksggt w ruym eb ryymbaf farfati
        W ruym eb ryymbaf farfati Lgjat Ferun ryymuz
        Kv Lgjat Ferun ryymuz c faea kh feeahgt lgflgho
        Yxatu'o vxa ruym eb ryymbaf farfati Lgjat Ferun ryymuz?
        """;

        assertEquals(encryptedMessage, Vigenere.encryptVariant("key", originalMessage));
        assertEquals(originalMessage, Vigenere.decryptVariant("key", encryptedMessage));
    }

    @Test
    void testKeyEncryption_AnyCase() {
        String message = "A simple variant is to encrypt by using the Vigenère decryption method and to decrypt by using Vigenère encryption.";
        final String ALL_CAPS_KEY = "CRYPTO";
        final String LOWER_CASE_KEY = "crypto";
        final String MIXED_CASE_KEY = "Crypto";

        assertEquals(Vigenere.encryptVariant(ALL_CAPS_KEY, message), Vigenere.encryptVariant(LOWER_CASE_KEY, message));
        assertEquals(Vigenere.encryptVariant(LOWER_CASE_KEY, message), Vigenere.encryptVariant(MIXED_CASE_KEY, message));
    }

    @Test
    void testKeyDecryption_AnyCase() {
        String message = "A simple variant is to encrypt by using the Vigenère decryption method and to decrypt by using Vigenère encryption.";
        final String ALL_CAPS_KEY = "CRYPTO";
        final String LOWER_CASE_KEY = "crypto";
        final String MIXED_CASE_KEY = "Crypto";

        assertEquals(Vigenere.decryptVariant(ALL_CAPS_KEY, message), Vigenere.decryptVariant(LOWER_CASE_KEY, message));
        assertEquals(Vigenere.decryptVariant(LOWER_CASE_KEY, message), Vigenere.decryptVariant(MIXED_CASE_KEY, message));
    }

Should these be in a single test case as a parameterized test because I am testing the same method - decrypt or encrypt - or should these be separate cases as I have them now?

3 Answers 3

1

I'll give you an overview of what tests should look like, but note that it might not be easy to achieve this without having a certain level of experience, and that it may be better to think of this as of something to work towards over time, rather than something to try an achieve immediately.

Think of test cases as testing (or exercising), in an isolated way, a particular scenario that's of interest to you. You'll have multiple such scenarios for the same unit under test (doesn't have to be a method). However, you have to think about these scenarios a bit more abstractly than just all the possible combinations of inputs and outputs, or you'll get bogged down, and your tests will tend to break easily and be a pain to maintain.

The basic idea is, for each scenario, you do "under these conditions, when I do this, this should happen" - the good ol' arrange-act-assert (or given-when-then).

So, what should these scenarios be? The details will depend on the problem, but generally, try to test the rules, rather than specific outputs (again, it might be hard to come up with what these should be). Each such scenario should also be rather specific (within reason, and to the extent your design will allow), so that when a test fails, you know exactly what the problem is (at least in terms of deviation from expected behavior in a particular scenario), as opposed to just the generic "something about the method doesn't work anymore".

For example, you might want to test the rule that says that encrypting a message, and then decrypting the result reproduces the original message. In such a test, you shouldn't really care what the encrypted message is. You don't care if it's all lowercase, uppercase, if it contains numbers or symbols, or whitespace, if it's meaningful or gibberish. You just want your implementation to not throw, and the message to be recoverable.

    @Test
    void testEncryptDecrypt() {
        String message = "how much wood can a woodchuck chuck until the woodchuck has chucked enough wood";

        String result = Vigenere.decryptVariant(
            "key", 
            Vigenere.encryptVariant("key", message)
        );

        assertEquals(message, result);
    }

You can then parameterize this test to use different messages and different keys - it should pass for all of them. In fact, it should even pass for different encryption algorithms (there's a clear example of LSP for you).

Since you're now splitting test scenarios by rules (the abstract "behavior"), you might let this test cover lowercase, mixed-case, uppercase input strings, or you might organize things differently (tests are code, so you can extract a shared method and call it from multiple tests). You're the designer, so really how the tests are organized and how these rules are conceptualized is up to you.

Some of your other tests are good in that they actually test a rule, rather than check for a specific output, like the test below.

    @Test
    void testKeyEncryption_AnyCase() {
        String message = "A simple variant is to encrypt by using the Vigenère decryption method and to decrypt by using Vigenère encryption.";
        final String ALL_CAPS_KEY = "CRYPTO";
        final String LOWER_CASE_KEY = "crypto";
        final String MIXED_CASE_KEY = "Crypto";

        assertEquals(Vigenere.encryptVariant(ALL_CAPS_KEY, message), Vigenere.encryptVariant(LOWER_CASE_KEY, message));
        assertEquals(Vigenere.encryptVariant(LOWER_CASE_KEY, message), Vigenere.encryptVariant(MIXED_CASE_KEY, message));
    }

From reading the test, it looks to me that this checks if the key is treated as case-insensitive, by testing if encrypting the message produces the same result every time. That is en example of a rule. I think the name of the test doesn't communicate that intent particularly well though - I'd rename it to something like testEncryptionKeyIsCaseInsensitive. In fact, since test methods aren't called from elsewhere in the codebase and since their sole purpose is to communicate to you what the rule is and what the test is all about, you can apply different naming conventions to them. E.g., you could just make it into a sentence: encryption_key_should_be_case_insensitive. The naming convention police is not coming to get you.

Some behaviors might be of special interest, like how your system under test handles specific edge cases, so I'd probably create those as separate tests. Think of tests as specifying the behavior: what should happen when the input is null? What when it's empty? What when it's all whitespace? Then again all of these but for the key parameter.

There are also some things that don't immediately come to mind, but that you could also test, if you have other parts of your system that rely on the algorithm having a particular property. That's what your test are for - they are meant to establish the contract between your component and its users. E.g. if you call encryptVariant on the same message with the same parameters twice, does it produce the same output or not? Should it produce the same output (or not)? (Many encryption algorithms are designed to produce different outputs, for security reasons.) So, in that light, think about what other parts of the system (callers) expect, and if you should capture that in your tests.

There might be a few more things to go through here, but let's say that that's about all the behavior that's "externally observable" through the Vigenere APIs. Everything else is an implementation detail as far as this particular test suite is concerned. Specific inputs and outputs are implementation dependent. If something depends on the internals and you think it's fairly likely to change at some point, ideally, you don't put it in your tests. At least, not in this set of tests. If you want to test the specifics of an algorithm that aren't covered here, have a separate set of tests for that (don't repeat the existing tests, but arrange things so that the implementation is covered by the two sets of tests).

What I outlined here is, more or less, what's known as behavioral testing. (I don't necessarily mean an approach like BDD, but the fundamental, bare bones notion of behavioral testing.) It's also related to property-based testing, as amon mentioned. The difference is less in philosophy, and more historical, and in terms of languages and tooling (not 100% sure, but I think one comes from math/comp.sci tradition, and the other from the Haskell community).

Now, if you're doing TDD, you might not have a fully formed vision of what the abstract behavior should be at the very beginning, so you might start by writing tests for a number of concrete examples and build up from there. That's fine.

But stop from time to time to see if there are any behavioral rules emerging, and try and re-express some of your tests in terms of those. Also, you might realize that some aspects of the original approach don't make sense in light of what you've learned since you've started, and that you need to reconceptualize things - and even delete some of the existing tests, and make new ones. That's fine. And sometimes, you'll feel as if your code is fighting your tests - e.g. the "arrange" part of the test might be quite long and involved, requiring a lot of setup before you can actually execute the test. That's your test telling you that your code knows too much about the other parts of the system - it's coupled to too many things and has deep dependencies. Such a test will easily break when something seemingly unrelated changes - but guess what, so will your code. In that case consider changing the design.

So, in summary: think of having behavioral tests as the goalpost, and try to move closer to it from time to time.

5

You are testing different aspects of the system under test, so those should be different test cases. However, you can use other techniques to reduce duplication. For example, you could extract the recurring pattern into a new function. For example:

void assertEncryptionEquals(String key, String plaintext, String ciphertext) {
  assertEquals(ciphertext, Vigenere.encryptVariant(key, plaintext));
  assertEquals(plaintext, Vigenere.decryptVariant(key, ciphertext));
}

@Test
void testConvertLowercase() {
  assertEncryptionEquals("key", "how much", "xky cqex");
}

@Test
void testConvertUppercase() {
  assertEncryptionEquals("key", "HOW MUCH", "XKY CQEX");
}

@Test
void testConvertMixedcase() {
  assertEncryptionEquals("key", "How MucH", "Xky CqeX");
}

@Test
void testConvertMultiline() {
  assertEncryptionEquals("key", "how\nMuch", "xky\nCqex");
}

Note also two other changes to your style of testing:

  • the examples are very short, making it easier for humans to understand the correct behaviour
  • the input/output data is spelled out explicitly – no use of automated transformations like calling toUpperCase(Locale.ROOT) which may obscure bugs if it doesn't do what you want it to do

If you are testing multiple examples of the same aspect, then you could consider parametrizing tests, or looping over multiple examples.

If you can phrase your test cases in more generic terms (rather than by specifying concrete examples), a style of testing called property-based testing might be useful. In a proptest, you specify a property that the system under test should satisfy, and a strategy for generating examples. The test framework then tries out multiple random examples in search for a counterexample. For example, a proptest to demonstrate that key case doesn't matter might look like:

@Property
void keyCaseDoesNotMatter(@ForAll String key, @ForAll String plaintext) {
  assertEquals(
    encrypt(key.toLower(), plaintext),
    encrypt(key.toUpper(), plaintext));
}

This kind of violates my above recommendation to spell out all examples explicitly, but here the focus is not on showing a simple example that you can debug in your head, but on doing a fairly thorough search through the potential input space in order to find potential edge cases you didn't think of. When such an edge case is found and the proptest fails, it makes sense to convert that edge case to an explicit example-based test.

2
  • In the proptest, how does the test framework choose what strings to put, considering there are millions of possible combinations and permutations of letters in a String, and many could be invalid (like including numbers)?
    – Marvin
    Commented Jul 30, 2023 at 19:18
  • 1
    @Marvin if not all Strings are valid inputs, you can use a constraint narrower than @ForAll
    – Caleth
    Commented Jul 31, 2023 at 8:21
4

Should these be in a single test case as a parameterized test because I am testing the same method - decrypt or encrypt - or should these be separate cases as I have them now?

No. These might best be implemented as parameterized tests, or they might best be implemented as distinct fixed tests with a bespoke name for each, but the fact that you are testing the same method isn't terribly important in choosing which approach you use.

TDD doesn't particularly care whether or not you are using parameterized tests - "red bar" and "green bar" work the same way in either case.

Most of the real, lasting, difference between the two is how things appear in your test report. For instance, does mixed case multi line need its own name in the test report? That's going to depend on audience - who is looking at the report, what are they trying to achieve when they look at that report, and so on.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.