0

In most definitions of negative testing, the idea is that we test outside what is specified/expected and it is highly related to robustness.

So basically, if the behavior for such conditions is defined, I would be positive testing as we verify behavior that is specified, isnt that so?

An example requirement:

REQ1: Name is a text field with maximal length of 32 chars.

In this case, I could try neative test cases with numbers, special characters etc.

However, what about this:

REQ1: Name is a text field with maximal length of 32 chars. If a non-alphabetical character is entered, except for a space, a message "Incorrect character entered" is shown.

In this case, I would say that now those tests are positive as I verify the spec.

I ask because I have read the following, in a book Fuzzing for Software security testing:

in a login feature..a positive tests would consist of trying a valid user name and a valid password. Everything else is negative testing.

That does not seem to be right to me. Usually the behavior for wrong login is pretty well defined and specified so strictly speaking, again I verify described behavior.

Is my understanding correct?

To support the definition I mean, I quote a few books below:

Positive testing is done to verify known test conditions and negative testing is done to break the product with unknowns.

Another one:

Most systems are designed with explicit and implicit restrictions and constraints. Negative test cases can be derived to test conditions outside of those restrictions and constraints.

2

Usually the behavior for wrong login is pretty well defined and specified so strictly speaking, again I verify described behavior.

In well-written software, ideally there should be no user-facing undefined behavior - in practice it is not possible to eliminate it completely (you cannot predict or control everything that could lead to it), but you do your best to get as close as you can. Even when something unpredictable happens, the software should fail gracefully. In that sense, the behavior in all of these scenarios is well-defined, so it's not about that.

It's about whether it's the main/appropriate use-case, where the user acts non-maliciously and performs actions correctly (positive path), or an "anomalous" use case - where the user makes an honest error, or is malicious, etc. (negative path).

Everything else is negative testing, including wrong user name, wrong password, someone else’s password, and so on.

P.S. Wikipedia says: "Negative testing is also known as failure testing or error path testing". The terminology doesn't really matter here, as long as you understand the concept, and test if the system behaves appropriately in these scenarios.

P.P.S. You said:

If the error/anomalous path is defined in the spec, then it is positive by definition.

The book doesn't define it in that way. Maybe you have some other definition in mind, and that's OK, but that just means that the terminology you are using is not the same as the one used in the book, even if you are using the same words.

REQ1: Name is a text field with maximal length of 32 chars.

In this case, I could try neative test cases with numbers, special characters etc.

This assumes that by "negative", they mean the opposite of (or otherwise in conflict with) what was specified, but the meaning they use is not about the wording of the specification but about the usage/path (ideal vs anomalous, as described above). I think that's the source of your confusion.

  • That does not answer the question, unfortunately. If the error/anomalous path is defined in the spec, then it is positive by definition. I agree it is not possible to fully cover that completely, this is question rather about the nomenclature. As for Wikipedia, that is rather a popular resource, I stick to books. – John V Jan 18 at 12:29
  • @JohnV: OK, maybe we are miscommunicating in some way. Your question is "Is my understanding correct?" - my answer is "No - because your premise is wrong.". Could you expand on what's unclear in my answer? Maybe I can find a way to improve. – Filip Milovanović Jan 18 at 12:32
  • @JohnV: I see you edited your comment above to include more info - I understand now what you mean. "If the error/anomalous path is defined in the spec, then it is positive by definition." - by what definition? The book you are citing clearly doesn't use that definition. – Filip Milovanović Jan 18 at 12:40
  • Not according to this book, but according to most others (and partially also it is a common sense - if that is specified, then I cannot handle it as unexpected and undefined input. – John V Jan 18 at 12:59
  • 1
    @JLewis At first, I thought the same. But more technical books put it that way (and it makes sense): positive test cases are obviously linked to requirements. If that error path is defined, it is a requirement, and thus must be tested to ensure full requirement coverage, meaning it is a positive test. – John V Jan 18 at 13:11
0

You quote a spec:

REQ1: Name is a text field with maximal length of 32 chars.

where is the rest of the spec? do they want you to truncate it? write it to an error log? play annoying music? note it as a failure? popup a message box?

You mention 32 characters and text. The spec doesn't mention not allowing numbers or special characters, plus the spec should tell you how to handle them.

Once they tell you what to do then you can design the proper test.

The book mentions:

in a login feature..a positive tests would consist of trying a valid user name and a valid password. Everything else is negative testing.

This assumes that the system specifications has told you how to handle the logins. But that specification has to address how many failures before lockout, are resets by email or security questions allowed, and what the error messages should be. Those specifications are not negative testing. You are expecting a specific behavior.

  • But that is exactly my point. In the first case, the requirement is vague, only allowing 32 chars - so negative test cases would be trying numbers etc. However, in the other example, the behavior for such inputs is specified. – John V Jan 18 at 13:05

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.