Regarding QA/Testing, can a bounds test be unrealistic? (See image for clarification)

Question

I ask this because I am interning as a QA Engineer and one of my bugs got rejected by a remote project manager because I performed an unrealistic test.

Basically, the bug involved a large text field to hold notes of 2000 characters max. I performed a test in which I entered a 2000 character long string to see how it would function then (bounds test), I imagined it would wrap around but instead it got cutoff and only the first 163 characters were displayed.

Now if I would enter "normal" length words (longest word in the English dictionary is 41 characters) it works.

Other websites wrap around the string such as FB, Twitter, WhatsApp etc.

Sample picture below. Thats the resulting saved note on top, and the bottom is the actual note.

Answer 1

I'd like to approach this from a non-conventional point of view.

The role of Q&A is to find what's not behaving regarding to what was required. That implies to me that a Q&A analyst must be aware of the requirements as much as the developer or even more.

One of the key aspects of the requirements is how a software must behave under the boundaries. If this is not defined, it's not the role of the Q&A to define it, but to seek clarifications from the Domain Experts (if this is a domain boundary) or Technical Experts (if this is a technical boundary) on how the software is expected to behave in these cases, and help the team to understand what's going on and to build a better product.

It's not the role of the Q&A to find an allegedly bug, report it and, let's say, forget it ("I did my job, I found the bug, I feel fine with myself, damn you developers do your work well!").

You're part of a team. You're helping building a product, and more often than not you're the last man standing between a release and a user. Understanding how your product must behave is just part of your job. The other part of your job is helping everyone on the team understand that too.

About your concrete case: What's the kind of information is expected in that field ? Is that a comment ? A question ? A BASE64 string to be used as a RSA Public Key ?

Depending upon the type of information you should have there, you may define that it's a bug (for a RSA Public Key, that would not allow you to copy it back) or not (if it's a comment field for English speakers only, it's not reasonable to think it as a bug). But either way, you should push people around to add the definition of the behavior to the formal requirements.

Answer 2

Rejection for an "unrealistic test" doesn't seem appropriate to me. Boundary testing is a perfectly legitimate way to design your test cases to find issues, and I would expect that QA engineers would test around boundaries on a regular basis.

How this issue is treated depends on your requirements. If you have a requirement that you need to be able view all of the characters/text entered in the text field, the software does not meet the requirements. However, since this is such a rare condition, it's likely to be a low priority issue. Alternatively, the issue could perhaps be in the requirements themselves and how the system is documented as behaving.

I wouldn't expect your issue to be rejected, but simply prioritized very low.

Answer 3

First thing to consider:

• does entering more than 163 characters cause severe issues like a buffer overflow? Then it is a clearly bug (and I hope that remote project manager gets fired).

But lets assume that is not the case:

• what is the specific text field used for? Single words like a user name? Then it is probably an acceptable limit and no bug. Whole sentences with potentially more than 163 characters? Then it is a bug. You don't know beforehand because it is used for different purposes? Then it should be treated as a bug, since one should expect use cases where 163 characters are not sufficient. Short notes, where an artificial limit makes sense? Then the limit should be somewhere in the spec, indeed.

Even if there is no explicit limit mentioned in the spec, for most real world cases, you can deduce a sensible limit from checking the meaning of the text field.

Finally, lets assume the limit of 163 is high enough for the intended use case, then I still would ask:

• is the limit transparent to the user?

Does the page / form prohibit entering of more than 163 characters immediately when the user tries to type character number 164, or does it accept 2000 characters and cuts off anything beyond 163 afterwards or (worse) secretly? If that's the case, then I would not call this issue a bug, but "unergonomic behaviour" or "not user friendly". Your team should have a policy how to deal with such issues (they will probably get a lower priority than "real" bugs, but if your team wants to produce high quality software, I recommend not to assign the lowest possible priority).

Answer 4

The test is fine. No test should be rejected. The only question is whether its a pass or fail.

Now correct me if I'm wrong but you say this test fails, not because the note isn't stored and displayed, but because the text wrapping doesn't break very long words?

I would have to ask whether the specification for the feature defines the type of word wrapping that should be used? If it doesn't then the test passes.