5

I ask this because I am interning as a QA Engineer and one of my bugs got rejected by a remote project manager because I performed an unrealistic test.

Basically, the bug involved a large text field to hold notes of 2000 characters max. I performed a test in which I entered a 2000 character long string to see how it would function then (bounds test), I imagined it would wrap around but instead it got cutoff and only the first 163 characters were displayed.

Now if I would enter "normal" length words (longest word in the English dictionary is 41 characters) it works.

Other websites wrap around the string such as FB, Twitter, WhatsApp etc.

Sample picture below. Thats the resulting saved note on top, and the bottom is the actual note.

|improve this question
  • 1
    It's only a bug if the specification says that single-words of 2000 characters must be shown somehow. If you get the biggest words on a German dictionary, does it display the text correctly ? – Machado Mar 24 '17 at 14:54
  • @Machado: who says the text field is used for single words only? And I doubt you find such a limit explicitly described in an average spec. – Doc Brown Mar 24 '17 at 14:54
  • 1
    @DocBrown, that's exactly the point. Who says (specify) what's the behavior of the field ? It's a bug if that's outside the expected behavior. If it's not in the requirements (undefined behavior), should Q&A define it (say it's a bug) or require a clarification of the requirements in order to understand better what should be put there ? – Machado Mar 24 '17 at 14:59
  • 1
    User stories tend to be very broad and general at my work place often lacking explicit definitions honestly. More often than not the QA Team and I have to use our best judgment to define what a bug is. – Pants Mar 24 '17 at 16:28
  • 5
    Quite simply: it's perfectly reasonable to raise this and it would be perfectly reasonable to decline to fix it as not being worth the effort. Whether your team is healthy enough to work in this (productive and pragmatic) manner is another question. The definition of a "bug" is not important - what's important is that your team understands how the system behaves and is then able to decide whether any of that behaviour is problematic . – Ant P Mar 24 '17 at 16:29
8

I'd like to approach this from a non-conventional point of view.

The role of Q&A is to find what's not behaving regarding to what was required. That implies to me that a Q&A analyst must be aware of the requirements as much as the developer or even more.

One of the key aspects of the requirements is how a software must behave under the boundaries. If this is not defined, it's not the role of the Q&A to define it, but to seek clarifications from the Domain Experts (if this is a domain boundary) or Technical Experts (if this is a technical boundary) on how the software is expected to behave in these cases, and help the team to understand what's going on and to build a better product.

It's not the role of the Q&A to find an allegedly bug, report it and, let's say, forget it ("I did my job, I found the bug, I feel fine with myself, damn you developers do your work well!").

You're part of a team. You're helping building a product, and more often than not you're the last man standing between a release and a user. Understanding how your product must behave is just part of your job. The other part of your job is helping everyone on the team understand that too.

About your concrete case: What's the kind of information is expected in that field ? Is that a comment ? A question ? A BASE64 string to be used as a RSA Public Key ?

Depending upon the type of information you should have there, you may define that it's a bug (for a RSA Public Key, that would not allow you to copy it back) or not (if it's a comment field for English speakers only, it's not reasonable to think it as a bug). But either way, you should push people around to add the definition of the behavior to the formal requirements.

|improve this answer
  • 1
    In other words, be the person who pushes the team to improve their overall awareness of software requirements. – rwong Mar 24 '17 at 15:17
  • I like this answer the most, you make great points but I want to hear from others before I accept this one. Thanks for the input! – Pants Mar 24 '17 at 16:22
3

Rejection for an "unrealistic test" doesn't seem appropriate to me. Boundary testing is a perfectly legitimate way to design your test cases to find issues, and I would expect that QA engineers would test around boundaries on a regular basis.

How this issue is treated depends on your requirements. If you have a requirement that you need to be able view all of the characters/text entered in the text field, the software does not meet the requirements. However, since this is such a rare condition, it's likely to be a low priority issue. Alternatively, the issue could perhaps be in the requirements themselves and how the system is documented as behaving.

I wouldn't expect your issue to be rejected, but simply prioritized very low.

|improve this answer
2

First thing to consider:

  • does entering more than 163 characters cause severe issues like a buffer overflow? Then it is a clearly bug (and I hope that remote project manager gets fired).

But lets assume that is not the case:

  • what is the specific text field used for? Single words like a user name? Then it is probably an acceptable limit and no bug. Whole sentences with potentially more than 163 characters? Then it is a bug. You don't know beforehand because it is used for different purposes? Then it should be treated as a bug, since one should expect use cases where 163 characters are not sufficient. Short notes, where an artificial limit makes sense? Then the limit should be somewhere in the spec, indeed.

Even if there is no explicit limit mentioned in the spec, for most real world cases, you can deduce a sensible limit from checking the meaning of the text field.

Finally, lets assume the limit of 163 is high enough for the intended use case, then I still would ask:

  • is the limit transparent to the user?

Does the page / form prohibit entering of more than 163 characters immediately when the user tries to type character number 164, or does it accept 2000 characters and cuts off anything beyond 163 afterwards or (worse) secretly? If that's the case, then I would not call this issue a bug, but "unergonomic behaviour" or "not user friendly". Your team should have a policy how to deal with such issues (they will probably get a lower priority than "real" bugs, but if your team wants to produce high quality software, I recommend not to assign the lowest possible priority).

|improve this answer
-1

The test is fine. No test should be rejected. The only question is whether its a pass or fail.

Now correct me if I'm wrong but you say this test fails, not because the note isn't stored and displayed, but because the text wrapping doesn't break very long words?

I would have to ask whether the specification for the feature defines the type of word wrapping that should be used? If it doesn't then the test passes.

|improve this answer

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.