Identify Enhancements early as a QA

Question

I am having a problem currently where product managers are raising improvements as bugs. Also, they expect us testers to catch them early. Our company doesn't have any requirement document and as QA we collect the information by brainstorming and talking to product managers. Recently the QA had a meeting with the manager and he is expecting us to catch these improvements or what they call as bugs early.

Example: in our web based application, after importing something, a message displays saying that "The "something" has been imported" and with that a progress bar will display below the message. The progress bar has a mix of rainbow colors. Now the bug that was raised was "Too many colors on the progress bar". Now my question is as a tester how do I catch those kind a things. Different people have different views and in my view, that seems to be ok.

What they say is we need to think as a customer and do the testing. I don't get it, is he implying that if I think as a customer I can find issues as the above example? Yes, we are testing based on how the customer will use the product, but how do we catch improvements early?

Answer 1

Product managers, end users or a product owner are usually the source of enhancements, rather than developers or testers. The project might benefit from usability testing in addition to QA testing.

"Catching bugs" is certainly a job for Quality Assurance. Whether issues are considered bugs is a matter of requirements. Developers and testers should be implementing the system according to requirements. Design requirements are typically in the form of mockups, which could be a PDF or image of what the final product should be.

If the mockup shows a green progress bar, and the developer made it rainbow colored, I would agree this is a bug. If there is no mockup, testers have nothing to judge that a rainbow color is a bug or enhancement. This is in conflict with the premise of your question. You cannot "catch bugs earlier" if you don't have proper requirements, whether they are in written form or a graphical mockup, because you have nothing to compare the application to.

Our company doesn't have any requirement document...

The first step towards "catching bugs earlier" is having requirements. Which leads to the second part of your sentence:

... as QA we collect the information by brainstorming and talking to product managers.

Product managers should be gathering and organizing requirements. This is not the job quality assurance is meant to perform. It appears the project is missing someone to fill the role of business analyst or product manager — the very people who want you to "catch bugs earlier" are the people who should be responsible for gathering, organizing and communicating requirements. Design requirements need mockups, and for that product managers will likely need a graphic designer.

To be clear, there is absolutely nothing you can do to fix this situation as a tester. The challenges you face are due to a lack of product management and designers. There is no testing technique or methodology that can fix this. Unfortunately this is more a workplace or process issue, and that requires management to make changes — not QA.

Answer 2

This question describes (at least) three different problems:

1. How to evaluate if certain change requests have to be classified as bugs or improvements?

2. How to priorize style improvements among other improvements (regardless of how they are called)?

3. How to deal with managers imposing they know better than you how to do your job and abuse the "bug classification" for trying to give their ideas a higher priority?

To solve #1, I think you should discuss with your team what categories you will be using in your issue tracker for what purpose. The category "bug" is usually restricted for issues where a certain functionality in the software cannot be reached any more, or does obviously not work as intended or described in the manual, or where a user interface gives a very wrong impression to the user of what should happen. If you get consensus about this with the team, it should be clear that stylistic issues usually don't fall into that category.

For #2, you need to know your product well, the market and user base you are aiming and the overall goals your company has with the product. For certain software products, ergonomics, good "look-and-feel" and consistent colors and style can indeed be a top priority, and QA people should put a focus on this. For other products, working on these aspects may be less important than working on others. This may be something you need to discuss with your superiors.

It may be also be a good idea to learn some foundations on ergonomics, if you haven't done already (ISO 9241 may be a good start). "Too many colors" can be indeed an issue which can be so distracting for a user that this should be changed. Of course, there is no hard-and-fast rule where to draw the line, and definitely none which fits to every kind of software.

#3 is more a workplace issue, but I guess if you get yourself through the discussions for #1 and #2 and settle the goals, you have all the arguments on your side to deal with question #3. But don't exclude the possibility your manager could actually be right: maybe they did point you to a problem which could be classified as a severe UI issue.

Answer 3

In my opinion, there is no fundamental difference between a "bug" and a "improvement". Both should be prioritized against other issues, and both can span the full range of severity. It is the job of QA to ensure that the product meets the target quality, or at least inform management about the current quality and let them decide if it meets the target. If "Too many colors on the progress bar" is reported as a bug or improvement is irrelevant, the important thing is if it should be fixed, and if so, when.

When considering if something is a bug or not, one guiding principle is "does it bug you?". So do you think the colorful progress bar fits the application? You should in principle report any issue that you think will affect the quality of the application. But in practice it is often useful to report the most serious issues, it might not be relevant to report poor color choice if every other button crashes the application. We have a saying that the number of reported bugs does not decrease as testing progresses, but the severity does.

Any design documents can be useful input when considering if something is a bug or not, but this should be treated as guidelines, if the UI design guidelines specify a colorful progress-bar for a serious business application, it is perfectly possible that the guidelines are wrong. If your company does not have such documents, you can look at other comparable applications, guidelines for the platform you are using etc. It can also be useful for QA to be somewhat involved in all stages of product development, to be able to catch potential issues early.

It might be useful to put on different 'hats' or perspectives when testing. What is the initial impression? How easy are common tasks accomplished? How does it compare to the competition? What would a novice user think? How about an experienced one? Does it look nice visually? Is it reliable? Does it work on different devices? Does it work well for a German customer? How about a Japanese one? How about some kind of visual impairment? etc...

Another important thing is to communicate. It you are uncertain about some issues it might be effective to make a list of them, and go to the product owner, UI designer, developer, or other stake holder and say, "Hi, I'm concerned about some things, do you mind taking a look together?", instead of blasting the bug-tracker with a ton of bugs. If the application is old it is likely a whole bunch of things that are bad, but to costly or unimportant to fix. It might take some time to get familiar with the product and business to get a better feel for what is important or not.