Software Engineering Stack Exchange is a question and answer site for professionals, academics, and students working within the systems development life cycle. It only takes a minute to sign up.
Sign up to join this community
I'm looking for assistance in figuring out the best approach to managing User Acceptance Testing (UAT) and how users raise defects/bugs - I've been in the situation before of having to deal with a free for all in that every small error is raised as a bug which takes valuable time from actual bug fix whilst I sift through lots of invalid defects/bugs.
Don't get me wrong, i'm not trying to short shift users by saying "not a defect" but trying to make the process smoother so that things get fixed.
First of all, if you aren't yet using an issue tracker tool, get one and make it accessible to users. This way they can enter their bugs into it, rather than pestering you directly via emails or phone calls. You should of course train them to use the tool correctly - this may be a significant initial investment, but it will quickly pay off.
Let users freely report anything they consider a bug, but have them assign severity and/or priority to each bug. This allows you to focus on the most important / urgent ones, but still keep track of all. All decent bug trackers have built in support for these properties, and allow you to query / filter / order bugs according to them. In case the severity or priority of a bug is incorrect, you can discuss this with the user and modify it. If some users often report issues which aren't real bugs, or aren't reproduceable, you (or your management) may need to discuss this with them, and/or - again - educate them on how to document items to make them usable in the development process.
A change control board (CCB) would be helpful to shield the developers from the user requests. A CCB is a group (although it could be an individual) that meets on a regular basis to review the latest submitted defects, determine their impact, prioritize them, and in some cases, assign them to a member of the development team.
The CCB would review all of the defects and feature requests submitted since the last meeting. For each one, they would determine if it's a valid request. If it's not, it would get closed with a reason why. If it is valid, it would be assigned a criticality and perhaps a milestone for incorporation or deadline date. The development lead at the CCB might even assign the task to a specific developer, depending on the process you are using.
Developers only need to care about the defects and feature requests approved by the CCB, and not all defects submitted by the user. The CCB would also be responsible for transforming user requests into the expected input to the development team such as a formal bug report, a prioritizied user story, or an updated requirements specification.
Depending on your situation, it might not be possible to provide your users/customers with access to your issue tracking tool. In such an instance, you would want another standardized method for everyone to submit reports with all of the information the CCB would need to turn it into a bug report or feature request in the issue tracking tool of your choice. Quality assurance would most likely still be submitting right into the issue tracking tool that you are using, but they would be reviewed, prioritized, and assigned by the CCB before a developer starts working on it.
I think the way to answer this question, is to take a step back, and define what is your development process. If you are using Scrum, or some other agile development methodology which has a defined release schedule, and a process for determining what goes in each release, you can push off dealing with the bugs until the next release planning session. (Of course, any "show stopper" bugs have to be fixed immediately.) The release planning session should be about prioritizing the bugs, together with any new feature requests, and deciding what gets in, what gets left out. If you are doing fairly quick release cycles (like every 1-2 weeks), users will get the confidence that the important stuff is getting fixed quickly.
What Peter said above, about having an Issue Tracking system, which allows users to capture the bugs, and set priorities, is critical. However, that is only the first step, since the bug fix won't get into a release unless the "team" agrees on it.
This is actually a great question, with no simple answer. Whatever solution you use needs to fit your organization.
You might find this link interesting reading, as a starting point for more investigation on what methodology would work best for you.
