In agile environment, how is bug tracking and iteration tracking consolidated.

Question

This topic stemmed from my other question about management-imposed waterfall-like schedule. From the responses in the other thread, I gathered this much about what is generally advised:

• Each story should be completed with no bugs. Story is not closed until all bugs have been addressed. No news there and I think we can all agree with this.
• If at a later date QA (or worse yet a customer) finds a bug, the report goes into a bug tracking database and also becomes a story which should be prioritized just like all other work.

Does this sum up general handling of bugs in agile environment?

If yes, the part I'm curious about is how do teams handle tracking in two different systems? (unless most teams don't have different systems).

I've read a lot of advice (including Joel's blog) on software development in general and specifically on importance of a good bug tracking tool. At the same time when you read books on agile methodology, none of them seem to cover this topic because in "pure" agile, you finish iteration with no bugs. Feels like there's a hole there somewhere.

So how do real teams operate? To track iterations you'd use (whiteboard, Rally...), to track bugs you'd use something from another set of products (if you are lucky enough, you might even get stuck with HP Quality Center). Should there be 2 separate systems? If they are separate, do teams spend time creating import/sync functionality between them? What have you done in your company? Is bug tracking software even used? Or do you just go straight to creating a story?

Answer 1

"...finds a bug, the report goes into a bug tracking database and also becomes a story which should be prioritized just like all other work.

The question is, should bug tracking and feature tracking be different, and can you use a single system to do both as well as schedule iterations/milestones/etc...

In terms of a "pure" Agile approach, you allow your team to use any combination of tools and processes that works well for them. Sure, you may find a single product that does everything, but perhaps it doesn't do some things as well as you'd like. If you run multiple systems, you need to determine just how integrated they need to be, and if any integration is needed, find the means to do it, and decide just how much information needs to be duplicated. It all boils down to a cost/benefit situation, so naturally any system employed needs to take into account the impact on a team's overall efficiency.

Where I work, we use a Redmine system to track bugs and features in a single system for multiple projects, with links between each project where dependencies exist. We create labels that relate to milestones, which for us are effectively long iterations that may range anywhere from a matter of weeks to a matter of months. For individual tasks and features, we tend not to track iterations too closely, so we have no need to worry about burn-down charts, white boards, sticky notes, feature cards and all of that stuff, as we've found that for our specific needs, some of this stuff is overkill. Each feature itself effectively represents small iterations of between 2-10 days duration, and for those that might care, we log our estimates of time versus actual time for later analysis. This may sound a little ad-hoc, but works for us and ultimately our real measure is working code within a series of time frames.

I suppose if we decided to employ another more formally "regimented" methodology, we might consider a tool to aid in tracking progress, but with what we currently have invested in our present method, we'd probably feed at a minimum the short feature descriptions and time data to another system, unless someone has developed a module for Redmine that does what we want it to, or if it became really important to us, we might create the Redmine module ourselves to avoid any nasty integration headaches that might concern us.

Answer 2

You can almost never realistically say with 100% certainty that no bugs exist. Even if every test case was meticulously thought out, tested for, verified, there will probably be that one situation that somebody didn't think of. Customer identified bugs happen, they aren't the end of the world, but when they do happen I make sure to create them in a user story and give them the highest priority.

I use Rally and love it. Typically throughout the sprint, testing of features is done as they are completed by QA resources. When QA finds bugs they usually write them up as Defects in Rally and then the developers attempt to resolve all the defects before the end of the sprint.

Ideally all Defects should be resolved, however for small or trivial Defects it may not be necessary to hold the sprint back by not Accepting the User Story. QA should make the judgement call. Do they accept the User Story with the open Defects or not?

Once the sprint is accepted any remaining Defects can be elevated to User Stories for the next sprint.

Another fallacy of your line of thinking I noticed is that you delineated QA as a seperate entity outside the development team. This is an inherently Waterfall mindset as you just assume that a large Testing phase must occur before release. This shouldn't be so. Testing should be occurring alongside the development team DURING the sprint and they should work closely with the developers preparing their test cases and interpreting their test results.

If Agile is done right, the sprint will be fully accepted by the end of the sprint meaning that it is ready for production release (It doesn't necessarily mean that every sprint should be a release, just that it should be production ready if the need arised for it).

Answer 3

Where I work we also have the principle, almost as an "article of faith", that defects are not raised during an iteration but simply that development is not complete until all issues are addressed. An issue can be anyting from a missing requirement to ambiguity in how the technology should be implemented to a good old fashioned bug in the code. Because, in true Agile fashion, our user stories are not specifications but rather "placeholders for a conversation" it is considered to be beneficial that requirements are finalised with Product Owners during development so that neither they or the developers are railroaded into a solution by upfront requirements specifications.

If and when an "issue" is identified, the encouraged practice is to track it by creating a new task on the project radiator board and in Rally. This serves to remind Product Owners to provide requirements, developers to implement it and QAs to incorporate it into their test plan. If an issue cannot be addressed in the time frame for development then it is upto to the Product Owner (not the QA) to accept the story with caveat that the issue will be become part of a new story or raised as a defect.

This is all great and works well on small projects with fairly small co-located teams who communicate face to face on a daily basis. However in my experience it becomes problematic on bigger projects where third parties and/or offshore teams are involved. This is because tasks on the board or in Rally do not have the benefits of formal defect reports, let us remind ourselves of those benefits; 1. A common frame of reference for the issue with a unique identifier (number). 2. Steps to recreate the issue with actual vs expected results so there can be no ambiguity as to what the issue actually is. 3. A searchable repository of known issues so that team members do not have constantly ask around if some purported bug they have found has already been reported. 4. Accountability; if we rely on word of mouth and/or email chains to report issues then things are bound to be missed, ignored, or forgotten especially when issues start mounting up.

As a QA I definitely believe there is room for some method of tracking issues during development. However whenever I try to raise this at work I am usually slapped down and told that "I do not understand Agile" or words to that effect. Interestingly it is usually programmers who raise such objections. This is understandable because programmers do not have the potential headache of reporting and tracking numerous issues. So I recommend that any discussion about this subject involves BAs and Project Managers as well as QAs because it is something we all have to work with on a daily basis. By contrast, a programmers' experience of a defect is usually just as an individual discrete piece of work that they have to fix within a given area of code.

Answer 4

Where I work, we use TFS for source control and work item tracking. Both the stories and the bugs are work items in TFS, as are the tasks necessary to complete either. The code, when checked in, is associated with the work items, and the builds are associated with both the check-in and the work items, so it's clear which builds are meant to fix which bugs.

Answer 5

I think so the best way is to add bugs to your sprints and release that way you can track your bugs.

I got this idea from a new tool Yodiz it allows you to add bugs to sprints and releases

Answer 6

I think the solution of this problem is to let team create Issues/bugs within user story, just like tasks.

Yodiz (www.yodiz.com) easily resolve this issue by allowing you to create Issues on Sprint board and assign them to a user story. Also same issue can be seen on Yodiz built-in issue tracker so now you can see the same issue at two places on your issue board as well as on your sprint or kanban board.

You can tread the issue as task and move it along the board just like you move your task from one problem state to another.

Again by doing so you never loose track of your bugs. There are times when testing teams start testing your completed tasks and in those situation it's very help to let you attach issues to user story. In case of Yodiz testers can create an issue on issue tracker and attach (associate) it to a user story and that bug would start showing up in developers sprint board.