Q: In agile, do we earn points for stories if they had defects? And what do we do with the defects we found?
- Decide "fix this iteration" or "add to backlog"
- Count points if story was still accepted
- Think about defect prevention
1. Decide "fix this iteration" or "add to backlog"
If the team thinks it can make the fixes now, let them. If the fixes can't be done before the iteration demo, estimate the sizes of the fixes using story points just like your user stories and add it to your product backlog. The defect can now be prioritized along with the remaining user stories. The product owner could decide to leave the defect there in favor of a more valuable story.
Remember, your product backlog can have: stories, epics, defects and even acceptance criteria.
2. Count points if story was still accepted
The defect can be categorized into four severities:
- Sev 1: user story can't be accomplished
- Sev 2: a work around exists, but the product owner rejected the user story as complete
- Sev 3: a work around exists and the product owner accepted the story as complete
- Sev 4: the defect was fixed in the same iteration it was introduced
So for Sev 1 and 2, the team would get no points. For Sev 3 and 4, the team would get full points.
Notice that the total product backlog size might go up when the defects are added to it, or some other user stories would have to be removed from scope to keep the product backlog size constant.
A nice quality metric here will be % of story points spent on defects vs stories.
3. Think about defect prevention
The purpose of test is not to find defects so that the developers can fix them. The purpose of test is to report on the quality level of the solution. To low a level and the product is considered bad. It could damage our reputation or even be harmful to our clients.
If you repair every defect found in test, there are dozens more defects that were NOT found that you will never repair. The number of test paths in your software is equal to 2n where n is the number of branch points in your code. So with just 3 branches, you have 8 test paths. With that many paths, testers will NEVER "test it all." And since each pass through a loop is a new path, if you have an "indeterminate loop" that can go 0 time to any number of times the user wishes, the number of test paths is now infinite.
Defect density suggests us that if we find 10 defects in 100 test paths, we will find 10 defects in every 100 test paths. If there are a million test paths, there are 100000 bugs. If test finds 100 of those and you fix them, you only improved quality by 0.1% or pretty much not at all.
The only way to prevent defects is to classify your defects, find a root cause, repair your process or use an automation to remove that type of error. Next iteration, test for that category of error again. If no new instances of that category appear, you can now use defect density to suggest that those defects also do not appear in the 999900 test paths you didn't explore too. Now THAT is a quality improvement.
Developers need to stop feeling OK about defects being found in their team's work. When a defect is found, the whole team needs to categorize them and find process or tool solutions that will prevent those types of defects from being introduced into the solution in all further iterations.
For example, if 10% of the defects found were due to unitialized variables and a tool is purchased that checks for that in the next iteration and at the end no such errors are found in new code, you have removed 10% of the defects from all of the new code including all the paths not tested.
Submit your Agile Antswers questions to firstname.lastname@example.org.