Have you heard people talk about defects and then you hear people talk about bugs? Some of us think it’s the same thing. It is most probably the most confusing thing but it is not the same.
My terminology for a defect and a bug is as follows. But you don’t have to agree, you can create your own. Although keep in mind, there is a difference between these two and you should be able to differentiate.
And then there is another category of issues that are caused by recent developments. I call those nasty little things “Escaping defects”.
Bug: a flaw found while testing a User Story (US) that prevents the US from being closed.
Action items for a bug:
- Fixed immediately during the sprint.
- Minimum documentation.
- No Sizing.
A bug is a classic just fix it situation. If the bug is not fixed you can’t close the US and you don’t meet the acceptance criteria. We never size bugs. The original US has all the size needed for designing, documenting, coding, automation testing, bug fixing and whatever else you may need to set it to “DONE”
It is up to your team to decide to document bugs or not. If you have excellent communication then you may get bugs fixed using collaboration and you can skip the paperwork. But if your team have more remote team members, different time zones or hectic work environments then you can adopt for more formal documentation by creating a bug and linking it to the original story.
But you should never ever count bugs as a metric for code quality or QA effectiveness. This will only create a barrier between development and QA teams. Let the team, together figure out what quality means and hold everyone accountable for the quality of their work. The only thing what matters is what when a US is closed at an end of a sprint it should be bug free. After all, we won’t want to send a new feature in to the world when you know it’s flawed. The goal at the end of the sprint should be zero bugs.
Defect: a flaw or failure found in production which prevents the software from functioning. Or, a condition in which the systems actual behavior doesn’t match the users (developers, PO’s) expectations.
Actions for a defect:
- Document a support issue.
- Diagnose why it happened and document to improve.
- Find a solution.
- Prioritize and add to backlog if it can wait.
- Pull in to sprints based on priority.
Defects are pre-existing issues fond in production. Generally, the older the product is the more defects you will have. But the nature of the bugs can be controversial.
- Users try to use the product in ways it was not designed for.
- The load is too much for the product to handle.
- Sometimes bugs which were there since inception emerge later through a different workflow.
- The system is so flexible users do things that you never expected them to do.
- It’s just a defect.
We can never aim for zero bugs unless the system is very small or very new. There will always be bugs. Especially as the software grows. It’s inevitable. Bugs are prioritized by the product owner and sized by the team. Always consider ROI on fixing a bug or implementing a new feature. We need to know details such as,
- How many users experienced that issue?
- What is the severity?
- How often does it happen?
- What happens if you decide not to fix it?
- The risk of it being there?
We pull bugs from the product backlog and take them in to the sprints based on priority.
I believe measuring the number of bugs maybe useful. Not in measuring development quality but how much do users expectations differ from our understanding of the product.
Escaping Bug: a defect introduced in recent development but missed in testing.
- Treat as a defect.
- Label as “escaping defect”.
- Perform root cause analysis.
- Address the root cause though quality, communication, technology, automation, process improvements, etc.
In Sprint A, the team works on a story and in Sprint C the user reports the issue. After analysis the team figures that it was introduced in Sprint A. This happens all the time. We fix something, we break something. Ideally it should have been caught in Sprint A but it was not, hence we call these Escaping Bugs. We tag these bugs in JIRA or TFS with the label “escaping bugs”.
The team will analyze the bug and come up with a mechanism to improve their process going forward.
Now the number of Escaping bugs is a good measure for the team’s quality. But aiming for zero escaping defects might not be realistic especially for legacy systems, large code and code written by developers not in the company any longer. But I think tracking the number across sprints should demonstrate improvements based on learning, automation, process improvement and refactoring.