Twenty years ago when I started out as a manual tester, the quality assurance team was the gatekeeper. We would decide whether or not a product was ready to ship and it was pretty stressful. What were the criteria for the go/no go decision? We had lots of metrics and more evolved over the years, but ultimately the defect count and severity, as well as “gut- feel” were often the inputs to the decision.
As the saying goes, “we’ve come a long way, baby.” Quality is a whole team concern. And not just in agile development teams. There are lots of initiatives to improve quality like
- Stakeholder feedback, after all if it’s not what they want why ship it?
- Slick, easy-to-use, intuitive functionality
- Requirements traceability
- Test driven development and/or no code is delivered without automated tests
- Continuous integration builds
And any other “fail early” technique you can think of. These really do improve quality. But how are we measuring quality? What metrics matter?
If quality is delivering what stakeholders want, what is the measurement for that? They just say so? Or proof that their requirements are met? Number of “likes” on a mobile app? Does it vary based on who the stakeholders are and what is being delivered? For example, when the stakeholder is a mobile app user getting something for free, is quality only that it basically works? Maybe that there’s no really bad problems (like crashing the phone)? Bug fixes are delivered in a matter of minutes? For any software that’s not free, and certainly for mission-critical software, organizations must have quality criteria. But what are they and how are they tracked?
Do those two circumstances cover your situation? What is your organization’s definition of quality and what are you using to measure it? Are you tracking things like
- Test coverage and/or code coverage?
- Defect counts and severity?
- Requirements rate of change?
- Requirements implemented and tested?
How does your organization reach a go/no-go decision on your
software? Are there numeric thresholds that have to be met and what are they? Please drop me a line or comment on this to tell me how your organization defines and measures quality.