I have worked as a manual and automation tester.My experience involved working on multiple projects from varied domains. This enabled me gain more knowledge and insights into the different QA processes and models followed by different teams depending upon their development cycles for example Waterfall, Iterative, Rapid Application Development and Agile. Each project and its development life cycle has helped me analyze the real issues and how and where the real improvements can help irrespective of the development model.
Based on my experience I am going to write about one of the most effective process “Defect Analysis”. If done at right time and with right perspective, it enhances the team deliverables and eventually the product quality.
Challenges faced by the QA team
1.Quality of defects.
2. Lengthy defect cycle before closure.
3. Number of defects
4. Quality of code
5. Pressure on the team towards the end.
6. Lack of consistency in QA and development deliverables.
Proposal based on the experience
Historically Defect Analysis is process done after the release. Lot of metrics are derived from all the defects to evaluate the QA and development team performance.
Proposal is to do Defect Analysis during QA cycle. To ensure ongoing quality improvement when it matters.
Project Phases for Defect Analysis
* Phase I- Initial project development. For agile projects we can say sprint 1 of QA
* Phase II - Integration testing stage
* Phase III - Regression cycle
Stake holders in Defect Analysis
* QA lead
* QA Manager
* Development lead
* Project Manager.
Defect Analysis when done during the release helps -
* understand what is going on
* what can be expected
* what needs to be worked upon
The following sections will cover the phases mentioned above in more detail.
Phase-1 - Project development just started, made builds available to QA for testing.
We have to Do Things, if done right the first time itself it saves lot of trouble later for all of us.
Lot of defects at this point could be because of following reasons
1. No Unit testing done
2.Requirements not frozen or not clear between QA and Development team Action
Development Lead need to act here. If the defects are valid, lead should ensure
* Code is Unit tested.
* Code reviews are done.
* Development team will catch as many defects at their end by self review.
If there are lot of invalid defects, QA team lead should ensure that there is a process streamlined that ensures that QA team has access to functional specification and is updated every time there is any change in design or functional requirement.
QA Lead too should check that QA members are entering defect in correct format. The least information that should go in a defect should be
1.Clear, precise title
2.Steps to reproduce
3.Test data if any
4.Test environment details
6.Logs, snapshot, etc to enable easy debugging.
7.Correct priority severity.
Having all the information in first go, helps faster closure of the defect.
Testers should read all the defects been logged. [Defect logged by peers. This should be a daily activity.].
This ensures that the team members don't land verifying/debugging affected components and are also spared from logging duplicate defects.This saves time for both the QA team as well as the development team. Duplicate defects introduces time cost. 2.Try new scenarios around the affected area, before and after fix to catch more defects.
Result of action:-
By giving all possible information and symptoms QA helps locating the problem and location which helps developers to get their act faster. More ever QA 's understand of internal work flows of the product gets clear.
Phase II - Half way through Project development cycle.
Defect count is definitely going to be high at this time and that means everybody is in action.
This is the time when Development Lead and QA Lead should ensure that timely action is taken on the defects. Usually Development Leads ensures that the blocker, critical, high severity defects are immediately resolved. The minor and normal defects may keep piling. That increases the pressure on the team as the defect count increase towards the end of the release.
“Any strategy like target 2 minor, 2 normal defect per week to each developer ensures application cleanup and helps keep defect count low.”
QA Lead should ensure that QA team is not taking long to respond on defect waiting for any action from their side. The defects should be verified and closed as soon as possible. Immediate defect verification is important because it helps in following scenarios:-
If the defect is to be reopened, the developer who worked on it, has all the thoughts fresh in his mind. Reopening the defect after a long duration is time consuming for the developer to resolve it again.
General testing around the fix, ensures that fix has not broken anything new.
QA metrics on response look great !!
Confirm that every defect has a associated test case, this is to help the QA team working on the next version of the product. In case of tight schedules a test case with brief title should be written. The detailed test cases can be taken up after release.
Result of action:-
1.Defect count continuous to be manageable.
Phase III - Thrashing ,Regression testing
Any Severity 1 defect at this time indicates need of more testing.
* This is sign of more defects in hiding. We need to catch them all.
* After a fix, to confirm that defect and related all subroutines are working as expected and no new defect has been introduced.
Ensure faster closure, pull in free developers to regress bugs and positive test cases.
Thumb rule - Any defect analysis done post release will not be as effective as done before release.
By been proactive and by being on the top of the problem from the start helps both QA and development team deliver quality products.
Next Steps and Recommendations
* All projects following Agile processes should make defect analysis an mandatory activity along with their other sprint exit activities.
* Performance and security defects generally crop up towards the end however if code review checklist include evaluation of code for these two factors as well, it would be like icing on the cake.
* Project Lead should continuously monitor defects to track trends and confirm that team strategies are in-line.
IBM Security Systems - India User Group
with Tags: defect X