Today's software development teams are wrestling with the increase in development costs associated with maintaining and modifying software. They are constantly challenged by the need to increase quality and deliver solid products to the business customer while reducing the overall cost to develop and maintain the software applications delivered. This has been the source of a great deal of tension in some organizations, with poor quality at the heart of the rework problem.
IBM® Rational® quality gates can help us, as developers, understand and control our software development lifecycle practices. The idea behind quality gates is that we are looking for some kind of green light before moving to the next phase or subphase in the software development lifecycle. As the checkpoints that we pass through between phases of a project, they help us improve quality by helping ensure that we are following best practices and then comparing measurements of results against previous projects. This level of scrutiny helps us spot trends in the software that we are developing and thus address quality issues earlier. Catching defects early makes them much cheaper to fix, which is likely to result in a reduction in your IT operating cost.
Thus, quality gates help increase quality yet, reduce overall cost. This solution is growing in popularity among software development practices.
If I told you that I got 25 MPG (miles per gallon) last week driving around town, would you say that is good or bad? Without any benchmarks, it is hard to say. What if I told you that in the previous weeks, I got between 18 and 22 MPG? Now, are you curious about what I did my car last weekend that increase my mileage? The effect of car care best practices, such as changing your oil or checking a tire's air pressure, is not implicit unless we have measurements to quantify what we gained or lost.
Software development is no different. Without a way to measure and record the results, it is difficult to understand whether a change had a positive or negative impact on the software practice. How do you know if the training course that you sent your developer to last week was any good? How can you see if the outsourced project compares to the quality of the code that you're producing in-house? The answer lies in how you establish your metrics, and the measurements that are gathered from your quality gates. In this way, you can see the impact that shortened deadlines have on quality; you can decide, based on meeting established metrics, if it is worth moving a project forward.
Typically, organizations have one quality gate sandwiched between the build phase and the delivery phase: the test phase. Organizations look to the test phases for final approval before deploying products to their customers. This places considerable pressure on the testing group to document every defect before deployment, which is unrealistic. Companies cannot rely on just one final test or quality gate for overall quality. They need to be more flexible, so that before they proceed deep into the development process they start thinking about quality.
Focusing on quality early in your software lifecycle is vital to increasing quality and decreasing costs, and Rational quality gates offer checkpoints along the development phases. Simply employing best practices is not good enough. You must employ measurements, benchmarking, and re-measurements to help ensure that your software demands are keeping pace with your quality needs. Having best practices and a good process is not enough if your stakeholders cannot understand how to use these practices in their decision-making processes. Best practices are frequently overlooked when left to the practitioners to evaluate. Early in my software development career my management team asked me to ensure that all code have a primary, author, and a secondary person associated with it for review — typically called a peer code review. In a peer code review the secondary author is responsible for reviewing and critiquing the primary author's work. While well meaning, the peer code review practice lasted for about a week. After several heated conversations between the secondary and the primary about the primary's code (combined with the lack of any real enforcement behind the process), this "best practice" was dropped.
Software development consists of a number of phases. These phases are essentially the same for all styles of practices, including the waterfall process, the Rational Unifies Process, OpenUp, and an agile approach. In each practice, there is a requirements phase, design/code phase, build phase, test phase, and release. More importantly, all phases could have a quality gate associated with it, because you could use multiple gates for each phase. For example, you might want to have a review quality gate in the middle of the testing phase for the automated test script framework authoring before executing any of the tests. Setting up this quality gates early in your software development lifecycle helps you find defects earlier, where it typically costs less to repair. In most cases, finding a requirement problem in a requirements review is far cheaper than waiting until the application is designed, coded, built, and in testing to find the problem.
The first phase is the requirements phase. A best practice in most software development lifecycle practices is the tracking requirements associated with your test case. This helps ensure that you understand your requirements and it helps provide a good user experience when you run the software. Writing test cases early assists you by identifying and jump- starting the testing process. This best practice allows you to prioritize test cases based on their complexity, risk, and functional area, which will create a quality estimate about how long the testing will take. This also allows time to look for reuse inside your testing and start building a testing framework.
This is where using a solution like the IBM® Rational® Quality Manager tool to capture the test cases can help you meet your quality goals. In order to do this, you will need to capture your requirements using Rational tools. Rational tooling offers multiple tools for tracking requirements. The majority of these tools will work, and Rational Quality Manager has basic requirement tracking capabilities built into the environment. Rational Quality Manager can help you simplify tracing which requirement has a test case associated with it, and the reporting functions give management visibility into the project to gauge the quality of the effort associated with it. Rational Quality Manager assists your process by capturing risk, complexity, and a host of other metadata to allow for prioritizing your testing efforts. Using this information, you can review the requirements to find which requirements are the most automatable. Rational Quality Manager also helps you create the groundwork for reusability. How do we turn this best practice into a quality gate? One way is to have this quality gate rolled up with other quality gates, and reported to management using a the IBM Rational Insight tool or with the Rational Quality Manager solution's built in reporting functions. Using the Rational Insight solution, you can roll up trace coverage with the metadata to gain visibility into reporting the percentage completed on the tasks outlined above.
Figure 2: A quality gate based on requirements test coverage using Rational Quality Manager
The design and coding phase of your software development lifecycle offers an opportunity for a quality gate designed around unit testing. Some software practices have developed a test driven development process, which offers an opportunity for a quality gate. Rational Quality Manager can help capture the metadata surrounding a unit test, additionally referencing the unit test so you can roll up the success of the unit test into your process. This information can be viewed and reported on, helping you identify if your code needs unit-test authoring and how successfully running the unit testing has been. Using the Rational Insight solution, you can roll up these tasks into a measured, benchmarked quality gate.
The building phase offers you another place to impart quality gates into your software development lifecycle. One way to setup a quality gate is to use static analysis to help tie Rational technology into the build process. Static analysis is similar to the spelling and grammar checker built into your Microsoft® Word® program. Static analysis scans the source code looking for common programming errors, comparable to how Microsoft Word scans a document looking for spelling and grammar errors,. More specifically, static analysis is able to look for coding generally concerned with coding style to identify rudimentary bugs. At this point, structural analysis can help identify inter-class dependencies and locate cyclical dependencies, hubs, and other structures. Additionally, structural analysis can assist in measuring the complexity of the software, offering data flow analysis, and reducing performance issues — all from the precompiled source code. The software can also be scanned for security issues, such as cross-site scripting and Structured Query Language (SQL) injection vulnerabilities. Rational offers two products for code review: the IBM Rational Software Analyzer enterprise RSAR and Ounce Lab analysis offers the value of expert code review without the confrontational aspects common to a peer code review process. Both tools can easily tie into the build process and can generate reports about how solid your source code is. These reports can serve as a quality gate, helping you determine if proceeding to the testing phase is realistic or if you should remediate your the source code. These quality gates can be rolled up with other quality gates, such as the requirement quality gate generated using the Rational Insight environment. Imparting quality on this level is good for outsourced projects to help ensure that the source code you are adopting is not too complex and meets your best coding practices.
Figure 3: A quality gate based on Ounce Labs in the build phase
Implementing quality gates during the testing phase offers your team several options. Test gates can be wrapped around almost any aspect of the testing phase. One quality gate you can consider implementing is wrapping the quality gate around your testing framework. This forces a review of the test script to insure reuse throughout the manual test steps. This testing framework quality gate could capture high-level walkthroughs of the test cases searching for reuse. Metrics should be gathered around the amount of reusable test steps and the quantity of monolithic non-repeatable test steps. This checkpoint will allow management to decide if automation should continue or if more time should be spent on authoring the test setup.
Figure 4: A quality gate based on testing execution status using Rational Quality Manager
In the final testing phase, the Rational Quality Manager solution can provide a report about passed and failed test cases, the defects produced, the number of severe defects open, and other statistics. These test results are part of the best practices for improving project quality and can be captured using the Rational Quality Manager environment, and they can be easily updated as your testing changes. This an area where the Rational Insight tool can help you improve your process by compiling this information into one report that can be easily tracked and evaluated by your management.
Facilitating the work done by business analysts, programmers, and testers, while making them more efficient and effective at the same time is vital to a successful software development process. Using the IBM Rational Jazz™ technology to store this input "behind the scenes", you can easily create quality gates using the Rational Insight solution. These quality gates are quite basic: they can be read and understood without meetings, inspections, or assessments this approach works because the Rational tools unobtrusively obtain information about your software development lifecycle without imposing it on your process.
Controlling and understanding your software development lifecycle approach improves when you use quality gates in your development process. Quality gates boost project quality by helping ensure that best practices are followed and measured against previous projects. This scrutiny helps you understand how the software you're developing is trending, giving you an opportunity to raise quality issues earlier in your development process. Experience shows that catching defects early makes them cheaper to fix, helping you reduce your IT operating costs.
For more on getting started with Rational Quality Manager, check out Managing your first project with IBM Rational Quality Manager.
For more about lab management options in Rational Quality Manager and how you can use to handle reporting, read Lab management using IBM Rational Test Lab Manager.
To find more information about IBM Rational Quality Manager and related products, check the Software testing and quality management page on IBM.com.
See the Rational Quality Manager product roadmap on developerWorks — a resource roadmap for all users.
Visit the Rational software area on developerWorks for technical resources and best practices for Rational Software Delivery Platform products.
Explore Rational computer-based, Web-based, and instructor-led online courses. Hone your skills and learn more about Rational tools with these courses, which range from introductory to advanced. The courses on this catalog are available for purchase through computer-based training or Web-based training. Additionally, some "Getting Started" courses are available free of charge.
Subscribe to the Rational Edge newsletter for articles on the concepts behind effective software development.
Subscribe to the IBM developerWorks newsletter, a weekly update on the best of developerWorks tutorials, articles, downloads, community activities, webcasts and events.
Browse the technology bookstore for books on these and other technical topics.
Get products and technologies
To find more information about IBM Rational Quality Manager, see the Rational Quality Manager page on developerWorks.
Download trial versions of IBM Rational software.
Download these IBM product evaluation versions and get your hands on application development tools and middleware products from DB2®, Lotus®, Tivoli®, and WebSphere®.
Find answers to your questions and share solutions in the Rational Quality Manager forum on developerWorks.
See also the Rational TestManager forum on developerWorks.
For tips, tricks, and happenings, monitor the IBM Rational Quality Manager blog.