Business leaders often view software quality as a luxury -- something that can be sacrificed, if necessary, for more functionality, faster development, or lower costs. However, in practice, software development organizations that have a firm commitment to quality can actually speed development, reduce costs, and add new features with greater ease. Organizations that develop low-quality software, whether for internal use or for sale, are always looking backward, spending time and money on fixing defects in "finished" products. In contrast, an organization that builds in product quality from the beginning can be forward-looking and innovative; it can spend its resources on pursuing new opportunities. Delivering quality is also a powerful differentiator in a marketplace where high-quality software is now the exception rather than the rule.
When you step back to assess the health of software development efforts in your organization, what do you see? Is there a focus on quality? Where is that focus built in, and where does it fall out? The fact is, quality is not a thing; it is -- well, a quality. Which means it should be a concern at every step in your process. Understanding this is the key to creating a truly successful process for application delivery.
But where do you start? Although the knee-jerk reaction to a demand for better quality is to expand the testing team, that is probably not the best approach. Instead, you can begin working to change the mindset within your organization to focus on quality, an effort that requires leadership from top management. This article will discuss the benefits of infusing concern for quality into an organization, as well as steps you can take and IBM tools you can use to support this key change.
For any organization, defining what you mean by quality is an important first step. Too often, software development organizations operate with a loose, general notion of quality and tolerate defects that most engineering disciplines would not allow. In contrast, a solid definition of quality that all team members understand and accept promotes thoroughness and attention to detail. In the domain of business applications, we can best define quality in terms of the target audience: the software users. Development organizations with a quality focus know that a "quality" application must do more than simply provide correct results without crashing. Does the application meet stakeholder requirements? Is it usable? Secure? Scalable? Reliable? Easily maintained? Easily extended? Simple to monitor?
James Juran's definition provides a good starting point: quality is "fitness for use."1 Juran goes on to say that a product is not high quality unless it adds value for both the consumer and the manufacturer.
Put slightly differently, quality encompasses added value plus attention to detail. Both a luxury car and an entry-level car will get you from point A to point B. But the luxury car offers features and capabilities that go beyond the essentials of transportation: usability, safety, comfort, reliability, and so on. Product quality also reflects the process behind the product. In the software world, a high-quality process can keep development organizations from losing time reworking, re-factoring, and rewriting software. These organizations can produce more innovative and creative products because they have more time to think about adding value and quality details.
Achieving quality results requires applying a high-quality process throughout development, integration, and testing. This applies equally well to projects involving packaged or homegrown applications, upgrades or greenfield development work, and extending, integrating, and modernizing legacy applications (see Figure 1).
Figure 1: All modes of software development require the same attention to quality.
Business benefits of focusing on quality
As Juran's definition implies, it is not just the customer who benefits from a focus on high quality. Businesses that value quality become more responsive and innovative, increase their competitive differentiation, and greatly reduce their total cost of development and ownership. Let's take a closer look at these benefits.
Quality enables responsiveness and innovation
Successful organizations recognize the importance of continuous innovation and differentiation, which require rapid responses to a constantly changing business landscape. Software's flexibility and malleability enables this rapid response, and organizations that embrace software development as a core business process quickly move ahead of the pack. However, software's greatest advantages are also what make it so vulnerable. One small change to one application can introduce a flaw that brings down an entire system and cripples the business. Knowing this, software development teams sometimes assume a "fix-it-later" mentality; they move forward with products they know are flawed -- a liberty that no other engineering discipline would permit.
When IBM's Global CEO Study 2004 2 interviewed 450 CEOs around the globe about their current strategic issues, ambitions, and concerns, they ranked "responsiveness" as their number one goal, followed closely by "new, differentiated products," "operational efficiency," and "improved business models." These goals are consistent with IBM's On Demand business model: In today's environment, businesses need to innovate, react quickly and efficiently to change, and differentiate themselves from the competition. If an organization's internal software systems are flawed, or if product development teams are constantly struggling to fix problems in last year's release, then the business cannot achieve these goals.
To operate successfully in an On Demand world, businesses must keep a laser-like focus on quality while developing and maintaining the software systems they rely on; quality systems are what enable businesses to react, adapt, deploy new solutions quickly, and maintain a competitive edge.
Quality is a differentiator
As a discipline, software development has quality standards far below those of other engineering disciplines. Managers and developers are good at rationalizing their products' shortcomings: The functionality they offer is "better than nothing" or "better than the former system," they say, and there are manual backups in case the system crashes. But would a structural engineer reason away the collapse of a building in a similar way? Would an aeronautical engineer merely shrug his shoulders if a control panel malfunctioned in mid-flight? To support a business effectively, software engineers must also move toward a zero tolerance policy for defects.
What difference can higher quality software make to a company in the marketplace? Consider the automobile industry in the late 1960s and 1970s. Japanese automakers had adopted a rigorous quality assurance (QA) program and soon gained a reputation for producing highly efficient, well-designed, reliable cars. This superior quality set their products apart and raised the industry's quality baseline. It also allowed Japanese automakers to develop further innovations in fuel efficiency, safety, and manufacturing processes as their competitors played catch up.
Similarly, when any company in any industry produces higher quality software for either internal use or external sale -- it raises the bar. The quality of an organization's internal human resources, financial, and customer relationship management systems affects the bottom line in ways that are harder to quantify than external sales, but the business impact is equally important. In fact, high-quality internal systems can be a business differentiator. Focused IT organizations typically purchase applications for automating common business functions and build only software and systems that are unique to their business. If those unique systems are also of high quality, the business will have an edge on its competitors.
Quality is free (almost)
A study commissioned by the U.S. Department of Commerce's National Institute of Standards and Technology (NIST) found that software defects cost the U.S. economy almost $60 billion annually. The study also found that about 80 percent of development funds are consumed by software developers identifying and correcting defects. 3 In another study, The Standish Group reports that canceled software development projects cost organizations US$55 billion dollars annually. 4 Clearly, poor software quality -- and poor software development processes -- are a major drain on business profitability.
The reasons for the high cost of poor quality vary according to the root cause. For example, poor problem domain analysis -- and thus poor requirements -- can lead to unplanned and costly reworking. Waiting until the eleventh hour to validate quality through testing inevitably extends both the schedule and budget. And operational downtime caused by reliability or performance problems can incur opportunity costs if customers are unable to access your system or the business is unable to do its work. Other root causes include: a poorly defined process with inconsistent staff buy-in; none or limited architectural and code QA; limited post-deployment monitoring and assessment.
But what is the cost of high quality? Expert Philip Crosby insists that quality is free. 5 Ideally, a software development organization would have these characteristics:
- A well-defined process with unanimous buy-in from team members.
- Team members who focus attention on value-added features, detail, and quality across the lifecycle.
- Managers who constantly evaluate end-user feedback to ensure reinvigoration of the product line and satisfied, repeat customers.
Although the intangibles that would make such an organization operate efficiently -- good management, effective staffing, consistent process, and so on -- would require up-front investments in terms of management time, planning, and training, they would not incur additional financial outlays.
One common misconception about quality is that you can trade it off to improve development speed, reduce costs, or add functionality. Too many people actually believe in Meskimen's tongue-in-cheek law of quality: "There's never time to do it right, but always time to do it over." In practice, however, most organizations find that the opposite is true. In the long run, it is improved quality that enables teams to deliver more projects on time, at lower cost, and with more features. A development team that continuously ensures quality does it right the first time. If you eliminate defects throughout the development process, you also eliminate the time and cost required to find and fix those defects later on.
Driving quality from the top down
Continuously ensuring quality requires commitment and dedication that is driven from the top of the business leadership chain on down. Although your methodology need not be as formal as organizational improvement methodologies, such as Six Sigma or Total Quality Management, it will require the same kind of management leadership and organizational involvement that more formal methodologies demand.6 Quality improvement is essentially a mindset issue; when managers from the top down instill a culture of quality throughout the organization, it permeates every project.7
Working within such a culture offers great benefits to managers. They no longer have to weigh the consequences of shipping products with known defects. And the rigorous processes, team responsibility, and objective metrics that contribute to producing quality also create predictability. Instead of endlessly re-scoping the project and still missing the deadline, teams can scope, estimate, and schedule accurately, and comfortably commit to delivering on time and according to specifications.
Achieving quality with the right process and platform
Establishing a quality-oriented mindset is a top management responsibility, but it is up to the software development organization to adopt and execute on that mindset. IT must embrace the right process and tools, and seek the right guidance or training to produce the results that management, customers, and end-users are seeking.
Strategies and IBM tool support for quality in RUP disciplines
IBM offers an integrated solution to help software development teams build higher quality software: The IBM Software Development Platform. This open and modular platform includes tools from all IBM software brands, including IBM Rational Unified Process,® or RUP.® This automated process guidance embodies software development best practices, including an iterative approach, change management, visual modeling, and continuous QA procedures.
The iterative phases of RUP emphasize one or more core process disciplines: analysis, design, development, test, and deployment. Each phase and each discipline emphasizes attention to quality and seeks to help teams identify problems early in the development lifecycle, when they are easiest to solve. The following sections examine how RUP and tools in the IBM Software Development Platform support quality-oriented practices in each process discipline.
Meta Group reports that up to 80 percent of the issues leading to customer dissatisfaction can be traced to poor understanding of requirements. For any software development project -- whether it's new application development, packaged application integration, or legacy modernization -- quality starts with analyzing the business to ensure that system requirements clearly and accurately reflect business and customer needs.
If a customer writes a list of requirements, is that good enough? Probably not, because understanding such a list would likely require domain knowledge that you can acquire only through extensive experience. As the people building the software do not typically work in that world, the requirements must be written in a manner that is understandable to them. Writing good requirements combines the efforts of customers, business users, and the development team.
Tools for this activity include IBM Rational RequisitePro®, a requirements management tool that enables teams to document requirements clearly (using Microsoft® Word) while leveraging the analytical capabilities of a database to perform requirements analysis, coverage, and change impact.
To define optimum, requirements-based business processes to drive your application development, you can use IBM WebSphere® Business Integration Modeler, a tool that helps you design, test, and communicate complex business processes. It simulates the operational efficiency of processes and analyzes potential business results, while IBM WebSphere Business Integration Monitor displays real-time information from a variety of environments to allow decisive business performance management and optimization.
In design, the primary quality focus is on architecture, the "soul" of software. Poor architecture can cause a wide array of quality problems, including fragility, lack of scalability, and resistance to modification. These problems become harder to solve as an application project matures; and the cost of correcting defects grows exponentially as applications progress from design to development, testing, and deployment. If software developers can efficiently detect, isolate, and resolve structural deficiencies during design and development, this work will pay dividends throughout the project. Developers should also ensure that the software is implemented in a way that preserves the architecture's integrity and flexibility, so that developers can make changes quickly, as business needs change.
Tools for this discipline include IBM Rational Software Architect,® which enables software architects and senior developers to create high-quality architectures for applications and services by leveraging model-driven development with the UML. IBM Rational Software Modeler® contains those features of Rational Software Architect dedicated to ensuring that specifications, architecture, and designs are clearly defined and communicated to stakeholders through the UML. Both products can help teams improve their understanding of the system and free up time to focus on quality issues during design and beyond.
On average, developers make 100 to 150 errors for every thousand lines of code they write. Of course, this number varies from developer to developer and project to project. Even if only a small fraction, say 10 percent, of these errors are serious, then a relatively small application of 20 thousand lines of code will have roughly 200 serious coding errors.
Partially in response to statistics like these, development organizations in recent years have placed a stronger emphasis on developer-led testing and analysis. This is epitomized by "agile" practices that endorse test-first development -- building tests before you code. Although unit testing and run-time analysis have become more mainstream, many managers still have the misconception that these procedures add unnecessary time to the schedule. In reality, schedules typically lengthen because of the time developers have to spend debugging code later in the lifecycle, after QA or customers find problems. For teams that want to reduce risk and increase predictability, a well-formed, proactive QA approach by the development team is a good solution. For Java developers, IBM Rational Application Developer for WebSphere Software provides comprehensive support for this approach. It combines Java™, Web, and enterprise development tools in a single, integrated development environment for building, testing, integrating, and deploying applications. IBM Rational XDE™ Developer also offers Java and managed .NET software designers and developers a rich set of model-driven development and run-time analysis capabilities for building quality software applications based on the Eclipse platform and in Microsoft Visual Studio® .NET environments. IBM Rational PurifyPlus™ has a complete set of automated, run-time analysis tools for improving the application reliability and performance of applications based on the Java, C/C++, managed .NET, and Visual Basic languages.
Conducting system-level functionality and performance tests is an integral part of continuously ensuring quality. The quality engineering (QE) team that performs these tests acts primarily as a customer advocate in verifying the application's ability to fulfill customer needs or business imperatives.
A development organization should neither overemphasize nor minimize the importance of system testing. Ensuring quality is not the sole responsibility of the QE team; nor is testing the exclusive domain of QE. Some tests can and should be run by developers and, in some cases, by architects.
Unfortunately, many organizations treat QE team members as bug sweepers, turning them loose on an application late in the lifecycle to find defects that other team members introduced. However, testing alone cannot create quality. If the application under test is riddled with architectural defects and bugs that should have been identified and eliminated during unit testing, then system and performance testing late in the lifecycle will not be effective. Similarly, QE cannot be expected to test efficiently without automated tools. To be most effective, QE teams must be able to apply appropriate testing tools as they focus on system-level quality issues.
Testers and business analysts who want to improve the speed, breadth, and reliability of their manual testing efforts should consider IBM Rational Manual Tester, a manual test authoring and execution tool. IBM Rational Functional Tester is a tool geared specifically for QA professionals who perform functional and regression testing of Java, .NET, Web, and terminal-based applications. And testers and IT operations professionals can use IBM Rational Performance Tester, a load generation tool, to ensure the scalability and reliability of Web-based applications before subjecting themselves to real-world user loads.
At some point, business applications must go live. Inevitably, functionality and reliability errors slip through the cracks of even the most thorough processes. If the application is to meet or exceed service level agreements (SLAs), then the IT operations team must assume responsibility for ensuring quality. Through constant monitoring and assessment, team members can ensure the viability of a deployed system and rapidly detect and correct performance problems.
IBM Tivoli® Monitoring products can help IT teams detect bottlenecks and potential problems, and also provide automatic recovery from critical situations. IBM Tivoli Monitoring for Transaction Performance® provides performance management capabilities for Web and enterprise infrastructures. And IBM Tivoli Provisioning Manager® automates the tasks of provisioning and configuring servers, operating systems, middleware, applications, storage, and network devices in both test lab and production environments.
Supporting team responsibilities for ensuring quality
Quality is the responsibility of each individual on the development team, but it is also the responsibility of the team as a whole. In an iterative process, each iteration is essentially circular -- each discipline receives input from, and provides output to, another discipline. At a high level, this circularity ensures constant reassessment of each artifact's quality. Teams must do everything they can to integrate workflows, establish traceability, and simplify communication. A breakdown in the chain that links team members leads to data loss, reworking, lack of clarity, inefficiency -- and ultimately lower quality software.
The IBM Rational Team Unifying Platform is an integrated suite of infrastructure tools and processes that unifies development teams by providing common access to development assets, communication alerts, and workflow processes. In addition to RUP and Rational RequisitePro, which we described earlier, this platform includes integrated tools for software configuration management, change management, test management, and reporting that enable requirements traceability throughout development, analysis, and testing. The software configuration management solution integrates software asset management from the IBM Rational ClearCase® family of products with defect and change tracking from the IBM Rational ClearQuest® family. Effective software configuration management is an essential tool for ensuring quality; it helps organizations ensure that software builds are repeatable and reliable, and that defects and change requests are managed properly.
The IBM Rational Team Unifying Platform also includes IBM Rational TestManager, which provides control, management, and reporting of all test activities from a single, central point. IBM Rational ProjectConsole™ enables managers to keep close tabs on quality control; it dynamically generates a project Web site and metrics dashboard with data automatically collected from the development environment. And IBM Rational SoDA® automatically generates and maintains comprehensive project documentation and reports.
IBM also offers a wide range of services to help development teams deliver quality software. Team members can access both instructor-led and Web-based training courses focusing on tool use and skill development via the IBM developerWorks® Web portal. And professional services consultants can work with development teams to create custom quality implementation plans spanning initial project assessment, installation, mentoring, training, and maintenance.
Reaping the extensive benefits of better software quality
When people think about what it takes to build mission-critical and safety-critical software, they often imagine that it requires a cumbersome process with oversight, standards, formality, and documentation. In truth, there are many ways to achieve quality without a heavy-duty process, such as Total Quality Management or Six Sigma.
Quality improvement, like software development, is an iterative process. You do not need to accomplish everything in a single step. Even small changes -- including adjustments in your organization's attitude towards quality -- can make a tangible difference. Although you don't need a burdensome process to achieve quality, you do need an organization mindset that focuses on quality, and that must be driven by top management.
The business benefits of including quality-oriented activities in all phases of your software development lifecycle are both broad and deep. These measures not only facilitate innovation and lower costs by increasing predictability, reducing risk, and eliminating rework, but they can also help to differentiate your business from its competitors. Most important, continuously ensuring quality will always cost less than ignoring quality considerations. In fact, raising product quality costs next to nothing if you do it correctly.
1 James M. Juran and Frank Gryna, Juran's Quality Control Handbook. McGraw-Hill, 1988.
2 IBM Business Consulting Services. "The Global CEO Study 2004." http://www-1.ibm.com/ services/bcs/globalstudy.html
3 NIST News Release, "Software Errors Cost U.S. Economy $59.5 Billion Annually: NIST Assesses Technical Needs of Industry to Improve Software-Testing." http://www.nist.gov/ public_affairs/releases/n02-10.htm
4 The Standish Group, "CHAOS Chronicles, v3.0." March, 2003.
5 Philip B. Crosby, Quality Is Free. New American Library, 1982.
6 Although organizations that produce mission-critical or life-critical software systems require a high degree of formality, in many organizations this is not a prerequisite for quality. RUP, a highly adaptable framework, can scale from highly structured/formal to loosely structured/informal environments.
7 Jack Welch and John Byrne, Jack: Straight from the Gut. Warner Books, 2001. Jack Welch, former General Electric CEO and champion of the Six Sigma approach, emphasized that executive sponsorship was the key driver behind the success of this initiative within GE.
8 See Watts S. Humphrey's article entitled "The Quality Attitude," published in the Software Engineering Institute (SEI ) newsletter, (2004, no.3): http://www.sei.cmu.edu/news-at-sei/columns/watts_new/watts-new.htm