Introduction

In this article, learn about four software architecture assessment methods defined by the Software Engineering Institute (SEI) at Carnegie Mellon University. The assessment methods can help you analyze whether a software architecture design is suitable for a given set of requirements. This is important because requirements provide the context from which quality expectations are determined; if the requirements of a software effort are satisfied, its quality goals should also be fulfilled.

Where appropriate, activities will be positioned within the software life cycle (SLC) model specified by the Rational Unified Process (RUP). The model was chosen because of its popularity across modern development efforts throughout the industry. For more information on integration with the methods described here, see Resources.

Back to top

Background

Before we discuss specific assessment methods, the following sections expand on key background concepts.

Quality

Defining quality is not as straightforward as it might seem. Initially it might be something like "quality is what is good," or "quality is good craftsmanship." Everyone agrees about the importance of quality in achieving successful results, in almost any domain. For example, in team sports, the quality of the team’s chemistry frequently signifies the difference between losing and winning. In cooking, quality ingredients often mark the difference between an average meal and a great one. Such positive connotations can be applied to software quality: the greater the quality, the greater the chances of project success.

Projects often have goals for enhancing quality, while also targeting an increase in features and decrease in schedule. This is typically not feasible, as changes are realistically possible for only two out of the three options, not all three. For example, an increase in quality and features requires more time to complete the tasks. If more time is not scheduled, people will have to spend less time on each task (including testing), thus possibly impacting the quality of their work.

Methods ease the dependency among the three areas. If, for instance, there were a method that allowed features to be added more efficiently (maybe the addition of a simple-to-use, specialized CASE tool), the schedule and overall quality might not be affected as much. The methods described later in this article help address the dependencies, with the focus on improving quality.

Quality attributes

Understanding how to enhance quality should be a priority on any software effort. Before quality can be improved, it needs to be measured and analyzed. Quality attributes provide a context for measuring and analyzing quality.

Quality attributes are the elements that characterize quality of a specific context, such as performance, security, portability, functions, and so on. Each of these attributes are not absolute quantities; their relevance is directly tied to the given circumstances. For example, if a customer does not care much about portability (perhaps all systems are running the same operating system), but cares a lot about performance, the performance attribute will be prioritized over portability. This allows for organizing tasks according to what is relevant to the customer.

To be able to properly measure the attributes, further task decomposition must occur. For instance, the performance attribute might be decomposed into data latency and transaction throughput. At this point, possible metrics to be used become more obvious. Each of these refined entities can be further decomposed into specific scenarios, which are a great way to elicit requirements information.

The relationship between requirements and quality attributes helps in understanding the suitability of software architectures. Without such mapping, it's harder to really understand why certain features were designed into architectures. Is it because it makes sense to the designer? Is it something that trade magazines have been proposing? Or is it something that has been specified by the stakeholders? The relationship between requirements and quality attributes also helps to effectively prioritize work, because it helps clarify what is important to the client.

You can use quality attributes to qualify specific scenarios, which serve as great eliciting tools for further design refinement.

Software architecture analysis

Software architecture enables the delivery of complex software systems. A software architect does not focus on every detail, but on those that have a high impact on the solution at hand. Like building architects, software architects don't care so much about the detailed techniques necessary for pouring concrete for constructing a house, but care about the feasibility of a specific house to be constructed. Given the interconnections between elements of a solution in modern software projects, it would be fairly difficult for one person to keep track of all of them. One of the most important tasks of a software architect is to break the complexity down into manageable parts by identifying the elements that are the most relevant for success. The next logical step is to research the dynamics among quality, quality attributes, and software architecture to better understand how quality can be enhanced. To efficiently achieve that goal, a relevant analysis approach, as described in this article, should be followed.

Back to top

Assessment methods

Successful architectures follow various guidelines and best practices. SEI has done extensive research in this area, leading to the creation of several methods for the improvement and assessment of architectures. The four most representative methods are:

• Quality Attributes Workshop (QAW)
• Architecture Tradeoff Analysis Method (ATAM)
• Software Architecture Analysis Method (SAAM)
• Active Reviews for Intermediate Designs (ARID)

QAW is executed before the architecture has been defined, ARID during design efforts, while ATAM and SAAM are executed after the architecture has been completed. The execution of the elicitation part of these methods is lead by a facilitator. See Resources for more information on these methods.

Quality Attributes Workshop method

The QAW method is an approach for discovering quality attributes before the software architecture is created. Achieving particular qualities, such as performance or security, is highly dependent on well-designed software architecture.

It is not uncommon for quality attributes to be often missing or not specified completely. This can spell disaster later in the life cycle, when the solution is implemented. For example, if system security is not well defined early on, it's hard to add it later because it is an attribute that affects many levels of a solution, from components to infrastructure elements.

The QAW elicitation activities are executed in a workshop composed of facilitators and system stakeholders. The QAW is divided into eight steps, as shown in Table 1.

The outputs of the QAW effort are a list of architectural drivers, the scenarios, a prioritized list of scenarios, and the refined scenarios. You can use this information to refine requirements, develop prototypes, influence design decisions, and so forth.

Architecture Tradeoff Analysis Method

The QAW method yielded documentation and prioritization of quality elements before an architecture was laid out. The ATAM assumes that an architecture has already been delivered. Within the ATAM, the QAW efforts can be reused to deliver a clear quality attribute definition. The ATAM includes "trade-offs" because it describes how well an architecture satisfies specific quality goals, but also provides insight into the trade-off those attributes have in the quality of the architecture.

ATAM provides a way to conduct architectural reviews to assess the suitability of the current architecture with its business drivers. It helps you to better synchronize stakeholder needs and solution design.

Software Architecture Analysis Method

SAAM, which pre-dates QAW and ATAM, is one of the earliest documented approaches for analyzing software architectures. It provides a simpler approach than ATAM. One of the motivators for SAAM was that many architects make claims that are hard to sustain, such as "my architecture is easily maintainable." Some of the SAAM steps match some of the ATAM steps; differences are noted in Table 3.

SAAM provides a way to attach measurable quality attribute scenarios to general attribute claims, enabling circumstances that are more directly tested.

Active reviews for intermediate design method

Both ATAM and SAAM focus on completed architectures. The ARID method focuses on incomplete architectures. The benefit is that you don't have to wait until the architecture design is complete to understand if it is heading in the right direction. The steps are shown in Table 4.

Among the methods described in this article, ARID is the newest. Because it shares many of the characteristics of the other methods, it reuses many of their proven techniques.

Back to top

Usage patterns

In real life these methods would be used within the context of a process framework. This is where RUP comes into the picture. RUP defines a proven software life cycle, with documented phases, well-defined disciplines, and pragmatic roles. A core principle of RUP is that it is an architecture-centric process: architecture is vital to the success of any application using RUP. The bulk of architecture development is delivered during the initial life cycle phases, and modified as needed in the later phases.

QAW fits best within RUP's inception phase, as it is then that the ideas shaping the to-be architecture are shaped. QAW can also be used in later phases to refine initial findings. The Requirements discipline would benefit from the extra focus on quality attribute analysis that QAW brings to RUP.

ARID should be executed in RUP's elaboration phase as it is at this phase that the software architecture is solidified. ARID would help in various review efforts executed as part of the RUP's Analysis and Design discipline.

ATAM/SAAM effort can occur anywhere where in the life cycle where an architectural review is needed. A good fit would be within RUP's construction phase, as it is at this phase that resources will begin executing on architecture plans.. ATAM's ability to compare delivered architectures against quality goals make it ideal for the reuse of QAW outputs.

Back to top

Summary

The following table summarizes the recommendations for using the methods described in this article. They are listed in order of usage in the RUP SLC phases. Roles and disciplines are also listed to better position the effort within existing knowledge areas.

Using assessment methods encourages a better understanding of current architectural designs, and enables a more efficient determination of quality in software architectures. Matching requirements to scenario-driven quality attributes encourages more accurate software architectures. The methods described in this article serve as catalysts for these types of connections, allowing for the externalization of key relationships not clearly seen before. Using the assessment methods can help ensure the suitability of architectural designs, and the enhancement of their quality.

Resources

Learn

• The Software Engineering Institute is a federally funded research and development center conducting software engineering research in acquisition, architecture and product lines, process improvement and performance measurement, security, and system interoperability and dependability.
  
  
• Quality Attributes (Technical Report CMU/SEI-95-TR-021, 1995) takes a small step in the direction of developing a unifying approach for reasoning about multiple software quality attributes. It defines software quality, introduces a generic taxonomy of attributes, discusses the connections between the attributes, and discusses future work leading to an attribute-based methodology for evaluating software architectures.
  
  
• Quality Attribute Workshops, 3rd Edition (Technical Report CMU/SEI-2003-TR-016) describes the QAW method.
  
  
• "Making Architecture Reviews Work in the Real World" (IEEE, 2002) explores the non-technical aspects of formal architecture reviews-social, psychological, and managerial.
  
  
• "Predicting Software Quality by Architecture-Level Evaluation" (Fifth International Conference on Software Quality, 1995) discusses the importance of evaluating a software design for fitness and achievement of quality as early as possible in the software development life cycle.
  
  
• Evaluating Software Architectures: Methods and Case Studies by Clements, Kazman, and Klein, 2002, provides detailed case studies that demonstrate the value and practical application of the ATAM, SAAM and the ARID methods to real-world systems.
  
  
• Integrating Software Architecture Centric Methods into the Rational Unified Process (Technical Report CMU/SEI-2004-TR-011) fits the Carnegie Mellon Software Engineering Institute's (SEI's) architecture-centric methods into the framework of the RUP.
  
  
• The Rational Unified Process – An Introduction (2nd Edition) by P. Kruchten provides an overview of Rational's prescription for whole-project health.
  
  
• In the Architecture area on developerWorks, get the resources you need to advance your skills in the architecture arena.
  
  
• Browse the technology bookstore for books on these and other technical topics.
  
  

Get products and technologies

• Download IBM product evaluation versions and get your hands on application development tools and middleware products from DB2®, Lotus®, Rational®, Tivoli®, and WebSphere®.
  
  

Discuss

• Participate in the discussion forum.
  
  
• Check out developerWorks blogs and get involved in the developerWorks community.
  
  

About the author

Jorge Diaz is a Senior Certified IT Architect within IBM Software Services for WebSphere, focusing on technology transformations and solution architecture consulting. He has extensive experience in a variety of industries, and a proven track record of leadership with large customers adopting service oriented architectures. Jorge holds a B.S. in Computer Science, a Master of Software Engineering, and a variety of technical certifications. He is a steering committee member of The Open Group's SOA Working Group, a senior member of IEEE, and a member of ACM and SEI.

Comments (Undergoing maintenance)

Back to top

Trademarks  |  My developerWorks terms and conditions

Table of contents

• Introduction
• Background
• Assessment methods
• Usage patterns
• Summary
• Resources
• About the author
• Comments

Local resources

• IBM Innovation Center - Austin, TX
• IBM Innovation Center - Chicago, IL
• IBM Innovation Center - Dallas, TX
• IBM Innovation Center - San Mateo, CA
• IBM Innovation Center - Waltham, MA
• Technical Briefings in North America

My developerWorks community

Dig deeper into IT architecture on developerWorks

Special offers