Improve project quality with Rational Team Concert 3.0 and ODC: Part 1. Classify and validate defects

Using Orthogonal Defect Classification (ODC) for analytics throughout the software development cycle gives the team a deeper insight into the defect backlog and defect trends. The set of ODC analytics functions can enrich almost all software development processes. This article, the first in a series of two, shows how to extend IBM® Rational Team Concert™ Version 3.0 to support ODC systematic defect data collection.

Share:

Anna M Bizoń-Adamczyk (anna.bizon@pl.ibm.com), Software Engineer, IBM

Author photoAnna Bizoń-Adamczyk is a software engineer for the IBM Krakow Software Laboratory, in Poland. She joined IBM in 2006. She is interested in project management and activities related to configuring Rational Team Concert for project progress tracking and status reporting.



03 May 2011

Also available in Chinese Russian Vietnamese

This article explains how to use IBM® Rational Team Concert™ Version 3.0 features to collect Orthogonal Defect Classification (ODC) data for further analysis. This first article in a series describes the steps required to enable the ODC process in an IBM® Rational® Jazz™ environment, in terms of tool capabilities for ODC data gathering. The second article will describe how to create a set of assessment charts and make them available to all team members within a Rational Team Concert environment.

Prerequisites

To follow the guidelines in this article, you need to have a Rational Team Concert 3.0 server installed and a corresponding Rational Team Concert Eclipse client configured. You will also need permissions for changing the process definition. If you do not have Version 3.0 of the Rational Team Concert server or client, please see the Resources section of this article to find information on obtaining, installing, and configuring the required software from the Jazz website.


Benefits of ODC

Orthogonal Defect Classification (ODC) is a powerful defect analysis method that development teams can use to monitor project progress and health. This technique was developed by the IBM Research group. The ODC process has been effectively applied to many projects, which have shown that it contributes to the successful delivery of high-quality software. ODC is built from a set of simple rules for defect classification that can be easily adapted to almost any project. The simplicity of the ODC method lowers the incurred costs of introducing it into the software development cycle. As a result, it provides real value to customers and development teams. Shortly after integrating ODC with your development environment, you are likely to see significant improvements in your project progress tracking, risk management, and quality that result in higher customer satisfaction.

The main assumption of ODC is to extend each defect record with a well-defined set of additional data. More information almost always means better decisions. After a certain period of time, the ODC data is systematically and comprehensively collected for each defect to allow the team to perform a wide analysis of the character of the defects found during different development activities and to analyze the defect trends. This analysis leads to the identification of key issues. Next comes the most important part: actions to prevent further occurrences of undesirable situations.

The ODC semantic information not only leads to better decisions but also leads to significant cost reduction. This is a simple and effective path for improving the quality of software by prevention of defects, increasing early defect removal, and lowering the defect escape rate. The ODC analysis can answer a wide set of questions that are often asked during the development process, for example:

  • Are we finding the right defects during each activity?
  • Where are defects usually located?
  • What is not covered by testing?
  • What is the quality of source code and testing over time?
  • Do we get the right level of customer attention?
  • Are we progressing in terms of completeness of our code?
  • Are we reaching better stability of our product?

ODC in action

The complete ODC classification and assessment process is well-defined and consists of four main phases that are shown in Figure 1.

Figure 1. ODC main process phases
Diagram of succession of the four activities

This process covers the following activities that occur in this order:

  1. Classify. The classification step that captures ODC data collecting. Collecting data requires a team commitment within two basic roles: a Submitter and a Responder. A tester usually takes the role of a Submitter, while development team members usually close the Responder sections. Classifying is a constant process of assigning the right attributes to a defect entity.
  2. Validate. This step must be executed by a fully qualified person with the right set of skills in terms of project-specific knowledge and ODC. The Validator role is to provide feedback to the team, based on the results of review of the classified defects.
  3. Assess. In terms of ODC, this assessment is focused on relationships between ODC attributes and other defect-related data, such as component or severity. It often includes either analysis of defect trends for incoming defects by certain ODC attribute or analysis based on the number of defects with a certain attribute set within another ODC attribute.
  4. Act. This is the stage of implementing actions as a result of all the analysis of the classified defects.

This article focuses on the Classify and Validate phases in terms of enabling the right features in Rational Team Concert to support the ODC scheme for work items of a defect type.

ODC success guidelines
These are all key factors to ensure ODC methodology success in your project:
  • Team commitment to the ODC method
  • Appropriate knowledge, because deficient education can lead to incorrect data
  • Defects must be classified systematically by all team members
  • Validation is performed regularly and feedback is provided to the team members.
  • Assessment is performed and leads to the identification of required actions that are executed in the team environment. Tools are properly configured and support the process at all stages.

Defect classification concepts

The ODC classification concept must be introduced in order to understand how the development environment should be configured. The defect lifecycle usually consists of several states in the development cycle, but the valuable information can be retrieved only twice: once when the defect is found and once when the fix is delivered by the developer.

When a defect is identified, the Submitter has to collect all required information and record it in the tracking tool before passing a defect to the development queue. Usually at this stage, the severity and other defect-related information is provided. Although all of the data is very useful in defect analysis, it can be extended with an initial ODC data set. There are three ODC- specific categories that capture the circumstances when a defect is opened. For an individual submitting a defect, the following three categories can be used to classify it:

Activity
The activity that is executed at the time that the defect was discovered. The set of available activities is defined by a project team.
 
Trigger
Condition fulfilled when the defect is discovered, strictly dependent on the activity. When the activity is dependent on the specific project, the set of triggers remains unchanged.
 
Impact
Assessment of the defect impact for the customer.
 

When a defect is closed by the Responder, additional data is made available that describes the exact nature and scope of the defect. This information has been categorized as one of these five types:

Target
Entity that was fixed
 
Defect type
Nature of the fix
 
Qualifier
Direct cause of the defect
 
Source
Identifies the origin of the target that had the defect.
 
Age
History of the target
 

Note:
Subsets of the information populated by each role are exclusive. Rational Team Concert should be configured to help keep them separated. This is one of our goals.


Rational Team Concert and defect data classification

As of the original publication date of this article, Rational Team Concert does not support ODC defect classification as you receive it; however, there are many ways to extend the functionality in order to achieve this goal. By extending the Rational Team Concert platform, you can enrich it with the ODC process.

To implement ODC data gathering in Rational Team Concert, you'll extend the Rational Team Concert work item artifact by defining custom attributes (this is a native capability of Rational Team Concert). This perfectly matches the simplicity of the ODC data model, which is focused on a well-defined set of defect-related attributes.

To enable ODC data collection, you need to customize the Defect type of work item. To do this, run the Rational Team Concert Eclipse client and a selected project definition or template in the Process Configuration Editor. You will work with the Work Items section of the Process Configuration:
Project Configuration > Configuration Data > Work Items

Enumerations

First, you'll need to add the required enumerations for ODC attributes. The process of creating a new enumeration is presented in this ODC Activity example:

  1. Open the Enumerations editor and, in the tab that opens, go to Choose enumeration to edit and click Add.
  2. A dialog window appears, as shown in Figure 2, so you can provide the enumeration ID and its name. After you enter the information and click OK, the new enumeration is available for editing.
Figure 2. Add a new enumeration for ODC Activity
Enumeration ID fields: ID, Name

Now you can define the enumeration literals and their order by adding them to the Enumeration Literals list.

  1. By clicking Add Literal for each enumeration entry, you define literals, including "Unassigned."
  2. Then, in the Enumeration section, set Default Literal and Unassigned Literal to the value Unassigned, as shown in Figure 3.
Figure 3. Add literals to the enumeration
Default Literal and Unassigned Literal drop-down menu

Repeat the steps to create enumerations for the rest of the enumerations required for the ODC attributes. Table 1 and Table 2 show literal entries for each enumeration that should be created. Table 3 shows enumeration IDs and names.

Table 1. Enumeration literals for ODC Submitter attributes
ODC ActivityODC TriggerODC Impact
Unassigned
Design review
Code inspection
Unit test
Build
BVT
CVT
SVT
IVT
GVT
TVT
DBCS IVT
Performance
Scalability
ID review
GUI review
Acceptance
Beta
Unassigned
Design conformance
Logic, flow
Backward compatibility
Lateral compatibility
Concurrency
Internal document
Language dependency
Side effect
Rare situations
Simple path
Complex path
Coverage
Variation
Sequencing
Interaction
Workload, stress
Recovery, exception
Start, restart
Hardware configuration
Software configuration
Screen text, characters
Navigation
Widget, GUI behavior
Unassigned
Installability
Standards
Serviceability
Integrity
Security
Migration
Reliability
Performance
Documentation
Requirements
Maintenance
Usability
Accessibility
Capability
Table 2. Enumeration literals for ODC Responder attributes
ODC targetODC defect typeODC qualifierODC ageODC source
Unassigned
Requirements
Design
Code
Build, package
Information development
National language support
Unassigned
Assignment
Checking
Algorithm, method
Function. class, object
Timing, serialization
Interface, O-O messages
Relationship
User interface
Translation
Unassigned
Missing
Incorrect
Extraneous
Unassigned
Base
New
Rewritten
Refixed
Unassigned
Developed in-house
Reused from library
Outsourced
Ported
Table 3. ODC custom attribute names and identifiers
NameID
ODCActivityodc.activity
ODCTriggerodc.trigger
ODCImpactodc.impact
ODCTargetodc.target
ODCDefectTypeodc.deftype
ODCQualifierodc.qualifier
ODCAgeodc.age
ODCSourceodc.source

The created enumerations are used as a type for defect ODC custom attributes.

Custom attributes

When you have all of the required enumerations defined, you can add ODC custom attributes to the Defect work item. To do this follow these instructions:

  1. Open Types and Attributes and make sure that the Defect type of work item is selected.
Figure 4. Types and Attributes editor
types and attributes, defect, and work item editor
  1. Scroll to the Attributes section of the Types and Attributes editor, and select Show only custom attributes.
  2. Click Add.
  3. In the new window (see Figure 5), provide the required information for the ODC Activity custom attribute, including appropriate enumeration and the same ID that was used for the corresponding enumeration.
Figure 5. Create a new custom attribute
Add Custom Attribute dialog window
  1. Repeat steps 1 and 2 for the rest of the ODC attributes.

Defect Presentation Editor customization

When all of the ODC Submitter attributes are created, you need to extend the Defect editor so that it displays the ODC data. Before that, you need to define a UI area for the Defect editor where you can edit the ODC data. To do this:

  1. Open the Types and Attributes section, check whether the Defect option is selected, make sure that in Work Item Types the Defect item is selected, and check which editor ID is displayed under the Work Item Editor section. Figure 4, above, shows the configuration of types and attributes.
  2. Open Editor Presentations and select the appropriate presentation for Defect.
  3. Click the Add Tab button and provide the Title, Layout, and ID in the appropriate sections (see Figure 6).
  4. Click OK.
Figure 6. Adding an ODC tab to the defect presentation editor
add tab dialog
  1. Click Add Section, name it ODC Submitter, and then choose one of the slot options.
  2. Repeat the previous step to add the ODC Responder section.
  3. For the ODC Submitter section, add presentations for ODC Activity, ODC Trigger, and ODC Impact. The Add Presentation button displays a dialog for creating the presentation. In the Attribute-based Presentation section, select an appropriate attribute and in the Kind drop down, select Enumeration. Optionally you can enter a label, description and an ID.
Figure 7. Add the ODC Activity presentation to ODC Submitter section
Add Presentation dialog window
  1. In the ODC Responder section, add ODC Target, ODC Defect Type, ODC Qualifier, ODC Age, and ODC Source.
  2. Create a new sample defect, and verify that the ODC tab and sections were created properly.

Dependent enumerations

The dependencies between ODC attributes can also be reflected in the ODC tab of the defect provider. One relationship, between ODC Activity and ODC Trigger, is shown in the Submitter section. The other, between the ODC Target and the ODC Defect Type, is in the ODC Responder section.

The possible values of ODC Trigger depend on the current value of the ODC Activity. The same can be observed in the case of ODC Target and ODC Defect Type pair. You can manage the relationship between them in the Attribute Customization editor Value Sets feature. To define the dependent value set, you need to create a new definition of the Dependent Enumerations value set and select the enumerations. For each selection from the source enumeration, you can check the values that should be available in the value set of the target enumeration if that value is selected. Make sure that Unassigned is always selected to prevent problems during initialization. Figure 8 presents an example of an ODC Activity and ODC Trigger dependency.

Figure 8. Dependent value set for the ODC activity and ODC trigger
customizing attribute dialog

ODC validation

You can also use the ODC tab, located in the defect editor, to track ODC validation. You can use it when you want to add a general section where other data can be stored. You can see the additional attributes that indicate whether a validation was performed and at which level (ODC validation enumeration: None, ODC Submitter, ODC Responder and ODC Submitter and ODC Responder).

You can also add an ODC Comments field to provide an originator and owner with feedback about a defect. Figure 9 shows this section with Validation and Comments fields for feedback is presented in Figure 9 and Figure 10 shows the result in the Defect Editor.

Figure 9. Additional section on the ODC tab for validation purposes
ODC tab in the Editor Presentations view
Figure 10. Complete extension of the Defect editor for ODC data tracking and validation
summary of defect

Summary

ODC is an effective way of classifying defects and saves project teams significant time during every project phase, but only if it is implemented and used correctly. Correct integration of the process with the development platform is critical to success. As you can see from this article, Rational Team Concert 3.0 can be extended in a way that empowers you to easily adopt the ODC scheme into Defect work items and validate activities. In the next article of this series, I'll be describing a method of enabling Rational Team Concert to support the ODC analysis phase.

The next article in this two-part series explains how to extend Rational Team Concert 3.0 to support the ODC analysis phase.


Acknowledgments

I would like to thank Michael Spisak, IT Architect, IBM Certified Consulting IT Specialist, and Master Inventor, for his technical review and Witold Kopel, Justyna Drozdz, and Rafal Adamczyk for general reviews and encouragement.

Resources

Learn

Get products and technologies

  • Visit the Jazz community to download Rational Team Concert 3.0 Client and Server trial versions and read articles.
  • Evaluate IBM software in the way that suits you best: Download it for a trial, try it online, use it in a cloud environment, or spend a few hours in the SOA Sandbox learning how to implement service-oriented architecture efficiently.

Discuss

Comments

developerWorks: Sign in

Required fields are indicated with an asterisk (*).


Need an IBM ID?
Forgot your IBM ID?


Forgot your password?
Change your password

By clicking Submit, you agree to the developerWorks terms of use.

 


The first time you sign into developerWorks, a profile is created for you. Information in your profile (your name, country/region, and company name) is displayed to the public and will accompany any content you post, unless you opt to hide your company name. You may update your IBM account at any time.

All information submitted is secure.

Choose your display name



The first time you sign in to developerWorks, a profile is created for you, so you need to choose a display name. Your display name accompanies the content you post on developerWorks.

Please choose a display name between 3-31 characters. Your display name must be unique in the developerWorks community and should not be your email address for privacy reasons.

Required fields are indicated with an asterisk (*).

(Must be between 3 – 31 characters.)

By clicking Submit, you agree to the developerWorks terms of use.

 


All information submitted is secure.

Dig deeper into Rational software on developerWorks


static.content.url=http://www.ibm.com/developerworks/js/artrating/
SITE_ID=1
Zone=Rational, DevOps
ArticleID=654330
ArticleTitle=Improve project quality with Rational Team Concert 3.0 and ODC: Part 1. Classify and validate defects
publish-date=05032011