Five ways to incorporate defect submissions into your tests on the fly

Use Rational Performance Tester or Rational Service Tester for SOA Quality to tightly integrate defect tracking into testing

Through five customer-oriented scenarios, you learn, step by step, how to use your IBM® Rational® Performance Tester and IBM® Rational® Service Tester for SOA Quality software, along with IBM® Rational Team Concert™, to create a defect tracking process that is fully integrated into your testing workflow. This increases efficiency and productivity while creating a more tightly integrated software testing process.

Share:

Marya Alston (malston@us.ibm.com), Staff Software Engineer, IBM

author photoMarya Alston is a software quality engineer with the IBM Rational System and Integration Test organization, where she currently works on Rational Performance Tester, focusing on the HTTP protocol. She has led major release activities.



Luc Auvray (luc.auvray@fr.ibm.com), Staff Software Engineer, IBM

author photoLuc Auvray is a software quality engineer and team lead at the IBM Rational organization's Toulouse, France, development lab. He currently works on the Rational Test RealTime, Rational Service Tester for SOA Quality, and Rational Performance Tester products in the System Verification team.



Ali Manji (amanji@ca.ibm.com), Software Developer, WebSphere Studio Application Developer Service, IBM Toronto Lab

photo of Ali Manji

Ali Manji is a software engineer at the IBM Toronto Lab and works with the WebSphere Studio Application Developer Service team. You can reach Ali at amanji@ca.ibm.com .


developerWorks Contributing author
        level

22 February 2011

Also available in Chinese

Software defect submission is more than a communication vehicle between development and system test groups. It is a method used to track defects, measure their scope and impact, and flush out related problems. Ultimately, it is a process that progressively improves software quality.

The System Verification Test (SVT) team for IBM® Rational® Performance Tester and IBM® Rational® Service Tester for SOA Quality has developed five scenarios that incorporate defect submissions "on the fly," either by using integration points (IPs) or by setting preferences within Rational Performance Tester. By following the steps in this article, you learn how you can use these two testing tools to create an efficient defect management process, plus how to get the most benefit from integrating a bug tracking tool into your test workflow.

IBM also offers several full-featured defect reporting and tracking tools, including IBM® Rational® ClearQuest®, IBM® Rational Team Concert™, and IBM® RETAIN® software. We discuss some of these in this article, too.

Internal defect tracking tools

Knowing what each of these tools offers is essential to smoothly integrating them into your testing processes.

Bugzilla

Bugzilla is a web-based general purpose bug tracker and testing tool that was originally developed and used by the Mozilla project.

Rational ClearQuest

The Rational ClearQuest toolset provides workflow-based, web-enabled project lifecycle defect management and tracking. It facilitates control and communication from defect report to resolution.

Rational Team Concert

The Rational Team Concert collaborative development environment provides tools for agile and traditional planning, project status checks, reporting, customizable process support, work item management, change management, source code management, and build management. You can implement it in either a unified or modular manner. It connects geographically dispersed development teams to help increase individual and team productivity, compress development cycles, and deliver high-quality software faster.

IBM RETAIN

The IBM RETAIN (REmote Technical Assistance Information Network) is a worldwide computer system that IBM employees, business partners, and vendors use to help solve the problems that clients have with hardware and software products. Although the Rational Performance Tester SVT team does not use RETAIN, it is frequently used by the IBM's product support representatives.


Value of customer scenarios to track defect reports

When evaluating product usability, one of the objectives of the System and Integration Test organization is to minimize the number of products launched and the clicks required to complete a user flow. Customers value a product's ease of use, so the more logical and less complicated the user flow, the higher the percentage of adoption.

Implementing customer scenarios to include the flow of defect submissions expands integration coverage. In addition, scenarios help through a strong demonstration of end-to-end user flows when you are using multiple IBM solutions. Integrating your defect tracking tool with existing IBM software adds value to your software development process in two key ways:

  • User productivity: A successful scenario is one that mimics how a customer uses the products and has a simple workflow. When these elements align, efficiency increases, and this results in more time allocated to finding problem areas within the software.
  • Integrated process: Using scenarios becomes an integrated process solution for supporting an identified customer green thread scenario. Integrating your testing software (whether Rational Performance Tester or Rational Service Tester and Rational ClearQuest) creates an association or linkage that accomplishes performance test objectives and grants the ability to submit defects on the fly. This, in turn, reduces the time required to separately launch your defect tracking tool and allows more time to focus on the tests (performance, functional, and so forth), as well as to determine the problem. An integrated process also leads to the following positive outcomes:
    • Productivity for the end user
    • Process implementation
    • Error reduction (which is dependent on valid input from the end user)

In determining your defect tracking submission scenarios as this article describes, it is vital that you are familiar with these IBM tools:

  • Rational ClearQuest
  • Rational Performance Tester
  • Rational Team Concert
  • Rational Quality Manager

The 5 scenarios

With a working knowledge of these, your team can make the right decision about which workflow to use.

Table 1. Defect tracking scenarios
ScenarioNameDescription
1Rational Team Concert defect/work item via Rational Performance Tester/Rational Service Tester Work Items perspectiveRational Team Concert defect submission by clicking on the "New Work Item" button via Rational Performance Tester
2Rational Team Concert defect/work item via Rational Performance Tester or Rational Service TesterRational Team Concert defect submission from test results via Rational Performance Tester
3Rational Team Concert defect from test results within Rational Quality Manager (on a configured CLM server)Rational Team Concert defect submission from test results via Rational Quality Manager after executing a Rational Performance Tester schedule (on a configured CLM server)
4Bugzilla defect via Rational Performance Tester or Rational Service TesterBugzilla defect submission from test results via Rational Performance Tester
5Rational ClearQuest defect via Rational Performance Tester or Rational Service TesterRational ClearQuest defect submission from test results via Rational Performance Tester

Scenario 1. Rational Team Concert defect or work item via the Rational Performance Tester or Rational Service Tester Work Items perspective

This user flow has been identified as the most efficient and simplest way to create a defect on the fly after selecting to install the new feature (the Jazz Eclipse client for Rational Team Concert and Rational Quality Manager Version 3.0) and then creating a repository connection for Rational Team Concert within Rational Performance Tester.


Note: Rational Performance Tester Version 8.2.0.2 is packaged with Jazz Eclipse client v3.0. Previous 8.2 releases are packaged with Jazz Eclipse client v2.0.

  1. During the installation of Rational Performance Tester, choose to install this feature: Jazz Eclipse Client for Rational Team Concert and Rational Quality Manager 3.0.
Figure 1. Selecting the Jazz Eclipse client
Screen capture: selecting during installation
  1. After installation finishes, launch Rational Performance Tester (full Eclipse).
  2. From the main menu, select Window > Open Perspective > Work Items.
  3. When the Work Items perspective is in view (Figure 2), you will see this message:
    You are not connected to a project area.
Figure 2. Work Items perspective (default) in Rational Performance Tester
Screen shows no connection to a Project Area
  1. Click Create a Repository Connection to open the Jazz Repository Connection wizard.
  2. Enter valid information in the URI, Name, User ID, and Password fields.
Figure 3. Jazz Repository Connection wizard
Location, Authentication and other values entered
  1. After connecting to a repository, click the arrow by the New Work Item icon (see Figure 4) to get a list of defect-related options (Figure 5).
Figure 4. New Work Item icon
Arrow points to New Work Item icon in top toolbar
Figure 5. Work item options
Screen capture: list of options
  1. Select Defect to open a New Defect Entry form.
Figure 6. New Work Item entry form
Fields to enter defect details

Scenario 2. Rational Team Concert defect or work item in Rational Performance Tester or Rational Service Tester

  1. In Rational Performance Tester, set Test Log Editor Preferences for Submit, Search, and Open URLs to point to a valid Rational Team Concert server. See the text that follows and Figure 10 for examples.
    1. Submit URL:
      https://<your jazz server: port number>/jazz/web/projects/RPT#action=com.ibm.team.workitem.newWorkItem
    2. Search URL:
      https://<your jazz server: port number>/jazz/web/projects/
      RPT#action=jazz.viewPage&id=com.ibm.team.workitem
    3. Open URL:
      https://<your jazz server: port number>/jazz/web/projects/RPT#action=com.ibm.team.workitem.viewWorkItem&id=
  2. Open an existing test log.
  3. Click the Events tab.
  4. Select an event (at the root or page level) within the test log.
  5. Click the appropriate button to Add, Submit, or Search a defect. (See Figure 11 for an example).

Using the URLs specified in Step 1, the appropriate view is launched on the specified Rational Team Concert server.

Scenario 3. Rational Team Concert defect from test results within Rational Quality Manager

There are various ways to submit a defect from Rational Team Concert. The steps outlined here support the overall flow when leveraging a server configured for Collaborative Lifecycle Management (CLM).

Prerequisites:
Although clicking the defect icon in Rational Quality Manager to launch Rational Team Concert to submit a defect seems seamless, there are steps required to make this function work efficiently.

  1. Configure both Rational Quality Manager and Rational Team Concert servers to include friendship links. These links are entered through the Jazz Server administration.
  2. In Rational Team Concert and Rational Quality Manager project administration, add integration links between the Rational Team Concert and Rational Quality Manager projects.

After configuring both servers, proceed with the steps that follow.

  1. Configure the Rational Quality Manager adapter:
    1. Set preferences in Rational Performance Tester for the Quality Manager Adapter by selecting Window > Preferences > Quality Manager Adapter, as Figure 7 shows.
Figure 7. Rational Quality Manager adapter preferences
Picture of preferences setting
  1. If not already in view, open the Adapter view by selecting Window > Show View.
  2. Validate the connection to the Rational Team Concert server by clicking the connection icon in the Rational Quality Manager Adapter view. A message indicating a successful connection will appear in the Messages log (Figure 8).
Figure 8. Adapter connection status
Graphic shows the adapter connection status.
    1. Launch Rational Quality Manager.
    2. Create a test case by selecting Construction > Create Test Case.
    3. Import your test script by selecting Construction > Import Test Script.
    Select the adapter that you created and connected to through Rational Performance Tester.
  1. Enter a project path or name.
  2. Select the test or schedule to run.
  3. Click Finish and Import.
    1. Open the test case that you created in Step 5.
    2. Under Table of Contents, select Test Script.
    3. Click the + (plus) sign, add the test script to the test case, and click Save.
    4. Select the test case, and click the Test Execution Record (TER).
    5. Click the green arrow > with the + sign to generate a new TER.
    6. Select to reuse an existing test environment.
    7. After TER creation, select the TER, and click the green arrow to run it.
    8. When the status bar appears, test execution begins and a Progress Summary is displayed.
  4. After test execution, in the event of a problem, select Defects from the left panel to choose either of these options:
    1. Click Create Defect, as Figure 9 shows, to create a new defect report (this automatically launches a Rational Team Concert project on a configured CLM server).
    2. Go to Work Items to query existing related defects (this automatically launches a Rational Team Concert project on a configured CLM server).
Figure 9. Defect selection from the Rational Team Concert panel
Defects icon (a blue bug labeled Defects)

Scenario 4. Bugzilla defect via Rational Performance Tester or Rational Service Tester

You can set preferences within the Rational Performance Tester to specify the default method or your tool of choice for submitting defect reports. By default, the preferences for the Test Log Editor (accessible in Rational Performance Tester by selecting Window > Preferences > Test > Test Log Editor) are automatically set to Bugzilla. However, the default URLs will automatically route the defect to the Eclipse project team. You can configure Test Log Editor Preferences to have defects sent to a standalone server setup and managed by you or your team.

  1. Install a local instance of Bugzilla. See the Mozilla wiki for instructions, or see the Bugzilla Setup Guide for an example.
  2. Set preferences in the Test Log Editor to specify the name of the server configured in Step 1. See Figure 10 for an example.
Figure 10. Test Log Editor preferences (default)
Picture shows Test Log preferences for Bugzilla.
  1. Apply changes.
  2. Open an existing test log.
  3. Click the Events tab.
  4. Select an event (at the root or page level) within the test log.
  5. Click the appropriate button to Add, Submit, or Search a defect.
Figure 11. Defects pane within the test log
Graphic of Add, Submit and Search buttons

Based on the default URLs, the user is prompted to authenticate and proceed with the action according to the button selected.

Scenario 5. Rational ClearQuest defect via Rational Performance Tester or Rational Service Tester

Enabling the integration point between the Rational ClearQuest and Rational Performance Tester environments requires installing the latest released version of "ClearQuest Client" and "Rational ClearQuest plug-ins." What follows are the steps for installing client and getting the required plug-ins.

  1. Install the latest released version of Rational ClearQuest native client.
    Note: If Rational Performance Tester already exists on your system, you will need to install the client in a different directory. Rational ClearQuest does not share a shell with Rational Performance Tester.
  2. Configure the Rational ClearQuest client to connect to a RATLC replica using Rational ClearQuest Maintenance Tool: click on File > Import Profile.
  3. Set up the replica for your site
  4. Provide a valid address for the SMTP server (mailrelay.myserver.mycompany.com).
    Note: Without an SMTP server, users are allowed to complete the configuration and enter information for submitting defects but not allowed to submit or save the defect information. Therefore, this step is critical in enabling the ability to submit a Rational ClearQuest defect through Rational Performance Tester.
  5. Install Rational Performance Tester if it is not already installed.
    Note: Rational Performance Tester Version 8.2 cannot be integrated with the latest released Rational ClearQuest Version 7.1.2.
  6. Install Rational ClearQuest plug-ins for Eclipse/Rational Performance Tester
    This is a key step. It is best to install the Rational ClearQuest Eclipse extension to your Rational Performance Tester solution by using the Integration Manager. By default, the Rational ClearQuest Test Manager perspective is hidden.
  7. Launch Rational Performance Tester in Full mode by selecting Start > Programs > IBM Software Delivery Platform > IBM Rational Performance Tester > IBM Rational Performance Tester > Full Eclipse.
  8. Open the Manage Connections window by selecting ClearQuest > Database from the menus.
    Important: This step allows the enablement of the Submit, Search, and Query buttons in the Test Log view. If this step is not administered, the buttons remain disabled.
  9. The RATLC replica (from Step 2) must exist in the available databases. Select the database, and connect.
  10. Switch to the Performance Test perspective and open a test log.
  11. Click the Events tab within the Test Log view.
  12. In the ClearQuest section, click the New button to open a new window titled "Create (ChangeRequest) RATLCxxx."
  13. Fill in the desired information and submit the defect.
  14. Click the newly created defect and click Open.
  15. The end user should be able to open the defect as if using the web-based Rational ClearQuest client.
  16. Click the Select button, and try to select a defect by following the dialog.

Summary

By using the approach outlined in these steps and scenarios, you have a pathway that enables you to use your Rational Performance Tester and Rational Service Tester for SOA Quality software to create a defect tracking process that is fully integrated into your test flow. As a result, you can help increase efficiency and productivity, while creating a more tightly integrated software testing process, too.


Acknowledgements

Thank you to the following people for taking time to provide technical reviews and comments: Dennis Moya, Qiang (George) Liu, Sanjay Joshi, Skye Bischoff, and Todd Merriweather.

Resources

Learn

Get products and technologies

Discuss

Comments

developerWorks: Sign in

Required fields are indicated with an asterisk (*).


Need an IBM ID?
Forgot your IBM ID?


Forgot your password?
Change your password

By clicking Submit, you agree to the developerWorks terms of use.

 


The first time you sign into developerWorks, a profile is created for you. Information in your profile (your name, country/region, and company name) is displayed to the public and will accompany any content you post, unless you opt to hide your company name. You may update your IBM account at any time.

All information submitted is secure.

Choose your display name



The first time you sign in to developerWorks, a profile is created for you, so you need to choose a display name. Your display name accompanies the content you post on developerWorks.

Please choose a display name between 3-31 characters. Your display name must be unique in the developerWorks community and should not be your email address for privacy reasons.

Required fields are indicated with an asterisk (*).

(Must be between 3 – 31 characters.)

By clicking Submit, you agree to the developerWorks terms of use.

 


All information submitted is secure.

Dig deeper into Rational software on developerWorks


static.content.url=http://www.ibm.com/developerworks/js/artrating/
SITE_ID=1
Zone=Rational
ArticleID=627027
ArticleTitle=Five ways to incorporate defect submissions into your tests on the fly
publish-date=02222011