Skip to main content

Using IBM Optim with IBM Rational testing tools to create test databases

Maximizing relevant data generation

Ben Parsons
Ben Parsons is a former IBM employee who participated in a co-operative education program working as a software developer for IBM Rational software. He is currently finishing his undergraduate degree and plans to continue his education in graduate school.
Joseph C. Leong (jcleong@us.ibm.com), Software Engineer, IBM
Joseph Leong
Joseph Leong is a software engineer for IBM and a computer engineering student at Purdue University. Has been involved with the IBM co-operative education program for nearly two years and has worked on both IBM WebSphere and IBM Rational software projects.

Summary:  This article shows you how to supplement the testing power of IBM Rational Performance Tester and IBM Rational Functional Tester with the database and data-generating capabilities of IBM Optim to simplify the testing process and expand the depth of the tests.

Date:  24 Sep 2009
Level:  Introductory
Activity:  1519 views
Comments:  

Introduction

Thorough testing is required for any software offering, and Rational testing tools help you create tests, automate the testing process, and report on the outcome of the tests. This article explains how you can supplement the Rational Performance Tester and Rational Functional Tester capabilities with the database and data-generating capabilities of Optim to help you simplify the testing process and expand the depth of the performed tests. It describes, step-by-step, how you can use IBM® Optim™ features to create expansive and customized test databases and then import the data into IBM® Rational® Functional Tester or IBM® Rational® Performance Tester testing tools to incorporate the data into tests.

This article is written in a modular format to accommodate readers who have varying degrees of experience with Optim, Rational Functional Tester, and Rational Performance Tester software. You can skip sections depending on your interests and needs.

Overview

We explain the steps that are required to customize and import data from an actual database into Rational Performance Tester and Rational Functional Tester setups:

  1. First, we cover creating an infrastructure or an object model that starts with a request, access definition, and column maps for data extraction.
  2. Then we explain generation of a Comma-Separated Values (CSV) file that starts with a request, table maps, and column maps.
  3. Next, we tell you how to import the CSV file into the Rational Functional Tester or Rational Performance Tester data pool creation wizard.
  4. In the last section, we describe how Optim can ensure the integrity of a database that you are testing with Rational Performance Tester.

Software used in these examples and preparation

  • IBM Optim Version 6.5.1
  • IBM® DB2® Version 9.5
  • IBM Rational Performance Tester Version 8.0.0
  • IBM Rational Functional Tester Version 8.0.0

Follow instructions in the Appendix to download, install, and configure the software if you don’t already have it installed. You need only one of the testing products, either Rational Performance Tester or Rational Functional Tester.


Software overview

Optim

IBM Optim software provides an array of database management features (see the link in Resources to learn more). In this article, however, we focus on the test-data management features. These features accelerate application deployment by streamlining the way that you create and manage test environments.

Optim integrated data management also helps in producing a referentially intact subset of data and migration data to build realistic and right-sized test databases. It helps you eliminate the expense and effort of maintaining multiple production clone databases. You can also alter confidential information to protect privacy. If you need further assistance or information, the Optim user information is a set of PDF documents that are installed in Your_Optim_Installation_Directory\RT\DOC.

Rational Performance Tester

This is a tool used to identify the presence and cause of system performance bottlenecks. Rational Performance Tester provides automated testing capabilities for functional testing, regression testing, GUI testing, and data-driven testing. See Resources for where to learn more.

Rational Functional Tester

Rational Functional Tester is an automated functional testing and regression testing tool. It provides automated testing capabilities for functional testing, regression testing, GUI testing, and data-driven testing. (See Resources.)


Creating the access definition

The next step is to create an access definition, which is the template or roadmap for data extraction. The access definition specifies a set of related data and metadata for Optim to process. Use the access definition to define relations for Optim procedures, such as restoring, archiving, moving, and comparing databases. This object is referred to every time that Optim reads the database sources.

To create the access definition:

  1. Open Optim by selecting Start > IBM Optim > Optim.
  2. Select Definitions > Access Definition to open the Access Definition Editor shown in Figure 1.
  3. Click the Table tab to open the Table page.
  4. On the Table page, specify the default qualifier. This is the alias and the ID of the user who created the table. This example uses ALIAS.BEN.
  5. Select the starting table. We use Customers for this example.
  6. Right-click the second row, and then click Add Tables.
  7. Select the Details table by selecting Ref Tbl. Although we do not use this table, it will be useful for exploring Optim.
  8. Save the access definition. This example uses ben.accdef.

Figure 1. Access Definition Editor
Shows Defaul Qualifier, Start Table selections


Creating the column map

The column map replicates columns from the source table in the destination table. You will use it when you convert the extracted data in the database to the new masked data with the masking definition that is contained in this file.

To create a column map:

  1. Select Definitions > Column Map to open the column map editor.
  2. Click New to create a new column map.
  3. Browse the drop-down menus for the Database Table Name fields, and select ALIAS.BEN.CUSTOMERS for the source and destination files as Figure 2 shows.

Figure 2. New column map window
Source and Destination database table selections

  1. Change the salesman ID to RAND(0,999999) to replace the ID with a random number.
  2. Save the column map. This example uses the name BEN.COLMAP.

Figure 3. Column Map Editor view
Nothing selected


Creating the table map

Table maps map source tables to target tables. Optim can use these table maps to alter data in the column maps. These map files are referred to when masking the data and when converting the data to a CSV file.

To create a table map:

  1. Select Definitions > Table Map to start the Table Map Editor (Figure 4).
  2. Click New to create a new table map.
  3. Make sure that the access definition is accurate, and then proceed.
  4. To specify the destination tables, right-click the destination table, and then click Populate.
  5. Fill in the Qualifier box with a database alias.
  6. Add the column map that you created in the previous procedure.
  7. Save the table map.

Figure 4. Table map editor
Arrow to Column Map or 'LOCAL' column cell


Extracting the data

In this step, you create an extraction request to define the extraction and then run the request. The extracted file is a binary-encoded file, which can and will be converted to a CSV file. If you want to repeat this step, be sure to save the extract request. Also, consider saving the extracted.xf file for data comparison in later steps.

  1. Select Actions > Extract to open the Extract Request Editor (Figure 5).
  2. Type a name for the extracted file.
  3. Select the access definition that you created previously.
  4. Save the extraction request.
  5. Click Run to extract the data.
  6. Verify that there are no errors in the report.

Figure 5. Extract request editor
Shows BEN.ACCDEF         selected from drop-down menu


Converting an .xf file to a CSV file

This procedure uses the column map to convert the extracted data into masked data. It then converts it into a CSV file, which can be imported into Rational testing software. This requested conversion process can be saved and reused. If you encounter a RUNSTATS error see DB2 Setup.

To convert an .xf file to a CSV file:

  1. Click Actions > Convert to open the Convert Request Editor (Figure 6).
  2. Select the table map that you created previously.
  3. Type the name of destination file. This is where the masked data will be stored until it is converted into a CSV file.
  4. Select the location for the control file.
  5. Run the conversion.
  6. Save the conversion request for future use.

Figure 6. The Convert Request Editor for the Customers table
Fields for Source, Control, and Destination files

  1. Click New to create a new conversion. The new conversion is the CSV file.
  2. Use the same control file as before, because it is overwritten over.
  3. Specify the source file for the .xf file that you just created by typing the source file name. This file will be the CSV file.
  4. Select the table map that you created previously.
  5. Click Create a Comma separated file to create a CSV file.
  6. Click the Comma Separated tab.
  7. Select Customers as the start table.
  8. Right-click the table name, and modify the column list so that it uses all of the columns.
  9. Run the conversion. Ignore the warning about the unused column map, because the column map was used in the first conversion.

Editing CVS files and creating data pools

Of course, you need to define how you are going to store your data.

Editing CVS files

Optim does not name the columns in the CSV file; therefore, you must give them names. Just add a row at the beginning of the CSV file to define the columns names. In the example in Figure 7, ID, Name, Address, City, State, and so on constitute the column names.


Figure 7. CSV file with column names
Screen capture of the code in the CVS file

Creating data pools

To create a data pool, begin by importing a CSV file into a Rational testing product:

  1. Open Rational Performance Tester or Rational Functional Tester.
  2. Right-click in the Test Navigator, and click New > Datapool [sic].
  3. Name the data pool, and use the default dimensions (you will override these dimensions later).
  4. Select the CSV file, and select First Record is Variable Information, because you added those earlier.
  5. Click Finish, and open the editor to view your imported data.

Figure 8. Rational Performance Tester with an imported data pool
Test Navigator view

Using Optim to help maintain test database integrity

In addition to importing data into Rational Performance Tester, you can use Optim with Rational Performance Tester to test the integrity of the data that Rational Performance Tester saves on your server. If you saved the original data from the .xf file before you converted it to a CSV file, you can connect to the database of your test server and compare the server-based data to the original data, which is what should have been imported. This comparison helps you ensure that the data is being placed into your server database correctly.

The following procedure demonstrates this principal with the two .xf files used in earlier procedures: data.xf, which has the original salesman ID, and data2.xf, which has a random number rather than the salesman ID.

To compare database entries:

  1. Click Actions > Compare to open the Compare Request editor.
  2. Select the location for the comparison file to be saved.
  3. Click the Source tab.
  4. Specify the source files, Data.xf and Data2.xf.
  5. Run the comparison.

As Figure 9 shows, a report window shows the differences in the two files.


Figure 9. Detailed report of the differences between the files compared
Screen capture of detailed table


Summary

This article outlined the procedure for using Optim to generate test data for Rational Performance Tester and Rational Functional Tester:

  1. You created a database alias to use to refer to your database.
  2. You created an access definition to define how to access the database for your purposes, and then created a column map and a table map to further define what you wanted to do with the data.
  3. After the definitions were completed, you extracted the data, converted the entries into a masked .xf file, and then converted the .xf file to a CSV file.
  4. You then edited the CSV file and imported it into Rational Performance Tester.

Note:
These could also have been easily imported into Rational Functional Tester.

With these steps, it is easy to see how you can easily and thoroughly create test data with Optim, Rational Performance Tester, and Rational Functional Tester. Most of the steps can be saved, so it is easy to repeat the process. Many of the files like the ones we used are already available if you are using Optim as your database management tool. Therefore, integration with Rational Performance Tester and Rational Functional Tester is fast and easy.


Appendix: Configuration and setup

Installation instructions follow for all of the various pieces of software used in this demonstration.

Rational Functional Tester and Rational Performance Tester

Use the IBM Installation Manager to install Rational Functional Tester or Rational Performance Tester. This wizard leads you through a simple process to download and install the necessary files. Prepare by downloading the software:

  1. Download the IBM Installation Manager.
  2. Download Rational Functional Tester or Rational Performance Tester.

IBM Installation Manager

  1. Start IBM Installation Manager and select Install.
  2. Select the product to be installed, and then click Next.

Figure 10. Selecting Rational Functional Tester for installation
IBM Installation Manager, Install Packages window
  1. Verify the installation directory, and then click Next.
  2. If you would like to use an existing version of Eclipse, specify it here by checking

the Extend an existing Eclipse box and provide the path. Then click Next.

  1. Select the language for the installation, and then click Next.
  2. Select all of the features that you want to install with Rational Functional Tester or Rational Performance Tester.
  3. Select how you would like to access support, and then click Next.
  4. Review the summary of packages to be installed, and then click Next.
  5. The Installation Manager will now proceed to download and install the products.

The DB2 database

Having the IBM® DB2® database on your computer is a prerequisite for Optim. You might need to set up a sample database manually if you did not specify one to be created during the wizard installation. We will use this database in the article.

You can create the sample database with the DB2 First Steps wizard:

Note:
if you already specified that you wanted to create a sample database during the DB2 installation, skip these steps.

  1. Go to http://www-01.ibm.com/software/data/db2/9/ and get a copy of IBM DB2 Version 9.5 Express-C.
  2. Extract the files and run setup.exe or the respective setup file.
  3. Review any prerequisite information and migration instructions that are included.
  4. Select Install a product and then click Install new.
  5. Follow the wizard instructions to completion to set up DB2 to your specifications.

Optim

You need to configure Optim before you use it. Follow these steps to set it up for the examples in this article:

  1. Select Options > Sign Optim Exit. Choose to sign the default option exit, and then click Next.
  2. Select Tasks > Configure the First Workstation, and then click Next.
    Note that Optim can be set up in a server configuration, but for this example, we will install it on one workstation.
  3. Type or paste in your license key. You can find the 30-day trial key in the readme.txt file.
  4. Create a New Optim Directory and Registry Entry (see Figure 11). You will use OPTIMDIR to work with the example in this article.
  5. Then click Proceed.

Figure 11. Creating the Optim directory
'Configure First Workstation' window
  1. Select the check box to keep the data in Unicode format, and then click Proceed.
  2. Type the database information. For the example in this article, use DB2 v9.5.
  3. Provide the information to connect to your database. In this article, connect to the DB2 sample database with the connection string SAMPLE, as Figure 12 shows.
  4. Provide the user ID and password. This authenticates with an account on the operating system.
  5. Then click Proceed.

Figure 12. Database connection information
'Connect to Database' window
  1. Leave the default settings to bind the plans, and then click Proceed.
  2. Click Proceed. Optim stores your database information.
  3. Skip the options for some of the additional functions until prompted about the user options file.
  4. Create the user options file with the default options and then continue to the end of the setup.

Create the database alias

You need to create a database alias so that Optim can connect to your database. For this example, DB2 has been installed with the sample data. This is the data that you will extract.

To create a database alias:

  1. Start the Optim configuration tool by clicking Start > IBM Optim > Optim Configuration.
  2. Click Tasks > Create/Update DB Alias to open the database alias editor shown in Figure 13.
  3. Select to use the existing Optim Directory and Registry Entry, OPTIMDIR. Then click Proceed.
  4. Select the Create/Update DB Alias check box, and then click Proceed.
  5. Click Create New, and type ALIAS in the Name field.
  6. Ensure that the DBMS specifications are automatically entered correctly.
  7. Connect to the database.
  8. Use the default values in the Bind/Drop Plans window.

Figure 13. Database alias and Blind/Drop Plans values
Shows entries described
  1. Skip always requiring a password. Do not share the database information.
  2. Click Proceed to finalize the database connection.
  3. You do not need to create the primary keys, sample tables, and the privacy tables.
  4. Do not select the option to create another database alias.
  5. Click Proceed to finish.

The first time that you start Optim, you will need to specify settings folders. You can create these folders in the IBM Optim folder, where the other Optim files are located.

Error resolution

During the Optim demonstration, you might encounter a RUNSTATS error when converting the database to a CSV fie.

To avoid this error, open the DB2 command window, and manually enter the RUNSTATS command to correct the table that is prompting errors:

  1. Open the DB2 command window.
  2. Type db2 to open the interactive mode.
  3. Type connect to sample to connect to the sample database.
  4. Type RUNSTATS ON TABLE table_name to perform RUNSTATS manually on the table that is prompting the error, where table_name is the name of your table.

Users need to open the DB2 command window and manually perform the RUNSTATS command to correct table prompting errors.


Figure 14. RUNSTATS action in progress on corresponding tables
ALT Screen capture of code output

Resources

Learn

Get products and technologies

Discuss

About the authors

Ben Parsons

Ben Parsons is a former IBM employee who participated in a co-operative education program working as a software developer for IBM Rational software. He is currently finishing his undergraduate degree and plans to continue his education in graduate school.

Joseph Leong

Joseph Leong is a software engineer for IBM and a computer engineering student at Purdue University. Has been involved with the IBM co-operative education program for nearly two years and has worked on both IBM WebSphere and IBM Rational software projects.

Comments



Trademarks  |  My developerWorks terms and conditions

Help: Update or add to My dW interests

What's this?

This little timesaver lets you update your My developerWorks profile with just one click! The general subject of this content (AIX and UNIX, Information Management, Lotus, Rational, Tivoli, WebSphere, Java, Linux, Open source, SOA and Web services, Web development, or XML) will be added to the interests section of your profile, if it's not there already. You only need to be logged in to My developerWorks.

And what's the point of adding your interests to your profile? That's how you find other users with the same interests as yours, and see what they're reading and contributing to the community. Your interests also help us recommend relevant developerWorks content to you.

View your My developerWorks profile

Return from help

Help: Remove from My dW interests

What's this?

Removing this interest does not alter your profile, but rather removes this piece of content from a list of all content for which you've indicated interest. In a future enhancement to My developerWorks, you'll be able to see a record of that content.

View your My developerWorks profile

Return from help

static.content.url=http://www.ibm.com/developerworks/js/artrating/
SITE_ID=1
Zone=Rational
ArticleID=430020
ArticleTitle=Using IBM Optim with IBM Rational testing tools to create test databases
publish-date=09242009
author1-email=bsparson@us.ibm.com
author1-email-cc=
author2-email=jcleong@us.ibm.com
author2-email-cc=

My developerWorks community

Tags

Help
Use the search field to find all types of content in My developerWorks with that tag.

Use the slider bar to see more or fewer tags.

Popular tags shows the top tags for this particular content zone (for example, Java technology, Linux, WebSphere).

My tags shows your tags for this particular content zone (for example, Java technology, Linux, WebSphere).

Use the search field to find all types of content in My developerWorks with that tag. Popular tags shows the top tags for this particular content zone (for example, Java technology, Linux, WebSphere). My tags shows your tags for this particular content zone (for example, Java technology, Linux, WebSphere).

Rate a product. Write a review.

Special offers