Programmatically modify Rational datapools

Automate mundane test data maintenance tasks

IBM Rational tester products provide an excellent repository for test data, known as datapools. However, maintenance of the contents of datapools can become a full-time task in cases where test data might not be static. Tim von Niessen describes a programmatic approach to modification of the contents of your datapools, using Java to update your test data from Rational Functional Tester, Rational Performance Tester, and Rational Service Tester for SOA Quality.

Share:

Tim von Niessen, Consulting IT Specialist, IBM Global Business Services

Photo of Tim von NiessenTim von Niessen is an IBM Certified Consulting IT Specialist for IBM Global Business Services. He has been using IBM Rational tools for testing since 2004 and recently led an initiative to convert Rational TestManager and Rational Robot tools to the new Eclipse-based versions. As a Java application developer, Tim focuses on implementing good programming practices when writing custom code for Rational Functional Tester, Rational Performance Tester, and Rational Service Tester for SOA Quality customizations.



26 June 2012

Also available in Chinese

The IBM® Rational® tester products that are based on automated test scripts provide a facility for maintaining test data within the test project in the format of datapools. This article describes programmatic modification of datapools. It is based on the assumption that you are already familiar with TPTP and Rational datapools.

There are several questions and newsgroup threads on the Internet for discussing ways to programmatically modify datapools for IBM Rational tester products. The answers in response tend to point to either manually updating the files in the datapool editor or importing new data through comma-separated value (CSV) files.

The manual approach completely bypasses any type of a programmatic solution, and the CSV approach is still a manual effort, particularly if you have a large number of datapools. You might also encounter problems with CSV-imported datapools, where you must delete your existing datapool and recreate it. This will break all script references, which must then be manually corrected.

There are utilities available for programmatically modifying datapools that will be discussed later in this article, which addresses these tools:

  • IBM Rational Functional Tester
  • IBM Rational Performance Tester
  • IBM Rational Service Tester for SOA Quality
  • Eclipse Test and Performance Tools Platform (TPTP) Project

Note:
A general knowledge of IBM Rational tester products and Java programming is assumed.

Why would you want to do this?

Before getting into solutions for this, one might ask "Why would you want or need to do this?" Normally in a testing environment, the optimal approach would be to have the test environment set back to a base state, so that all test execution would be against identical data, thereby providing apples-to-apples comparisons. This utopian scenario is not always possible.

You could be in a situation where the test data is date- or time-sensitive. Maybe there are multiple teams working from the same application, and the data is continually in flux. There could be data content or format changes occurring for a new build or release of the application under test.

You may be in a situation where data is regularly changing through a conversion process. In the case where you need to go back to perform regression tests, you might also need to back-date your test data.

Generating large quantities of datapools can often occur when using Rational Functional Tester or Rational Service Tester for SOA Quality (a functional and regression for service-oriented architecture, or SOA-based applications). You could easily end up with hundreds of different datapools, thus making the manual CSV import an unrealistic solution.

An entirely different approach might be to do away with the datapool as a source of testing data, using either a CSV-formatted file or some other type of flat file, and accessing the test data through custom code or a test data server utility. This would require the creation of a facility to load the data and deliver it to the executing test script. But that is a completely different topic from this article.

Regardless of the reasons that make it desirable, this article investigates various means of performing the task of programmatically modifying Rational datapools.

The *.datapool file is actually an archive file, which can be opened by many compression utility programs. There is a single file inside the archive named “ResourceContents. The ResourceContents file is a XML Metadata Interchange or XMI file. This type of file is a container for a form of metadata that is represented in XML format. In the case of these datapool files, the XMI represents a simple table.

Note:
Rational Functional Tester also uses a TPTP-based datapool format, although the format is very different from Rational Service Tester and Rational Performance Tester datapools. Rational Functional Tester creates an XML-formatted text file that represents the contents of the datapool. The file has an rftdp suffix.

There are editors in these testing tools that you can use to manually edit the files. They behave much like a spreadsheet program. It is also possible to import data into all of these datapools through CSV files.

The following screen captures show the two different datapool editors.

Figure 1. TPTP RPT_RST datapool editor (Rational Performance Tester, Rational Service Tester)
Shows key string, value string, value 2 string columns
Figure 2. Rational Functional Tester datapool editor
EquivalenceClass1 tab

Although the data appears very similar, the actual contents of the files are quite different.


Looking inside the datapool files

Unless you are planning to create a custom program to directly modify the XMI or XML contents of the datapool, the following code is not likely more than a point of interest. However, later in this article, we will be looking at methods for doing exactly that.

Code Listings 1 and 2 show the differences in the core content of the datapool types. The order of elements has been modified for the sake of clarity.

Listing 1. Rational Performance Tester and Rational Service Tester TPTP datapool XMI, ResourceContents
<?xml version="1.0" encoding="UTF-8"?>
<Common_Datapool:DPLDatapool xmi:version="2.0" xmlns:xmi="http://www.omg.org/XMI" 
        xmlns:Common_Datapool="http://www.eclipse.org/hyades/models/common/datapool.xmi" 
        id="A1E10A110B8F7BA0EFDCE26335653537" 
        description="Sample TPTP Datapool" 
        name="RPT_RST_Datapool">

    <datapoolSpec  id="A1E10A110B8FA2B0EFDCE26335653537">
        <variables id="A1E10A110B90DB30EFDCE26335653537" 
                    name="RPT_RST_Key"     type="String"/>
        <variables id="A1E10A110B912950EFDCE26335653537" 
                    name="RPT_RST_Value_1" type="String"/>
        <variables id="A1E10A110B915060EFDCE26335653537" 
                    name="RPT_RST_Value_2" type="String"/>
    </datapoolSpec>
  
    <equivalenceClasses id="A1E10A110B8FF0D0EFDCE26335653537" name="EquivalenceClass1">
        <records>
            <cells value="<Literal>1st KEY</Literal>"           
                    variable="A1E10A110B90DB30EFDCE26335653537"/>
            <cells value="<Literal>RPT_RST #1</Literal>"        
                    variable="A1E10A110B912950EFDCE26335653537"/>
            <cells value="<Literal>The first record</Literal>"  
                    variable="A1E10A110B915060EFDCE26335653537"/>
        </records>
       ...

    </equivalenceClasses>

</Common_Datapool:DPLDatapool>
Listing 2. Rational Functional Tester datapool, *.rftdp
<?xml version="1.0" encoding="UTF-8"?>
<Datapool L="Datapool">
    <Variables L="DpVariables">
        <Variable L="DatapoolVariable">
            <Name>RFT_Key</Name>
            <Id>V0.hXs1seXm3Z7:2Asi9F:NFfWUid:8WW</Id>
            <Description T="null"/>
            <SuggestedType L="DatapoolSuggestedType">
                <TypeCode>0</TypeCode>
                <SuggestedClassName>String</SuggestedClassName>
            </SuggestedType>
            <Role>-1</Role>
        </Variable>
        <Variable L="DatapoolVariable">
            <Name>RFT_Value_1</Name>
            <Id>V1.hXs1seXm3Z7:2Asi9F:NFfWUid:8WW</Id>
            <Description T="null"/>
            <SuggestedType L="DatapoolSuggestedType">
                <TypeCode>0</TypeCode>
                <SuggestedClassName>String</SuggestedClassName>
            </SuggestedType>
            <Role>-1</Role>
        </Variable>
        <Variable L="DatapoolVariable">
            <Name>RFT_Value_2</Name>
            <Id>V2.hXs1seXm3Z7:2Asi9F:NFfWUid:8WW</Id>
            <Description T="null"/>
            <SuggestedType L="DatapoolSuggestedType">
                <TypeCode>0</TypeCode>
                <SuggestedClassName>String</SuggestedClassName>
            </SuggestedType>
            <Role>-1</Role>
        </Variable>
        ...

    </Variables>

    <EquivalenceClasses L="DpEquivalenceClasses">
        <EquivalenceClass L="DatapoolEquivalenceClass">
            <Name>EC1</Name>
            <Id>EC1.hXs1seXm3Z7:2Asi9F:NFfWUid:8WW</Id>
            <Description T="null"/>
            <Records L="DpRecords">
                <Record L="DatapoolRecord">
                    <Cells L="DpCells">
                        <Cell>1st KEY</Cell>
                        <Cell>RFT #1</Cell>
                        <Cell>The first record</Cell>
                    </Cells>
                </Record>
                ...

            </Records>
        </EquivalenceClass>
    </EquivalenceClasses>
    
    <DefaultEquivalenceClass>0</DefaultEquivalenceClass>
    <EquivalenceClassNextId>2</EquivalenceClassNextId>
    <VariableNextId>3</VariableNextId>

</Datapool>

The TPTP style datapool is defined as an XMI object; whereas, the Rational Functional Tester datapool uses a standard XML format, starting with the <Datapool> tag.

As Listing 3 shows, the TPTP style datapool uses a <datapoolSpec> element to define the variables.

Listing 3. TPTP <datapoolSpec> element
<datapoolSpec  id="[ID for the datapool]">
        <variables id="[ID for the variable]">" 
                    name="[name of the variable]”	type="[variable format]"/>
        <variables ...

The Rational Functional Tester style datapool defines the variables in a <Variables> element, as Listing 4 shows.

Listing 4. Rational Functional Tester <Variables> element
<Variables L="DpVariables">
        <Variable L="DatapoolVariable">
            <Name>[name of the variable]</Name>
            <Id>V[ID for the column].[ID for the datapool]</Id>
            <Description T="null"/>
            <SuggestedType L="DatapoolSuggestedType">
                <TypeCode>0</TypeCode>
                <SuggestedClassName>[variable format]</SuggestedClassName>
            </SuggestedType>
            <Role>-1</Role>
        </Variable>
        <Variable ...

Actual datapool data is included in equivalence class elements, again with notable differences. For TPTP, the element used is <equivalenceClasses>.

Listing 5. TPTP <equivalenceClasses> element
 <equivalenceClasses id="[ID for the equivalence class]" name="EquivalenceClass1">
        <records>
            <cells value="<Literal>[variable value]</Literal>"           
                    variable="[ID for the variable]"/>
            <cells ...

For the TPTP implementation, the [ID for the variable] will match the value for the variables definition in the <datapoolSpec> element.

Rational Functional Tester implements the equivalences classes in a different format, as shown in Listing 6.

Listing 6. Rational Functional Tester <EquivalenceClasses> element
   <EquivalenceClasses L="DpEquivalenceClasses">
        <EquivalenceClass L="DatapoolEquivalenceClass">
            <Name>EC1</Name>
            <Id>EC[ID for the row].[ID for the datapool]</Id>
            <Description T="null"/>
            <Records L="DpRecords">
                <Record L="DatapoolRecord">
                    <Cells L="DpCells">
                        <Cell>[variable value]</Cell>
                        <Cell ...
                    </Cells>
                </Record>
                <Record ...

Tip:
The key difference is that the TPTP relationship is based on a reference to the definitions in the <datapoolSpec> element; whereas, Rational Functional Tester uses more of a column and row type of design.

If you are directly editing the XMI or XML in the datapool contents, this is a notable consideration. There will be more on this approach later in this article.


Accessing Rational datapools through Java APIs

There are full application programming interfaces (APIs) available for both TPTP and Rational Functional Tester formats, providing Java classes for accessing datapools programmatically. However, using these interfaces will lead to development of utilities in the custom code area of the test scripts. The test script editors have a rich set of tools that make accessing the datapools a simple process.

It is my belief that using these built in datapool access tools is the best approach, particularly if you have testers who might not have the technical skills to perform complex customization in Java.

The APIs seem more appropriate for accessing datapools for JUnit testing or for use within Rational Functional Tester, which has a much more Java-based test scripting environment. However, these APIs can also be used to modify the datapool contents as a batch process, thus keeping the relationship between the test scripts and the datapools intact.

TPTP datapool API

The Eclipse API for reading datapools is located in the TPTP Public API Specification. Additional datapool utilities and classes for writing to datapools is included in the TPTP Internal API Specification.

Also in the internal API is the CSVImportExportUtil class, which, as the name implies, you can use to import or export data to and from a TPT datapool by using a CSV-formatted file as either the input source or output target. You could use this to create a process for importing large quantities of CVS files to many datapools.

There are several examples of basic reading of TPTP datapools on the Internet, many of which use them in JUnit testing. This is very useful, because there is no specific environment required to use the API, with the exception of importing necessary Eclipse Java archive (.jar) files into your project’s build path.

You can access TPTP style datapools from any Eclipse-based tool that supports a Java development environment. These Eclipse library components are required for TPTP API datapool access:

  • org.eclipse.emf.common_version.jar
  • org.eclipse.emf.ecore.xmi_version.jar
  • org.eclipse.emf.ecore_version.jar
  • org.eclipse.equinox.common_version .jar
  • org.eclipse.hyades.test.ui_version.jar
  • org.eclipse.tptp.platform.models.version.jar
  • org.eclipse.tptp.platform.models_version.jar

Some or all of these libraries might be included in your software. If not, all are included in the sample code package. See the section about The sample code package at the end of this article for more details.


Reading and updating a TPTP datapool

The Java code snippet in Listing 7 displays a simple example of reading a TPTP format datapool.

Listing 7. TPTP datapool, example
// set the Datapool File value
    File inputTptpFile = new File(
        "/Datapool_Workspace/Common/datapool/TPTP_Datapool.datapool");
    System.out.println("Read a TPTP (RPT/RST) format Datapool: " + 
        inputTptpFile.getAbsolutePath());

    try {
        // create an IDatapoolFactory object
        IDatapoolFactory tptpDatapoolFactory = new Common_DatapoolFactoryImpl();
        // create an Datapool object
        IDatapool tptpDatapool = 
            (IDatapool) tptpDatapoolFactory.load(inputTptpFile, true);

        // capture the Datapool column header values 
        int datapoolColumnCount = tptpDatapool.getVariableCount();
        String[] header = new String[datapoolColumnCount];
        aStringBuilder.append("\nHEADER  :: ");
        for (int i = 0; i < datapoolColumnCount; i++) {
            header[i] = tptpDatapool.getVariable(i).getName();
            aStringBuilder.append(header[i] + " :: ");
        }
        System.out.println(aStringBuilder.toString());

        // Create an IDatapoolIterator object and populate with 
        //    the Datapool as an iterator
        IDatapoolIterator tptpDatapoolIterator = tptpDatapoolFactory.
            open(tptpDatapool, 
        "org.eclipse.hyades.datapool.iterator.DatapoolIteratorSequentialPrivate");

        // initialize datapool iterator    
        tptpDatapoolIterator.dpInitialize(tptpDatapool, 0);
        int count = 1; // int counter for demonstration purposes only
        while(!tptpDatapoolIterator.dpDone()) {
            // reset the temporary container for captured values
            aStringBuilder.setLength(0);
            aStringBuilder.append("ROW #" + count + "  :: ");
            String[] nextRow = new String[datapoolColumnCount];
            for (int i = 0; i < datapoolColumnCount; i++) {
                /* NOTE: this example assumes value to be of type String
                 *    value can be captured via:
                 *    IDatapool.getVariable(int).getSuggestedType() */
                nextRow[i] = tptpDatapoolIterator.dpCurrent().getCell(
                    header[i]).getStringValue();
                aStringBuilder.append(nextRow[i] + " :: ");
            }
            tptpDatapoolIterator.dpNext();
            count++;
            System.out.println(aStringBuilder.toString());
            // unload the Datapool
            tptpDatapoolFactory.unload(tptpDatapool);
        }
    } catch (DatapoolException e) {
        e.printStackTrace();
    }

This is a simple approach to capturing the contents of a TPTP format datapool and, in this case, writing it to the console. At this point, you can iterate through your datapool, using the values as you see fit, or build in logic to perform a search for specific values. Field types can also be captured to ensure integrity of data types when updating the values.

Updating a TPTP datapool

The datapool reader program uses this class:
org.eclipse.hyades.execution.runtime.datapool.IDatapool

To extend the above program that reads a datapool to directly edit the datapool, use this class:
org.eclipse.hyades.edit.datapool.IDatapool

This is part of the TPTP internal API (see the Public API Specification for reference).

Minor modifications to this code will give you the ability to perform modifications to the datapool. The first item of note to perform an edit rather than a read is changing the references in the import section of the program.

Listing 8. TPTP datapool edit imports
        /* NOTE: To update a Datapool the edit version of the classes must be used */
        import org.eclipse.hyades.edit.datapool.IDatapool;
        import org.eclipse.hyades.edit.datapool.IDatapoolFactory;
        //import org.eclipse.hyades.execution.runtime.datapool.IDatapool;
        //import org.eclipse.hyades.execution.runtime.datapool.IDatapoolFactory;

The edit classes behave slightly different from the runtime classes. The first notable change is located at the start of the try or catch block, as the example in Listing 9 shows.

Listing 9. TPTP datapool edit instantiation
        try {
            // create an org.eclipse.hyades.edit.datapool.IDatapoolFactory object
            IDatapoolFactory tptpDatapoolFactory = new Common_DatapoolFactoryImpl();
            // create an org.eclipse.hyades.edit.datapool.IDatapool object
            IDatapool tptpDatapool = (IDatapool) tptpDatapoolFactory.loadForEdit(
                inputTptpFile, true);

When creating the IDatapool object, the file is loaded through the loadForEdit(File, Boolean) method. Also, when creating the IDatapoolIterator, different behavior is found in the edit classes (Listing 10).

Listing 10. TPTP datapool edit iterator
                /* Create an IDatapoolIterator object and populate with the Datapool 
             *     as an iterator
             * NOTE: for edit variations of the IDatapool* objects the IDatapoolFactory
             *     must be cast to "Common_DatapoolFactoryImpl" to make the appropriate 
             * open(IDatapool, String) method visible*/
            IDatapoolIterator tptpDatapoolIterator = 
               ((Common_DatapoolFactoryImpl)tptpDatapoolFactory).open(tptpDatapool, 
               "org.eclipse.hyades.datapool.iterator.DatapoolIteratorSequentialPrivate");

As the comment states, the IDatapoolFactory must be cast to Common_DatapoolFactoryImpl to provide access to the open(IDatapool, String) method. The code snippet in Listing 1 illustrates the capture of a specific cell in the datapool, and then changing the value.

Listing 11. TPTP datapool edit iteration
                for (int i = 0; i < datapoolColumnCount; i++) {
                // look for the last column in the Datapool -- to be updated
                if (i + 1 == tptpDatapool.getVariableCount()) {
                    // create a new value
                    String newCellValue = "Updated value for row " + count +
                        " at " + dateFormat.format(new Date().getTime());
                    // capture the cell as an IDatapoolCell
                    IDatapoolCell aCell = (IDatapoolCell)tptpDatapoolIterator.
                        dpCurrent().getCell(header[i]);
                    aCell.setCellValue(newCellValue);
                    // sleep to demonstrate timestamp update in Datapool
                    Thread.sleep(1000);
                }
                nextRow[i] = tptpDatapoolIterator.dpCurrent().
                    getCell(header[i]).getStringValue();

After processing the datapool, perform a save and unload on the datapool (Listing 12).

Listing 12. TPTP datapool edit finalization
                // save updates to the Datapool file 
                tptpDatapoolFactory.save(tptpDatapool); 
                tptpDatapoolFactory.unload(tptpDatapool);

Exporting a TPTP datapool to CSV

Another approach that you might want to take it to export your datapool to a comma separated values-formatted, or CSV, file, make the modifications there, and then import the modified file back into the datapool.

The TPTP internal API describes the classes and methods associated with this process, which are illustrated in the following code snippet.

Listing 13. TPTP datapool to CSV
    try {
        // create an IDatapoolFactory object
        IDatapoolFactory tptpDatapoolFactory = new Common_DatapoolFactoryImpl();
        // create an IDatapool object
        IDatapool tptpDatapool = 
            (IDatapool) tptpDatapoolFactory.load(inputTptpFile, true);

        // create an instance of  CSVImportExportUtil
        CSVImportExportUtil aCSVImportExportUtil = CSVImportExportUtil.getInstance();

        // perform export of the Datapool to CSV
        aCSVImportExportUtil.exportCSV(tptpDatapool, 
            tptpDatapoolAsCsvFilename, true, false, false, "");
        System.out.println(this.getClass().getSimpleName() + 
            " created output to: " + tptpDatapoolAsCsvFilename);
    } catch (IOException e) {
        e.printStackTrace();
    }

The process is very simple. Listing 14 shows the export arguments.

Listing 14. TPTP datapool to CSV file export options
    CSVImportExportUtil.exportCSV(
        org.eclipse.hyades.edit.datapool.IDatapool datapool, 
        java.lang.String csvFilePath, 
        boolean includeVariableInfo, 
        boolean includeEquivalenceClassInfo, 
        boolean includeTags, java.lang.String encoding)

Table 1 explains the fields.

Table 1. TPTP datapool to CSV export fields
FieldDescription
datapool The datapool that is exported to the CSV file.
csvFilePath The absolute path of the CSV file.
includeVariableInfo If the first row of the CSV file contains the variable information (name/type), otherwise false.
includeEquivalenceClassInfo If the first column of the CSV file contains the equivalence class information (name/row index), otherwise false.
includeTags If the cell values are enclosed in tags, otherwise false.
encoding The encoding of the CSV file, otherwise null.

The import process is very similar. Listing 15 shows the import arguments.

Listing 15. TPTP datapool to CSV import options
    CSVImportExportUtil.importCSV(
        org.eclipse.hyades.edit.datapool.IDatapool datapool, 
        java.lang.String csvFileName, 
        boolean isFirstRowVariableInfo, 
        boolean isFirstColumnEquivalenceClassInfo, 
        java.lang.String encoding)

Table 2 explains the fields.

Table 2. TPTP Datapool to CSV Import Fields
FieldDescription
datapool The datapool that the CSV file is imported into.
csvFilePath The absolute path of the CSV file.
isFirstRowVariableInfo If the first row of the CSV file contains the variable information (name/type), otherwise false.
isFirstColumnEquivalenceClassInfo If the first column of the CSV file contains the equivalence class information (name/row index), otherwise false.
encoding The encoding of the CSV file, otherwise null.
numberOfColumns The number of columns of the CSV file to import into the datapool, otherwise -1
numberOfRows The number of rows of the CSV file to import into the datapool, otherwise -1

See the TPTP internal API, Class CSVImportExportUtil, for more details.

If this is the approach that you choose to take, describing the steps for modifying the CSV contents is beyond the scope of this article.


Reading and updating a Rational Functional Tester datapool

Given that the Rational Functional Tester datapool does extend the TPTP datapool classes, one would hope that performing the same process would be equally simple. This is not quite the case. The extended utilities for Rational Functional Tester datapools reside within the Rational Functional Tester product framework, and they are very dependent on the test being executed from within that framework.

If you decide you would like to take a run at resolving configuration issues, prepare yourself for a long exercise in Java build path modifications, extensions of undocumented classes, and a stream of exceptions, ultimately leading to complete failure. You can trust me on that, because I attempted to go down that path, only leaving myself frustrated.

My personal favorite exception out of the many was this completely obscure one:

Exception in thread "main"
com.rational.test.ft.sys.Transaction$NotInTransactionException

Feel free to take this as a challenge, and if you are able to make this work, I would love to hear about it. If you do get past the Transaction$NotInTransactionException, prepare for an onslaught of datapool parsing errors.

That said, you can take a very similar approach to the one identified above for TPTP datapools by executing the Java as a Rational Functional Tester Functional test script. Of course, if you want to alter Rational Functional Tester datapools, you need access to the Rational Functional Tester software.

Full documentation for the Rational Functional Tester API is available online:

By taking the approach of performing the Java steps in Rational Functional Tester, you can use very similar code.
Because the program is to be executed through Rational Functional Tester, it must be created as a Rational Functional Tester test script, identifiable in the Java class definition, as shown in Listing 16.

Listing 16. Rational Functional Tester test script class definition
/** <p>Example of datapool reading via API from a RFT Test Script.</p> */
public class DatapoolReader_RFT extends DatapoolReader_RFTHelper {

It also needs the extension of <testScriptName>_Helper, and it must be executed by the method shown in Listing 17.

Listing 17. Rational Functional Tester test script execution method
    /**
     * <p>Standard RFT testMain(Object[]) execution method.</p>
     * @param args input arguments
     */
     public void testMain(Object[] args) {

Do not use the standard Java main(String[]) method. Instead, use the Rational Functional Tester testMain(Object[]). Within the testMain method, the code is very similar to the above TPTP example, although it uses some of the Rational Functional Tester built-in methods for generation datapool objects, as Listing 18 shows.

Listing 18. Rational Functional Tester datapool reading example
try {
        // create an org.eclipse.hyades.execution.runtime.datapool.IDatapool object
        //    utilizing the DatapoolScriptSupport.dpFactory() method
        IDatapool rftDatapool = dpFactory().load(inputRftFile, false);

        // capture the Datapool column header values 
        int datapoolColumnCount = rftDatapool.getVariableCount();
        String[] header = new String[datapoolColumnCount];
        aStringBuilder.append("\nHEADER  :: ");
        for (int i = 0; i < datapoolColumnCount; i++) {
            header[i] = rftDatapool.getVariable(i).getName();
            aStringBuilder.append(header[i] + " :: ");
        }
        System.out.println(aStringBuilder.toString());

        // create an IDatapoolIterator, again using the RFT:
    //    DatapoolScriptSupport.dpFactory() method
        IDatapoolIterator rftDatapoolIterator = dpFactory().open(rftDatapool, "");
            
        // initialize the IDatapoolIterator
        rftDatapoolIterator.dpInitialize(rftDatapool,0);
        int count = 1;
        while(!rftDatapoolIterator.dpDone()) {
            // reset the temporary container for captured values
            aStringBuilder.setLength(0);
            aStringBuilder.append("ROW #" + count + "  :: ");
            String[] nextRow = new String[datapoolColumnCount];
            for (int i = 0; i < datapoolColumnCount; i++) {
                /* NOTE: this example assumes value to be of type String
                 *     value can be captured via:
                 *     IDatapool.getVariable(int).getSuggestedType() */
                nextRow[i] = rftDatapoolIterator.dpCurrent().getCell(
                header[i]).getStringValue();
                aStringBuilder.append(nextRow[i] + " :: ");
            }
            rftDatapoolIterator.dpNext();
            count++;
            System.out.println(aStringBuilder.toString());
        }
        // unload the Datapool
        dpFactory().unload(rftDatapool);
    } catch (DatapoolException e) {
        e.printStackTrace();
    }

Note:
These lines are required for implementation of Rational Functional Tester components:

  • IDatapool rftDatapool = dpFactory().load(inputRftFile, false);
  • IDatapoolIterator rftDatapoolIterator = dpFactory().open(rftDatapool, "");

Run the test as a Rational Functional Tester test script, and you will get the same results as from the previous example for reading a TPTP-style datapool.

Updating a Rational Functional Tester datapool

As with the TPTP examples, there are some minor modifications.

As is the case with the TPTP, you will need to use the org.eclipse.hyades.edit.datapool classes, rather than the org.eclipse.hyades.execution.runtime.datapool classes.

Since Rational Functional Tester has built-in extensions for accessing the Eclipse datapool classes, the approach is somewhat different from the previous TPTP example for editing datapool values.

First, notice in Listing 19 that the imports section needs specific references to the Eclipse datapool edit classes.

Listing 19. Rational Functional Tester datapool edit imports
import org.eclipse.hyades.edit.datapool.IDatapool; 
import org.eclipse.hyades.edit.datapool.IDatapoolCell;

When creating the datapool object for editing in Rational Functional Tester, the built-in DatapoolScriptSupport.dpFactory() generates a read-only type of DatapoolFactory. To allow further edit functions, you need to create a proper edit type by casting the output from dpFactory(), as Listing 20 shows.

Listing 20. Rational Functional Tester datapool edit instantiation
    try {
        // create a com.rational.test.ft.datapool.DatapoolFactory object,
        //        cast from IDatapoolFactory to the proper DatapoolFactory class
        DatapoolFactory rftDatapoolFactory = (DatapoolFactory)dpFactory();
        // create an org.eclipse.hyades.edit.datapool.IDatapool object
        IDatapool rftDatapool = rftDatapoolFactory.loadForEdit(updateRftFile, true);

The remainder of the code for updating is almost identical to the TPTP update code. See the sample code in the Downloads section of this article for full details.

Remember to make sure you save your datapool updates, else any changes made will not be persisted to the datapool file.

Listing 21. Rational Functional Tester datapool edit, finalization
        // save updates to the Datapool file
        rftDatapoolFactory.save(rftDatapool);
        // unload the Datapool
        rftDatapoolFactory.unload(rftDatapool);    
    } catch (DatapoolException e) {
        e.printStackTrace();
    } catch (InterruptedException e) {
        e.printStackTrace();
    }

Export a Rational Functional Tester datapool to a CSV file

Classes and methods for exporting Rational Functional Tester datapools to CSV files are defined in the API. The code snippet in Listing 22 demonstrates the simple process.

Listing 22. Rational Functional Tester datapool to CSV file
    try {
        // create an IDatapool object utilizing the RFT 
        //     DatapoolScriptSupport.dpFactory() method            
        IDatapool rftDatapool = (IDatapool) dpFactory().load(inputRftFile, false);
        // save the Datapool to CSV file utilizing the RFT DatapoolUtilities class
        DatapoolUtilities.storeCSV(rftDatapool, 
            new File(rftDatapoolAsCsvFilename), ",", true);
        // unload the Datapool
        dpFactory().unload(rftDatapool);
    } catch (DatapoolException e) {
        e.printStackTrace();
    }

Listing 23 shows the export arguments.

Listing 23. Rational Functional Tester datapool export options
     DatapoolUtilities.storeCSV(IDatapool datapool, 
    java.io.File csvFile, 
    java.lang.String separator, 
    boolean firstRowIsHeaders)

Table 3 explains the arguments.

Table 3. Rational Functional Tester datapool to CSV export fields
FieldDescription
datapool The datapool that is exported to the CSV file.
csvFile The file name of a file in CSV format. If this name is null or the file it represents does not exist, an appropriate exception is thrown.
separator The cell separator used in the file being parsed. If null, then a single-character comma is used.
firstRowIsHeaders Flag to parser indicating to use the first row of values as the column headers.

The import process is very similar, with the arguments being those shown in Listing 24.

Listing 24. Rational Functional Tester datapool import options
         DatapoolUtilities.loadCSV(java.io.File csvFile,
        java.lang.String separator,
        boolean firstRowIsHeaders)

Parameter values match those for the export (store) method.

This behavior differs from TPTP in that this method returns a datapool object, for which you must create code to save; whereas, the TPTP export accepts a datapool as input.

Details for using the DatapoolUtility class are available in the API.


Differences in datapools exported as CSV files

Although both TPTP and Rational Functional Tester have built-in CSV export utilities, the output is slightly different, as shown in the exported CSV data in Tables 4 and 5.

Table 4. TPTP RPT_RST datapool as a CSV file (Rational Performance Tester, Rational Service Tester for SOA Quality)
RPT_RST_Key::StringRPT_RST_Value_1::StringRPT_RST_Value_2::String
1st KEY RPT_RST #1 The first record
2nd KEY RPT_RST #2 The second record
3rd KEY RPT_RST #3 The third record
Table 5. Rational Functional Tester datapool as a CSV file
RFT_Key[][STRING]RFT_Value_1[][STRING]RFT_Value_2[][STRING]
1st KEY RFT #1 The first record
2nd KEY RFT# 2 The second record
3rd KEY RFT #3 The third record

The differences in the CSV output headers relate to the column data type. This is a minor parsing exercise, which would allow you to edit the values in the CSV formatted datapools for TPTP (Rational Performance Tester, Rational Service Tester) and Rational Functional Tester datapools by using the same approach.

Whether you decide to directly edit the datapool or to create a process to export CSV and modify the CSV files, you should consider defining conventions for naming headers for a more controlled approach to field identification. You might also want to define a separate column to identify different row types. For example, you might want to retain some data rows for static values, where other rows might be dynamically updated. This is for you to decide, based on the plan that you design for updating test data.


Approaching datapools as XML

A bit of background on this article:

The original requirement that I had was to update test data for Rational Service Tester, where there were hundreds of datapools. The plan was to capture the test data from the application under test’s back-end database, and then to populate the datapools with relevant data.

Upon discovering the archive nature of the *.datapool files, each of which contained a single ResourceContents XMI file, I began by capturing the values by using standard Java libraries.

To capture the contents of the *datapool file, I used the standard java.util.zip.* package. After the contents were captured, I proceeded to treat the XMI file as a standard org.w3c.dom.Document, using additional classes from org.w3c.dom, javax.xml.parsers, and javax.xml.transform to complete the code by depending on no non-IBM or open source components.

Although the APIs contain update methods for the datapools, TPTP (Rational Performance Tester, Rational Service Tester) and Rational Functional Tester need to be executed in different environments. This issue leads me to prefer the approach of dealing with the datapools as XML documents, thereby allowing for a pure Java implementation that can be completely externalized from the testing tool. (There are dozens of articles relating to dealing with the processing of XML documents, so I will not go further into that topic in this article.)

At that point, it becomes a purely Java application development exercise. The outcome is a fully functional datapool file that, with proper handling of XML attributes, will retain all object references to existing test scripts.


Further thoughts

As useful as datapools are for maintaining tabular data for references in Rational and TPTP test scripts, there might be times that you need to support a more complex test data model by capturing data from various sources, such as spreadsheets or CSV files, XML representations of data, and databases.

One option would be to build a custom test data server utility. Creating a class that executes as a singleton can reduce overhead by instantiating values on the first call. The test data server delivers a variety of data to test scripts in a sequential or random form, or based on specific criteria passed to the test data server component.

However, you should be aware of potential overhead issues when creating such a component for performance testing. You also need to factor in the requirements for connectivity from remote performance test execution agents.

An additional thought on the topic of test data is injecting the test scripts with real-time test data from a database or some other dynamic state value.

These items are beyond the scope of this article, but they might be worth thinking about when designing your test data strategy.


Eclipse TPTP end-of-life cycle

According to the Test & Performance Tools Platform (TPTP) Project Plan 2010, the final release of TPTP is Version 4.7.2, which was released in February 2011.

With TPTP deeply entrenched in the IBM Rational tester products, it will be interesting to see where the TPTP components go in the future.


Download

DescriptionNameSize
Samples for this articlesamples.zip5MB

Resources

Learn

Get products and technologies

Discuss

Comments

developerWorks: Sign in

Required fields are indicated with an asterisk (*).


Need an IBM ID?
Forgot your IBM ID?


Forgot your password?
Change your password

By clicking Submit, you agree to the developerWorks terms of use.

 


The first time you sign into developerWorks, a profile is created for you. Information in your profile (your name, country/region, and company name) is displayed to the public and will accompany any content you post, unless you opt to hide your company name. You may update your IBM account at any time.

All information submitted is secure.

Choose your display name



The first time you sign in to developerWorks, a profile is created for you, so you need to choose a display name. Your display name accompanies the content you post on developerWorks.

Please choose a display name between 3-31 characters. Your display name must be unique in the developerWorks community and should not be your email address for privacy reasons.

Required fields are indicated with an asterisk (*).

(Must be between 3 – 31 characters.)

By clicking Submit, you agree to the developerWorks terms of use.

 


All information submitted is secure.

Dig deeper into Rational software on developerWorks


static.content.url=http://www.ibm.com/developerworks/js/artrating/
SITE_ID=1
Zone=Rational
ArticleID=822085
ArticleTitle=Programmatically modify Rational datapools
publish-date=06262012