Simplify GUI test automation by using a labor-saving design pattern

Learn how you can employ a design pattern that helps automatically test your graphical user interfaces (GUIs) — by using the IBM® Rational® Functional Tester environment coupled with the IBM Corporation's automation test framework. This solution provides a simple architecture, offering a comprehensive set of functional tests for any GUI-based product, despite its underlying platform. The design pattern leverages pre-defined input XML files that describe the test case flow and the final expected output results. The tests can be deployed seamlessly to run on a variety of platforms, generating detailed trace and logs in the process. In order to provide a real world example of how this works, this article uses the IBM Database Add-Ins for the Microsoft® Visual Studio® product to supply examples about how design pattern for automation testing can be achieved.

Sandhya Anantharamiah (sananth@us.ibm.com), Software Engineer, IBM

Author photoSandhya Anantharamiah is a Software Engineer at the Silicon Valley Lab at San Jose. Anantharamiah has worked on web related, Java, C++ and C# development projects at IBM and is currently working as the QA Lead for the IBM Database Add-Ins for the Microsoft Visual Studio tool.



05 October 2010

Also available in Russian

With customers requiring faster development times and improved software quality, organizations must find ways to automate and streamline their processes to stay competitive. This article describes a test automation design pattern that can be used to automate the testing of graphical user interface (GUI) based tools thereby significantly reducing the time required to run quality assurance (QA) testing. Moreover, the design pattern seeks to alleviate these challenges by helping to provide an ideal environment for automation testing requirements, offering these features:

  • Platform agnostic: Via a set of well defined configuration templates, automation testing is possible on multiple platforms and server connections.
  • Extensible: Helps enable a seamless addition of new tests with no impact on your existing tests and artifacts.
  • Reporting and regression comparison: Seamless automatic comparison reporting of expected and actual test results.
  • Easy debugging: Aids your efforts to easily generate logs and trace for debugging purposes.

The IBM® Database Add-Ins for Microsoft® Visual Studio® environment helps provide application developers with an intuitive means to interact and integrate database components for applications under development. Using powerful graphical designers, wizards and menu options, this tool offers an integrated development environment for IBM database objects (including tables, views, stored procedures, functions, etc.) lifecycle and maintenance. The tool supports all types of IBM DB2 servers and IDS servers: more information about this tool can be found in the What's new with the IBM Database Add-Ins for Visual Studio 2005 guide.

System requirements

In order to work with the design pattern, you will require the following technologies and development environment artifacts:


Automation testing design pattern

What does the design pattern look like? The design pattern discussed in this paper is depicted in the architectural diagram below.

Figure 1. Architectural overview
Flow diagram shows relationships between components

Architectural components explained

In order to proceed, having an understanding of what is inside the design pattern is essential. The design pattern's architectural components include:

Test Case Schema
This file defines the XML schema and the metadata for how test data is formulated.
Test Case XML
Based on the Test Case Schema, this file contains the actual test data that will be read by the automation test scripts to run tests.
Binding Compiler & Schema Derived Classes & Interfaces
The JAXB binding compiler auto-generates the required packages to integrate with the test automation scripts.
Automation Framework and JAXB API
Automation framework script imports the package(s) generated by the biding compiler. JAXB APIs are used to read the test case XML data.
Test Application
This is the underlying application that is being tested.
Trace Log and Result Log
The output of automation test framework is a set of well understood Trace and Result logs. These logs show the control flow and stack traces during test failures.

A case scenario

This section illustrates how the design pattern is used to develop an automation testing mechanism for IBM Visual Studio Add-ins. For a brief overview of IBM Visual Studio Add-Ins GUI designers can review this document:What's new with the IBM Database Add-Ins for Visual Studio 2005 guide.

Step 1. Creating the XML schema for test case data

Begin by creating a schema for the test case data. The schema below is designed to hold data required to test stored procedures using the IBM Visual Studio Add-In tool.

  • testCaseData is the root element.
  • Servers element contains the serverType, which will hold different server types that need to be tested. There can be more than one server type.
  • Procedures are the element that will hold all the data related to create a stored procedure like the name, the parameters and their properties, plus the body of the stored procedure and the expectedResult.

Listing 1. XML schema for test case data

<?xml version="1.0"?>
<xsd:schema xmlns:xsd="http://www.w3.org/2001/XMLSchema">
  <xsd:element name="testCaseData">
    <xsd:complexType>
      <xsd:sequence>
        <xsd:element name="servers">
          <xsd:complexType>
            <xsd:sequence>
              <xsd:element maxOccurs="unbounded" name="serverType" type="xsd:string" />
            </xsd:sequence>
          </xsd:complexType>
        </xsd:element>
        <xsd:element name="procedures">
          <xsd:complexType>
            <xsd:sequence>
              <xsd:element maxOccurs="unbounded" name="sp">
                <xsd:complexType>
                  <xsd:sequence>
                    <xsd:element name="name" type="xsd:string" />
                    <xsd:element name="description" type="xsd:string" />
                    <xsd:element maxOccurs="unbounded" name="param">
                      <xsd:complexType>
                        <xsd:sequence>
                          <xsd:element name="name" type="xsd:string" />
                          <xsd:element name="mode" type="xsd:string" />
                          <xsd:element name="type" type="xsd:string" />
                          <xsd:element name="value" type="xsd:string" />
                        </xsd:sequence>
                      </xsd:complexType>
                    </xsd:element>
                    <xsd:element name="body" type="xsd:string" />
                    <xsd:element name="expectedResult" type="xsd:string" />
                  </xsd:sequence>
                </xsd:complexType>
              </xsd:element>
            </xsd:sequence>
          </xsd:complexType>
        </xsd:element>
      </xsd:sequence>
    </xsd:complexType>
  </xsd:element>
</xsd:schema>

Next, run the JAXB binding compiler to generate the Java package that contains all the Java classes based on the test case schema.

Once this is complete compile the Java package to generate class files and import the generated classes into the test automation script.

These steps are described with an example in the next section of the article.

Step 2. Creating the server specification file

Fill in the server details in the server specification file. This file is specific to the application under test. The server data is not included in the test case xml file because the data here does not change as frequently as the test case file. This file is common to all the test cases, and the information in this file will change only if there are any changes to the server connection parameters. This server information is referred to in the serverType element of the schema mentioned in Step 1.

Listing 2. Server specification file
#yes/no to test LUW97
gsTestLUW97=yes
#Test LUW97Server URL
gsTestLUW97URL=server1.test.ibm.com:50000
#Test LUW97user ID
gsTestLUW97UserID=UserName
#Test LUW97user password
gsTestLUW97Password=password
#Test LUW97dbname
gsTestLUW97DBName=SAMPLE
#Test LUW97dbname version info
gsTestLUW97DBVersion=[DB2/NT 09.07.0001]

#yes/no to test LUW95
gsTestLUW95=yes
#Test LUW95Server URL
gsTestLUW95URL=server2.test.ibm.com:50000
#Test LUW95user ID
gsTestLUW95UserID=UserName
#Test LUW95user password
gsTestLUW95Password=password
#Test LUW95dbname
gsTestLUW95DBName=SAMPLE
#Test LUW97dbname version info
gsTestLUW97DBVersion=[DB2/NT 09.05.0005]

In the above properties file all statements that begin with '#' are treated as comments.

  • gsTestLUW97URL holds the server URL for a LUW v97 connection.
  • gsTestLUW97UserID and gsTestLUW97Password will hold the user id and the password to connect to this server.

Similarly, other information required to make a connection to a server is provided in this file.

Step 3. Creating sample test case XML file

Next, create sample test case XML. This XML file contains the actual test data based on the above mentioned schema. This is the file that a user must create to add a new test case. The test case below provides stored procedure and the server information. In the <serverType> element the user has to specify one or more of the servers mentioned in step one.

Listing 3. Sample test case XML
<?xml version="1.0"?>
<testCaseData>
  <servers>
    <serverType>LUWV97</serverType>
  </servers>
  <procedures>
    <sp>
      <name>vsai_fvt0001</name>
      <description>Stored Procedure with 1 INTEGER IN parameters</description>
      <param>
        <name>Zipcode</name>
        <mode>IN</mode>
        <type>INTEGER</type>
        <value>95054</value>
      </param>
      <body></body>
      <expectedResult>\expectedResult\SP\vsai_fvt0001.xml</expectedResult>
    </sp>
  </procedures>
</testCaseData>

The above XML document is based on the schema, described in Step 1.

  • <testCaseData> is the root element of the document.
  • <ServerType> holds the server name LUWV97 on which the test cases are to be executed.
  • <Procedures> holds the stored procedures <sp> that are to be created. In the above example there is one <sp>.
  • <sp> contains the name vsai_fvt0001, parameters details in the <param> tag and the path<expectedResult>, where the expected results file is located for the stored procedure. The example procedure has one input parameter with name ZIPCODE, which is of type INTEGER with the value of 95054.

Step 4. Executing the test automation scripts

Run the automation script containing the code for reading the test case data from your Rational Functional Tester environment. This will start IBM Database Add-Ins for Visual Studio, and the application under test creates a stored procedure with the data specified in the test case XML file, as shown below.

Figure 2. Stored procedure designer in Visual Studio
Screenshot shows the stored procedure GUI dialog.

Now, look at the logs to determine the result of the test.

Figure 3. Result log file
Graphics shows test case PASSED/FAILED result.


If the test has failed, you can take a look at the generated trace file to debug the problem. A sample trace file is shown below.

Figure 4. Trace log file
Screen output of the test case execution flow.

Implementation of the automation framework

This section describes the implementation of the test automation framework. The tool used to develop the automation script is the Rational Functional Tester 8.1.0 product's Java Scripting. The automation follows the three tiered IBM Framework, containing AppObjects, Tasks and test cases.

Generating test case Java classes using JAXB

The below binding command takes the testCaseData.xsd file described in Step 1 generates the package jaxb.testData which will contain all the derived classes based on the schema.

Binding:

Xjc –p jaxb.testData testCaseData.xsd –d autoClasses

The -p option identifies a package for the generated classes, and the -d option identifies a target directory. So for this command, the classes are packaged in jaxb.testData within the autoClasses directory.

The binding compiler generates a set of classes that represents the elements in the schema.

Compiling:

javac jaxb/testData/*.java

This compiles all the java files output by the binding process and generates the class files.

The automation script has to import the package generated by the binding process and can access the data in the testCaseData.xml file using JAXB unmarshal API, as shown below in Listing 4.

Listing 4. Using the JAXB unmarshall APIs to access test case data
private TestCaseData GetTestCaseData() {
TestCaseData tcData = null;
try {
	JAXBContext jc = JAXBContext.newInstance("jaxb.testData");
	Unmarshaller u = jc.createUnmarshaller();
	// Validating schema with xml file
	u.setEventHandler(new javax.xml.bind.helpers.DefaultValidationEventHandler());
	tcData = (TestCaseData) u.unmarshal(new FileInputStream(
					"TestCaseData.xml"));
			return (tcData);
} catch (JAXBException je) {
	je.printStackTrace();
} catch (IOException ioe) {
	ioe.printStackTrace();
}
	return (null);
}

Once the xml file has been unmarshalled, the data in the xml file can be used as objects as Listing 5 shows.

Listing 5. Using test case data objects
// Get the testCaseData xml file as a object
TestCaseData tcData = GetTestCaseData();

// Get all the servers expected to test
List serverList = tcData.getServers().getServerType();
List spList = tcData.getProcedures().getSp();

The advantage of this technique is that the user can add new test cases without needing to understand the Rational Functional Tester tool, or the automation framework. By using JAXB, the test case data is automatically converted to Java classes, helping provide easy integration into the framework. Likewise, any automation framework that is written in Java can make use of this design to read the test case document.

Running the test cases

The automation framework executes the following steps for each test case:

  • Opens the application under test.
  • Starts logging trace.
  • Reads the test case data file.
  • Loops through all the servers and adds a connection to the database server in the IBM Database Add-Ins tool based on the data in the test case data file.
  • For every server type mentioned in the test case data file, iterates through all the stored procedure data and creates a stored procedure.
  • Executes the stored procedure and logs the result in the result log file.
  • Adds an entry to the trace log file at the beginning and end of every method called and also when there is an exception.
  • Compares the result of the test with the expected result and updates the result log.

Extensibility

Now that you are able to work in the automation framework, your GUI design environment can benefit from several advantages provided by this solution. By using this automation framework, you can benefit from these features:

  • Adding new test cases is made simple.
  • Modifying existing test cases is easier.
  • By modifying the server file, any server info modification can be done.
  • User interface changes like size, position, and so forth do not require any changes in the automation implementation.
  • Extending automation to other components is easy.
  • The framework allows you to collaboratively development automation scripts.

Summary

Leveraging the test pattern installed using the steps outlined above, you can now take advantage of a test automation framework that allows for the automated testing of IBM Database Add-Ins for Visual Studio. Currently, the framework is being implemented for the automated testing of stored procedures. Future work will include extending the framework to cover the automated testing of other features like tables, functions and views.

Resources

Learn

Get products and technologies

  • Download the latest IBM Database Add-Ins for Visual Studio 2005 from DB2 UDB for .NET.
  • Download a free trial version of DB2 Universal Database.
  • Now you can use DB2 for free. Download DB2 Express-C, a no-charge version of DB2 Express Edition for the community that offers the same core data features as DB2 Express Edtion and provides a solid base to build and deploy applications.
  • Build your next development project with IBM trial software, available for download directly from developerWorks.

Discuss

Comments

developerWorks: Sign in

Required fields are indicated with an asterisk (*).


Need an IBM ID?
Forgot your IBM ID?


Forgot your password?
Change your password

By clicking Submit, you agree to the developerWorks terms of use.

 


The first time you sign into developerWorks, a profile is created for you. Information in your profile (your name, country/region, and company name) is displayed to the public and will accompany any content you post, unless you opt to hide your company name. You may update your IBM account at any time.

All information submitted is secure.

Choose your display name



The first time you sign in to developerWorks, a profile is created for you, so you need to choose a display name. Your display name accompanies the content you post on developerWorks.

Please choose a display name between 3-31 characters. Your display name must be unique in the developerWorks community and should not be your email address for privacy reasons.

Required fields are indicated with an asterisk (*).

(Must be between 3 – 31 characters.)

By clicking Submit, you agree to the developerWorks terms of use.

 


All information submitted is secure.

Dig deeper into Rational software on developerWorks


static.content.url=http://www.ibm.com/developerworks/js/artrating/
SITE_ID=1
Zone=Rational
ArticleID=548824
ArticleTitle=Simplify GUI test automation by using a labor-saving design pattern
publish-date=10052010