Software testing is an indispensable yet expensive part of the software development lifecycle. At Wipro Technologies, where we provide software development services to various clients worldwide, we wanted to put a software testing system in place that would be both economical and effective. When it came time for us to run some regression tests on a generic workflow system we were developing based on the client/server model, we chose to use the IBM Rational® Robot and TestManager software development tools as our automation system. In this article we share our process of selecting these tools, our modular approach to developing a functional test automation suite, and what we learned about automating scenarios that are unfit for automation.
Selection of an Automation System/Tool
The purpose of test automation is to make software testing cheaper and faster. With this in mind, we set about evaluating the various automation systems/tools on the market, such as WinRunner, SilkTest, and the Rational tools. We also kept in mind that a good test automation system ought to do the following:
- provide for data independence
- provide for project maintainability
- offer functional modularity
- offer a high level of object recognition
- be traceable to the test requirements
- be cost effective
- be extensible
We learned that Rational offers the following tools to enable development of an effective functional test suite:
- Rational® TestManager, to design the automation test plan and manage the datapool (test data input) for achieving data independence for the functional test suite. This tool includes Rational® Administrator to manage a test project and Rational® LogViewer to view test results and test logs.
- Rational® Robot, to develop and execute functional test scripts. This tool integrates the compiler for the scripting language and an object recognition tool called Inspector to facilitate script design.
When we carried out a Proof of Concept (POC) to analyze the feasibility of automating our testing with a tool, we found that the Rational tools were the most economical among similar automation systems available and were highly suitable for us in all respects. Together they form one of the best automation systems for testing client/server and Web-based applications. The features that made this a particularly effective automation system for our project are as follows:
- Data independence by means of datapools -- Datapools (that is, collections of application inputs) can be used to vary test scenarios on the basis of input data.
- Ease of project organization and management -- Storage of all the assets/artifacts of a Rational test project is managed by the automation system itself. The administration facility provided by TestManager supports a high level of maintainability.
- Provision for the deployment of projectwide libraries -- A Rational test project provides for designing a projectwide library. Such a library can be designed for the functions that either involve interaction with the controls or are repetitive in nature in various scenarios. This provides not only ease in the development of test scripts but also standardization of the code to some extent, hence supporting maintainability for the test scripts.
- Ability to recognize and interact with the controls provided in the application -- All the controls provided in the application fall within Robot's recognition capability.
- Ability to access database objects -- This is important to us since a number of database transactions are required at various points in our project.
- Use of SQA Basic as a scripting language -- SQA Basic, the language used by Robot to code scripts, is a superset of Visual Basic and provides a rich library of verification functions that help both in testing an application and analyzing the results. Support for these verification functions has been provided for various controls and objects specific to the application type. The tool broadly supports Web-based application systems and hence can handle applications like ours.
A Modular Approach to Developing the Test Suite
To maintain a structured and object-oriented approach to automation, we developed our test suite by creating the following components:
- Projectwide function library -- This contains functions for commonly performed GUI actions and tasks to be used in the script files. Any commonly performed back-end task can also be designed as a function and included in the library to be used in the test scripts.
- Header files -- These files contain declarations for the functions defined in the library files and can be included in the test scripts using specific libraries.
- Test script files -- Each test script is organized into a
Sub Mainprocedure that calls scenario-related functions defined in the script itself. Thus, each task in a scenario can be broken down into elementary functions, and theSub Mainprocedure can call these functions with various data values (extracted from datapools). This modularity provides extensibility to a test script -- any change in functionality of a scenario involves changing only the related functional task in the test script. - Shell script files -- Scripts belonging to a particular module can be organized to be executed in a batch mode by means of a shell script. Similarly, a test harness can be provided for the whole test suite that executes all the testing scenarios for all the modules of the application/system.
- Datapools -- Each test script takes input from a datapool. Thus, different scenarios can be tested by merely changing the datapool values. This also provides a way to extensively test all the aspects of the application in terms of test data.
The pseudocode in Listing 1 demonstrates how we might use some of these modules in a test script implementing a sample scenario.
//Inclusion of the header files
$Include "sqautil.sbh"
$Include "<Basic Functions Library>.sbh" 'Set of functions defined in the
library to interact with GUI controls of the application
$Include "<Module Business Functions Library>.sbh" 'Set of module-specific
business functions
//Declaration of script-specific global variables (Alternatively, various
global variables to be used across the scripts in the test repository can be
declared in a projectwide header file. This header file can be included in the
individual scripts.)
Global sModuleMainWindow As String
Global Const sLoginDomainComboBoxName As String ="LoginOptions"
//Declaration of the functions that will be defined in the same script
Declare Function FunctionName(Parameters)
'------------------------------------------------------------------------------
------------------------------------
' Script Name:
' Function Description:
'------------------------------------------------------------------------------
------------------------------------
// Start of the main method
Sub Main
// Declaration of local variables
Dim iSum As Integer
Dim sURL As String
Dim lDatapoolId As Long
// Calls to functions/procedures defined in various libraries This section
also contain commands and functions to implement the test flow.
. . .
. . .
// End of the main method
End Sub
// Definition of script-specific functions
Function ScriptFunction(Parameters)
// Function definition
End Function |
Listing 1: Pseudocode for a test script implementing a sample scenario
Approaches to Automating Problematic Scenarios
Besides being data-independent, a test script should be free from any dependency on the execution environment like desktop screen resolution - for example, it should avoid the use of screen coordinates. Further, an effective automation suite shouldn't require any manual intervention. Now, the majority of the scenarios in our project involved manual intervention and hence made it unfit for automation. But we handled these kinds of situations by way of some innovative approaches like the following:
- For a Web-based application, the time it takes to load a new page isn't very predictable. Hence, a test script should be reasonably independent of page-loading time. A time-out mechanism can be devised for page loading in these cases, or alternatively, commands like Browser New Page with sufficiently large time-out values can be used.
- For controls like HTML images that represent buttons in various pages of our application, the developers have assigned no
Nametag. Under normal circumstances, we're left with the option of using only the screen coordinates. But this can be avoided by devising a way to identify the Index property of the HTML image in the window that relates to the GIF image and then capturing theHeightandWidthproperties of the object with the tool. In this way, the coordinates can be determined at run time. - The presence of some asynchronous events in the application might make it unfit for automation. Introduction of relevant polling mechanisms can go a long way to help in such situations, increasing the scope for automation.
All those scenarios that otherwise might be considered unfit for automation can be worked out with similarly novel approaches.
The right choice of tools (Rational TestManager and Robot) along with good planning, hard work, and commitment paid off for us at Wipro Technologies. We successfully completed the test automation suite for both functional and performance testing of our application and delivered it to the client a couple of weeks before the planned project-ending date. We hope that the information we've shared in this article will help make a successful conclusion possible for you, too.
The authors are grateful to Mr. Gangadharaiah C. P. and Mr. Anand Moorthy of Interops Solutions for providing the infrastructure and facility to conduct this work.
Dr. Sanjay Gupta received a Ph.D. in metallurgical engineering and materials science from the Indian Institute of Technology in Bombay, India, and worked as a research associate for four years in the Department of Physics there. He works now as an Architect in Talent Transformation for Wipro Technologies in India. A Sun-certified Java programmer and Sun Certified Trainer, he's been teaching Java since 2000. He has published research papers in various international journals and presented his research findings at many international and national conferences. His current areas of research, study, and knowledge dissemination are Java, Swings, J2EE technology, magnetic materials and high-temperature superconductivity, and tools like Rational Purify, Rational PureCoverage, and Rational Robot.
Table of contents
Local resources
My developerWorks community
Tags
Use the slider bar to see more or fewer tags.
Popular tags shows the top tags for this particular content zone (for example, Java technology, Linux, WebSphere).
My tags shows your tags for this particular content zone (for example, Java technology, Linux, WebSphere).
Popular article tags |
My article tagsSkip to tags list
Popular article tags |
My article tags
