I was recently consulted by an insurance company that wanted to automate regression testing of applications that run on a 5250 terminal. We ran a proof of concept to look at the suitability of using IBM® Rational® Functional Tester with its Extension for Terminal-based Applications, an extension to automate tests for applications running on either a 3270 terminal to an IBM zSeries (System 390) host or a 5250 terminal to an iSeries (AS/400) host. We found it to be a suitable tool for automating the regression tests. In this article I'll share with you what I learned about using Rational Functional Tester with the extension. To get the most out of it, you should have a basic understanding of Rational Functional Tester and use of terminal emulation programs for accessing "green screen" applications.
Editor's Note: This article is based on Version 2003.06.13 of IBM Rational Functional Tester and Version 2004.06.30 of IBM Rational Functional Tester Extension for Terminal-based Applications.
As in testing any other type of application, your first step in planning to automate testing of green screen applications is to rank the areas of risk. When you have your list of tests to automate, the simplest approach is to record your actions in Rational Functional Tester and then modify the resulting script to extend the test coverage. Because most green screen applications are procedural -- that is, you enter data into a field and then move to the next -- they lend themselves particularly well to data-driven testing.
Ranking risks as a basis for your plan
You need to ensure that you don't expend effort trying to build regression tests for areas that are unlikely to change, balanced against the risk to the product. In an insurance system, the highest risk areas are those that provide premium calculations. Areas affecting brand credibility must also be considered high risk. If customers are sent incorrect invoices or letters requesting information on deceased family members, customers lose faith in the company. Loss of brand credibility affects the sale of new policies and may even cause customers to move to another brand. System fragility is another area of risk; there are often parts of a system that always cause defects whenever they're changed. If you're new to the system, reviewing the defect history will help you uncover these areas. Once you've identified all the risks, rank them from the highest to the lowest. This will be the basis of your test automation plan.
Some risks can't be suitably addressed in automated testing; you need to note these as such and make sure they're part of a manual test plan. This includes risks related to reliability, such as running out of disk space, hardware failure, network failure, running out of memory, and such. Tests of reliability normally need someone to intervene at the right time to cause the wanted failure. You also can't automatically verify output that's gone to a printer, but you can automate the process that creates an invoice or letter. Create a set of master invoices and letters, then execute your suite. Do a comparison between the master, the expected printout, and the actual printout to verify that the printouts are correct.
Using the "test oracle" approach to automation
The test automation architecture you build to support data-driven testing also allows you to extend your test coverage. You can easily add the most common scenarios specific to the application under test into your data-driven tests. The simplest way of identifying these common scenarios is by talking with those who will be using the application. You can also use reporting and data-mining tools to build a profile of other scenarios.
To build a set of regression tests quickly, we used the latest release as the "test oracle." This means that we arbitrarily decided that this version would serve as the baseline and that we would compare the results of testing later versions with the results of testing this version to identify changes. In this approach, we created scenarios with empty results and then extended the data-driven framework so that the results were recorded back into the test oracle. These results were then used as a basis of comparison.
The drawbacks to this approach are as follows:
- The test oracle codifies incorrect states. The test oracle only models the expected state, not necessarily the correct state. Any time a failure occurs, the tester must investigate the failure to ensure that it doesn't actually indicate that the product has now been corrected.
- The approach creates unlikely scenarios. It's difficult to know if a scenario is unlikely or not. The fact that it doesn't appear in the current data would seem to indicate that the scenario isn't used, but we don't know that it won't happen in the future. If leaving unlikely scenarios in causes little impact, they can be left in. But if it takes a minute for each scenario to execute, scenarios should be broken up into different sets and the set of the most common scenarios should be executed first.
- The expected results are only as good as the last release. This method only proves that the quality of the latest release is as good as the quality of the test oracle release. To improve the quality of subsequent releases, we need to review and verify the scenarios we're running, using our risk list to prioritize the scenarios to review.
Installing and launching the extension
After you've done some planning, you'll be ready to install the extension. Follow these steps:
- Install IBM Rational Functional Tester but don't start it.
- Insert the IBM Rational Functional Tester Extension for Terminal-based Applications installation CD and follow the instructions provided by the installation program.
- Start Rational Functional Tester. You may be prompted that there are updates pending.
- Click Yes and wait for the Configuration Changes window to appear.
- Select the changes listed in the Configuration Changes window and then click Finish.
- Restart Rational Functional Tester.
- Check your installation by selecting Configure>Configure Applications for Testing. You should see that Extension for Terminal Applications has been added to the applications list in the Application Configuration Tool window, as shown in Figure 1.
Figure 1. Checking the applications list in the Application Configuration Tool window
Once you've installed the extension, you can run it and become acquainted with the interface before you start to record test scripts. In the Application Configuration Tool window, select Extension for Terminal Applications and then click the Run button to launch the extension. As you can see in Figure 2, you're presented with an interface similar to other terminal emulation packages. The difference here is that the extension makes the terminal fields visible to Rational Functional Tester as objects with properties.
Figure 2. Extension for Terminal-based Applications interface
To connect to your application, you'll enter the host name, select the terminal type from the drop-down list, and click the Connect button.
Recording a script
Recording a script with the extension is much like recording a script with Rational Functional Tester. You'll choose "Extension for Terminal Applications -- java" in the Start Application dialog (see Figure 3) and click OK.
Figure 3. Choosing the extension in the Start Application dialog
In your test script, you'll see the following entry:
startApp("Extension for Terminal Applications");
|
Once the extension has started, you'll enter the host name and terminal type of the machine running the green screen application. You'll then be presented with the normal application window, so that you can log in and navigate as you please. (See Figure 4.)
Figure 4. A typical green screen application's login window
Navigation will be entered into your test script using the inputKeys method -- for example:
TFrame().inputKeys("username{TAB}");
TFrame().inputKeys("password{ENTER}");
|
Creating a verification point is similar to doing so when testing other applications. You'll start the Verification Point and Action Wizard by clicking the appropriate icon in the Recording dialog. This displays the Select an Object tab of the Verification Point and Action Wizard, as shown in Figure 5.
Figure 5. Verification Point and Action Wizard window, Select an Object tab
Moving the Object Finder hand over the area of the screen you want to interact with will cause the extension to signal your selection by placing a red box around the object, as demonstrated in Figure 6.
Figure 6. Highlighted screen object for testing
Releasing the mouse button will display the Perform an Action tab (Figure 7) and allow you to select the type of verification point you want.
Figure 7. Verification Point and Action Wizard window, Perform an Action tab
Making manual modifications to the script
There are a couple of different kinds of modifications to the script that you'll need to make after you've recorded it in Rational Functional Tester with the Extension for Terminal-based Applications. You'll need to correct object states and separate commands from command streams.
When you're creating a host connection script and you're entering keystrokes (for example, you might type logout and then press the Enter key), this puts something like the following in the script:
TFrame().inputKeys(logout{ENTER});
|
At playback time, Extension for Terminal-based Applications reaches that line in the script and waits for the TFrame object to be ready before it sends the keystrokes.
While recording your script, you may be able to enter your keystrokes before the synchronization algorithm for the terminal determines that the presentation space is loaded. In this case, something like the following is entered into the script instead, indicating that the TFrame object wasn't initialized and wasn't in the ready state when you started typing the command:
TFrame(ANY, UNINITIALIZED).inputKeys(logout{ENTER});
|
This line could potentially cause a problem when the script is played, because at playback time Extension for Terminal-based Applications sends those keystrokes as soon as it reaches that line in the script, regardless of the state of the terminal. To fix this, simply remove the state information from the object in the script:
//TFrame(ANY,UNINITIALIZED).inputKeys(logout{ENTER});
TFrame().inputKeys(logout{ENTER});
|
Separating commands from command streams
During recording, there are occasions when the extension won't capture the state of the terminal correctly. This means that during playback the extension won't synchronize with the frame. You can resolve these issues by changing your recorded script so that the keystrokes that cause the host screen to update are separated into two command streams. For example, in the following instance, the keystroke "A" causes the cursor to move to the next field, where "12" is input.
TFrame().inputKeys("A12{ENTER}");
|
This needs to be separated into two calls to inputKeys, so that the extension can synchronize correctly, as shown:
//TFrame().inputKeys("A12{ENTER}");
TFrame().inputKeys("A");
TFrame().inputKeys("12{ENTER}");
|
In instances like the following, where all the keystrokes are bunched together in the input string, the {TAB} command causes the cursor to move to the next field.
TFrame().inputKeys("{TAB}SomeHostCommand{ENTER}");
|
This needs to be separated into two calls to inputKeys, as shown:
//TFrame().inputKeys("{TAB}SomeHostCommand{ENTER}");
TFrame().inputKeys("{TAB}");
TFrame().inputKeys("SomeHostCommand{ENTER}");
|
Always log out of your application and use the Disconnect button to ensure that the application states stay synchronized, rather than using File > Exit.
As this article has demonstrated, Rational Functional Tester with its Extension for Terminal-based Applications is an effective tool for automating tests against green screen applications. Because green screen applications are procedural, they're very good candidates for data-driven testing. You might also want to investigate using the latest application release as a test oracle for regression testing the next version with Rational Functional Tester.
-
Read more about test oracles in
Testing Object-Oriented Systems: Models, Patterns, and Tools by Robert V. Binder (Addison-Wesley, 2000)
-
Share your questions and views on this article with the author and other readers in the Rational discussion forums.
-
To learn more about IBM Rational, visit the developerWorks Rational area. You'll find technical documentation, how-to articles,education, downloads, product information, and more.
Get involved in the developerWorks community by participating in developerWorks blogs.
-
Get Rational books at discounted prices from the Developer Bookstore.
Carl Bray is a specialist test consultant at Equinox Ltd. in New Zealand with extensive experience in software delivery. He has managed all aspects of software quality assurance, including test management, test automation, and test training. Carl has loved mountains and skied all his life. He took up scuba diving while living in Columbia and climbed to a height of 4,800 meters while traveling in Peru. Carl can be reached by e-mail.
Comments (Undergoing maintenance)
Table of contents
Next steps from IBM
My developerWorks community
Tags
Use the slider bar to see more or fewer tags.
Popular tags shows the top tags for this particular content zone (for example, Java technology, Linux, WebSphere).
My tags shows your tags for this particular content zone (for example, Java technology, Linux, WebSphere).
Popular article tags |
My article tagsSkip to tags list
Popular article tags |
My article tags
