Using IBM Rational Functional Tester to test financial services applications

With Web sites becoming critical to the mission of financial services companies, Web development (and consequently Web testing) is quickly becoming a core competency. That's where automated testing with IBM Rational Functional Tester comes in.


Michael Kelly (, Senior SQA Specialist, CTI Group

Mike Kelly is currently a senior SQA specialist at CTI Group. He's had experience managing software automated-testing teams and has been working with the Rational tools since 1999. His primary areas of interest are software development lifecycles, software test automation, and project management. Mike can be reached by e-mail.

14 November 2005 (First published 13 June 2005)

Firms in the financial industry (banking, insurance, equities/stock brokerage, and transaction services companies) are increasingly moving services to the Web as technology becomes cheaper and consumer demand for Web-enabled solutions grows. Offering services such as online checking and bill payment, online insurance quoting, and online stock trading gives such companies a way to expand sales channels and expose new and existing products. With Web sites becoming mission critical, Web development (and consequently Web testing) is quickly becoming a core competency of financial services companies. That's where automated testing with the IBM® Rational® Software Development Platform comes in.

Automated testing can reduce the complexity of Web development, reduce the costs of testing in an iterative environment, speed up the detection of meaningful bugs, and help to increase communication between developers and testers. Furthermore, it can help financial services companies avoid financial losses related to Web sites that are hard to use or unreliable. According to Michal Blumenstyk in his article Web Application Development -- Bridging the Gap between QA and Development, financial services companies could lose up to "$6.5 million per hour in brokerage operations, and $2.4 million per hour in credit card sales applications" due to "poorly functioning" Web sites.

Both functional and performance testing play crucial roles in creating a well-functioning Web site. In this article, we'll focus on functional testing of financial services applications with IBM® Rational® Functional Tester. In particular, we'll look at uses of data-driven functional testing and ways to use functional tests to check security.

Editor's note: This article is based on version 6.0 of the IBM Rational Software Development Platform.

About functional testing and Rational Functional Tester

Functional test automation is an easy way to lower the costs of test execution over time. The same test can be reused multiple times, saving tester resources and allowing fast turnaround. Automation done right is worth the effort, as shown in the abundant literature on the topic of return on investment (ROI) for automation.

For financial services applications, there's often a lot of low-hanging fruit for automated test suites.

  • Simple data-entry screens without many complex controls or much graphics processing are ideal candidates for automated testing because most automation tools interface nicely with the technologies used to develop them.
  • Straightforward business models lend themselves to scenario-based testing, ideal for automation.
  • Report-oriented or quote-oriented results lend themselves to automation because automated scripts can interface directly with the application and the databases to verify accuracy.

In particular, using data-driven techniques (discussed below) is one of the most effective ways to increase ROI for automation on financial applications. These techniques allow you to turn one test into many.

IBM Rational Functional Tester is an object-oriented automated testing tool that you can use to test a variety of applications. Rational Functional Tester's object-oriented technology makes it possible for you to identify objects by their internal object properties, not their screen coordinates, when you generate scripts by recording against the application under test. This means you can test any object in the application under test, including the object's properties and data. In the financial services industry, you can easily extract data from controls and fields in order to verify its accuracy or to store it for comparison at a later point in the test. You can test IDE-specific test objects in .NET, Java, and Web-based applications, whether they're visible or hidden in the interface.

Rational Functional Tester offers you a choice of scripting language and editor -- Java in Eclipse or Microsoft Visual Basic .NET in Visual Studio .NET. That means that regardless of the language or development platform your development staff has chosen, you should be able to integrate with them and leverage some of their expertise as you develop your automated tests. For more a more detailed look at Rational Functional Tester and the differences between the Java and Visual Basic .NET implementations, see my article Introduction to IBM Rational Functional Tester.

Applying data-driven testing to financial services applications

Data-driven testing is a technique where a single test script is used repeatedly with varying input and response data that comes from a predefined data set. The nature of the data used by financial services applications lends itself nicely to data-driven techniques. We'll take a look at one example of using data-driven testing in the financial services world and then discuss how to select the data to use. I'll also show you how useful it can be to separate default data from test data. For more information on implementing data-driven testing, see "Resources" at the end of this article.

An example of data-driven testing

Suppose your financial services site presents the credit card application shown in Figure 1.

Figure 1. A credit card application on a financial services site
A credit card application on a financial services site

If you wanted to test different scenarios based on age, state, income, and years with employer, you could generate a large number of test cases.

  • To test all 50 states, you would have 50 values for state.
  • To test valid and invalid ages, you would have two values (above 18 and below 18 to keep it simple).
  • To test different incomes, you would develop brackets based on your business rules -- say four income brackets, giving four possible value ranges for income.
  • To test years with employer (again based on your business rules), you'd again have to use value ranges -- say under a year and over a year.

Combining all of these variables in all possible ways would give you something like 800 test cases (50 x 2 x 4 x 2). While you would most likely use some technique to reduce the number of test cases you would execute, if you used a data-driven technique you could use the same set of test scripts to execute all 800 test case variations (or whatever number of data variations you decided on).

In Rational Functional Tester, data-driven testing can be implemented via datapools. A datapool is a collection of related data records that supplies values for the variables in a test script during script playback. When you use datapools, Rational Functional Tester delivers a different set of test data to a script each time you play it back. You use datapools to supply realistic data and to stress an application with a variety of data and with a realistic amount of data. Figure 2 shows an example of a datapool that you could use to test your credit card application.

Figure 2. A datapool to use in testing the credit card application
A datapool to use in testing the credit card application

Let's look now at how you might use a datapool to do functional testing of some aspect of your financial services site. Listing 1 is a sample functional test script that tests the credit card application shown in Figure 1 with values from the datapool shown in Figure 2.

Listing 1. A sample script testing our credit card application
{ public void testMain(Object[] args)
      //First Name

      mnametext().click(atPoint(60,6));       submitAnApplicationwindow().inputChars(dpInt("MI"));

      //Last Name
      lnametext().click(atPoint(60,6));       submitAnApplicationwindow().inputChars(dpInt("LastName"));


      ssntext().click(atPoint(60,6));       submitAnApplicationwindow().inputChars(dpInt("SSN"));


When you run this script, Rational Functional Tester will enter each of the values from the current row of the datapool into the appropriate fields of the application. Every time the script is executed, assuming the datapool cursor is advanced, the next row of data will be entered. Ideally, your script will stop executing once the datapool no longer contains data that hasn't been entered.

Selecting data to use

Data selection is the first step down the road of data-driven testing. You'll need to select the data that either drives the navigation of your application, represents the data that gets entered into your application, or both. Here we'll quickly examine five different ways to select data for testing: based on risk, based on requirements, based on availability, using production data, or using randomly generated data.

Selecting data based on risk. The number one criterion for test data selection should be risk. When you identify risks, you consider what can go wrong. You're looking for the events that might occur that would decrease the likelihood that you'll be able to deliver the project with the right features and the requisite level of quality on time and within budget. There are three ways to categorize risks:

  • By the impact of the risk -- the deviations of schedule, effort, or cost from plan if the risk materializes
  • By likelihood of occurrence -- the probability that the risk will materialize (usually expressed as a percentage)
  • By risk exposure -- the impact multiplied by the likelihood of occurrence

Selecting data based on requirements. You can also select data that will allow you to test a requirement or a set of requirements explicitly. Look for data that will allow you to exercise feature sets, capabilities, and security features in your application. If your application has different roles, what data would you need to exercise each role? What features do you want to include in your test coverage and what data do you require to use them? In addition, consider what impact the various target deployment environments will have on the elements to be tested. Your list of selected data should include data for both the application under test and the target environment(s). What data will you need to test hardware devices, device drivers, operating systems, network and communications software, third-party base software components (for example, e-mail software, Internet browsers), and various configurations and settings related to the possible combinations of all these elements? When dealing with financial applications, what data will you need to test internationalization?

Selecting data based on availability. You may want to select data that's readily available. This could be production data (discussed in depth below) that's in an easy-to-access format, data from past iterations, spreadsheets used by manual testers for your project, data from other projects or teams in your company, or data from some data generation source (also discussed below). The idea here is that if the data is easily accessible, as well as usable and meaningful, including this data in your testing can save time and money. I emphasize usability and meaningfulness because it's important that you don't select data just because it's there and ready to be used.

Using production data. Another strategy to gather test data is to use production data. Although you shouldn't rely solely on this type of data, it can be one of the richest sources of scenarios for automated testing, both because the data is representative of real scenarios the application will face and because it will most likely provide a high number of different scenarios. On a recent project, we read production data from a holding table about once a week, and we were rewarded with 300 to 500 scenarios with little to no effort. You can load the data straight into the test environment, read it into data files for processing later, or read it in real time and convert it as you use it. Production data is also an excellent source for parallel testing. If you use production data in the system you're developing, you'll quickly know if that system works like the system in production. With financial applications, this can especially help in finding problems with floating-point values, conversion ratios, and lengths associated with data types.

There are some caveats about using production data, however. Production data will most likely not contain many of the special cases you'll want to test for, and it's not a replacement for well-thought-out test scenarios. There are also potentially some legal issues surrounding use of production data (for more on this, see "Resources" at the end of this article). Especially if you outsource some of your testing, you'll want to be sure to check your company's policies on the use of production data; if no formal policy exists, consult someone in your legal department. Even if you can't directly use production data, odds are you'll be able to change some values (names, social security numbers, and such) and use the rest of the data.

Using random data generation. Many tools include test data generators, and Rational Functional Tester is one of them. Random data generators can be especially helpful in generating large sets of financial data or customer data. For example, if you need to test for rounding errors on financial information (and I'm guessing you do), you can randomly generate a large set of data, run the data through your application, run it through the Windows calculator (assuming the Windows calculator has no rounding errors) simulating the same calculations, and then compare the results to make sure they match. Or if you need to generate a set of 500 different customer names and addresses, you can have a generator create a table of names and addresses.

One last note: You should avoid trying to select data for all test cases at once. Instead, take an incremental and iterative approach to working with test data, focusing your efforts on the data that you think is most likely to produce useful evaluation information for the given test cycle. This helps to mitigate the risk of devoting too much time to a single set of data or type of testing, to the neglect of other data or testing tasks, and minimizes the risk of expending effort on data for test ideas that may later prove of little interest.

Separating default data from test data

By separating default data from test data, you can save space and time as well as simplify the test data set, making it easier to read and troubleshoot. To illustrate this concept, let's go back to the example of the credit card application shown in Figure 1. Say you're in New York and want to test the processing of out-of-state applications. Your test data for the Address Information section may look something like the data shown in Table 1.

Table 1. Test data for the Address Information section of the credit card application
Street #Street NameApart #CityStateZip CodePhoneStatus
1234Washington Blvd Penn YanNY14527555-345-4776Owner
1234Washington Blvd BerneIN46711555-345-4776Owner
1234Washington Blvd Big LakeTX76932555-345-4776Owner
1234Washington Blvd YakimaWA98901555-345-4776Owner
1234Washington Blvd WellstonOH45692555-345-4776Owner

The default data set, as shown in Table 2, consists of one full entry. For each column in the table (or each field on the screen), there's a value.

Table 2. Default data set
Street #Street NameApart #CityStateZip CodePhoneStatus
1234Washington Blvd Penn YanNY14527555-345-4776Owner

The test data set, shown in Table 3, contains only the data that varies from entry to entry. If a value doesn't differ from the value in the default data set, you don't include it in the test data set.

Table 3. Test data set
Street #Street NameApart #CityStateZip CodePhoneStatus
   Big LakeTX76932  

Looking at these tables, you can very quickly see what's being tested and what values are used. If this were a much larger table (containing all the states, more cities per state, and so forth), you would more readily appreciate what a difference the simplification makes.

When implementing this model, you'll need a process that reads from both the test and the default data set. That process should populate data using the default data whenever it encounters an empty cell (or null value) in the test data set. The pseudocode in Listing 2 illustrates this process.

Listing 2. A simple process for populating data using the default and test data sets
Get the value from the TestDatapool

If the value returned from the TestDatapool is NULL Then
   Get the value from the DefaultDatapool
   Return the DefaultDatapool value
   Return the TestDatapool value
End If

Testing the security of a financial services site

Security testing is intended to ensure confidentiality (privacy, authorization, and exclusivity), integrity (correctness, completeness, validity, authenticity, and auditability), and availability (continuity, timeliness, contingency, reliability, and robustness). The nature of financial applications makes security testing particularly important. No one tool will automate security testing -- if one existed, hackers would be using it to get at your financial information -- but functional and performance testing tools can find many security-related bugs.

The best investment in security testing you can make is to learn how to think like a security tester. As you're scripting something ask yourself, "How can I vary this code to test some aspect of security?" Following is a short list of techniques you can apply using Rational Functional Tester with data-driven testing.

  • Test fields at the boundaries and test with bad data.

For example, you might have a datapool that looks like the one shown in Figure 3 to test the boundaries of a birthdate field on a credit card application. (Assume that this datapool was created on 11/04/2004. Actually, a better strategy would be to use dates that don't age in the datapool.)

Figure 3. A datapool to test boundaries
A datapool to test boundaries
  • Test common user IDs and passwords.

You might have a datapool that contains common user IDs and passwords that you can use to log in with. Or if you're more devious, use a datapool to test to ensure that when you create a user ID it conforms to minimum standards for user IDs and passwords, such as the password standards for West Chester University. To test these password standards, you might create the datapool shown in Figure 4.

Figure 4. A datapool to test password standards
A datapool to test password standards
  • Test for buffer overflows.

Create datapools (or perhaps here it would make sense to use some other storage method due to string size -- or even perhaps to create the string programmatically) with very long strings of data. In their book How to Break Software Security, Whittaker and Thompson go into detail on this type of "attack" and show you sample strings that you can use as well as give you tips on how to select meaningful values for this type of testing.

  • Create tests that use alternate routes for the same (or similar) tasks.

If there are multiple ways to navigate to a particular page on your site, make sure you're testing all of them and not just the one that you think will be used most often.

  • Create tests that force all error messages.

If you have rules in your application that are associated with error messages, create test cases to exercise those messages. For example, if your application requires a user to log in, your tests might force error messages such as these:

You have entered an invalid user ID.
You have entered an invalid password.
You have exceeded the maximum number of login attempts.

You can also use the other IBM Rational tools to test security. With IBM® Rational® Performance Tester, you can do the following:

  • Modify the GET and POST commands for the Web site and change values, paths, and data to see if you can bypass security.
  • Create a load and then perform security testing on the application while it's under stress.
  • Review scripts to see if you can find any security problems contained in the GET and POST commands.

With one of the scripting tools, you can test security in these ways:

  • Parse Web page source code looking for information leaks (paths, IDs, protected information, and so on).
  • Enter predefined SQL injection attacks into various input fields.
  • Replace files that the application uses at run time.

Certainly there are more uses than these (and probably some better), but this should get you thinking about how you can use these tools (and others) to help with security testing. Some of my favorite resources for security testing are listed in the "Resources" section at the end of this article.

Next steps

This article has discussed just a subset of the types of tests that lend themselves to testing financial applications. Static analysis, component testing, and runtime analysis top my list of other valuable types of testing that have a large suite of tools available for them. But before you look into those, get comfortable with the capabilities of Rational Functional Tester.

I'd suggest that you start recording scripts, building object maps for your application, and building datapools to reflect the data and decision points in the application. As you're building these test assets, start to think about how you can combine them by using business scenarios and/or models to create more complicated tests. If your application is made of multiple components, start to explore how Rational Functional Tester might interface with each of them. The sooner you know this, the better off you'll be down the road.



developerWorks: Sign in

Required fields are indicated with an asterisk (*).

Need an IBM ID?
Forgot your IBM ID?

Forgot your password?
Change your password

By clicking Submit, you agree to the developerWorks terms of use.


The first time you sign into developerWorks, a profile is created for you. Information in your profile (your name, country/region, and company name) is displayed to the public and will accompany any content you post, unless you opt to hide your company name. You may update your IBM account at any time.

All information submitted is secure.

Choose your display name

The first time you sign in to developerWorks, a profile is created for you, so you need to choose a display name. Your display name accompanies the content you post on developerWorks.

Please choose a display name between 3-31 characters. Your display name must be unique in the developerWorks community and should not be your email address for privacy reasons.

Required fields are indicated with an asterisk (*).

(Must be between 3 – 31 characters.)

By clicking Submit, you agree to the developerWorks terms of use.


All information submitted is secure.

Dig deeper into Rational software on developerWorks

ArticleTitle=Using IBM Rational Functional Tester to test financial services applications