Skip to main content

skip to main content

developerWorks  >  Rational  >

Model-based testing using IBM Rational Functional Tester

developerWorks
Document options

Document options requiring JavaScript are not displayed

Discuss


My developerWorks needs you!

Connect to your technical community


Rate this page

Help us improve this content


Level: Intermediate

Jeff Feldstein (jbf@sherpas.com), Manager, Software Development, Cisco Systems

07 Oct 2005
Updated 28 Nov 2005

The classical method of test automation takes an application through the same sequence of test cases until all the bugs have been fixed. The problem with this method is that a customer who uses the functions of the program in a different order may run into bugs the test automation never encountered. Model-based testing, on the other hand, generates tests at random from a description of the application's behavior and thus has the advantage of testing every possible sequence a customer might try. If you implement model-based testing with IBM® Rational® Functional Tester for Java® and Web, you have the power of Java programming at your fingertips to apply to your automation effort.

In this article we'll first explore the case for model-based testing (not to be confused with the development approaches of model-driven development and model-driven architecture, which use UML) and then look at the basic data structures, logic flow, and navigation algorithms required for model-based testing of Java and Web applications using Rational Functional Tester. A working version of a model-based test system and all of its source code is available from my model-based testing Web page. I'll be showing you parts of that here to illustrate how model-based testing works. I'd suggest that you download and run the model after reading the article. You can also download an evaluation version of Rational Functional Tester for Java and Web if you want to take it for a test drive. related developerWorks content. USE LINK EXAMPLE FOR LINKS IN extended abstract

The case for model-based testing

Most software testing begins with a test engineer sitting down at the computer, starting the application, randomly navigating its features, searching for bugs, and tripping across crashes. Since the test engineer knows (or learns while testing) what the application is supposed to do, he can file a bug when the application doesn't get it right. In addition, he'll file bugs for all the crashes he's able to reproduce. Eventually, the test engineer generates enough test cases that he wants to remember to rerun on the next build or release that he's able to write a comprehensive test plan. The core of the test plan is the description of all the test cases to run on each build.

After running through the test plan a few times, the good test engineer decides that the computer can do a faster, more accurate job of running some of these test cases, so he automates as many as he can. He might then logically sequence these test cases and run through the entire use case, checking for errors and crashes. This automation forms the basis of his regression test. The good engineer might even data-drive the test cases so that the same logic can be reused against large sets of input and expected-result data. I'll refer to this type of automation as "classic test automation."

One problem with this evolution is that the random sequencing that was implicit (and probably unconscious) in the manual testing is gone. As soon as the application can run through the sequence of test cases with no apparent errors or crashes, it's shipped. Now the customer, having just purchased and installed the application, uses functions in an order that differs from the automation and finds some nasty bugs. The customer calls support, support reports the sequence to development, and development fixes the bug. The test engineer, in reviewing the bug, sees what he missed in his automation, adds this new sequence to his tests, and is happy -- at least until the next customer tries the same functions in yet a different order and breaks the application in yet a new way. Right about now, management is probably thinking that maybe an army of manual testers wasn't such a bad idea.

There is, however, another approach. Imagine that the test automation knew where the current feature (or screen) being tested lived in the application relative to the other features (or screens). Imagine further that it knew all of the possible ways to transit out of the current feature and where in the application it's supposed to be after this transition. Based on this behavioral model, the automation could execute the current feature, check for the correct results, pick a transition at random, end up in a new feature, test this new feature, and randomly pick yet another transition. This process could then be repeated indefinitely, bringing back the random testing that was lost when the human was removed from running the tests. The term commonly used in the software test industry for this type of testing is model-based testing.

Model-based testing is a form of automated testing that implements random behavior in your test automation software. Each application feature (or screen) becomes a "state," and each button, link, menu selection, or command leading to the next function becomes a "transition." Your application's model is a memory-based representation of each state, all of the possible transitions, and the destination state that should result when a particular transition is selected. In a Web or graphical Java application, a screen or message box usually corresponds to a state. All the buttons, links, tabs, and other objects that take the user to another state or screen are the transitions. Validation of the business or application logic at each state is achieved in the same way as in classic test automation.

By applying various navigation algorithms to the model (for example, randomly choose an unvisited state, visit all states in order), the test exercises the application in new ways or more complete ways than in classic automation. In addition, with model-based testing, adding computers increases the test coverage over a given time by generating even more paths through the code. In classic automation, no matter how many times you run through the automation it's always the same. In model-based testing, since the navigation can be random, computers running in parallel to each other will be executing different paths through the code. In Java or Web applications where many clients run against a single server, this method is a valuable tool to test multiuser access as well.



Back to top


Model-based testing for Java or Web applications

Model-based testing can be used for many types of software or application testing. The concepts remain the same whether you're testing a cell phone's embedded system or a browser-based Internet application. There's a problem, however, when you try to implement model-based testing for Java or Web applications. Most of the legacy test tools are designed primarily for script recording and playback. Although the logic can be hand-manipulated and tuned by directly editing the source code, the native languages of these tools don't allow for easy declaration and manipulation of large-scale data structures, and object-oriented design is nearly impossible.

These issues are solved with Rational Functional Tester for Java and Web, because Java itself is the test script language. This isn't some subset of Java or any "Javalike" language -- it's the real thing. It runs on a variety of JVMs and is implemented as a plug-in of the Eclipse® 3.0 IDE and runtime environment. The test engineer, therefore, has all of the programming power afforded to application programmers. This environment lends itself to highly engineered, highly scalable, flexible, and reliable test automation software.

To take advantage of the power of model-based testing, you need a level of skill beyond that required of the normal manual tester or automation script writer. You need a good foundation in software engineering as well as a sound basis in Java programming. The implementation described below, although written entirely in Rational Functional Tester for Java and Web, makes heavy use of native Java concepts. You can grasp the value of model-based testing without detailed knowledge of Java, but to adapt the implementation to a real-world application, you'll need to have a high level of comfort with Java programming techniques.



Back to top


Implementation details

Now let's consider the details of how to implement model-based testing with Rational Functional Tester. I'll just touch on the highlights here. For a good tutorial on model-based testing, see "Intelligent Test Automation" by Harry Robinson. Take a look at Figure 1. It shows a simple Web portal to illustrate a browser-based application consisting of six tabs. Each tab brings you to a new screen. Each screen is a state. Since all six tabs are available from each other, you can navigate from any state to any other by clicking its associated tab. Each tab becomes a transition to a new state.
Figure 1: A simple Web portal with six tabs = six states
A simple Web portal with six tabs = six states

Figure 2 illustrates a three-state model using just three of the tabs -- Home, Images, and News. Each screen is a state. From the Home state, three transitions are shown -- Home, News, and Images. A transition from a state is shown as an arrow leading out of a state. Each arrow points to its corresponding destination state. This model shows that when you click the News tab, you should transition to the News screen; the Images tab should take you to the Images screen, and the Home tab should return you to the Home state. Similar information is stored for the Images and News states. These states, transitions, and destinations form the model of the application that we can test against.


Figure 2: A three-state model of the Web portal
Figure 2: A three-state model of the Web portal
Table 5. Table using a caption tag, alternating color rows
From StateTransitionTarget State
Home screenHome screenHome screen
News tabImages tabHome tab
News screenImages screenHome screen
  1. List item 1
  2. List item 2
  3. List item 3

Use monospace code font for names of commands, such as the make install command, as well as for classes, objects, methods, code snippets, and any text the user types in. You can also use bold and italics for emphasis or clarity in text.



Back to top


Another major heading

Code listings are common in developerWorks content.


Listing 1. Caption goes here

<!-- Please limit code lines to 90 characters max. -->
<!-- The following line is the max length: -->
|-------10--------20--------30--------40--------50--------60--------70--------80--------90
000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
Line of code
Line of code
Line of code
Line of code
  Line of indented code
  Line of indented code
Line of code with <angle brackets>
Line of code with an ampersand (&)
Line of code with something in bold
      

Minor heading

When to use a sidebar
Use a sidebar for a content detour or reinforcement. For example, you might define terms or provide a bit of historical background in a sidebar. Or you might summarize the key points of a lengthy discussion. For a sidebar in action, see the article "Open source development using C99".

Technical graphics such as screen captures, diagrams, and photographs can enhance your content. See "Illustrating your article or tutorial for developerWorks" to learn how to create and deliver effective graphics.

To keep readers engaged with your content, and not wandering away mid-article to pursue non-developerWorks content, try to minimize off-site links in the body of your article. Instead, in the body, link to the Resources section and include related content (articles, tutorials, developerWorks views, technical briefings, Webcasts, demos, books, Web sites, or offers) and urls there. Likewise, for code (downloads, demos, etc.), don't include the urls in the body; rather, link to the Downloads section and include the urls there.



Back to top


Table examples

Use any of the following styles for tables in your article. Table 1 uses a <caption> tag and left-aligned columns, and is the recommended style.



Download


Resources

Learn

Get products and technologies

Discuss


About the author

Jeff Feldstein has been a software developer, tester, and manager for 24 years. Jeff's current position is at Cisco Systems, managing a team of 38 test engineers who test an internally developed operating system used by Cisco's network management products. A frequent speaker at test and user conferences throughout the United States, Jeff is a strong advocate of applying the principles of software engineering employed in the best applications to the test automation of those systems.




Rate this page


Please take a moment to complete this form to help us better serve you.



 


 


Not
useful
Extremely
useful
 


Share this....

digg Digg this story del.icio.us del.icio.us Slashdot Slashdot it!



Back to top