Eclipse Test & Performance Tools Platform, Part 3: Test applications

Test, profile, and monitor your applications with an advanced, extensible platform

In this installment of the "Eclipse Test & Performance Tools Platform" tutorial series, learn how to use Eclipse Test & Performance Tools Platform (TPTP) testing features to measure the response time of your Web application. Then, discover how to create and manage improved JUnit tests and create computer-directed manual testing.

Martin Streicher (martin.streicher@linux-mag.com), Editor-in-Chief, Linux Magazine

Martin Streicher is the Editor-in-Chief of Linux Magazine. He earned a master's degree in computer science from Purdue University and has been programming UNIX-like systems since 1982 in the Pascal, C, Perl, Java, and (most recently) Ruby programming languages.



22 August 2006

Also available in

Before you start

About this series

Writing the code for an application is but the first stage in the long process required to deliver robust production-quality programs. Code must be tested to vet its operation and accuracy. Code must often be profiled to remove bottlenecks that impede performance and to remove wasteful or inadvertent use of resources, especially memory. Code must also be monitored -- to pinpoint failures, of course, but also to identify usage patterns, opportunities for further enhancement and optimization, and attempted and actual intrusions.

The Eclipse Test & Performance Tools Platform (TPTP) is a software architecture and several realized components (so far) that extend the Eclipse platform to include test, performance, and monitoring tools. This "Eclipse Test & Performance Tools Platform" tutorial series explores its capabilities. Part 1 demonstrates how to profile a Java™ technology application. Part 2 shows how to capture and transform arbitrary log files to the widely supported Common Base Events (CBE) format. Here in Part 3, find out how to manage application testing.

About this tutorial

This third installment in the "Eclipse Test & Performance Tools Platform" tutorial series demonstrates how to use TPTP's testing features to measure the response time of your Web application, shows how to create and manage improved JUnit tests, and describes how to create computer-directed manual testing.

Objectives

In this tutorial, you learn how to use the Eclipse TPTP testing features to measure the response time of your Web application. Then, you discover how to create and manage improved JUnit tests and learn how to create computer-directed manual testing.

Prerequisites

You should have experience with software development and the entire software development life cycle. You should also have experience installing software from the command line, and know how to set and manage shell and system environment variables, such as the shell's PATH and LD_LIBRARY_PATH variables, and the Java CLASSPATH variable. Acquaintance with the Java programming language, the Eclipse integrated development environment (IDE), and the Eclipse user interface (UI) is advantageous, but not required.

System requirements

You can run Eclipse on any system that offers or supports a Java Virtual Machine (JVM), which includes Solaris, Linux®, Mac OS X, and Microsoft® Windows®. If you haven't installed a JVM on your system, make sure you have at least 250 MB of free disk space for the Java software (in some cases, far less is required). The core Eclipse software and all the plug-ins used here consume another 50 MB.

In addition to disk space, you need enough free physical memory to run the JVM. In general, 64 MB or more is recommended for satisfactory performance.

Here's everything you need:

Note: Even if you already have a Web browser, if you want to use the computer-directed manual test suites, you must install Mozilla V1.7.8 or later because it includes UI components that the Eclipse TPTP manual testing tools use.


Getting started

Install Eclipse and the Eclipse TPTP

After you install Mozilla (directions are available on the Mozilla Web site and won't be repeated here) and download the required components, you can install Eclipse and the Eclipse TPTP. The installation instructions that follow should suffice for Linux and UNIX® systems; consult Eclipse.org for installation instructions for Windows and other platforms.

Assuming that you've downloaded these files:

  • jre-1_5_0_07-linux-i586.bin
  • eclipse-SDK-3.2-linux-gtk.tar.gz
  • xsd-runtime-2.2.0.zip
  • tptp.platform.runtime-TPTP-4.2.0.zip
  • tptp.test.runtime-TPTP-4.2.0.zip
  • emf-sdo-runtime-2.2.0.zip
  • tptpdc.linux_ia32-TPTP-4.2.0.zip

and assuming those files are in your /tmp directory, complete the steps shown below to install Eclipse and prepare your environment to run the Eclipse TPTP.

Listing 1. Install Eclipse and the Eclipse TPTP
#
#
# Unpack the JRE in a new directory, ~/java-tptp
#
$ mkdir ~/java-tptp
$ cd ~/java-tptp
$ sh /tmp/jre-1_5_0_07-linux-i586.bin

#
#
# Unpack Eclipse, Eclipse TPTP, and all the prerequisites 
#
$ tar xzvf /tmp/eclipse-SDK-3.2-linux-gtk.tar.gz
$ unzip /tmp/xsd-runtime-2.2.0.zip
$ unzip /tmp/tptp.platform.runtime-TPTP-4.2.0.zip
$ unzip /tmp/tptp.test.runtime-TPTP-4.2.0.zip
$ unzip /tmp/emf-sdo-runtime-2.2.0.zip

#
#
# Unpack, configure, and launch the Eclipse TPTP Agent Controller
#
$ mkdir tptpd
$ cd tptpd
$ unzip /tmp/tptpdc.linux_ia32-TPTP-4.2.0.zip
$ chmod +x ./{bin,lib}/*

#
#
# Set essential environment variables
#
$ export JAVA_HOME=~/java-tptp/jre1.5.0_07
$ export CLASSPATH=$JAVA_HOME
$ export PATH=$JAVA_HOME/bin:$JAVA_HOME/tptpd/bin:$PATH
$ export MOZILLA_FIVE_HOME=/usr/lib/mozilla
$ export LD_LIBRARY_PATH=$MOZILLA_FIVE_HOME:\
$JAVA_HOME/tptpd/lib:$LD_LIBRARY_PATH

#
#
# Start the Eclipse TPTP Agent Controller
#
$ cd ./bin
$ ./SetConfig.sh
$ ./RAStart.sh

#
#
# Finally, launch Eclipse
#
$ ~/java-tptp/eclipse/eclipse

Note: Don't type the comment lines in Listing 1. They're merely informational.

When prompted, answer "Yes" to the Java licensing terms, then click All each time you're prompted to overwrite the Eclipse manifest files.

Launch Eclipse

When you launch Eclipse and the Eclipse Workspace window appears, name your workspace testing. (Don't worry about this name; you can maintain multiple workspaces in Eclipse -- say, one for each project -- and you can change from one workspace to another from within Eclipse.) In a few seconds, you should see the main Eclipse window, as shown below. Click the arrow icon to go to the Eclipse Workbench.

Figure 1. The main Eclipse window
Main Eclipse window

Of course, if you've already installed Eclipse, you can use the built-in Eclipse Update Manager to install the Eclipse TPTP. To do so, complete these steps from within Eclipse

  1. Click Software Updates > Find and Install.
  2. Select the Search for new features to install option, then click Next.
  3. Click New Remote Site, then type TPTP Update Site and http://eclipse.org/tptp/updates/site.xml for the name and URL, respectively.
  4. Click Finish, choose the features to install, then click Next.
  5. Accept the license, click Next, then click Finish.
  6. At the confirmation prompt, click Install All.
  7. When the installation is complete, restart Eclipse.

You should now be ready to work with the Eclipse TPTP.

Eclipse TPTP testing tools

Given the variety of applications and libraries developed using Java technology, the Eclipse TPTP testing tools include four kinds of testing software:

URL performance tools
The Hypertext Transfer Protocol (HTTP), or URL, tools automate performance testing of Web applications. Using the URL tools, you can capture a browser session as a repeatable JUnit test. The URL tools automatically add instrumentation to each test to measure the response time of your Web software. Results are captured in graphs you can share among your project team.
JUnit tools
Using the Eclipse TPTP JUnit tools, you can create and manage JUnit tests within Eclipse. Further, the JUnit test provides in-depth interactive reports to let you plumb quickly and directly into failed tests.
Manual testing tools
The manual testing tools provide a system to document those tests you must perform manually. You can capture instructions, expected results, and checklists to make manual testing less irregular and more reliable.
Automated GUI Recorder
The Eclipse TPTP Automated Graphical User Interface (GUI) Recorder (AGR) can capture and play back interactions with an application's GUI. You can capture interactions using raw screen coordinates (not recommended because such recordings are inexorably tied to screen resolution) or, if the application is written with the Standard Widget Toolkit (SWT), you can record interactions using the unique properties of individual widgets. Custom subroutines, called verification hooks, assert the veracity of the application during test playback. You can also build collections of data called datapools to vary the input used in recorded tests.

Note: The AGR is quite expansive and worthy of its own tutorial. To learn more about the AGR, refer to "An Introduction to TPTP's Automated GUI Recorder" online in the Eclipse TPTP Test Tools documentation.

This tutorial demonstrates the URL tools, the JUnit tools, and the manual testing tools.


Measuring the performance of a Web application

Prior to the introduction of the World Wide Web, most applications ran "locally" on an individual computer. Benchmarking, or measuring the performance, of such applications was fairly simple. If you held the processor speed and amount of memory constant, you could readily quantify the impact of code changes and various compilers. Better yet, for UNIX-based systems, the command line, shell scripts, and other utilities provided a rich framework for running tests repeatedly and recording results.

Web applications are radically different, mostly because many components are involved in the effectuation of a Web -- more specifically, an HTTP -- operation. The components include the local Web browser, the network, the Web server, the remote host, the Web application, and perhaps more "moving parts," such as a database. Performance testing a Web application requires you to minimize the influence of so many actors -- something you can easily achieve if your tests run on a LAN. However, there's still the issue of how to repeat the tests.

The Eclipse TPTP URL tools are designed to automate and benchmark interaction with a Web application. The URL tools are able to record -- through proxy -- each "click" and measure the delay between request and reply.

Using the URL tools

Listing 2 shows a simple PHP application that burns cycles -- and ties up your Web server. You can choose to sleep a quarter-second, a half-second, or one second. Simply place this code in your DocRoot.

Listing 2. A simple PHP application
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN"
        "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
    <html xmlns="http://www.w3.org/1999/xhtml">
    <head>
      <title>Consume Cycles Example</title>
    </head>

    <body>
<?php
      $time = 0;
      
      switch($_GET['mode'])
      {
        case 'short':
          $time=250;
          break;
          
        case 'medium':
          $time=500;
          break;
          
        case 'long':
          $time = 1000;
          break;
      }
    
      $time *= 1000;

      if ($time > 0) {
        usleep($time);
        print "<p>I snoozed for $time microseconds</p>";
      }
?>

    <h3>Choose how long you want the application to sleep:</h3>
    <ul>
      <li><a href="index.php?mode=short">A short time</a></li>
      <li><a href="index.php?mode=medium">A little while</a></li>
      <li><a href="index.php?mode=long">A long time</a></li>
    </ul>
    </body>
</html>

Deploy the PHP code as $DOCROOT/burn/index.php on your local machine and type http://localhost/burn/index.php into your browser's address bar (see Figure 2). Click a link, then click Back to return to the index page.

Figure 2. The PHP application to benchmark
PHP application to benchmark

Build a URL test

Begin by creating a test to measure the PHP application shown in Listing 2. Launch Eclipse and create a new general project named PHP.

Record a browser session

First, set your browser and proxy port preferences within Eclipse. Eclipse uses these settings to launch a proxy, and automatically configure and launch your browser to route all traffic through the proxy. As you'll see, the proxy facilitates HTTP recording. To set your preferences:

  1. Click Window > Preferences, expand the Test arrow, then click HTTP Recording.
  2. Alter the settings in the rightmost pane to resemble those shown in Figure 3.
    Figure 3. Configure the browser and proxy
    Configure the browser and proxy
    Using the settings in Figure 3, the HTTP Recorder launches Mozilla and directs it to use port 1080 to connect to the (SOCKS 4) proxy.
  3. Click OK.
  4. Click New > Other, expand the Test arrow, then expand the Recording arrow and choose HTTP Recording. Your window should resemble Figure 4.
    Figure 4. Create a new HTTP recording
    Create a new HTTP recording
  5. Click Next.
  6. Name your recording file Simple, then click Finish.
    In a moment, the Recorder Control pane appears in Eclipse (as shown in Figure 5). An empty Mozilla window also appears. (If you open Mozilla's preferences, you can discover the proxy settings being used.)
    Figure 5. The Recorder Control pane controls HTTP recording
    The Recorder pane controls HTTP recording
  7. Change focus to the Mozilla window, then type the URL of the sample PHP application -- for example, http://localhost/burn/index.php.
    The application should immediately appear in the browser.
  8. Click the A short time link, then click Back.
  9. Click A little while, then click Back.
  10. Click A long time.
  11. Switch back to Eclipse, and click Stop recording (the red square at the far right of Figure 5).

Eclipse halts recording and generates a new test case called Simple.testsuite. Eclipse also saves the recording. Your Eclipse window should now look something like Figure 6.

Figure 6. An HTTP performance test suite
An HTTP performance test suite

Review the captured HTTP test

Click the HTTP Requests tab (at the bottom of the top-most pane), then choose one of the four HTTP requests that were captured. If you expand the Eclipse window, you'll see the internals of the request in great detail, as shown in Figure 7. You can use the HTTP Requests view to alter the request -- say, to add extra headers or change GET parameters. The requests on this page will be used in the next step -- defining the behavior of the test suite. (Think of the HTTP requests as actions that you first capture, then customize, if necessary. The test suite uses the actions to drive the Web application.)

Figure 7. Editing a series of HTTP requests
Editing a series of HTTP requests

Now click the Behavior tab and expand all the arrows. Loop controls are marked with a recycle icon; HTTP requests (those you previously edited) are the actions. You may notice that you can add, remove, reorder, and insert additional steps. (Add creates a new child; Insert creates a new sibling.) Click a loop, then click a request. The respective property panes are shown in Figure 8.

Figure 8. Editing a loop and a request
Editing a loop and a request

The settings in both panes are fairly self-explanatory. You can control the iterations of a loop and alter the details of a request. You can also include other tests from other suites. To try this, click Loop 1 at the top of the leftmost pane, and change Number of iterations to 10. Use Ctrl+S to save your work.

Compile your test suite

To run your test suite, you must compile it -- a process that converts your test case to Java code (actually, a JUnit test.) To run the suite, create a Java project in your current workspace, and name it Testing. To generate Java code from the test suite:

  1. Switch to the Test perspective by clicking Windows > Open perspective and navigating through the menus until you can choose Test.
    Your Eclipse window should change to reveal the Test Navigator, as shown in Figure 9.
    Figure 9. The Test Navigator
    The Test Navigator
  2. Right-click Simple, then click Generate. A window similar to Figure 10 appears.
    Figure 10. Generate a JUnit test from your HTTP recording
    Generate a JUnit test
  3. Click the top Browse button, then choose the Testing Java project.
  4. Click Finish. Two folders should appear in your Testing project.
  5. And before you move on, switch back to the Java perspective.

Run the URL test

You're now ready to run your Web application performance test. The Eclipse TPTP tools converted your URL test to Java code. Now you must execute that code:

  1. Expand the arrow next to the Testing project, then expand the arrow next to the folder named test, which contains one Java class called Simple.java.
    If you expand Simple.java, you'll see that it closely resembles a traditional JUnit test. Indeed, that's how to run it.
  2. Right-click Simple.java, then click Run As > JUnit test.
    The result looks like Figure 11. The bar is green, so the tests -- 40 in total -- ran clean.
    Figure 11. The results of running the HTTP performance test suite
    Results

Running your test as a JUnit test proves that the test works, but to get valuable metrics, you must run the test as an Eclipse TPTP test. To do that, switch back to the Test perspective, right-click the test folder, then click Run As > Run. In the next window, expand Test, then choose Simple. At this point, the right-most pane should mirror Figure 12. Click Run. A progress meter like the one shown in Figure 13 remains on screen until the test is complete.

Figure 12. Launch the HTTP performance test suite
Launch the HTTP performance test suite
Figure 13. The progress meter
Progress meter

Analyze the results

So far, you've captured a Web session and created a testing script, and you've generated and executed the test case. The next step is to analyze the results.

Your Test perspective should now have a new resource: a test log named Simple. Double-click the log to open it. Your Eclipse workbench should look like Figure 14.

Figure 14. A log of the test suite run
A log of the test suite run

Click the Events tab, and expand the loops and pages until your view approximates Figure 15.

Figure 15. The events of the test suite run
The events of the test suite run

Each item contains a good amount of information, but you may be most interested in the message elements because each one reports response time (in milliseconds). Open several invocation elements to see how long each request took. Here, in the contrived example, you're likely to see quick responses (say, to open the index page) and lengthy responses when the test chooses to sleep for a long time.

Generate reports

The Eclipse TPTP HTTP Performance tool has one more trick up its sleeve: reports. Take some time to run your new test suite three to five times. Each run produces its own log. To view a summary report:

  1. Right-click Simple in the Test Navigator, then click Options.
  2. Choose Report.
  3. In the next window, shown below, click HTTP Page Response Time.
    Figure 16. A list of reports to choose from
    A list of reports to choose from

If you're struggling trying to benchmark the responsiveness of your Web application, the HTTP Recorder should come in handy. You can likely find other uses for it as well -- a link validator (that is, to determine whether if the link works), a site "heartbeat" (that is, a monitor), and others.


Creating and managing JUnit tests

The previous section presented tools to test your Web application after it's been deployed. If you notice performance problems, you can use other Eclipse TPTP tools to profile your code and reveal bottlenecks. If you want to monitor the ongoing operation of your application, Eclipse TPTP offers log analysis tools, as well.

However, most developers -- especially Java developers -- typically want to squash bugs long before the code lands on a production server. Moreover, many Java development projects focus on developing infrastructure to prevent bugs throughout the lifetime of a body of code. Source control tracks software changes. Nightly builds surface problems -- including performance degradations and inaccuracies -- sooner, rather than later. Additionally, unit tests validate that each "unit" -- down to an individual class -- runs correctly. Indeed, writing tests first, coding later is widely used to capture expectations first. Then, if the code passes its tests, the unit is finished.

The Eclipse core (without Eclipse TPTP installed) provides tools to generate (the skeleton of) JUnit tests from individual classes. Eclipse TPTP goes further by providing detailed logs and a special log viewer, which you can use to filter out certain results that aren't of interest (say, ignore all tests that passed). It is also far easier to jump quickly to a particular event using the Eclipse TPTP tools. You can even navigate immediately to the questionable code. You'll surely notice other features and benefits as you proceed through this section.

Get started with the Eclipse TPTP JUnit tools

Let's take a tour of the Eclipse TPTP JUnit tools by way of example. The code in Listing 3 is a simple Java class named Account that models a bank account. It has a "getter" named balanced, and deposit(amount) and withdrawal(amount) method affect the balance. Additionally, each time you call the compoundAnnually(rate) method, the principal in Account is increased by the interest earned at a certain rate for one year.

Listing 3. A Java class for a simple bank account
package demo;

import java.lang.Math; 

public class Account {

    private double balance = 0;
    
    public Account() {
        this.balance = 0;
    }
    
    public Account(double deposit) {
        this.balance = 0;
        
        if (deposit > 0) {
            this.balance = deposit;
        }
    }
    
    public double deposit (double deposit) {
        if (deposit < 0) {
            return this.withdrawal(deposit);
        }
        
        this.balance += deposit;
        
        return(this.balance);
    }
    
    public double withdrawal (double withdrawal) {
        if (withdrawal < 0) {
            return(this.deposit(-withdrawal));
        }
        
        if (this.balance >= withdrawal) {
            this.balance -= withdrawal;
        }
        
        return(this.balance);
    }
    
    public double balance() {
        return this.balance;
    }
    
    public double compoundAnnually(double rate) {
        int n = 356 * 24;
        
        if (rate > 0) {
            this.balance *= Math.pow((1 + (rate/n)), n);
        }
        
        return(this.balance);
    }
}
  1. Using Eclipse, create a new Java project named Demo by clicking File > New > Other, then choosing Java project.
  2. Within the Java project folder, create a new class named Account and place it in a package named demo.
  3. Cut and paste the source code in Listing 3 as needed to replace the members and methods of the Account class.
  4. Create an Eclipse TPTP JUnit test for the Account class. An Eclipse TPTP JUnit test is a special and distinct form of JUnit test, and you'll find it under the Test folder whenever you choose File > New > Other (see Figure 17).
    Figure 17. An Eclipse TPTP JUnit test
    Eclipse TPTP JUnit test
  5. Right-click Account.java in the Java perspective, then click New > Other.
  6. In the resulting window, choose TPTP JUnit Test. If a window identical to Figure 18 appears, click Yes to add three vital JAR files to your current Java CLASSPATH variable.
    Figure 18. Add mandatory classes to you configuration
    Add mandatory classes
  7. Name your new test AccountTest and choose to edit the test within the test editor (see Figure 19). When you're finished, click Finish.
    Figure 19. Name your Eclipse TPTP JUnit test case
    Name your Eclipse TPTP JUnit test case
    When the new Eclipse TPTP JUnit Test case has been created, your Java perspective should resemble Figure 20.
    Figure 20. The framework for your new JUnit test suite
    The framework for your new JUnit test suite
  8. Click the Test Methods tab. On this tab, you add individual tests. Click Add three times to create three tests: test 1, test 2, and test 3. Rename test 1 testCreateEmpty, rename test 2 createBalance, and rename test 3 to testVerifyDeposit.
  9. Save your work by using Ctrl+S. If you click the AccountTest.java Eclipse TPTP JUnit Test case and expand its arrow, you may be pleased to see that three new methods named after your tests were created in the class. The Eclipse TPTP JUnit test tools try diligently to keep the Java class and the test methods in the test editor in sync. For example, if you rename the test methods in the test editor, the names in the Java class automatically change. (But be careful: Many operations can get the two out of sync.)
  10. Open the AccountTest.java class and edit the new methods to recreate the code in Listing 4. When you're finished, use Ctrl+S to save your work.
Listing 4. Three test cases for your new Account class
public void testCreateEmpty()
throws Exception
{
  Account zeroAccount= new Account();
  assertEquals("balance not zero", zeroAccount.balance(), (double) 0, 0.01);
}


public void testCreateBalance()
throws Exception
{
  double startingBalance = 510.96;
  Account newAccount= new Account(startingBalance);
  
 assertEquals("starting balance incorrect", \
 newAccount.balance(), startingBalance, 0.01);
}


public void testVerifyDeposit()
throws Exception
{
  double startingBalance = 987.91;
  double deposit = 1560.98;
  
  Account newAccount = new Account(startingBalance);
  
  assertEquals("deposit not credited", 
      newAccount.deposit(deposit), deposit+startingBalance, .01);
}

Now that you have a Java class and a JUnit test with several cases, you can move on to detail how you want the test to run. Click AccountTest.testsuite, then click the Behavior tab; this is where you detail how to execute the test suite. Much like you did with the URL tools, your next steps are to create loops and method calls to run the individual test cases.

To configure how you want to execute the loops and method calls:

  1. On the Behavior tab, click Add.
  2. In the next window, shown in Figure 21, choose the three Java methods in the top-most pane, then click OK.
    You should see your three test methods in the left-most pane.
    Figure 21. Choose the cases you want to run in the current JUnit test
    Choose the cases you want to run
  3. Right-click AccountTest.testsuite, then click Run As > Test.
    As with the URL tools, the progress meter appears to tracks the suite as it's running. When complete, it leaves a log of the results in a new Eclipse resource named AccountTest_#.execution, where # is a unique identifier.
  4. Double-click AccountTest_#.execution, then click the Events tab. You should see a window like Figure 22.
    Figure 22. The results of the Account testing class
    The results of the Account testing class
    Click around and look at the events. You can click back to Overview to see a summary: the verdict, the duration, and more.

So, what happens when a test case fails? Introduce an error into testVerifyDeposit() to make the assertEquals() method in that method fail. Then save your edits and rerun the AccountTest.testsuite. Running the suite again generates a new AccountTest_#.execution file. Double-click that file.

Navigate to Events and expand the arrows until you find the failure. If you click the failure, you can see the details of what went wrong, including a stack trace, as shown below.

Figure 23. What happens when a test suite fails?
What happens when a test suite fails?

As you can see, the Eclipse TPTP JUnit tools provide a supportive environment in which to craft, run, and analyze JUnit tests. In addition, if you already have JUnit tests, you can import them into the Eclipse TPTP and use the improved tools. Just click File > Import in the Import window, click JUnit tests to TPTP, as shown below.

Figure 24. Import an existing JUnit test into the Eclipse TPTP
Import an existing JUnit test

With Eclipse TPTP's help, there's no reason not to adopt test-first coding.


Creating manual tests

The Eclipse TPTP HTTP Performance tools and integrated JUnit tools help automate your benchmarking and regression testing, respectively. But not all tests can be automated. Some require human intervention, and others may be so complex as to elude rote capture and playback. For those (not-so-rare) instances, the Eclipse TPTP provides tools to create directed, manual tests and test suites.

Like the HTTP Performance tools, you can create a test case and reuse it in several test suites. You can order and reorder steps, and document when certain steps should be repeated in a loop. Better yet, you can "run" a manual test suite, and the Eclipse TPTP will walk you through each step while monitoring and recording outcomes.

Build a manual test suite

Start by creating a simple manual test suite that can lead another member of your team through the verification of the PHP application introduced earlier. To do so:

  1. Click the PHP folder, then click File > New > Other.
  2. Click Manual TPTP Test.
  3. Give the manual test a descriptive name and click Finish.
    You should see a pane like the one shown below. Optionally, you can type a description in the Description field.
    Figure 25. Create a manual test suite
    Create a manual test suite
  4. Click Add, then fill in the fields of the pane to match Figure 26.
    The Description field is a rich-text field, and you can use HTML to make your instructions more readable.
    Figure 26. Create a manually operated test case
    Create a manually operated test case
  5. Click Add three more times to emulate Figure 27.
    Figure 27. Edit the behaviors of a manual test case
    Edit the behaviors of a manual test case
  6. Click the first test case, Home Page responds, then click the Behavior tab, which you use to create the test suite.
  7. Create a new step in the suite by clicking Add > Loop, as shown below.
    Figure 28. The menu of test steps
    The menu of test steps
  8. Click Add > Invocation to add another step.
  9. In the window that appears, click Home Page responds, as shown below, then click OK.
    Figure 29. Invoke a test
    Invoke a test
  10. To make Home Page responds part of the loop, drag its entry onto the loop.
  11. Click Insert three times to append the three "Click" test cases to the end of the loop.

When you've completed all the steps, your Behavior pane should resemble Figure 30.

Figure 30. The outline of the manual test suite
The outline of the manual test suite

If you want the user of your test suite to loop more than once, go back and click Loop 1 (you can change its name, too), then type a new number in the Number of iterations field. For safekeeping, use Ctrl+S to save your work.

Run your manual test suite

Your test is now ready to run.

  1. Switch to the Test Perspective. Right-click Manual Web test in the PHP folder, then click Run As > Run, as shown in Figure 31.
    Figure 31. Choose a manual test suite
    Choose a manual test suite
  2. In the resulting window, which Figure 32 shows, choose Manual Web test in the TPTP Manual Test folder, click local_deployment, then click Run.
    Figure 32. Launch a manual test suite
    Launch a manual test suite
    The familiar test suite progress bar appears. Don't panic! Behind the scenes, the Eclipse Agent Controller is launching a separate application (based on Mozilla's SWT UI toolkit) to lead you through the manual tests. The new application should appear when the progress bar is near 50 percent. The Manual Test View application looks like Figure 33. Do not click Cancel below the progress meter unless you want to abort the manual test view.
    Figure 33. Manual test suite application leads you through each test
    Manual test suite application leads you through each test

    The left pane shows your "score" and the individual test cases in this suite. The top-right pane describes what to do. The two bottom panes, Verdict and Reason, allow you to record whether the test case passed, along with why or why not.

  3. To annotate the results, type that information in the Text field. To attach files to record success or allow the failure to be reproduced, use the Attachments feature.
  4. When you've finished one test case, click Next to move on. Assuming that all your test cases pass, the final window looks like Figure 34.
    Figure 34. All the manual tests pass
    All the manual tests pass
  5. Click Stop (the red square) to return to Eclipse. The Test Navigator should show check marks next to each test case that passed. After you click Stop, you can see a summary of results in the main Eclipse window, as shown in Figure 35.
    Figure 35. Summary of a successful test
    Summary of a successful test

The next time you need to create a checklist, try using the Eclipse TPTP manual test tools. Of course, it's good for software testing, but you may find that you can capture any complex task in the tool.


Summary

Building software is both an art and a science. Without your creativity, code wouldn't run its fastest, difficult problems would remain complex, and computers might be little more than glorified typewriters.

Even if software isn't a physical machine composed of gears, actuators, springs, and pulleys, it nonetheless has many "moving parts" that must mesh seamlessly. And while you might like to smash obfuscated code with a hammer, don't. Instead, reach for the Eclipse TPTP to edit and build your code, test it, tune it, and keep a watchful eye over its performance and stability. The smart code builder keeps Eclipse close at hand.

Resources

Learn

Get products and technologies

  • Download the entire Eclipse TPTP runtime. Discover all the features of the Eclipse TPTP, as well as extensive documentation, tutorials, presentations, and screencasts that illuminate the capabilities of the Eclipse TPTP.
  • Visit Sun Microsystems to download the Java 2 Platform, Standard Edition (J2SE) V1.4 SDK and the JRE.
  • Download Eclipse, the open source and freely available, extensible IDE.
  • The Eclipse TPTP depends on the EMF SDK V2.1 and the XSD SDK V2.1.
  • If you're using Eclipse V3.1.1, download V1.1.1 of UML2 and unpack its archive file in the same directory that contains Eclipse.
  • Download the TPTP Agent Controller, a vital component if you want to profile your applications.
  • Innovate your next open source development project with IBM trial software, available for download or on DVD.

Discuss

  • Connect with Eclipse developers and other users in the Eclipse mailing lists and newsgroups. (You must register to read the newsgroups, but membership is free, and the registration process is easy.)
  • Get involved in the developerWorks community by participating in developerWorks blogs.

Comments

developerWorks: Sign in

Required fields are indicated with an asterisk (*).


Need an IBM ID?
Forgot your IBM ID?


Forgot your password?
Change your password

By clicking Submit, you agree to the developerWorks terms of use.

 


The first time you sign into developerWorks, a profile is created for you. Information in your profile (your name, country/region, and company name) is displayed to the public and will accompany any content you post, unless you opt to hide your company name. You may update your IBM account at any time.

All information submitted is secure.

Choose your display name



The first time you sign in to developerWorks, a profile is created for you, so you need to choose a display name. Your display name accompanies the content you post on developerWorks.

Please choose a display name between 3-31 characters. Your display name must be unique in the developerWorks community and should not be your email address for privacy reasons.

Required fields are indicated with an asterisk (*).

(Must be between 3 – 31 characters.)

By clicking Submit, you agree to the developerWorks terms of use.

 


All information submitted is secure.

Dig deeper into Open source on developerWorks


static.content.url=http://www.ibm.com/developerworks/js/artrating/
SITE_ID=1
Zone=Open source, Java technology
ArticleID=153952
ArticleTitle=Eclipse Test & Performance Tools Platform, Part 3: Test applications
publish-date=08222006