 | Level: Introductory Andrew Glover (aglover@stelligent.com), President, Stelligent Incorporated
26 Sep 2006 Writing logically repeatable tests is especially tricky when testing Web applications that incorporate a servlet container. In his continued quest to improve code quality, Andrew Glover introduces Cargo, an open source framework that automates container management in a generic fashion, so you can write logically repeatable system tests every time.
By their very nature, test frameworks like JUnit and TestNG
facilitate the creation of repeatable tests. Because these frameworks
leverage the reliability of simple Boolean logic (in the form of
assert methods), it's possible to run tests without human
intervention. In fact, automation is one of the primary benefits of test
frameworks -- I can write a fairly complex test asserting specific
behaviors, and if those behaviors ever change, the framework reports
an error that anyone can interpret.
Utilizing a mature test framework gives you the benefit of
framework repeatability, right out of the box. But logical
repeatability is up to you. For example, consider the challenge of
creating repeatable tests that verify Web applications. A few JUnit
extension frameworks (such as JWebUnit and HttpUnit) excel at
facilitating automated Web testing. But it's the developer's job to make
the plumbing of the test repeatable, and that's hard to do when
it comes to deploying Web application resources.
The actual construction of a JWebUnit test is fairly simple, as
demonstrated by Listing 1:
Listing 1. A simple JWebUnit test
package test.come.acme.widget.Web;
import net.sourceforge.jwebunit.WebTester;
import junit.framework.TestCase;
public class WidgetCreationTest extends TestCase {
private WebTester tester;
protected void setUp() throws Exception {
this.tester = new WebTester();
this.tester.getTestContext().
setBaseUrl("http://localhost:8080/widget/");
}
public void testWidgetCreation() {
this.tester.beginAt("/CreateWidget.html");
this.tester.setFormElement("widget-id", "893-44");
this.tester.setFormElement("part-num", "rt45-3");
this.tester.submit();
this.tester.assertTextPresent("893-44");
this.tester.assertTextPresent("successfully created.");
}
}
|
This test communicates with a Web application and
attempts to create a widget based on the interaction. The test then
verifies that the widget was successfully created. Readers of previous
articles in this series may notice a subtle repeatability issue with the
test, however. Do you see it? What do you think will happen if this test
case is run twice consecutively?
Judging by the identification aspect of the widget instance (that is,
widget-id), it's safe to assume that database
constraints in the application will likely prohibit the creation of an
additional widget that already exists. Without a process for removing
the test case's intended widget before another test is run, there's a
high probability that the test case will fail if run twice in a row.
Fortunately, as I've discussed in a previous article, there is a mechanism that facilitates database-dependent test case repeatability -- namely DbUnit.
DbUnit goes to work
Enhancing the test case from Listing 1 to employ
DbUnit is fairly simple. All DbUnit needs is some data to insert into a
database and a corresponding database connection, as shown in Listing 2:
Listing 2. Database-dependent testing with DbUnit
package test.come.acme.widget.Web;
import java.io.File;
import java.io.IOException;
import java.sql.Connection;
import java.sql.DriverManager;
import java.sql.SQLException;
import org.dbunit.database.DatabaseConnection;
import org.dbunit.database.IDatabaseConnection;
import org.dbunit.dataset.DataSetException;
import org.dbunit.dataset.IDataSet;
import org.dbunit.dataset.xml.FlatXmlDataSet;
import org.dbunit.operation.DatabaseOperation;
import net.sourceforge.jwebunit.WebTester;
import junit.framework.TestCase;
public class RepeatableWidgetCreationTest extends TestCase {
private WebTester tester;
protected void setUp() throws Exception {
this.handleSetUpOperation();
this.tester = new WebTester();
this.tester.getTestContext().
setBaseUrl("http://localhost:8080/widget/");
}
public void testWidgetCreation() {
this.tester.beginAt("/CreateWord.html");
this.tester.setFormElement("widget-id", "893-44");
this.tester.setFormElement("part-num", "rt45-3");
this.tester.submit();
this.tester.assertTextPresent("893-44");
this.tester.assertTextPresent("successfully created.");
}
private void handleSetUpOperation() throws Exception{
final IDatabaseConnection conn = this.getConnection();
final IDataSet data = this.getDataSet();
try{
DatabaseOperation.CLEAN_INSERT.execute(conn, data);
}finally{
conn.close();
}
}
private IDataSet getDataSet() throws IOException, DataSetException {
return new FlatXmlDataSet(new File("test/conf/seed.xml"));
}
private IDatabaseConnection getConnection() throws
ClassNotFoundException, SQLException {
Class.forName("org.hsqldb.jdbcDriver");
final Connection jdbcConnection =
DriverManager.getConnection("jdbc:hsqldb:hsql://127.0.0.1",
"sa", "");
return new DatabaseConnection(jdbcConnection);
}
}
|
With the addition of DbUnit, the test case is truly repeatable. In the handleSetUpOperation() method, DbUnit executes a CLEAN_INSERT on the data every time a test case is run. This operation essentially cleanses a database of data and inserts a fresh set, thereby removing any previously created widgets.
|
DbUnit is what again?
DbUnit is a JUnit extension that facilitates putting a database into a known state between test runs. Developers use XML seed files to insert specific data into a database, which a test case can then rely on. Thus, DbUnit facilitates the repeatability of test cases that rely on one or more databases.
|
|
But that doesn't mean I'm done with the topic of test case repeatability. In fact, I'm just getting started.
Repeating system tests
The test cases defined in Listing 1 and Listing 2 are what I like to call system tests.
Because system tests exercise a fully installed application, such as a Web
application, they usually incorporate a servlet container and an associated
database. These tests tend to verify that external interfaces (such as Web
pages in the case of a Web application) operate end-to-end as designed.
|
Prioritizing flexibility
As a general rule of thumb, avoid test case inheritance whenever possible. Many JUnit extension frameworks offer specialized test cases that can be inherited from to facilitate testing a particular architecture. Test cases
that inherit classes from a framework suffer from inflexibility, however, because of the Java™ platform's single-inherence paradigm. More often than not, these same JUnit extension frameworks offer a delegation API, which makes it easy to combine various frameworks without taking on a rigid inheritance structure. |
|
Because they're designed to test fully functional applications,
system tests tend toward lengthy run times, not the least of which
involves the total time to set up a test. For example, the logical test shown in Listing 1 and Listing 2 requires the following steps before it is even run:
- Create a war file containing all associated Web artifacts, such as JSP files, servlets, third-party jar files, images, etc.
- Deploy the war file to the targeted Web container. (If the container isn't started, start it.)
- Start any associated databases. (If a database needs its schema updated, update that before starting.)
Now, that's a lot of leg work for one measly test! If this process
proves time-consuming, how often do you think the test will be run?
Faced with the requirement to make system tests logically repeatable (say,
in a continuous integration environment) this list of steps is certifiably daunting.
Introducing Cargo
The good news is that you can automate all the major setup steps in
the previous list. In fact, if you happen to be doing Java Web development,
you're probably already automating Step 1 with Ant, Maven, or some other
build tool.
Step 2 presents an interesting hurdle, however. Automating a Web
container can be a bit tricky. For instance, some containers have custom
Ant tasks that facilitate their auto-deployment and running, but the
tasks are container specific. What's more, these tasks make some
assumptions, such as about where the container is installed and, more
importantly, that the container is installed.
Cargo is an innovative open source project that aims to automate
container management in a generic fashion, such that the same API used
to deploy a WAR file to JBoss can also start and stop Jetty. Cargo also
can download and install a container automatically. You can utilize Cargo's
API in a few different fashions, ranging from Java
code to Ant tasks to Maven goals.
Using a tool like Cargo takes care of one of the major challenges to
writing logically repeatable test cases. Furthermore, you can construct
a build that harnesses the power of Cargo to automatically do the
following:
- Download a desired container.
- Install the container.
- Start the container.
- Deploy a selected WAR or EAR file to the container.
Pretty easy, huh? At a later point, you can also have Cargo stop a selected container.
'Speaking' Cargo
It's a good idea to get the basics of Cargo down before diving in
head first. Namely, you want to be sure you understand the concept of
containers and container management as Cargo relates to them.
For starters, there is obviously the notion of a container. Containers are servers that host applications. Those applications can be Web-based, EJB-based, or both, which is why we have Web containers and EJB containers. Tomcat is a Web container and JBoss could be considered an EJB container. Consequently, Cargo supports quite a few containers, but for my example, I'll use Tomcat version 5.0.28. (Cargo would call this a "tomcat5x" container.)
Next, if a container isn't already installed, you can use Cargo to
download and install a particular one. To do this, you need to supply
Cargo with a download URL. Once you've installed a container, Cargo also
allows you to use configuration options to configure it. These
options take the form of name-value pairs.
Lastly, there is the notion of a deployable resource, which in
the case of my example is a WAR file. Note that it could just as easily be
an EAR file.
Keep these concepts in mind and let's see what we
can do with Cargo.
Cargo in action
The example for this article involves using Cargo in Ant, which entails wrapping the
system test I defined earlier with Cargo Ant tasks. These tasks then install, start, deploy, and stop the container. We'll do the setup first, run the test, and then stop the container.
The first step required to use Cargo in an Ant build is to provide a
task definition for all the Cargo tasks. This step allows you to
reference Cargo's tasks later in your build file. There are a number of
ways to handle this step. Listing 3 simply loads the tasks from a
properties file found in Cargo's JAR files:
Listing 3. Loading all Cargo tasks in Ant
<taskdef resource="cargo.tasks">
<classpath>
<pathelement location="${libdir}/${cargo-jar}"/>
<pathelement location="${libdir}/${cargo-ant-jar}"/>
</classpath>
</taskdef>
|
Once you've defined Cargo's tasks, the real action begins. Listing
4 defines Cargo tasks to download, install, and start a Tomcat
container. The zipurlinstaller task downloads
and installs Tomcat from http://www.apache.org/dist/tomcat/tomcat-5/v5.0.28/bin/
jakarta-tomcat-5.0.28.zip to a local temporary directory.
Listing 4. Downloading and starting Tomcat 5.0.28
<cargo containerId="tomcat5x" action="start"
wait="false" id="${tomcat-refid}">
<zipurlinstaller installurl="${tomcat-installer-url}"/>
<configuration type="standalone" home="${tomcatdir}">
<property name="cargo.remote.username" value="admin"/>
<property name="cargo.remote.password" value=""/>
<deployable type="war" file="${wardir}/${warfile}"/>
</configuration>
</cargo>
|
Note that to start and stop a container from within different
tasks, as you'd like to do, you must associate the container with a unique
id, which is the id="${tomcat-refid}" aspect
of the cargo task.
Also note that configuration aspects of Tomcat are handled inside the
cargo task. In Tomcat's case, you must set the
username and password properties. Finally, you define a pointer to a WAR
file using the deployable
element.
Properties of Cargo
All the properties used in your Cargo tasks are shown in
Listing 5. For example, tomcatdir defines one
of two locations where Tomcat will be installed. This particular
location is a mirrored structure, which will be referenced by the actual
downloaded and installed Tomcat instance (which is found in a temporary
directory). The property tomcat-refid helps
associate a unique instance of a container as well.
Listing 5. Cargo properties
<property name="tomcat-installer-url"
value="http://www.apache.org/dist/tomcat/tomcat-5/v5.0.28/bin/
jakarta-tomcat-5.0.28.zip"/>
<property name="tomcatdir" value="target/tomcat"/>
<property name="tomcat.username" value="admin"/>
<property name="tomcat.passwrd" value=""/>
<property name="wardir" value="target/war"/>
<property name="warfile" value="words.war"/>
<property name="tomcat-refid" value="tmptmct01"/>
|
To stop a container, you can define a task that references the tomcat-refid property, as shown in Listing 6:
Listing 6. Stopping a container a la Cargo
<cargo containerId="tomcat5x" action="stop"
refid="${tomcat-refid}"/>
|
Wrapping with Cargo
Listing 7 combines the code in Listing 4 and Listing 6,
wrapping a test target with two Cargo tasks: one for starting and one for stopping Tomcat. The task antcall calls the target named _run-system-tests, which is defined in Listing 8.
Listing 7. Wrapping a test target with Cargo
<target name="system-test" if="Junit.present"
depends="init,junit-present,compile-tests,war">
<cargo containerId="tomcat5x" action="start"
wait="false" id="${tomcat-refid}">
<zipurlinstaller installurl="${tomcat-installer-url}"/>
<configuration type="standalone" home="${tomcatdir}">
<property name="cargo.remote.username" value="admin"/>
<property name="cargo.remote.password" value=""/>
<deployable type="war" file="${wardir}/${warfile}"/>
</configuration>
</cargo>
<antcall target="_run-system-tests"/>
<cargo containerId="tomcat5x" action="stop"
refid="${tomcat-refid}"/>
</target>
|
Listing 8 defines your test target, called _run-system-tests. Note that this task only runs system tests, which are placed in the test/system directory. For example, the test case defined in Listing 2 lives in this directory.
Listing 8. Running JUnit via Ant
<target name="_run-system-tests">
<mkdir dir="${testreportdir}"/>
<junit dir="./" failureproperty="test.failure"
printSummary="yes" fork="true"
haltonerror="true">
<sysproperty key="basedir" value="."/>
<formatter type="xml"/>
<formatter usefile="false" type="plain"/>
<classpath>
<path refid="build.classpath"/>
<pathelement path="${testclassesdir}"/>
<pathelement path="${classesdir}"/>
</classpath>
<batchtest todir="${testreportdir}">
<fileset dir="test/system">
<include name="**/**Test.java"/>
</fileset>
</batchtest>
</junit>
</target>
|
In Listing 7, you fully configured your Ant build
file to wrap your system tests with Cargo's deployment voodoo. The code in
Listing 7 ensures that all the system tests in the test/system directory from Listing 8 are logically
repeatable. You can run these system tests on any machine at any time,
which is perfect for a Continuous Integration environment. The tests
make no assumptions about the container -- not about its location or even
if it's running! (Of course, these tests still make one assumption I
haven't addressed, namely that the underlying database is properly
configured and running. But that's a subject for another day.)
Repeatable results
In Listing 9, you can see the results of your efforts. When you issue the system-test command to an Ant build, your system tests are executed. Cargo handles all the details of managing the container of your choice, without making repeatability-crushing assumptions about the test environment.
Listing 9. The enhanced build
war:
[war] Building war: C:\dev\projects\acme\target\widget.war
system-test:
_run-system-tests:
[mkdir] Created dir: C:\dev\projects\acme\target\test-reports
[junit] Running test.come.acme.widget.Web.RepeatableWordCreationTest
[junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 4.53 sec
[junit] Testcase: testWordCreation took 4.436 sec
BUILD SUCCESSFUL
Total time: 1 minute 2 seconds
|
Keep in mind that Cargo also works in Maven builds. Furthermore, the
Cargo Java API facilitates programmatic management of containers in
anything from normal applications to test cases. And Cargo isn't only
for JUnit (even though the code examples are written in JUnit). TestNG
users will be happy to know that Cargo works just as well for their test
suites. In fact, it doesn't matter what your tests are written in: just
wrap them with Cargo and your container management issues will be
automated out of the box!
In conclusion
It's up to you to ensure that your tests are logically repeatable,
but you've seen this month that Cargo can really help. Cargo manages the
container environment so you don't have to. Incorporate Cargo into your
testing routine -- it will definitely lighten the load of crafting repeatable tests that verify Web applications.
Resources Learn
- "Effective Unit Testing with DbUnit" (Andrew Glover, OnJava, January 2004): Introduces database-dependent testing with DbUnit.
- Hip system tests with Cargo (Andrew Glover, thediscoblog.com, April 2006): Learn more about Cargo's Java API.
- "Make Ant easy with Eclipse" (Prashant Deva, developerWorks, April 2006): Discover the Ant integration features in the Eclipse IDE.
- "Practically Groovy: Ant scripting with Groovy" (Andrew Glover, developerWorks, December 2004): Combine Groovy with Ant and Maven for more expressive and controllable builds.
- In pursuit of code quality: JUnit 4 vs. TestNG (Andrew Glover, developerWorks, April 2006): Has JUnit 4 rendered TestNG obsolete? Find out why not.
- In pursuit of code quality series (Andrew Glover, developerWorks): Learn more about code metrics, test frameworks,
and writing quality-focused code.
- developerWorks: Hundreds of articles about every aspect of Java programming.
Get products and technologies
Discuss
About the author | 
| | Andrew Glover is president of Stelligent Incorporated, which helps companies address software quality with effective developer testing strategies and continuous integration techniques that enable teams to monitor code quality early and often. Check out Andy's blog for a list of his publications. |
Rate this page
| |
