 | Level: Intermediate Brian Goetz (brian@quiotix.com), Principal Consultant, Quiotix
22 Aug 2006 The first two installments in this series showed how static analysis tools like FindBugs can provide greater leverage in managing software quality by focusing on entire categories of bugs rather than on specific bug instances. In this final installment on testing, Brian Goetz examines another technique for smoking out bugs that violate design rules: aspects.
Aspect-oriented programming (AOP) is a new and promising technology,
but adopting new technologies can be risky (as, of course, can
not adopting new technologies). As with all new technologies, it
is often best to follow a risk-managed path toward their
adoption. You can benefit from reduced risk with AOP if you use it for policy enforcement and testing. Because the
aspects never go into production, there is no risk that the technology
will destabilize the production code or the development process, but
it can still assist you in developing better quality software. Using
aspects for testing is also a great way to learn how aspects work and
gain some experience with this exciting new technology.
Combining testing methodologies
As I discussed in Part
I, the goal of QA is not to find all possible bugs -- because
that's impossible -- but instead to improve our confidence that the
code works as expected. The challenge of managing an effective QA
organization is maximizing the return -- confidence -- on the
resources expended. Because all testing methodologies eventually
exhibit diminishing returns (less incremental confidence for the same
incremental effort) and different methodologies are suited to finding
different types of errors, spreading your QA effort over testing, code
review, and static analysis is likely to yield a better return than
spending your entire QA budget on just one of these approaches.
Static analysis tools like FindBugs are inexact, but inexact analysis
can still be quite useful and effective for improving software
quality. They may emit false positives, such as triggering on harmless
constructs, or false negatives, such as failing to identify all bugs
matching a particular bug pattern. But they still find real bugs and,
so long as the false positive rate isn't so high that users are
overwhelmed with warnings, they still offer a valuable return on
testing effort.
From the perspective of testing, using AOP to verify design rules has
a lot in common with static analysis. Rather than crafting a test case
for a specific method or class, both static analysis and
aspect-oriented testing encourage you to identify entire categories of
rule violations and create artifacts that allow you to find violations
in any body of code. Another similarity is that they need not be
perfect to be useful; even if bug detectors or testing aspects do not
identify all possible bugs or sometimes emit false positives, they
can still be useful as tools for verifying that the code works as
expected. Some bug patterns are more easily identified with static
analysis while others are more easily identified with aspects -- making aspects a useful
methodology to add to the QA mix.
A simple testing aspect
Static analysis tools like FindBugs audit the code without executing
it; aspect-oriented tools offer both static and dynamic
instrumentation of classes. Static aspects can generate warnings or
errors at compile time; dynamic aspects can insert error-checking code
into your classes.
In Part
1, I offered a simple FindBugs detector to find calls to
System.gc() that might be lurking in libraries. Many bug
patterns that can be detected by static analysis, including this one,
can also be detected by aspects; depending on the particular bug
pattern, it might be easier to do so with static analysis or with
aspects, so having both in your arsenal can increase your leverage.
Listing 1 shows a simple dynamic aspect that throws an AssertionError
when System.gc() is about to be called. (Because an
important role of bug detectors like this one is to find errors not
only in your own code but also in libraries that your code depends on,
you may need to tell your tools to analyze or instrument those
libraries as well.)
Listing 1. Dynamic aspect for enforcing the "don't call System.gc()" rule
public aspect GcAspect {
pointcut gcCalls() : call(void java.lang.System.gc());
before() : gcCalls() {
throw new AssertionError("Don't call System.gc!");
}
}
|
The dynamic approach Listing 1 illustrates is less effective for
testing than static analysis because it requires your program to
actually execute a call to System.gc() before the aspect
will disclose a problem, rather than the program simply containing a
call to System.gc(). However, as you'll soon see, dynamic
aspects are more flexible because they can execute arbitrary testing
code at the point where the aspect is triggered, enabling
finer-grained control over declaring a problem than is possible with a
static approach.
It is also easy to create a static aspect that flags calls to
System.gc() at compile time, as shown in Listing 2.
Again, if you wish to detect occurrences of this bug pattern in
library code, you must instrument not only code from your project, but
from libraries it uses as well.
Listing 2. Static aspect for enforcing the "don't call System.gc()" rule
public aspect StaticGcAspect {
pointcut gcCalls() : call(void java.lang.System.gc());
declare error : gcCalls() : "Don't call System.gc!";
}
|
Detecting violations of the Swing single-thread rule
One design rule that is nearly impossible to enforce statically is
thread confinement -- that a given object only be accessed from a
single thread (sometimes a specific thread, such as the Swing event
thread). The correctness of Swing programs relies on thread
confinement, but you won't get any help from the compiler, runtime, or
class library for enforcing this rule. If you violate it, your program
is broken, but it may not always be obvious because it may appear to
work correctly in testing.
The Swing single-thread rule states that:
Swing components and models should be created, modified,
and realized only from the event-dispatching thread.
Earlier formulations of the single-thread rule allowed Swing
components and models to be accessed from other threads before they
are realized on the screen, but this approach led to thread-safety
problems and so the rule was strengthened. With the SwingUtilities.isEventDispatchThread() method, Swing offers a means to ask
"is the current thread the event-dispatching thread?" So if we
were to insert code before every call to a Swing method checking that
the call is being made from an appropriate thread and throwing an
AssertionError if it was not, we could catch violations of
the single-thread rule in testing rather than having them cause
confusing failures in the field.
Listing 3 shows an aspect that can detect many violations of the
single-thread rule. It has two parts: the list of methods that should
not be called from outside the event thread and the code to insert
before each call to one of those methods. The advice -- the code to be
inserted -- is quite simple: check if the current thread is the event
thread and if it is not, throw an AssertionError. This aspect instruments all calls to methods from the Swing packages plus
any methods in classes that extend the most important Swing classes (so as to capture user-provided components and models), but it excludes methods in those classes that are known to be (or required to be) safe to call from multiple threads. The list of safe methods is not
exhaustive; constructing an exhaustive list would require spending
some additional time with the Swing Javadoc to find all the methods
that are documented to be thread-safe.
Listing 3. Aspect for enforcing Swing's single-thread rule
public aspect SwingThreadAspect {
pointcut swingMethods() : call(* javax.swing..*.*(..))
|| call(javax.swing..*.new(..));
pointcut extendsSwing() : call(* javax.swing.JComponent+.*(..))
|| call(* javax.swing..*Model+.*(..))
|| call(* javax.swing.text.Document+.*(..));
pointcut safeMethods() : call(void JComponent.revalidate())
|| call(void JComponent.invalidate(..))
|| call(void JComponent.repaint(..))
|| call(void add*Listener(EventListener+))
|| call(void remove*Listener(EventListener+))
|| call(boolean SwingUtilities.isEventDispatchThread())
|| call(void SwingUtilities.invokeLater(Runnable))
|| call(void SwingUtilities.invokeAndWait(Runnable))
|| call(void JTextPane.replaceSelection(..))
|| call(void JTextPane.insertComponent(..))
|| call(void JTextPane.insertIcon(..))
|| call(void JTextPane.setLogicalStyle(..))
|| call(void JTextPane.setCharacterAttributes(..))
|| call(void JTextPane.setParagraphAttributes(..));
pointcut edtMethods() : (swingMethods() || extendsSwing()) && !safeMethods();
before() : edtMethods() {
if (!SwingUtilities.isEventDispatchThread())
throw new AssertionError(thisJoinPointStaticPart.getSignature()
+ " called from " + Thread.currentThread().getName());
}
}
|
The swingMethods() pointcut includes all calls to methods
in the javax.swing package, including constructors. The
extendsSwing() pointcut represents all calls to methods
in classes that extend JComponent or any of the Swing
model classes. The safeMethods() pointcut represents
(some of) the Swing methods known to be safe to call from any thread.
SwingThreadAspect isn't perfect, but that's all
right. The safeMethods() pointcut does not fully
enumerate the thread-safe methods, and the extendsSwing()
pointcut probably does not include all of the frequently extended
Swing classes. But we're not using it for production -- we're using it
for testing. It finds bugs without having to create new test cases for
each program, and that's what counts. And, like most bug detectors, it
is likely to find bugs in programs that you probably thought were
correct.
Swapping in debugging objects
Another good application for aspects is swapping in "debugging"
versions of a class for the regular version. It is fairly common to
create a debugging version of a class, such as one with more logging
or error checking, that is not suitable for production use because of
its side effects or performance. But swapping in the debugging version
when you need it can be tedious and error-prone. If the object is
instantiated through its constructor, you have to find all the places
in the code where the constructor is invoked. A common technique for
mitigating the inconvenience of modifying all constructor invocations
is to instantiate objects through factories instead, but using
factories only for the purpose of choosing between the production
version and the debugging version can add complexity or introduce
safety or security holes.
If the goal is to be able to say "whenever we would have instantiated
a Foo, instantiate a DebuggingFoo instead,"
aspects provide a very simple mechanism for doing so reliably and
without modifying the program. As an example, Listing 4 shows an
aspect for assisting in the process of spotting deadlocks, replacing
all instantiations of a ReentrantLock with a
DebuggingLock. (Note that AspectJ only modifies calls
within the code that you've asked the AspectJ compiler to instrument;
instantiations of ReentrantLock within the Java™
class libraries themselves will not be replaced unless you've gone
out of your way to weave your aspects into the platform libraries as
well.)
Listing 4. Aspect that replaces all instantiations of a ReentrantLock with a DebuggingLock
public aspect ReentrantLockAspect {
pointcut newLock() : call(ReentrantLock.new());
pointcut newLockFair(boolean fair) :
call(ReentrantLock.new(boolean)) && args(fair);
ReentrantLock around() : newLock() {
return new DebuggingLock();
}
ReentrantLock around(boolean fair) : newLockFair(fair) {
return new DebuggingLock (fair);
}
}
|
As of Java SE 6, the runtime performs deadlock detection upon
request, either through the ThreadMXBean interface in
java.lang.management or when a thread dump has been
requested. Listing 5 shows one possible implementation of
DebuggingLock that performs the deadlock detection every
time a lock is acquired, so you can be alerted to deadlocks more
quickly. The locking performance is worse than with
ReentrantLock because more work is being done on every
lock attempt, so this approach may not be suitable for production
use. (Also, the synchronization inherent in maintaining the waitingFor
data structure may perturb the timing of your application, changing
the likelihood of deadlock.)
Listing 5. Debugging version of ReentrantLock that throws AssertionError when deadlock is detected
public class DebuggingLock extends ReentrantLock {
private static ConcurrentMap<Thread, DebuggingLock> waitingFor
= new ConcurrentHashMap<Thread, DebuggingLock>();
public DebuggingLock() { super(); }
public DebuggingLock(boolean fair) { super(fair); }
private void checkDeadlock() {
Thread currentThread = Thread.currentThread();
Thread t = currentThread;
while (true) {
DebuggingLock lock = waitingFor.get(t);
if (lock == null || !lock.isLocked())
return;
else {
t = lock.getOwner();
if (t == currentThread)
throw new AssertionError("Deadlock detected");
}
}
}
public void lock() {
if (tryLock())
return;
else {
waitingFor.put(Thread.currentThread(), this);
try {
checkDeadlock();
super.lock();
}
finally {
waitingFor.remove(Thread.currentThread());
}
}
}
}
|
For the version of DebuggingLock in Listing 5 to be
helpful, your program has to actually deadlock in testing. Because
deadlocks are often timing- and environment-dependent, the approach in
Listing 5 may not be enough. Listing 6 shows another version of
DebuggingLock, which not only determines whether a
deadlock has occurred, but whether a given pair of locks is ever
acquired in an inconsistent order by multiple threads. Each time a
lock is acquired, it looks at the set of already held locks, and for
each one, remembers that some thread has asked for those locks before
this one. Before attempting to acquire a lock, the lock()
method looks at the locks already held and throws an
AssertionError if any of those locks has ever been
acquired after this one. The space overhead of this implementation is
much larger then the previous version (because of the need to keep track
of all locks that have been acquired before a given lock), but it can
detect a broader range of bugs. It doesn't detect all possible
deadlocks -- only those that result from inconsistent ordering between
two specific locks, which is the most common case.
Listing 6. Alternate version of DebuggingLock that can detect inconsistent lock ordering even if a deadlock does not result
public class OrderHistoryLock extends ReentrantLock {
private static ThreadLocal<Set<OrderHistoryLock>> heldLocks =
new ThreadLocal<Set<OrderHistoryLock>>() {
public Set<OrderHistoryLock> initialValue() {
return new HashSet<OrderHistoryLock>();
}
};
private final Map<Lock, Boolean> predecessors
= new ConcurrentHashMap<Lock, Boolean>();
public OrderHistoryLock() { super(); }
public OrderHistoryLock(boolean fair) { super(fair); }
public void lock() {
boolean alreadyHeld = isHeldByCurrentThread();
for (OrderHistoryLock lock : heldLocks.get()) {
if (lock.predecessors.containsKey(this))
throw new AssertionError("Possible deadlock between "
+ this + " and " + lock);
else if (!alreadyHeld)
predecessors.put(lock, Boolean.TRUE);
}
super.lock();
heldLocks.get().add(this);
}
public void unlock() {
super.unlock();
if (!isHeldByCurrentThread())
heldLocks.get().remove(this);
}
}
|
Summary
The aspects described here fall into the category of policy
enforcement aspects. Some policies are part of your application's
design, such as "these methods should only be called from class X" or
"don't use System.out or System.err for
anything." Other policies are part of the interface contract for an
API, such as Swing's single-thread rule or the requirement that EJBs
should not create threads or make calls to AWT. In all of these cases,
you can use aspects in development and testing to identify whether
these policies are inadvertently violated. Whether or not you are
using aspects in production, they make a great addition to your
testing toolbox.
Resources Learn
-
Java theory and practice: "Testing
with leverage, Part 1" (Brian Goetz, developerWorks, June 2006): Explores the role of static analysis in
testing.
-
Java theory and practice: "Testing
with leverage, Part 2" (Brian Goetz, developerWorks, July 2006): Illustrates creating and tuning a FindBugs bug pattern detector.
-
AOP@Work series
(developerWorks, February 2005 - April 2006): AOP experts offer practical information on tools, techniques, and applications.
-
AspectJ in Action: Practical Aspect-Oriented Programming (Manning Publications, Ramnivas Laddad, July 2003): Describes applications and techniques for using AOP
effectively.
- The Java technology zone: Hundreds of articles about every aspect of Java programming.
Get products and technologies
-
FindBugs: Download FindBugs and try it out on your code.
-
AspectJ: Download the
AspectJ compiler and Eclipse plug-in for creating aspects and weaving
them into your project.
Discuss
About the author | | | Brian Goetz has been a professional software developer for 20 years. He is a Principal Consultant at Quiotix, a software development and consulting firm located in Los Altos, California, and he serves on several JCP Expert Groups. Brian's book, Java Concurrency In Practice, was published in May 2006 by Addison-Wesley. See Brian's published and upcoming articles in popular industry publications. |
Rate this page
| |
