Java theory and practice: Testing with leverage, Part 3

Verifying design constraints with aspects

Brian Goetz (brian@quiotix.com), Principal Consultant, Quiotix

Brian Goetz has been a professional software developer for 20 years. He is a Principal Consultant at Quiotix, a software development and consulting firm located in Los Altos, California, and he serves on several JCP Expert Groups. Brian's book, Java Concurrency In Practice, was published in May 2006 by Addison-Wesley. See Brian's published and upcoming articles in popular industry publications.

Summary: The first two installments in this series showed how static analysis tools like FindBugs can provide greater leverage in managing software quality by focusing on entire categories of bugs rather than on specific bug instances. In this final installment on testing, Brian Goetz examines another technique for smoking out bugs that violate design rules: aspects.

View more content in this series

Date: 22 Aug 2006
Level: Intermediate
Activity: 1288 views
Comments:

Aspect-oriented programming (AOP) is a new and promising technology, but adopting new technologies can be risky (as, of course, can not adopting new technologies). As with all new technologies, it is often best to follow a risk-managed path toward their adoption. You can benefit from reduced risk with AOP if you use it for policy enforcement and testing. Because the aspects never go into production, there is no risk that the technology will destabilize the production code or the development process, but it can still assist you in developing better quality software. Using aspects for testing is also a great way to learn how aspects work and gain some experience with this exciting new technology.

Combining testing methodologies

As I discussed in Part I, the goal of QA is not to find all possible bugs -- because that's impossible -- but instead to improve our confidence that the code works as expected. The challenge of managing an effective QA organization is maximizing the return -- confidence -- on the resources expended. Because all testing methodologies eventually exhibit diminishing returns (less incremental confidence for the same incremental effort) and different methodologies are suited to finding different types of errors, spreading your QA effort over testing, code review, and static analysis is likely to yield a better return than spending your entire QA budget on just one of these approaches.

Static analysis tools like FindBugs are inexact, but inexact analysis can still be quite useful and effective for improving software quality. They may emit false positives, such as triggering on harmless constructs, or false negatives, such as failing to identify all bugs matching a particular bug pattern. But they still find real bugs and, so long as the false positive rate isn't so high that users are overwhelmed with warnings, they still offer a valuable return on testing effort.

From the perspective of testing, using AOP to verify design rules has a lot in common with static analysis. Rather than crafting a test case for a specific method or class, both static analysis and aspect-oriented testing encourage you to identify entire categories of rule violations and create artifacts that allow you to find violations in any body of code. Another similarity is that they need not be perfect to be useful; even if bug detectors or testing aspects do not identify all possible bugs or sometimes emit false positives, they can still be useful as tools for verifying that the code works as expected. Some bug patterns are more easily identified with static analysis while others are more easily identified with aspects -- making aspects a useful methodology to add to the QA mix.

A simple testing aspect

Static analysis tools like FindBugs audit the code without executing it; aspect-oriented tools offer both static and dynamic instrumentation of classes. Static aspects can generate warnings or errors at compile time; dynamic aspects can insert error-checking code into your classes.

In Part 1, I offered a simple FindBugs detector to find calls to System.gc() that might be lurking in libraries. Many bug patterns that can be detected by static analysis, including this one, can also be detected by aspects; depending on the particular bug pattern, it might be easier to do so with static analysis or with aspects, so having both in your arsenal can increase your leverage.

Listing 1 shows a simple dynamic aspect that throws an AssertionError when System.gc() is about to be called. (Because an important role of bug detectors like this one is to find errors not only in your own code but also in libraries that your code depends on, you may need to tell your tools to analyze or instrument those libraries as well.)

Listing 1. Dynamic aspect for enforcing the "don't call System.gc()" rule


public aspect GcAspect {
    pointcut gcCalls() : call(void java.lang.System.gc());
 
    before() : gcCalls() {
        throw new AssertionError("Don't call System.gc!");
    }
}

The dynamic approach Listing 1 illustrates is less effective for testing than static analysis because it requires your program to actually execute a call to System.gc() before the aspect will disclose a problem, rather than the program simply containing a call to System.gc(). However, as you'll soon see, dynamic aspects are more flexible because they can execute arbitrary testing code at the point where the aspect is triggered, enabling finer-grained control over declaring a problem than is possible with a static approach.

It is also easy to create a static aspect that flags calls to System.gc() at compile time, as shown in Listing 2. Again, if you wish to detect occurrences of this bug pattern in library code, you must instrument not only code from your project, but from libraries it uses as well.

Listing 2. Static aspect for enforcing the "don't call System.gc()" rule


public aspect StaticGcAspect {
    pointcut gcCalls() : call(void java.lang.System.gc());

    declare error : gcCalls() : "Don't call System.gc!";
}

Detecting violations of the Swing single-thread rule

One design rule that is nearly impossible to enforce statically is thread confinement -- that a given object only be accessed from a single thread (sometimes a specific thread, such as the Swing event thread). The correctness of Swing programs relies on thread confinement, but you won't get any help from the compiler, runtime, or class library for enforcing this rule. If you violate it, your program is broken, but it may not always be obvious because it may appear to work correctly in testing.

The Swing single-thread rule states that:

Swing components and models should be created, modified, and realized only from the event-dispatching thread.

Earlier formulations of the single-thread rule allowed Swing components and models to be accessed from other threads before they are realized on the screen, but this approach led to thread-safety problems and so the rule was strengthened. With the SwingUtilities.isEventDispatchThread() method, Swing offers a means to ask "is the current thread the event-dispatching thread?" So if we were to insert code before every call to a Swing method checking that the call is being made from an appropriate thread and throwing an AssertionError if it was not, we could catch violations of the single-thread rule in testing rather than having them cause confusing failures in the field.

Listing 3 shows an aspect that can detect many violations of the single-thread rule. It has two parts: the list of methods that should not be called from outside the event thread and the code to insert before each call to one of those methods. The advice -- the code to be inserted -- is quite simple: check if the current thread is the event thread and if it is not, throw an AssertionError. This aspect instruments all calls to methods from the Swing packages plus any methods in classes that extend the most important Swing classes (so as to capture user-provided components and models), but it excludes methods in those classes that are known to be (or required to be) safe to call from multiple threads. The list of safe methods is not exhaustive; constructing an exhaustive list would require spending some additional time with the Swing Javadoc to find all the methods that are documented to be thread-safe.

Listing 3. Aspect for enforcing Swing's single-thread rule


public aspect SwingThreadAspect {

 pointcut swingMethods() : call(* javax.swing..*.*(..))
        || call(javax.swing..*.new(..));
 
 pointcut extendsSwing() : call(* javax.swing.JComponent+.*(..))
        || call(* javax.swing..*Model+.*(..))
        || call(* javax.swing.text.Document+.*(..));
 
 pointcut safeMethods() : call(void JComponent.revalidate())
        || call(void JComponent.invalidate(..))
        || call(void JComponent.repaint(..))
        || call(void add*Listener(EventListener+))
        || call(void remove*Listener(EventListener+))
        || call(boolean SwingUtilities.isEventDispatchThread())
        || call(void SwingUtilities.invokeLater(Runnable))
        || call(void SwingUtilities.invokeAndWait(Runnable))
        || call(void JTextPane.replaceSelection(..))
        || call(void JTextPane.insertComponent(..))
        || call(void JTextPane.insertIcon(..))
        || call(void JTextPane.setLogicalStyle(..))
        || call(void JTextPane.setCharacterAttributes(..))
        || call(void JTextPane.setParagraphAttributes(..));
 
 pointcut edtMethods() : (swingMethods() || extendsSwing()) && !safeMethods();
 
 before() : edtMethods() {
  if (!SwingUtilities.isEventDispatchThread())
   throw new AssertionError(thisJoinPointStaticPart.getSignature() 
     + " called from " + Thread.currentThread().getName());
 }
}

The swingMethods() pointcut includes all calls to methods in the javax.swing package, including constructors. The extendsSwing() pointcut represents all calls to methods in classes that extend JComponent or any of the Swing model classes. The safeMethods() pointcut represents (some of) the Swing methods known to be safe to call from any thread.

SwingThreadAspect isn't perfect, but that's all right. The safeMethods() pointcut does not fully enumerate the thread-safe methods, and the extendsSwing() pointcut probably does not include all of the frequently extended Swing classes. But we're not using it for production -- we're using it for testing. It finds bugs without having to create new test cases for each program, and that's what counts. And, like most bug detectors, it is likely to find bugs in programs that you probably thought were correct.

Swapping in debugging objects

Another good application for aspects is swapping in "debugging" versions of a class for the regular version. It is fairly common to create a debugging version of a class, such as one with more logging or error checking, that is not suitable for production use because of its side effects or performance. But swapping in the debugging version when you need it can be tedious and error-prone. If the object is instantiated through its constructor, you have to find all the places in the code where the constructor is invoked. A common technique for mitigating the inconvenience of modifying all constructor invocations is to instantiate objects through factories instead, but using factories only for the purpose of choosing between the production version and the debugging version can add complexity or introduce safety or security holes.

If the goal is to be able to say "whenever we would have instantiated a Foo, instantiate a DebuggingFoo instead," aspects provide a very simple mechanism for doing so reliably and without modifying the program. As an example, Listing 4 shows an aspect for assisting in the process of spotting deadlocks, replacing all instantiations of a ReentrantLock with a DebuggingLock. (Note that AspectJ only modifies calls within the code that you've asked the AspectJ compiler to instrument; instantiations of ReentrantLock within the Java™ class libraries themselves will not be replaced unless you've gone out of your way to weave your aspects into the platform libraries as well.)

Listing 4. Aspect that replaces all instantiations of a ReentrantLock with a DebuggingLock


public aspect ReentrantLockAspect {

  pointcut newLock() : call(ReentrantLock.new());

  pointcut newLockFair(boolean fair) : 
    call(ReentrantLock.new(boolean)) && args(fair);
		 
  ReentrantLock around() : newLock() {
    return new DebuggingLock();
  }

  ReentrantLock around(boolean fair) : newLockFair(fair) {
    return new DebuggingLock (fair);
  }
}

As of Java SE 6, the runtime performs deadlock detection upon request, either through the ThreadMXBean interface in java.lang.management or when a thread dump has been requested. Listing 5 shows one possible implementation of DebuggingLock that performs the deadlock detection every time a lock is acquired, so you can be alerted to deadlocks more quickly. The locking performance is worse than with ReentrantLock because more work is being done on every lock attempt, so this approach may not be suitable for production use. (Also, the synchronization inherent in maintaining the waitingFor data structure may perturb the timing of your application, changing the likelihood of deadlock.)

Listing 5. Debugging version of ReentrantLock that throws AssertionError when deadlock is detected


public class DebuggingLock extends ReentrantLock {
    private static ConcurrentMap<Thread, DebuggingLock> waitingFor 
        = new ConcurrentHashMap<Thread, DebuggingLock>();

    public DebuggingLock() { super(); }
    public DebuggingLock(boolean fair) { super(fair); }

    private void checkDeadlock() {
        Thread currentThread = Thread.currentThread();
        Thread t = currentThread;
        while (true) {
            DebuggingLock lock = waitingFor.get(t);
            if (lock == null || !lock.isLocked())
                return;
            else {
                t = lock.getOwner();
                if (t == currentThread)
                    throw new AssertionError("Deadlock detected");
            }
        }

    }

    public void lock() {
        if (tryLock())
            return;
        else {
            waitingFor.put(Thread.currentThread(), this);
            try {
                checkDeadlock();
                super.lock();
            }
            finally {
                waitingFor.remove(Thread.currentThread());
            }
        }
    }
}

For the version of DebuggingLock in Listing 5 to be helpful, your program has to actually deadlock in testing. Because deadlocks are often timing- and environment-dependent, the approach in Listing 5 may not be enough. Listing 6 shows another version of DebuggingLock, which not only determines whether a deadlock has occurred, but whether a given pair of locks is ever acquired in an inconsistent order by multiple threads. Each time a lock is acquired, it looks at the set of already held locks, and for each one, remembers that some thread has asked for those locks before this one. Before attempting to acquire a lock, the lock() method looks at the locks already held and throws an AssertionError if any of those locks has ever been acquired after this one. The space overhead of this implementation is much larger then the previous version (because of the need to keep track of all locks that have been acquired before a given lock), but it can detect a broader range of bugs. It doesn't detect all possible deadlocks -- only those that result from inconsistent ordering between two specific locks, which is the most common case.

Listing 6. Alternate version of DebuggingLock that can detect inconsistent lock ordering even if a deadlock does not result


public class OrderHistoryLock extends ReentrantLock {
    private static ThreadLocal<Set<OrderHistoryLock>> heldLocks = 
      new ThreadLocal<Set<OrderHistoryLock>>() {
        public Set<OrderHistoryLock> initialValue() {
            return new HashSet<OrderHistoryLock>();
        }
    };

    private final Map<Lock, Boolean> predecessors 
    		 = new ConcurrentHashMap<Lock, Boolean>();
    
    public OrderHistoryLock() { super(); }
    
    public OrderHistoryLock(boolean fair) { super(fair); }

    public void lock() {
    		 boolean alreadyHeld = isHeldByCurrentThread();
        for (OrderHistoryLock lock : heldLocks.get()) {
            if (lock.predecessors.containsKey(this))
                throw new AssertionError("Possible deadlock between " 
                  + this + " and " + lock);
            else if (!alreadyHeld) 
                predecessors.put(lock, Boolean.TRUE);
        }
        super.lock();
        heldLocks.get().add(this);
    }

    public void unlock() {
        super.unlock();
        if (!isHeldByCurrentThread()) 
        		 heldLocks.get().remove(this);
    }
}

Summary

The aspects described here fall into the category of policy enforcement aspects. Some policies are part of your application's design, such as "these methods should only be called from class X" or "don't use System.out or System.err for anything." Other policies are part of the interface contract for an API, such as Swing's single-thread rule or the requirement that EJBs should not create threads or make calls to AWT. In all of these cases, you can use aspects in development and testing to identify whether these policies are inadvertently violated. Whether or not you are using aspects in production, they make a great addition to your testing toolbox.

Resources

Learn

Java theory and practice: "Testing with leverage, Part 1" (Brian Goetz, developerWorks, June 2006): Explores the role of static analysis in testing.
Java theory and practice: "Testing with leverage, Part 2" (Brian Goetz, developerWorks, July 2006): Illustrates creating and tuning a FindBugs bug pattern detector.
AOP@Work series (developerWorks, February 2005 - April 2006): AOP experts offer practical information on tools, techniques, and applications.
AspectJ in Action: Practical Aspect-Oriented Programming (Manning Publications, Ramnivas Laddad, July 2003): Describes applications and techniques for using AOP effectively.
The Java technology zone: Hundreds of articles about every aspect of Java programming.

Get products and technologies

FindBugs: Download FindBugs and try it out on your code.
AspectJ: Download the AspectJ compiler and Eclipse plug-in for creating aspects and weaving them into your project.

Discuss