Evolutionary architecture and emergent design: Leveraging reusable code, Part 2

Capturing idiomatic patterns

Once you use the techniques described in previous Evolutionary architecture and emergent design installments to discover emergent design in code, you need a way to harvest and leverage those design elements. This article covers two techniques for harvesting idiomatic patterns: capturing patterns as APIs and using metaprogramming techniques.

Neal Ford, Software Architect / Meme Wrangler, ThoughtWorks Inc.

Photo of Neal FordNeal Ford is a software architect and Meme Wrangler at ThoughtWorks, a global IT consultancy. He also designs and develops applications, instructional materials, magazine articles, courseware, and video/DVD presentations, and he is the author or editor of books spanning a variety of technologies, including the most recent The Productive Programmer. He focuses on designing and building large-scale enterprise applications. He is also an internationally acclaimed speaker at developer conferences worldwide. Check out his Web site.

11 May 2010

Also available in Chinese

Several previous installments of this series focus on the obvious first step in emergent design: discovering idiomatic patterns. Once you've found one, what do you do with it? The answer to that question is the focus of this installment, which is the second part of a multipart article. Part 1 — a discussion of the relationship between code and design — covers the theoretical basis of the perspective that design in software really refers to the entire source code for the solution. Once you shift your perspective to think about all the code as the real design, you can start thinking about consolidating design elements at the language level rather than purely in the realm of diagrams and other design-aid artifacts. Here I'll talk about what to do once you've unearthed reusable design in code, covering techniques for harvesting those patterns. I'll start with harvesting them as simple APIs and then illustrate a harvesting technique that enables you to differentiate these elements from other code.

About this series

This series aims to provide a fresh perspective on the often-discussed but elusive concepts of software architecture and design. Through concrete examples, Neal Ford gives you a solid grounding in the agile practices of evolutionary architecture and emergent design. By deferring important architectural and design decisions until the last responsible moment, you can prevent unnecessary complexity from undermining your software projects.

Harvesting patterns as APIs

The easiest way to capture idiomatic patterns is to extract them as their own API or framework. Most of the open source frameworks you use are sets of related idiomatic patterns that have to do with solving a particular problem. For example, Web frameworks contain all the API elements you need to build Web applications, preharvested from other working Web applications. Spring, for example, is a collection of technical idiomatic patterns for handling dependency injection and construction, and Hibernate encapsulates patterns for object-relational mapping (see Resources).

Of course, you can do the same things in your code. This is by far the simplest technique because you're just changing your code's structure (frequently through refactoring support in your IDE of choice). Numerous examples of this technique appear in Part 1, as well as in "Language, expressiveness, and design, Part 2," which discusses design patterns.

Avoiding structural duplication

APIs tend, accidentally, to encourage structural duplication. Using APIs can be noisy because you frequently must use host objects to invoke the API. Consider the example in Listing 1 (which invokes an API relating to railroad cars):

Listing 1. Accessing a Car API
Car2 car = new CarImpl();
MarketingDescription desc = new MarketingDescriptionImpl();
desc.setAttribute("length", "50.5");
desc.setAttribute("ladder", "yes");
desc.setAttribute("lining type", "cork");

Forcing the user to type the host object (desc) adds needless noise to the code. Most APIs include host objects as the entry point to the API, and you must carry them around to access the API.

A couple of techniques exist to mitigate this problem in APIs. One uses a little-known Java syntax that allows you to "carry" a host object via the scoping of an anonymous inner class, as shown in Listing 2:

Listing 2. Using an anonymous inner class to carry a host object
MarketingDescription desc = new MarketingDescriptionImpl() {{
    setAttribute("length", "50.5");
    setAttribute("ladder", "yes");
    setAttribute("lining type", "cork");


For you to understand Listing 2, I must delve into a bit of trivia about how the Java language handles initialization. Consider the code in Listing 3:

Listing 3. Initializers in the Java language
public class InitializerDemo {
    public InitializerDemo() {
        out.println("in constructor");

    static {
        out.println("in static initializer");

        out.println("in instance initializer");

    public static void main(String[] args) {
        out.println("in main() method");
        new InitializerDemo();

The example in Listing 3 illustrates four different initialization techniques in the Java language:

  • In the main() method
  • In the constructor
  • In a static initializer block, which executes when the class is loaded
  • In an initializer block, which executes just before the constructor

This execution order is illustrated in Figure 1:

Figure 1. Initialization order in the Java language
illustration of initialization order in Java

The static initializer runs first, as the class is loaded, followed by the main method (which is also static). After that, the Java platform gathers up all the instance initializer blocks and executes them before the constructor, followed finally by the constructor itself. Instance initializers allow you to execute construction code for an anonymous inner class. In fact, it's the only real initialization mechanism, because it is impossible to write a constructor for an anonymous inner class — the constructor must have the same name as the class, but the class for an anonymous inner class has no name.

By using what is essentially a stupid Java trick, you avoid reusing the host name of the series of methods you want to execute. However, this is at the expense of a strange syntax that may leave your colleagues scratching their heads.

The downside

Extracting idiomatic patterns as APIs is a perfectly valid technique and probably the most common way to leverage the reusable gems you uncover. The downside of this approach lies in its normalcy: it becomes hard to distinguish the design elements you've extracted because they look like all your other code. Your successors on a project will have a hard time understanding that the API you've created represents something slightly different from the code around it, so your detective work in uncovering the pattern may go to waste. However, if you can make the idiomatic pattern stand out from other code, it's easier to see that it is in fact something different.

Using metaprogramming

Metaprogramming offers a great way to differentiate pattern code from implementation code because you express your pattern by using code that is about code. A nice technique offered by the Java language is attributes. You can define attributes to create declarative metaprogramming tags. Attributes offer a concise way to express concepts. You can pack a lot of functionality into a small space by defining it as an attribute and decorating the pertinent portion of your class.

Here's a good example. One common technical idiomatic pattern in most projects is validation, which lends itself well to declarative code. If you harvest validation patterns as attributes, you can mark up your code with clear validation constraints that don't interfere with the code's main thrust. Consider the code in Listing 4:

Listing 4. MaxLength attribute
public class Country {
	private List<Region> regions = new ArrayList<Region>();
	private String name;
	public Country(String name){
		this.name = name;
	@MaxLength(length = 10)
	public String getName(){
		return name;
	public void addRegion(Region region){
	public List<Region> getRegions(){
		return regions;

The ability to mark code elements with attributes declares your intent that something external operates on the code that follows. This in turn makes it easier to distinguish between the pattern portion and the implementation portion. Your validation code sticks out because it doesn't look like the other code around it. This partitioning of code by functionality makes it easy to identify particular responsibilities, refactor, and maintain.

The MaxLength validator specifies that a Country name must not exceed 10 characters. The attribute declaration itself appears in Listing 5:

Listing 5. MaxLength attribute declaration
public @interface MaxLength {
	int length() default 0;

The MaxLength validator's real functionality resides across two classes: an abstract class named Validator and a concrete implementation of it named MaxLengthValidator. The Validator class appears in Listing 6:

Listing 6. Abstract attribute-based Validator class
public abstract class Validator {

    public void validate(Object obj) throws ValidationException {
        Class clss = obj.getClass();
        for(Method method : clss.getMethods())
            if (method.isAnnotationPresent(getAnnotationType()))
                validateMethod(obj, method, method.getAnnotation(getAnnotationType()));

    protected abstract Class getAnnotationType();
    protected abstract void validateMethod(
        Object obj, Method method, Annotation annotation);

This class iterates over methods in a class by looking at getAnnotationType() to determine if the methods are decorated with a particular attribute; when it finds one, it executes the validateMethod() method. The implementation of the MaxLengthValidator class appears in Listing 7:

Listing 7. The MaxLengthValidator class
public class MaxLengthValidator extends Validator {

    protected void validateMethod(Object obj, Method method, Annotation annotation) {
        try {
            if (method.getName().startsWith("get")) {
                MaxLength length = (MaxLength)annotation;
                String value = (String)method.invoke(obj, new Object[0]);
                if ((value != null) && (length.length() < value.length())) {
                    String string = method.getName() + " is too long." + 
                        "Its length is " + value.length() + 
                        " but should be no longer than " + length.length();
                    throw new ValidationException(string);
        } catch (Exception e) {
            throw new ValidationException(e.getMessage());


    protected Class getAnnotationType() {
        return MaxLength.class;

This class checks to see if the method subject to potential validation starts with get, then harvests the metadata from the annotation, finally checking the value of the attribute's length field against the declared length, throwing a validation error upon rule violation.

Attributes can do some very sophisticated work. Consider the example in Listing 8:

Listing 8. Class with uniqueness validation
public class Region {
    private String name = "";
    private Country country = null;
    public Region(String name, Country country) {
        this.name = name;
        this.country = country;

    public void setName(String name){
        this.name = name;
    @Unique(scope = Country.class)
    public String getName(){
        return this.name;
    public Country getCountry(){
        return country;

The declaration of the Unique attribute, shown in Listing 9, is quite simple:

Listing 9. The Unique Attribute
public @interface Unique {
	Class scope() default Unique.class;

The Unique attribute implementation class extends the abstract Validator class shown in Listing 6. Its source appears in Listing 10:

Listing 10. Unique validator implementation
public class UniqueValidator extends Validator{

  protected void validateMethod(Object obj, Method method, Annotation annotation) {
    Unique unique = (Unique) annotation;
    try {
      Method scopeMethod = obj.getClass().getMethod("get" + 
      Object scopeObj = scopeMethod.invoke(obj, new Object[0]);
      Method collectionMethod = scopeObj.getClass().getMethod(
          "get" + obj.getClass().getSimpleName() + "s");
      List collection = (List)collectionMethod.invoke(scopeObj, new Object[0]);
      Object returnValue = method.invoke(obj, new Object[0]);
      for(Object otherObj: collection){
        Object otherReturnValue = otherObj.getClass().
            getMethod(method.getName()).invoke(otherObj, new Object[0]);
        if (!otherObj.equals(obj) && otherReturnValue.equals(returnValue))
          throw new ValidationException(method.getName() + " on " + 
            obj.getClass().getSimpleName() + " should be unique but is not since");
    } catch (Exception e) {
      throw new ValidationException(e.getMessage());

  protected Class getAnnotationType() {
    return Unique.class;

This class must do a fair amount of work to make sure that a country name's value is unique, but it is a good example of just how powerful attributes are in Java programming.

Attributes are a welcome addition to the Java language. They allow you to concisely define behavior that has wide-ranging impact with very little syntactic hangover in the target class. However, they are still limited compared to the kinds of things you can do in more expressive languages on the JVM, such as JRuby.

Sticky attributes using JRuby

The Ruby language also has attributes (although they don't have a special name like "attributes" — they are one of many metaprogramming techniques Ruby offers). Here's an example. Consider the test class in Listing 11:

Listing 11. Testing a complex calculation
class TestCalculator < Test::Unit::TestCase
  def test_complex_calculation
    assert_equal(4, Calculator.new.complex_calculation)

Let's say that the complex_calculation method takes so long that you only want to run it when doing acceptance tests, not during unit test runs. One way to limit it appears in Listing 12:

Listing 12. Limiting testing scope
class TestCalculator < Test::Unit::TestCase

    def test_complex_calculation
       assert_equal(4, Calculator.new.complex_calculation)    

This is a technical idiomatic pattern relating to testing that I can easily anticipate needing in lots of contexts. Wrapping the method declaration in an if block adds some ugly complexity to my code, because now not all method declarations appear at the same level of indentation. Instead, I'll capture that pattern using an attribute, as shown in Listing 13:

Listing 13. Declaring an attribute in Ruby
class TestCalculator < Test::Unit::TestCase  
  extend TestDirectives 
  def test_complex_calculation
    assert_equal(4, Calculator.new.complex_calculation)        

This version is much cleaner and easier to read. The implementation, shown in Listing 14, is trivial:

Listing 14. Attribute declaration
module TestDirectives
  def acceptance_only
    @acceptance_build = ENV['BUILD'] == 'ACCEPTANCE'
  def method_added(method_name)
    remove_method(method_name) unless @acceptance_build
    @acceptance_build = false

It's remarkable how much you can achieve with so little code in Ruby. Listing 14 declares a module, which is Ruby's version of a mix-in. A mix-in contains functionality that you can include into a class in order to add that functionality to the class. You can think of it as a sort of interface, but one that can include code. This module defines a method called acceptance_only, which checks the BUILD environment variable to see which testing phase is executing. Once this flag has been set, the module leverages a hook method. Hook methods in Ruby execute at interpretation time (not run time), and this particular hook method fires every time you add a new method to a class. When this method executes, it removes the method just defined if the acceptance_build flag is set. It then sets the flag back to false. (Otherwise, this attribute will affect all subsequent method declarations because the flag would remain true.) If you wanted it to affect a block of code that encompasses several methods, you remove the resetting of the flag, allowing this behavior to remain until something else (such as a user-defined unit_test attribute) changes it. (These are informally called sticky attributes.)

To illustrate this mechanism's power, the Ruby language itself uses sticky attributes to declare private, protected, and public class-scope modifiers. That's right — class scoping designations in Ruby aren't keywords, they are merely sticky attributes.


In this installment, I demonstrated the use of APIs and attributes as techniques for harvesting idiomatic patterns. If you can find a way to make harvested patterns stand out from other code, it's easier to read both kinds of code because they don't contaminate each other.

In the next installment, I continue showing how to harvest idiomatic patterns, with a collection of techniques usually used for building domain-specific languages.



  • The Productive Programmer (Neal Ford, O'Reilly Media, 2008): Neal Ford's most recent book expands on a number of the topics in this series.
  • "Ruby off the rails" (Andrew Glover, developerWorks, December 2005): Get to know Ruby from a Java developer's perspective.
  • Hibernate is a popular open source object-relational mapping framework that encapsulates many handy idiomatic patterns.
  • Spring: The Spring framework is considered one of the most useful frameworks in all of Javadom.
  • Browse the technology bookstore for books on these and other technical topics.
  • developerWorks Java technology zone: Find hundreds of articles about every aspect of Java programming.



developerWorks: Sign in

Required fields are indicated with an asterisk (*).

Need an IBM ID?
Forgot your IBM ID?

Forgot your password?
Change your password

By clicking Submit, you agree to the developerWorks terms of use.


The first time you sign into developerWorks, a profile is created for you. Information in your profile (your name, country/region, and company name) is displayed to the public and will accompany any content you post, unless you opt to hide your company name. You may update your IBM account at any time.

All information submitted is secure.

Choose your display name

The first time you sign in to developerWorks, a profile is created for you, so you need to choose a display name. Your display name accompanies the content you post on developerWorks.

Please choose a display name between 3-31 characters. Your display name must be unique in the developerWorks community and should not be your email address for privacy reasons.

Required fields are indicated with an asterisk (*).

(Must be between 3 – 31 characters.)

By clicking Submit, you agree to the developerWorks terms of use.


All information submitted is secure.

Dig deeper into Java technology on developerWorks

Zone=Java technology,
ArticleTitle=Evolutionary architecture and emergent design: Leveraging reusable code, Part 2