Evolutionary architecture and emergent design: Language, expressiveness, and design, Part 1

How expressiveness in your code enables emergent design

The ability to see and harvest idiomatic patterns is critical for emergent design. Also vitally important to design is code's expressiveness. In a two-part article, Neal Ford discusses the intersection of expressiveness and patterns, demonstrating these concepts with both idiomatic patterns and formal design patterns. He recasts some of the classic Gang of Four patterns in dynamic languages for the JVM to show how more expressive languages enable you to see design elements obscured by more opaque languages.

Neal Ford, Software Architect / Meme Wrangler, ThoughtWorks Inc.

Neal FordNeal Ford is a software architect and Meme Wrangler at ThoughtWorks, a global IT consultancy. He also designs and develops applications, instructional materials, magazine articles, courseware, and video/DVD presentations, and he is the author or editor of books spanning a variety of technologies, including the most recent The Productive Programmer. He focuses on designing and building large-scale enterprise applications. He is also an internationally acclaimed speaker at developer conferences worldwide. Check out his Web site.



28 July 2009

Also available in Chinese Vietnamese

One of the primary enablers of emergent design is the ability to see and harvest idiomatic patterns: processes, structures, and idioms that repeat in nontrivial ways within your code base. However, sometimes these patterns hide from you. In the first installment of the Evolutionary architecture and emergent design series, I described problems that obscure the visibility of these patterns, such as rampant genericness. Building multilayered applications might be effective for the kind of separation of concerns that enables scalability and partitioning, but it hides idiomatic patterns because now you must find them across multiple layers. Becoming a good designer and architect requires developing "eyes" to discern these patterns.

About this series

This series aims to provide a fresh perspective on the often-discussed but elusive concepts of software architecture and design. Through concrete examples, Neal Ford gives you a solid grounding in the agile practices of evolutionary architecture and emergent design. By deferring important architectural and design decisions until the last responsible moment, you can prevent unnecessary complexity from undermining your software projects.

Another inhibitor for harvesting patterns lies with the expressiveness of the language itself. For example, it is difficult to harvest patterns from assembly language because the language's characteristics fight expressiveness. Even if you've learned to read assembly language as well as your native tongue, the draconian constraints on the way you write code obscure your ability to gain a holistic view. For example, passing variables into and out of registers instead of being able to create well-named variables and methods means you spend a lot of time dealing with the overhead inherent in the language.

Comparing the Java™ language to assembly is a stretch, but the expressiveness of computer languages resides along a spectrum. Some languages are more expressive than others, making it easier to see patterns more effectively. To that end, this article — the first of two parts — uses a dynamic language for the JVM (Groovy) to demonstrate alternative implementations of some of the Gang of Four patterns.

Design patterns revisited

One of the seminal books in software development is Design Patterns: Elements of Reusable Object-Oriented Software by Eric Gamma, Richard Helm, Ralph Johnson, and John Vlissides (see Resources). The book consists of two distinct parts: descriptions of common problems encountered during software development and examples of their solutions. The first part is valuable as a catalog of generic issues, but the implementations of the patterns necessarily show a bias toward a particular language. The sample implementations appear in both C++ and Smalltalk, yet they take advantage of few of Smalltalk's advanced language features. In many ways, the implementations highlight limitations of C++ and workarounds required to solve problems inherent in that language.

The nomenclature aspects of the Gang of Four book remain valuable today, but the implementations are outdated. Many of the problems that the implementations solve structurally (by building a hierarchy of interacting classes) have more elegant solutions in languages that offer more power and expressiveness.

Another interesting change has occurred since the Gang of Four book's publication. Many languages have subsumed the patterns into the language itself. For example, the Java language changed its collection-iteration style between JDK 1.1 and 1.2, replacing Enumerator with the Iterator interface to align the iterators in the Java language more closely with the Gang of Four's Iterator pattern. Languages tend to incorporate patterns and other common idioms, where they disappear as part of the language abstraction itself.

The first couple of examples show this exact embrace of Gang of Four patterns in more modern Java-based languages, incorporating the Iterator and Command patterns directly into the language.


The Iterator pattern

The Gang of Four book defines the Iterator pattern as:

Provide a way to access the elements of an aggregate object sequentially without exposing its underlying representation.

Groovy iterators

For a detailed introduction to Groovy iterators, see "Reaching for each" in the Practically Groovy series on developerWorks.

The Iterator pattern was one of the first to appear as an addition to the Java language, in the form of the Iterator interface and implementation. Groovy has taken this one step further and added internal iterators as part of the collections API. Thus, you can iterate over a collection quite easily using the each method in conjunction with a code block, as shown in Listing 1. This listing illustrates an internal iterator (also known as a push iterator because it pushes each element in turn to the code block).

Listing 1. The Groovy each operator
def numbers = [1,2,3,4]

numbers.each { n ->
  println n
}

Groovy allows iteration to work for all kinds of collections, including hashes, as shown in Listing 2:

Listing 2. Iteration over a hash
def months = [Mar:31, Apr:30, May:31]

months.each { 
  println it
}

Groovy also implements the handy default behavior of automatically supplying a parameter for your iteration named it, which you can reference within the code block.

And Groovy supports an external iterator (also known as a pull iterator because you must request the next item in the collection), shown in Listing 3. This is the exact same iterator built into the Java language itself.

Listing 3. The pull iterator
iterator = numbers.iterator()
while (iterator.hasNext()) {
  println iterator.next()
}

Iterator is so common that it has ceased to be a formal pattern at all; it is just a feature of the language. This is a common occurrence in the emergent design of computer languages themselves.


The Command pattern

The Gang of Four book defines the Command pattern as:

Encapsulate a request as an object, thereby letting you parameterize clients with different requests, queue or log requests, and support undoable operations.

The common implementation of this pattern in the Java language creates a Command class that includes an execute() method. The Command design pattern appears in Groovy as a code block, which is anything defined within free-standing curly braces ({ and }). Rather than force you to create a new class and corresponding method, you can execute the code block either by calling its call() method or by placing parentheses after the variable name holding the code block (with or without parameters). Listing 4 shows an example:

Listing 4. Command pattern with code blocks in Groovy
def count = 0
def commands = []

1.upto(10) { i ->
    commands.add { count++ }
}

println "count is initially ${count}"
commands.each { cmd ->
    cmd()
}
println "did all commands, count is ${count}"

Supporting undo

One of the advantages of using code blocks over a similar mechanism like anonymous inner classes lies in its conciseness. Because specifying undoable operations is such a common need, the syntax becomes important. Consider the Groovy code in Listing 5, which shows how you would support undoable operations using code blocks in conjunction with the Command design pattern:

Listing 5. Using code blocks to support undoable operations
class Command {
    def cmd, uncmd
    
    Command(doCommand, undoCommand) {
        cmd = doCommand
        uncmd = undoCommand
    }
    
    def doCommand() {
        cmd()
    }
    
    def undoCommand() {
        uncmd()
    }
}

def count = 0
def commands = []
1.upto(10) { i ->
    commands.add(new Command({count++}, {count--}))
}
println "count is initially ${count}"
commands.each { c -> c.doCommand() }
commands.reverseEach { c -> c.undoCommand() }
println "undid all commands, count is ${count}"
commands.each { c -> c.doCommand() }
println "redid all command, count is ${count}"

It is trivial to pass code blocks as parameters, allowing the terse yet still readable commands.add(new Command({count++}, {count--})) syntax.

Code blocks, expressiveness, and idiomatic patterns

Although the difference between code blocks and anonymous inner classes may seem like mere semantics, it has an impact on your code's readability and therefore the ease with which you can harvest idiomatic patterns. Consider this example of an idiomatic pattern I refer to as Unit of Work. First, the Java version (using anonymous inner classes) appears in Listing 6:

Listing 6. The Unit of Work Pattern with an anonymous inner class
public void wrapInTransaction(Command c) throws SQLException {
    setupDataInfrastructure();
    try {
        c.execute();
        completeTransaction();
    } catch (RuntimeException ex) {
        rollbackTransaction();
        throw ex;
    } finally {
        cleanUp();
    }
}

public void addOrderFrom(final ShoppingCart cart, final String userName,
                         final Order order) throws SQLException {
    wrapInTransaction(new Command() {
        public void execute() throws SQLException{
            add(order, userKeyBasedOn(userName));
            addLineItemsFrom(cart, order.getOrderKey());
        }
    });                
}

Now, consider the same example written in Groovy, shown in Listing 7, which takes advantage of the more concise syntax offered with code blocks:

Listing 7. Using code blocks to implement the Unit of Work pattern
public class OrderDbClosure {
   def wrapInTransaction(command) {
     setupDataInfrastructure()
     try {
       command()
       completeTransaction()
     } catch (RuntimeException ex) {
       rollbackTransaction()
       throw ex
     } finally {
       cleanUp()
     }
   }
   
   def addOrderFrom(cart, userName, order) {
     wrapInTransaction {
       add order, userKeyBasedOn(userName)
       addLineItemsFrom cart, order.getOrderKey()
     }
   }
}

While the code in Listing 7 that defines wrapInTransaction() is approximately the same as in Listing 6, the calling code is much cleaner. The Java version requires creating lots of syntax to implement the anonymous inner class; that syntax obscures the meaning of what I am trying to accomplish. The more syntax you must wade through to see the design elements, the more difficult it becomes to realize that a pattern is present. The Groovy version has minimal syntax to implement the pattern, leaving only pertinent content.


The Strategy pattern

The Gang of Four book defines the strategy pattern as:

Define a family of algorithms, encapsulate each one, and make them interchangeable. Strategy lets the algorithm vary independently from clients that use it.

The traditional implementation of the Strategy pattern in the Java language requires an interface that defines the algorithm's semantics, and concrete classes that provide the implementation. A Java implementation of Strategy to multiply numbers appears in Listing 8:

Listing 8. Multiplication strategies in the Java language
public interface Calc {
    public int product(int x, int y);
}

public class CalcByMult implements Calc {
    public int product(int x, int y) {
        return x * y;
    }
}

public class CalcByAdds implements Calc {
    public int product(int x, int y) {
        int result = 0;
        for (int i = 1; i <= y; i++)
            result += x;
        return result;
    }
}

The Java language forces you to create structure to solve the problem. In fact, the Gang of Four solutions are heavily biased toward creating structures to implement the pattern solutions — have you noticed that every pattern includes a UML diagram showing the solution? But building structure isn't always the most clear or concise way to attack problems. Consider Listing 9, which implements the same pattern in Groovy:

Listing 9. Multiplication strategies in Groovy
interface Calc {
    def product(n, m)
}

def multiplicationStrategies = [
    { n, m -> n * m } as Calc,
    { n, m -> def result = 0
      n.times { result += m }
      result 
    } as Calc
]

def sampleData = [
    [3, 4, 12],
    [5, -5, -25]
]

sampleData.each{ data ->
    multiplicationStrategies.each{ calc ->
        assert data[2] == calc.product(data[0], data[1])
    }
}

In the Groovy example, you aren't required to create the extra classes explicitly to implement the interface that defines the call's semantics. The powerful as operator in Groovy takes a code block and generates a new class that implements the interface, which you can then call as if it were a concrete class implementing the interface. Thus, in this sample execution, all of the code blocks that define a strategy on the fly can still act like formal concrete classes that implement the Calc interface.


The Interpreter pattern

The Gang of Four Interpreter pattern is a special case. The definition is:

Given a language, define a representation for its grammar along with an interpreter that uses the representation to interpret sentences in the language.

This pattern is essentially a "get out of jail free" pattern. It is the formal admission of something better espoused by Philip Greenspun as Greenspun's Tenth Rule (see Resources):

Any sufficiently complicated C or Fortran program contains an ad hoc, informally-specified, bug-ridden, slow implementation of half of Common Lisp.

He means that when you build more and more complex software in weaker languages, you are really implementing ad-hoc features from more powerful languages (like Lisp), one feature at a time, without realizing it. The Interpreter pattern is the admission that your base language is perhaps insufficient for the task at hand, and in that case the best solution is to use this language to build a better language on top of it.

This pattern shows the age of the Gang of Four book and its thinking. The Gang advocates abandoning your core language and building an entirely new one atop it, creating your own lexer, parser, grammar, and so on. However, an intermediate stage of this pattern has emerged into the mainstream over the past few years (even though it has been around since Lisp): making your language more expressive by building a domain-specific language (DSL) on top of it.

Building DSLs on top of the Java language is difficult because the language syntax is quite rigid, and it has few language-level extension points. It's common to build DSLs in languages like Groovy and Ruby because the syntax is both extensible and more forgiving.

Listing 10 shows a demonstration application for a small recipe DSL written in Groovy:

Listing 10. Recipe DSL in Groovy
def recipe = new Recipe("Spicy Bread")
recipe.add 1.gram.of("Nutmeg")
recipe.add 2.lbs.of("Flour")
println recipe

Groovy metaprogramming

Learn more about ExpandoMetaClass and Groovy's other metaprogramming features in "Practically Groovy: Metaprogramming with closures, ExpandoMetaClass, and categories."

The interesting lines of code in Listing 10 are the middle ones, which define the recipe ingredients. Groovy allows you to add new methods to any class (including java.lang.Integer, which is how Groovy treats number literals). This is how I am able to call methods on the numeric values. To add new methods to an existing class, you can use a Groovy mechanism called ExpandoMetaClass, shown in Listing 11:

Listing 11. Adding methods to Integer via ExpandoMetaClass
  Integer.metaClass.getGram { ->
    delegate
  }
  Integer.metaClass.getGrams {-> delegate.gram }

  Integer.metaClass.getPound { ->
    delegate * 453.29
  }

  Integer.metaClass.getPounds {-> delegate.pound }
  Integer.metaClass.getLb {-> delegate.pound }
  Integer.metaClass.getLbs {-> delegate.pound }

In Listing 11, I define a new property named getGram (which allows me to call it from Groovy without the get prefix) on the meta class of Integer. Within the property definition, delegate refers to the value of this instance of the Integer; I'm keeping all my units of measure within my DSL in grams, so it returns the integer value. One of the goals of DSLs is fluency, so I also define a plural version of the gram property called getGrams, making for more readable DSL code. I also need to support pounds as a unit of measure, so I also define a family of pound properties.

The new properties handle the first part of my DSL, leaving the of method as the only remaining part. of is also a method added to Integer, and it appears in Listing 12. This method accepts a single parameter, assigns the name of the ingredient, sets the quantity, and returns the newly created ingredient object.

Listing 12. The of method added to Integer
  Integer.metaClass.of { name ->
    def ingredient = new Ingredient(name);
    ingredient.quantity = delegate
    ingredient
  }

Some subtlety exists in the code in Listing 10 that is exposed by the code in Listing 12. Although the first line of the DSL works fine now (recipe.add 1.gram.of("Nutmeg")), the second one fails because the of method I've defined is no longer applicable. Once the call to of occurs in the sample line recipe.add 2.lbs.of("Flour"), the calling type has changed from Integer to BigDecimal, Groovy's default format for floating-point numbers. How did this happen? In the call to the pounds, the return type is now a floating-point number (2 * 453.29). Thus I need an additional of method attached to BigDecimal, as shown in Listing 13:

Listing 13. The of method added to BigDecimal
  BigDecimal.metaClass.of { name ->
    def ingredient = new Ingredient(name);
    ingredient.quantity = delegate
    ingredient
  }

This issue pops up surprisingly often in DSL implementations. Lots of DSLs need quantities of things: 1 week, 2 pounds, 6 dollars. Adding methods to Integer allows you to create much more expressive code because you can use real numbers to represent numeric values. Lines of code in DSLs frequently start with a quantity, call some intermediate methods to get work done, and eventually return an instance of the interesting final type. In Listing 10, the quantity starts the method call, which moves through Integer, then BigDecimal, finally returning an Ingredient. DSLs tend to create more compact code, boiling away needless verbose syntax. Removing syntax helps readability, which it turn makes it easier to see the design elements hiding in your code, obscured by necessary but cluttering syntax.


Conclusion to Part 1

In this installment, I covered how expressiveness of language affects readability and the ability to see (and therefore harvest) idiomatic patterns in your code — the real design elements you want to find and reuse. I discussed how several of the Gang of Four patterns can be implemented in more expressive languages (like Groovy). In Part 2, I'll continue discussing the intersection of expressiveness and language, ways in which formal design patterns can be better expressed, and how certain languages give you capabilities that literally don't exist in less expressive languages.

Resources

Learn

Get products and technologies

Discuss

Comments

developerWorks: Sign in

Required fields are indicated with an asterisk (*).


Need an IBM ID?
Forgot your IBM ID?


Forgot your password?
Change your password

By clicking Submit, you agree to the developerWorks terms of use.

 


The first time you sign into developerWorks, a profile is created for you. Information in your profile (your name, country/region, and company name) is displayed to the public and will accompany any content you post, unless you opt to hide your company name. You may update your IBM account at any time.

All information submitted is secure.

Choose your display name



The first time you sign in to developerWorks, a profile is created for you, so you need to choose a display name. Your display name accompanies the content you post on developerWorks.

Please choose a display name between 3-31 characters. Your display name must be unique in the developerWorks community and should not be your email address for privacy reasons.

Required fields are indicated with an asterisk (*).

(Must be between 3 – 31 characters.)

By clicking Submit, you agree to the developerWorks terms of use.

 


All information submitted is secure.

Dig deeper into Java technology on developerWorks


static.content.url=http://www.ibm.com/developerworks/js/artrating/
SITE_ID=1
Zone=Java technology
ArticleID=416869
ArticleTitle=Evolutionary architecture and emergent design: Language, expressiveness, and design, Part 1
publish-date=07282009