Pointillism meets pixelation

Using the Java 2D API to animate art

Paul Reiners shows how to animate images in unexpected and artistic ways using the Java™ 2D API and cellular automata. In the process, he demonstrates implementation of an image operator in Java code and explains cyclic space, a type of 2D cellular automaton. You can use the ideas from this article to create your own image operators and artistic programs using Java technology.

Paul D. Reiners (paul.reiners@gmail.com), Software Engineer, EMC

Paul ReinersPaul Reiners is a Sun Certified Java Programmer and Developer. He is the developer of several open source programs, including Automatous Monk, Twisted Life, and Leipzig. Reiners received his M.S. in Applied Mathematics (Theory of Computation) at the University of Illinois at Urbana-Champaign in May, 1991. He lives in Minnesota and in his spare time practices the electric bass and plays in an employee jazz band.



18 November 2008

Also available in Chinese Russian Japanese

This article explains how to write a custom Java 2D image-processing class by implementing the BufferedImageOp interface. It uses a 2D cellular automaton (CA) — cyclic space — in constructing the image-processing application. The CA "operates" on an image (for example, a JPEG file), causing the image to transform in interesting ways over time. I hope the article will open your eyes to the possibility of writing a whole new class of image-processing applications.

2D cellular automata

2D cellular automata are composed of cells in a 2D grid commonly called the universe. Each cell has a state, which you can think of as an integer between 0 and n. Listing 1 shows how to declare a cellular automaton universe in Java code:

Listing 1. Definition of TwoDCellularAutomaton.universe
protected int[][] universe;

At each tick of an imaginary clock, cells update their states simultaneously. The new state of a cell depends on that cell's current state and the current states of its neighbor cells, using a specified rule. Listing 2 updates the universe at each tick of the clock:

Listing 2. TwoDCellularAutomaton class (partial listing)
public void update() {
    int[][] newUniverse = new int[rowCount][colCount];
    for (int row = 0; row < rowCount; row++) {
        for (int col = 0; col < colCount; col++) {
            newUniverse[row][col] = updateCell(row, col);
        }
    }
    for (int row = 0; row < rowCount; row++) {
        for (int col = 0; col < colCount; col++) {
            universe[row][col] = newUniverse[row][col];
        }
    }
}

protected abstract int updateCell(int row, int col);

The rule that governs how an individual cell is updated is different for different types of CA. Its definition is delegated to subclasses.

Cyclic space

Cyclic space was discovered by David Griffeath of the math department of the University of Wisconsin at Madison and popularized by A. K. Dewdney in a column in Scientific American (see Resources).

In cyclic space, each cell can have one of n states. Each cell's initial state is usually defined to be random — that is, a random number between 0 and n - 1 (inclusive). A cell's neighbors are defined to be the von Neumann neighborhood: the four cells above, below, to the left, and to the right of the cell.

Listing 3 defines a cell's von Neumann neighborhood by giving the difference between the coordinates of each of the cell's neighbors and the cell itself:

Listing 3. Definition of TwoDCellularAutomaton.VON_NEUMANN_NEIGHBORHOOD
protected static final int[][] VON_NEUMANN_NEIGHBORHOOD = { { -1, 0 },
        { 1, 0 }, { 0, -1 }, { 0, 1 } };

Cyclic space is defined by the following rule:

If a cell in state k has a neighbor in state k + 1, then that cell has a new state of k + 1 at the next tick of the clock. Otherwise, the cell's state remains the same.

This rule is cyclic. So, if a cell is in state n - 1 and has a state 0 neighbor, it has state 0 at the next clock tick.

ConvolveOp is a cellular automaton — almost

The Java 2D API's ConvolveOp class represents a spatial convolution: Each destination pixel's color is determined by combining the colors of the corresponding source pixel and its neighbors.

Sound familiar? That's almost the same thing as a 2D cellular automaton, but not quite. For example, the states (colors) are continuous rather than discrete. (That's not completely true: The number of RGB values is finite, but it's close enough to true.) This makes it more like continuous automata. Also, you don't have the fine-grained control of the new state based on the current states of the cell and its neighbors that you have with CA.

For these reasons, you couldn't define cyclic space, for example, with a ConvolveOp, but it's still interesting. It's another way of looking at ConvolveOp.

This simple rule leads to unexpected and complex behavior. Listing 4 implements the rule for updating a cell in cyclic space:

Listing 4. Definition of CyclicSpace.updateCell(int, int)
protected int updateCell(int row, int col) {
    int[] neighborStates = getNeighborStates(row, col, neighborhood);
    int currentState = universe[row][col];
    for (int i = 0; i < neighborStates.length; i++) {
        int neighborState = neighborStates[i];
        if (neighborState == (currentState + 1) % n) {
            return neighborState;
        }
    }

    return currentState;
}

As I've said, the initial state of a cyclic space universe is random. Cells get 'eaten' by 'bigger' cells and eventually cycle back to state 0 again. As this happens, regions self-organize and spread, becoming waves. Eventually, a stable pattern of waves emerges. These waves move diagonally across the universe and look a little like pinwheels.


Creating an image operator

The java.awt.image.BufferedImageOp interface allows you to create your own image operator (also called a filter). This article is concerned with only one method in BufferedImageOp:

BufferedImage filter(BufferedImage src, BufferedImage dest)

src and dest are 2D grids of pixels. When implementing this method, you can build up dest from src any way you want. The usual approach is to iterate through the pixels in src and create the corresponding pixels in dest according to some rule. This is what I'll do in my image-processing application, which I call Seurat after the famous French painter Georges-Pierre Seurat (see Download for the full sample code).

Seurat application

It might have occurred to you that there is a natural mapping between pixels of an image and cells in a CA. They both exist in 2D grids, and each has state, which in the case of a pixel is its red-green-blue (RGB) values. I'll exploit this natural mapping in my implementation of filter(BufferedImage src, BufferedImage dest). For each pixel in src, I'll combine that pixel's RGB values with the state of the corresponding cell in a CA according to some rule to create new RGB values for the corresponding pixel in dest. This rule will define a filter.

Listing 5 shows how to iterate over all the pixels in src and construct the pixels in dest. The abstract getNewRGB(Color) method is defined by individual filters. It calculates and returns the filtered RGB values for an input color.

Listing 5. CellularAutomataFilter class (partial listing)
public BufferedImage filter(BufferedImage src, BufferedImage dest) {
    if (dest == null)
        dest = createCompatibleDestImage(src, null);

    int srcHeight = src.getHeight();
    int srcWidth = src.getWidth();
    for (int y = 0; y < srcHeight; y++) {
        for (int x = 0; x < srcWidth; x++) {
            // Get the pixel in the original image.
            int origRGB = src.getRGB(x, y);
            Color origColor = new Color(origRGB);

            // Get the new RGB values from the filter.
            int[] newRGB = getNewRGB(origColor);

            // Convert the pixel coordinates to the CA coordinates by
            // scaling.
            int cAY = (int) ((double) twoDCellularAutomaton
                    .getRowCount()
                    / (double) srcHeight * y);
            int cAX = (int) ((double) twoDCellularAutomaton
                    .getColCount()
                    / (double) srcWidth * x);
            // Get the state of the corresponding CA cell.
            int state = twoDCellularAutomaton.getState(cAY,
                    cAX);
            // Determine the weight of the filtered RGB values depending on
            // the state.
            double filterProportion = (double) state
                    / (double) twoDCellularAutomaton.getN();

            // Determine the weighted average between the filtered RGB
            // values and the image RGB values.
            int weightedRed = (int) Math.round(newRGB[0] * filterProportion
                    + origColor.getRed() * (1.0 - filterProportion));
            int weightedBlue = (int) Math.round(newRGB[1]
                    * filterProportion + origColor.getBlue()
                    * (1.0 - filterProportion));
            int weightedGreen = (int) Math.round(newRGB[2]
                    * filterProportion + origColor.getGreen()
                    * (1.0 - filterProportion));

            // Set the pixel in dest with this weighted average.
            dest.setRGB(x, y, new Color(weightedRed, weightedBlue,
                    weightedGreen).getRGB());
        }
    }

    return dest;
}

abstract protected int[] getNewRGB(Color color);

You might have noticed that I don't use a one-to-one mapping between the pixels in the image and the cells in the CA. The CA is more coarse-grained (in most cases, at least). I originally did this for performance reasons. However, using different sizes of CA universes can give interesting pixelation effects.

Listing 6 shows one particular implementation of getNewRGB(Color). It computes what I call the "RGB complement," which is not the true color complement. (A filter computing the true color complement would also be an interesting filter, but not quite so straightforward to code.)

Listing 6. RGBComplementFilter class (partial listing)
protected int[] getNewRGB(Color c) {
    int red = c.getRed();
    int newRed = getComplement(red);
    int green = c.getGreen();
    int newGreen = getComplement(green);
    int blue = c.getBlue();
    int newBlue = getComplement(blue);

    return new int[] { newRed, newGreen, newBlue };
}

private int getComplement(int colorVal) {
    // 'Reflect' colorVal across the mid-point 128.
    int maxDiff = colorVal >= 128 ? -colorVal : 255 - colorVal;
    // Divide by 2.0 to make the effect more subtle. Could also just use
    // maxDiff for a more garish effect.
    int diff = (int) Math.round(maxDiff / 2.0);
    int newColorVal = colorVal + diff;

    return newColorVal;
}

I could have generalized getNewRGB(Color) to pass in not only the color of the pixel to be transformed, but also the colors of the eight neighboring pixels. This would allow me to create certain other effects, such as blurring or edge detection, where the filtered color of the pixel depends on the colors of its neighbors. This would be a nice enhancement.

Finally, I animate the image by updating it in tandem with the tick of the CA clock. I use a javax.swing.Timer for this. (This is a simple way to animate the changing image, but not the best way. Jonathan Knudsen's book Java 2D Graphics presents a better, more complicated way to create animations; see Resources.)

Running Seurat

Figure 1 is a photograph of Georges Seurat's 1884 pointillistic masterpiece "A Sunday Afternoon on the Island of La Grand Jatte":

Figure 1. Georges Seurat's "A Sunday Afternoon on the Island of La Grand Jatte"
Sample figure containing an image

Now I'll run the Seurat application on the Seurat painting using the RGB-complement filter. Figure 2 shows the filtered painting when cyclic space is in its initial, random state:

Figure 2. Painting filtered with cyclic space in disorganized random state
Sample figure containing an image

Figure 3 shows the filtered painting when the cyclic space is starting to self-assemble into patterns of order, but still with much randomness:

Figure 3. Painting filtered with cyclic space in intermediate state
Sample figure containing an image

Figure 4 shows the filtered painting in its final, steady state:

Process art versus algorithmic art

I read a lot about Jackson Pollock in preparation for writing Seurat (which I first called "Blue Poles"). I kept running across the term process art. I naively assumed it meant algorithmic or rule-based art, whereby an artist devises a set of rules and runs them on some set of initial conditions, thereby generating an artistic artifact, such as a painting. But I found that its usual meaning in the art world is not quite the same. According to the Guggenheim Collection Glossary:

"Process art emphasizes the 'process' of making art (rather than any predetermined composition or plan) and the concepts of change and transience, as elaborated in the work of such artists as Lynda Benglis, Eva Hesse, Robert Morris, Bruce Nauman, Alan Saret, Richard Serra, Robert Smithson, and Keith Sonnier."

So, algorithmic or rule-based art can, if the intermediate states of the algorithm (or program) are artistically interesting, be considered a form of process art while the algorithm is in the process of running.

As programmers, we're uniquely suited to create process art or to help artists create it.

Figure 4. Painting filtered with cyclic space in steady state
Sample figure containing an image

The still pictures don't really do the filter/CA justice, though. (After all, this application was written to animate still images.) I encourage you to run the actual Java applet to see the filter/CA in action (see Resources for a link to a live demo).

Aesthetic considerations

Some people might consider running an image-filtering application on a great painting such as "A Sunday Afternoon on the Island of La Grand Jatte" a desecration. I certainly have a lot of sympathy for this view. I'm simply using this painting as an example. My main goal was to show how images can be animated in interesting and complex ways using a simple cellular automaton, and a familiar painting makes a good example.

I've run Seurat on many types of paintings, with interesting results on both abstract and representational art. However, it seems to work best with modern art — in particular, pop art. Very interesting patterns emerge when you run it on Jasper Johns' "Flag" paintings, for example. The diagonal lines of cyclic space work well against the straight lines in the "Flag" paintings. Seurat also yields interesting results when it's run on Jackson Pollock's drip paintings. For example, as the cyclic space CA flows over Pollock's "Blue Poles," it hides, reveals, and again hides details of this intricate painting, concentrating your attention on different parts at different times. Photographs work well too. I've enjoyed running it on the surreal photos of Ralph Eugene Meatyard.

You can vary three dimensions when running an application like Seurat: the type of 2D cellular automaton, the filter, and the original image. In this article, I've only used cyclic space, but other types of 2D cellular automata, such as Hodgepodge, could be used. In programming a filter, you're limited only by your imagination. I've mainly experimented with filters that operate on color, but it would be interesting to experiment with filters that change spatial relationships in the image. For example, you could program a filter that warps the surface of the image, creating an effect like that used on the cover of the Beatles' Rubber Soul album. Finally, you can use whatever images you want — photographs, for example. For a given image, different combinations of filters and types of CA could produce better or worse results. I hope this article will encourage you to experiment in this area.


Acknowledgment

I'd like to thank Julia Braswell for getting me interested in the visual arts.


Download

DescriptionNameSize
Java files for this articlej-j2D.zip19KB

Resources

Learn

  • The Magic Machine: A Handbook of Computer Sorcery (A. K. Dewdney, W. H. Freeman, 1990): This book, a collection of Dewdney's "Computer Recreations" columns in Scientific American, includes a chapter on cyclic space and another on Hodgepodge. When the columns first appeared in the 1980s, I programmed all their algorithms in AmigaBASIC on my Amiga 500.
  • Primordial Soup Kitchen: Learn more about cellular automata from David Griffeath, who discovered cyclic space.
  • Seurat: Try out the Seurat Java applet.
  • Java 2D Graphics (Jonathan Knudsen, O'Reilly Media, 1999): This book is an excellent introduction to the subject.
  • Java 2D API: Find documentation, examples, and other resources for Java 2D.
  • Art: The Way It Is, 3rd ed. (John Adkins Richardson, Prentice Hall and Harry N. Abrams, 1973): This is the book to read if you want to learn more about Seurat and other artists.
  • "Cellular automata and music" (Paul Reiners, developerWorks, May 2004): Read about using the Java language and cellular automata for algorithmic music composition.
  • "Introduction to Java 2D" (Mitch Goldstein, developerWorks, July 2002): This tutorial offers a step-by-step guide to the advantages of advanced drawing, text layout, and image manipulation that Java 2D brings to GUI programming.
  • "2D animation with image-based paths" (Barry Feigenbaum and Tom Brunet, developerWorks, January 2004): Combine lossless images, Swing technology, and the authors' own Java-based animation engine to generate movement sequences for fixed objects in 2D animation.
  • "Creating Java2D composites for rollover effects" (Joe Winchester and Renee Schwartz, developerWorks, September 2002): Learn more about image creation and manipulation using the Java 2D API.
  • Browse the technology bookstore for books on these and other technical topics.
  • developerWorks Java technology zone: Find hundreds of articles about every aspect of Java programming.

Discuss

Comments

developerWorks: Sign in

Required fields are indicated with an asterisk (*).


Need an IBM ID?
Forgot your IBM ID?


Forgot your password?
Change your password

By clicking Submit, you agree to the developerWorks terms of use.

 


The first time you sign into developerWorks, a profile is created for you. Information in your profile (your name, country/region, and company name) is displayed to the public and will accompany any content you post, unless you opt to hide your company name. You may update your IBM account at any time.

All information submitted is secure.

Choose your display name



The first time you sign in to developerWorks, a profile is created for you, so you need to choose a display name. Your display name accompanies the content you post on developerWorks.

Please choose a display name between 3-31 characters. Your display name must be unique in the developerWorks community and should not be your email address for privacy reasons.

Required fields are indicated with an asterisk (*).

(Must be between 3 – 31 characters.)

By clicking Submit, you agree to the developerWorks terms of use.

 


All information submitted is secure.

Dig deeper into Java technology on developerWorks


static.content.url=http://www.ibm.com/developerworks/js/artrating/
SITE_ID=1
Zone=Java technology
ArticleID=352308
ArticleTitle=Pointillism meets pixelation
publish-date=11182008