This article introduces CruiseControl, open source software you can use to automate the build and unit-testing process for multideveloper software projects. I'll explain why automatic builds are essential for successful development teams and take you step-by-step through configuration, installation, and maintenance of a continuous-integration system running CruiseControl.

Why automatic builds?

A common practice these days is to use a version-control system such as CVS or Subversion (see Resources). With multiple developers working on the same system, such coordination is essential. Another increasingly common practice is to write unit tests and run them as part of the build. The Maven build tool, for example, runs JUnit unit tests as part of its normal build process (see Resources). But adopting these practices is just the start. They form the basis of many of the lightweight and pragmatic software-development methodologies that have been developed over the past few years.

When a number of developers work on a single project it's important to make sure that the latest version of the code in the version-control system (the trunk) will always build. This is a good practice for projects with closed development teams; when developers periodically synchronize their working areas with the trunk, a source tree that won't build will hold up development until it can be fixed. For open source projects, keeping the trunk working is vital. Potential new developers could check out the code at any time, but if it won't build, they are likely to be dissuaded from contributing.

Extreme Programming (XP) methodology advocates continuous integration. Developers integrate their code into the trunk as often as possible -- typically every few hours -- while making sure that all the unit tests pass. Other agile methodologies echo this advice.

To adopt continuous integration and unit testing, you need your team to buy into the method and practices, but often that's not enough. The practices rely on manual steps -- integrating code, running tests, and checking code in at the right times -- that can lead to mistakes. Having an automated system build your code and run the unit tests can be a more reliable solution.

Back to top

Configuring a build server

The rest of this article guides you through the steps involved in configuring a build server for your Java projects using CruiseControl, a piece of software that manages the automatic build process (see Resources). You need a reliable machine with plenty of spare disc space, but it doesn't need to be particularly fast. (You want regular builds, but it doesn't matter whether they take 2 minutes or 20.) The server you'll build is based on Fedora Core 4, a community-developed Linux distribution sponsored by Red Hat (see Resources), so a bit of Unix experience is assumed. The main tasks this article covers are:

• Initial configuration of the system and setting up a user account to run CruiseControl
• Installing CruiseControl and configuring a first build
• Making CruiseControl run all the time
• Simplifying the CruiseControl configuration
• Setting up an optional browser-based interface for monitoring CruiseControl builds

Back to top

Initial configuration

The first order of business is to make sure that all the software you need for basic Java development is installed on your system. Fedora Core 4 includes a Java tool chain based on gcj, the Java compiler from the GNU Compiler Collection (gcc) project, but for compatibility reasons, it's probably wise to install a JDK from either IBM or Sun. The tidiest approach is to build and install your own Java RPMs by following the instructions at jpackage.org (see Resources). The xerces-j2 package that Fedora Core 4 ships was built incorrectly, preventing the Xalan XSLT implementation from working. So you also need to install the updated xerces-j2 packages from the Fedora development repository (see Resources).

You'll also use some other pieces of software:

• XMLStarlet, a useful command-line program for manipulating XML documents (see Resources). You'll use it later on to simplify maintenance of the CruiseControl configuration file.
• CVS and Subversion: You need these tools installed to download updates for the source trees you build. Fortunately, both tools are included with Fedora Core 4.

You must be logged in as root to perform these steps. First, here are the RPMs you should have on the system:

[root@fcvm ~]# ls
java-1.4.2-sun-1.4.2.08-1jpp.i586.rpm
java-1.4.2-sun-alsa-1.4.2.08-1jpp.i586.rpm
java-1.4.2-sun-demo-1.4.2.08-1jpp.i586.rpm
java-1.4.2-sun-devel-1.4.2.08-1jpp.i586.rpm
java-1.4.2-sun-fonts-1.4.2.08-1jpp.i586.rpm
java-1.4.2-sun-jdbc-1.4.2.08-1jpp.i586.rpm
java-1.4.2-sun-plugin-1.4.2.08-1jpp.i586.rpm
java-1.4.2-sun-src-1.4.2.08-1jpp.i586.rpm
xerces-j2-2.6.2-5jpp_2fc.i386.rpm
xerces-j2-demo-2.6.2-5jpp_2fc.i386.rpm
xerces-j2-javadoc-apis-2.6.2-5jpp_2fc.i386.rpm
xerces-j2-javadoc-dom3-2.6.2-5jpp_2fc.i386.rpm
xerces-j2-javadoc-impl-2.6.2-5jpp_2fc.i386.rpm
xerces-j2-javadoc-other-2.6.2-5jpp_2fc.i386.rpm
xerces-j2-javadoc-xni-2.6.2-5jpp_2fc.i386.rpm
xerces-j2-scripts-2.6.2-5jpp_2fc.i386.rpm
xmlstarlet-1.0.1-1.i586.rpm
[root@fcvm ~]# 

Install the Java, Xerces, XMLStarlet, and Subversion packages:

[root@fcvm ~]# rpm -ivh java-1.4.2-sun-1.4.2.08-1jpp.i586.rpm \
java-1.4.2-sun-alsa-1.4.2.08-1jpp.i586.rpm \
java-1.4.2-sun-devel-1.4.2.08-1jpp.i586.rpm \
java-1.4.2-sun-fonts-1.4.2.08-1jpp.i586.rpm \
java-1.4.2-sun-plugin-1.4.2.08-1jpp.i586.rpm \
java-1.4.2-sun-src-1.4.2.08-1jpp.i586.rpm
Preparing...                ################################# [100%]
   1:java-1.4.2-sun         ################################# [ 17%]
   2:java-1.4.2-sun-alsa    ################################# [ 33%]
   3:java-1.4.2-sun-devel   ################################# [ 50%]
   4:java-1.4.2-sun-fonts   ################################# [ 67%]
   5:java-1.4.2-sun-plugin  ################################# [ 83%]
   6:java-1.4.2-sun-src     ################################# [100%]
[root@fcvm ~]# java -version
java version "1.4.2_08"
Java(TM) 2 Runtime Environment, Standard Edition (build 1.4.2_08-b03)
Java HotSpot(TM) Client VM (build 1.4.2_08-b03, mixed mode)
[root@fcvm ~]# rpm -Uvh xerces-j2-2.6.2-5jpp_2fc.i386.rpm \
xerces-j2-demo-2.6.2-5jpp_2fc.i386.rpm \
xerces-j2-javadoc-apis-2.6.2-5jpp_2fc.i386.rpm \
xerces-j2-javadoc-dom3-2.6.2-5jpp_2fc.i386.rpm \
xerces-j2-javadoc-impl-2.6.2-5jpp_2fc.i386.rpm \
xerces-j2-javadoc-other-2.6.2-5jpp_2fc.i386.rpm \
xerces-j2-javadoc-xni-2.6.2-5jpp_2fc.i386.rpm \
xerces-j2-scripts-2.6.2-5jpp_2fc.i386.rpm
Preparing...                ################################# [100%]
   1:xerces-j2              ################################# [ 13%]
   2:xerces-j2-demo         ################################# [ 25%]
   3:xerces-j2-javadoc-apis ################################# [ 38%]
   4:xerces-j2-javadoc-dom3 ################################# [ 50%]
   5:xerces-j2-javadoc-impl ################################# [ 63%]
   6:xerces-j2-javadoc-other################################# [ 75%]
   7:xerces-j2-javadoc-xni  ################################# [ 88%]
   8:xerces-j2-scripts      ################################# [100%]
[root@fcvm ~]# rpm -ivh xmlstarlet-1.0.1-1.i586.rpm
Preparing...                ################################# [100%]
   1:xmlstarlet             ################################# [100%]
[root@fcvm ~]# yum install subversion
[...]
Installed: subversion.i386 0:1.2.3-2.1
Complete!
[root@fcvm ~]# 

You also need to create a new user account on the server to own the files and processes involved in running CruiseControl:

[root@fcvm ~]# useradd cruise
[root@fcvm ~]# su - cruise
[cruise@fcvm ~]$ pwd
/home/cruise
[cruise@fcvm ~]$ 

Finally, because some of the projects you'll build use the Maven build tool, you need to download it, install it, and set the appropriate environment variables (see Resources). (JAVA_HOME should be set to /usr/lib/jvm/java.) I use the convention of placing external packages such as Maven and CruiseControl into a directory called pkg. Full installation instructions are available on the Maven Web site, so I won't cover this step in detail:

[cruise@fcvm ~]$ mkdir pkg
[cruise@fcvm ~]$ cd pkg
[cruise@fcvm pkg]$ [install Maven]

Back to top

Installing CruiseControl

The next job is to download CruiseControl (see Resources) and install it in the pkg directory:

[cruise@fcvm pkg]$ wget -q http://heanet.dl.sourceforge.net/\
sourceforge/cruisecontrol/cruisecontrol-2.2.1.zip
[cruise@fcvm pkg]$ unzip cruisecontrol-2.2.1.zip
Archive:  cruisecontrol-2.2.1.zip
   creating: cruisecontrol-2.2.1/
   creating: cruisecontrol-2.2.1/contrib/
[...]
  inflating: cruisecontrol-2.2.1/reporting/jsp/webcontent/xsl/testdeta
ils.xsl
  inflating: cruisecontrol-2.2.1/reporting/jsp/webcontent/xsl/unittest
s.xsl
[cruise@fcvm pkg]$ rm cruisecontrol-2.2.1.zip
[cruise@fcvm pkg]$ 

You don't need to build CruiseControl, because the distribution includes a prebuilt JAR file.

Now you can get your first automated build working. You'll use the XStream project source tree as an initial example (see Resources). Later, you'll learn how to add more projects from local and remote source-code repositories. CruiseControl reads information about projects it should build from a file called config.xml in the directory where it starts up. In your installation, this is the home directory, /home/cruise. Listing 1 shows the content of the simple config.xml file that you'll start with. Create it by copying the text in Listing 1 into a new file:

<?xml version="1.0"?>
<cruisecontrol>
  <project name="xstream" buildafterfailed="false">
    <listeners>
      <currentbuildstatuslistener
           file="log/build/xstream/status.txt"/>
    </listeners>
    <modificationset>
      <filesystem folder="/home/cruise/force-build/xstream"/>
      <svn LocalWorkingCopy="src/xstream"/>
    </modificationset>
    <schedule interval="3600">
      <ant antscript="/usr/bin/ant"
           uselogger="true"
           antworkingdir="src/xstream"
           multiple="1"
           target="library"/>
      <ant antscript="/usr/bin/ant"
           uselogger="true"
           antworkingdir="src/xstream"
           multiple="5"
           target="clean library"/>
    </schedule>
    <log dir="log/build/xstream"/>
    <dateformat format="dd/MM/yyyy HH:mm:ss"/>
  </project>
</cruisecontrol>

The configuration file gives CruiseControl three main pieces of information for each project it is to build:

• How to build the project, specified in the <schedule> element:
  • Try building the project every 3,600 seconds (that is, every hour).
  • Use Ant to drive the build process.
  • On every fifth build, clean the source tree of build artifacts (class files and the like from previous builds).
• How to detect when the project should be built, specified in the <modificationset> element:
  • Use Subversion (svn) to check if the local working copy of the source tree is out of date. (You don't need to build the project if the source code hasn't changed.)
  • Check the timestamp of a file called xstream in the force-build directory. This lets you manually force the next scheduled build to happen even if the source tree hasn't changed. (I'll talk about the times you might need this manual override later in this article.)
• What to do with the results from the build, specified in the <listeners> and <log> elements:
  • Put the output from the build process in timestamped files in the log/build/xstream directory.
  • Write the overall status of the build into a file in that directory.

Now you need to check out the XStream source tree from the project's Subversion repository. To be consistent, check out all your source trees as subdirectories of the /home/cruise/src directory, and put the XStream source in src/xstream, as specified in the config.xml file:

[cruise@fcvm pkg]$ cd
[cruise@fcvm ~]$ mkdir src
[cruise@fcvm ~]$ cd src
[cruise@fcvm src]$ svn co https://svn.codehaus.org/\
xstream/trunk/xstream
A    xstream/LICENSE.txt
A    xstream/continuous-integration.xml
[...]
A    xstream/build.xml
 U   xstream
Checked out revision 614.
[cruise@fcvm src]$ 

Then, set up the force-build subdirectory:

[cruise@fcvm src]$ cd ..
[cruise@fcvm ~]$ mkdir force-build
[cruise@fcvm ~]$ touch force-build/xstream
[cruise@fcvm ~]$ 

This last step is necessary because CruiseControl will refuse to start up if the file you specified in the config.xml file's <filesystem> element does not already exist.

It's possible that the build tools aren't working correctly, or that some dependencies you're not aware of are missing. So at this point, it's worth doing a manual check to ensure that the XStream source tree will build successfully:

[cruise@fcvm ~]$ cd src/xstream
[cruise@fcvm xstream]$ ant library
Buildfile: build.xml

compile:
    [mkdir] Created dir: /home/cruise/src/xstream/build/java
     [echo] Java version used for compile: 1.4.2_08
    [javac] Compiling 150 source files to /home/cruise/src/xstream/bui
ld/java

[...]

library:

BUILD SUCCESSFUL
Total time: 1 minute 44 seconds
[cruise@fcvm xstream]$ 

Also, as you add new projects, you'll need to find out the names of the targets that are used to build the source and to clean it of built artifacts. You must put that information in the config.xml file.

Now you should be ready to let CruiseControl perform this build automatically. Just start CruiseControl, sit back, and wait:

[cruise@fcvm xstream]$ cd
[cruise@fcvm ~]$ java -jar \
pkg/cruisecontrol-2.2.1/main/dist/cruisecontrol.jar
[cc]Aug-24 20:09:31 Main          - CruiseControl Version 2.2.1
[cc]Aug-24 20:09:32 trolController- projectName = [xstream]
[cc]Aug-24 20:09:32 trolController- No previously serialized project f
ound: /home/cruise/xstream
[cc]Aug-24 20:09:32 Project       - Project xstream:  reading settings
 from config file [/home/cruise/config.xml]
[cc]Aug-24 20:09:32 BuildQueue    - BuildQueue started
[cc]Aug-24 20:09:32 Project       - Project xstream starting
[cc]Aug-24 20:09:32 Project       - Project xstream:  idle
[cc]Aug-24 20:09:32 Project       - Project xstream started
[cc]Aug-24 20:09:32 Project       - Project xstream:  next build in 1 
hours
[cc]Aug-24 20:09:32 Project       - Project xstream:  waiting for next
 time to build
[cc]Aug-24 21:09:33 Project       - Project xstream:  in build queue
[cc]Aug-24 21:09:33 BuildQueue    - now adding to the thread queue: xs
tream
[cc]Aug-24 21:09:33 Project       - Project xstream:  reading settings
 from config file [/home/cruise/config.xml]
[cc]Aug-24 21:09:33 Project       - Project xstream:  bootstrapping
[cc]Aug-24 21:09:33 Project       - Project xstream:  checking for mod
ifications
[cc]Aug-24 21:09:59 Project       - Project xstream:  No modifications
 found, build not necessary.
[cc]Aug-24 21:09:59 Project       - Project xstream:  Building anyway,
 since build was explicitly forced.
[cc]Aug-24 21:09:59 Project       - Project xstream:  now building
Buildfile: build.xml
[cc]Aug-24 21:11:29 Project       - Project xstream:  merging accumula
ted log files
[cc]Aug-24 21:11:30 Project       - Project xstream:  build successful
[cc]Aug-24 21:11:30 Project       - Project xstream:  publishing build
 results
[cc]Aug-24 21:11:30 Project       - Project xstream:  idle
[cc]Aug-24 21:11:30 Project       - Project xstream:  next build in 1 
hours
[cc]Aug-24 21:11:30 Project       - Project xstream:  waiting for next
 time to build
[stop CruiseControl using Ctrl-C]
[cruise@fcvm ~]$ 

Back to top

Making CruiseControl run all the time

You have CruiseControl running now, but you can't let it run unattended yet. It's running within a terminal window, so the cruise user would need to be permanently logged in to keep it running. The terminal is also the only way you can control the program: You can stop CruiseControl by pressing Ctrl+C and restart it by running the program again. You can't do either of these things remotely unless you use a Virtual Network Computing (VNC) session or something similar. If CruiseControl (or the JVM) crashes, you'll need to restart it manually. And when you reboot the machine, CruiseControl won't restart until you go through the manual steps to establish a new session, create a terminal, and start the program again. You need something to keep CruiseControl running as a service, or daemon, in Unix terminology.

Numerous ways of getting a program to run continuously under Linux are available. Probably the most common approach is to start the program up at system startup time by hooking a suitable script into the init system-initialization process. These scripts can start and stop a program, but they do not automatically restart the program if it fails.

The approach I used was to download and install Daniel J. Bernstein's daemontools (see Resources). This is a small package of programs that take care of starting a set of services and keeping them running. You need to be logged in as root to perform your own installation of daemontools:

[root@fcvm ~]# mkdir -p /package
[root@fcvm ~]# chmod 1755 /package
[root@fcvm ~]# cd /package
[root@fcvm package]# wget -q http://cr.yp.to/\
daemontools/daemontools-0.76.tar.gz
[root@fcvm package]# gunzip daemontools-0.76.tar.gz
[root@fcvm package]# tar -xpf daemontools-0.76.tar
[root@fcvm package]# rm daemontools-0.76.tar
rm: remove regular file 'daemontools-0.76.tar'? y
[root@fcvm package]# 

You must make a slight tweak to the package's C source code before it can build cleanly on Fedora Core 4. Using any text editor, change line 6 of src/error.h from extern int errno; to #include <errno.h>. Here's how you'd do it with ed:

[root@fcvm package]# cd admin/daemontools-0.76
[root@fcvm daemontools-0.76]# ed src/error.h
595
6
extern int errno;
c
#include <errno.h>
.
wq
596
[root@fcvm daemontools-0.76]# 

You can now complete the installation:

[root@fcvm daemontools-0.76]# package/install
Linking ./src/* into ./compile...
Compiling everything in ./compile...
[...]
Creating /service...
Adding svscanboot to inittab...
init should start svscan now.
[root@fcvm daemontools-0.76]# ps -ef | grep svs
root     21160     1  0 16:09 ?        00:00:00 /bin/sh /command/svsca
nboot
root     21162 21160  0 16:09 ?        00:00:00 svscan /service
root     21173 20051  0 16:10 pts/1    00:00:00 grep svs
[root@fcvm daemontools-0.76]# 

daemontools provides a daemon process called svscan that takes care of managing a collection of services. Each service is represented by a directory in the /service directory, so you need to create a directory there for the CruiseControl service. For each subdirectory of /service, svscan starts a child process running the supervise program.

supervise is the program that deals with managing an individual service such as CruiseControl. It starts the service by creating a child process running the run program within the service's subdirectory (/service/cruisecontrol/run, for example). If the child process ever terminates, supervise will restart it. supervise can also stop and restart its child process by sending signals to it.

daemontools also provides two mechanisms to handle logging for the services that it manages. First, the program called readproctitle captures output written to the standard error stream (System.err in the Java world) and copies it into a small buffer that is part of the process title shown by the ps command:

[root@fcvm daemontools-0.76]# ps -ef | grep proctitle
root     25040 25037  0 20:58 ?        00:00:00 readproctitle service 
errors: ..............................................................
......................................................................
......................................................................
......................................................................
......................................................................
..........................................................
root     25047 24006  0 20:59 pts/1    00:00:00 grep proctitle
[root@fcvm daemontools-0.76]# 

The buffer is initialized to contain dots at startup, but they are replaced by error messages as they are generated. This mechanism is fine for small quantities of information such as critical error messages. But the small buffer size makes it unsuitable for larger quantities of log information, and the logged information isn't stored to disk so it's hard to analyze over a period of time. daemontools provides the second mechanism -- the multilog program -- for this kind of bulk logging. It writes lines from its standard input to log files under the control of instructions you enter as command-line arguments. It includes controls for log rotation, keeping a fixed amount of log information so storage doesn't run out. For example, the simple multilog /home/cruise/log command logs information to a file in the /home/cruise/log directory, rotating the log files when the log file's size reaches 99,999 bytes and keeping 10 old log files.

multilog is intended to be managed by supervise just like any other service. In each service directory that svsccan finds, it looks for a subdirectory called log and creates a supervise process to manage execution of the run script within that directory. It also arranges for a pipe to carry the standard output from the main service to the log process's standard input.

So, what do you need to do to let daemontools manage CruiseControl? You must create the directory structure for the service and its multilog partner. And you must create run scripts for each of them and create a directory for the log files. Initially, you'll name the service directory .cruisecontrol. The leading dot causes svscan to ignore the directory, giving you time to set things up before starting the service for the first time:

[cruise@fcvm ~]$ mkdir -p log/cruisecontrol
[cruise@fcvm ~]$ su -
Password: [enter root password]
[root@fcvm ~]# cd /service
[root@fcvm service]# mkdir .cruisecontrol
[root@fcvm service]# cd .cruisecontrol
[root@fcvm .cruisecontrol]# mkdir log
[root@fcvm .cruisecontrol]# 

Then, create a directory called env. You use this directory's contents to set environment variables for CruiseControl and the other processes that it will start. This is where you make sure that JAVA_HOME has a suitable value. And it's where you set the environment variables, such as MAVEN_HOME, needed by the build tools you're using:

[root@fcvm .cruisecontrol]# mkdir env
[root@fcvm .cruisecontrol]# cd env
[root@fcvm env]# echo /usr/lib/jvm/java >JAVA_HOME
[root@fcvm env]# echo /home/cruise/pkg/maven-1.0.2 >MAVEN_HOME
[root@fcvm env]# ls
JAVA_HOME  MAVEN_HOME
[root@fcvm env]# cd ..
[root@fcvm .cruisecontrol]# 

Listing 2 shows the /service/cruisecontrol/run script:

#!/bin/sh
svc=`pwd`
cd /home/cruise
exec 2>&1
exec setuidgid cruise \
        envdir ${svc}/env \
        java -jar pkg/cruisecontrol-2.2.1/main/dist/cruisecontrol.jar

The script is relatively simple. It performs these steps:

1. Saves the name of the service directory (/service/cruisecontrol in this case) for later use.
2. Changes the current directory to /home/cruise.
3. Makes the standard error stream write to the pipe to the multilog process that is already connected to the standard output stream.
4. Starts the JVM to run CruiseControl, running the process as the cruise user and setting up the environment from the files you created in the /service/cruisecontrol/env directory.

Listing 3 shows the /service/cruisecontrol/log/run script, which is even simpler. It runs multilog as the cruise user:

#!/bin/sh
exec setuidgid cruise multilog /home/cruise/log/cruisecontrol

Note that you must use chmod to make both scripts executable. Also, both scripts are careful to use the exec shell command, which replaces one program with another but does not create new processes. This is important because supervise manages only its immediate child process. If you did not use exec, the JVM would start as a child process of the shell that is executing the run script. If supervise were to send a signal to kill its child process, the shell would receive the signal and exit, but the JVM would carry on running and become an orphan. supervise would be unaware of this and might then start a second copy of the daemon -- not what you want.

After the service directory has been set up, you can rename it to remove the leading dot. svscan then starts CruiseControl automatically, and its output should appear in the log file:

[root@fcvm .cruisecontrol]# cd ..
[root@fcvm service]# mv .cruisecontrol cruisecontrol
[root@fcvm service]# cat /home/cruise/log/cruisecontrol/current
[cc]Aug-24 21:45:45 Main          - CruiseControl Version 2.2.1
[cc]Aug-24 21:45:46 trolController- projectName = [xstream]
[cc]Aug-24 21:45:46 Project       - Project xstream:  reading settings
 from config file [/home/cruise/config.xml]
[cc]Aug-24 21:45:47 BuildQueue    - BuildQueue started
[cc]Aug-24 21:45:47 Project       - Project xstream starting
[cc]Aug-24 21:45:47 Project       - Project xstream:  idle
[cc]Aug-24 21:45:47 Project       - Project xstream started
[cc]Aug-24 21:45:47 Project       - Project xstream:  next build in 1 
hours
[cc]Aug-24 21:45:47 Project       - Project xstream:  waiting for next
 time to build
[root@fcvm service]# 

Back to top

Simplifying the CruiseControl configuration

You now have CruiseControl running automatically in a well-controlled environment. Chances are you'll want to add your own projects to the configuration. As you might imagine, all the entries in the config.xml file will look very similar, other than the details of which tools to use to build the project. You can maintain the config.xml file by hand by using copy-and-paste in a text editor, but a less error-prone technique is to generate the config.xml file from a simpler XML document using an XSLT stylesheet. The collection of files that implement this scheme are available to download as a compressed tar file (see Download). Unpack these files into the /home/cruise directory:

[cruise@fcvm ~]$ ls
config.xml         force-build  pkg  xstream.ser
cruisecontrol.log  log          src
[cruise@fcvm ~]$ tar xvzf [...]/simple-cc.tar.gz
meta-config-params.xsl
meta-config.xsl
meta-config.xml
mkconfig
[cruise@fcvm ~]$ ls
config.xml         meta-config-params.xsl  pkg
cruisecontrol.log  meta-config.xml         src
force-build        meta-config.xsl         xstream.ser
log                mkconfig
[cruise@fcvm ~]$ 

The simplified configuration file is named meta-config.xml. This file is transformed using the meta-config.xsl stylesheet to produce CruiseControl's config.xml file. A simple script called mkconfig performs the transformation using the XMLStarlet tool that you installed earlier. Run mkconfig to regenerate CruiseControl's config.xml file.

Build monitoring

The simple CruiseControl configuration that you started with was sufficient to get your first build to work, but it doesn't do much to help you monitor the build process. The most common requirement is for the results of the integration build to be e-mailed to the relevant developers. The meta-config.xsl stylesheet generates a configuration that will send an e-mail message, but to do so it requires some information about your local environment. It must know a number of variables that are read from the meta-config-params.xsl file; you should alter this file accordingly before you start. The settings in the file are as follows:

• home: The home directory of the build process. The default should be okay if you've used the directory layout described in this article.
• cruisecontrol-home: The directory that the CruiseControl distribution was unpacked into. Again, the default should be okay.
• ant-home: The directory that Ant is installed into. To use the copy of Ant that ships with Fedora Core 4, this should be /usr.
• maven-home: The directory where Maven is installed if you need to use it. The default value assumes that you have unpacked it into the /home/cruise/pkg directory.
• return-address: The return e-mail address for the build e-mails from CruiseControl.
• return-name: The name for the return address in the the build e-mails.
• developers-address: An e-mail address that should always get a copy of the build e-mail, in addition to the developers who committed since the last successful build.

CruiseControl is quite flexible about whom it should e-mail when a build succeeds or fails. The configuration you're using here will send e-mail to each developer who has committed a change to the version-control system since the last build occurred. Your continuous-integration build might also include open source projects that are developed elsewhere (I'll refer to them as remote projects), in which case you probably don't want the build system e-mailing the project's developers when they break something. In this case, you can send the build e-mail to an address that would typically be a mailing list that members of your team can subscribe to if they always want to know the build's status. This lets team leaders find out as soon as possible that the build has been broken.

Listing 4 shows the general syntax of the meta-config.xml file:

<projects>
  <project name="project-name" [interval="seconds"]>
    <svn/>|<cvs/>
    <ant/>|<maven/>
    <clean>goals or targets to clean source tree</clean>
    <build>goals or targets to build</build>
    [<srcdir>source directory</srcdir>]
    [<remote-project/>]
    [<repo-dependency>groupId</repo-dependency>*]
    [<srcdir-dependency>project-name</srcdir-dependency>*]
    [<modificationset>CruiseControl elements</modificationset>]
  </project>*
</projects>

The configuration file is basically a list of <project> elements. Each project has a name attribute. The optional interval attribute overrides the default CruiseControl build interval of five minutes. You should increase the build interval for remote projects to reduce the load on their version-control repositories.

Using either the <svn/> or <cvs/> empty element, each project must specify the version-control tool that should be used to update its source tree. It must also specify the build tool to be used, either <ant/> or <maven/>. The project must also contain two elements that say which targets (or goals for Maven) should be used to clean and build the source tree. For Maven, typical values might be <clean>clean</clean> and <build>jar:install-snapshot</build>. For Ant, you would need to examine the build.xml file to find the target names.

A project's source is assumed to be in a directory in /home/cruise/src named after the project, so a project whose name attribute is my-project would have its source in /home/cruise/src/my-project. Some projects have large source trees with subdirectories that can be built separately; to handle these cases, a <project> element can contain an optional <srcdir> element that specifies a specific subdirectory of the /home/cruise/src directory. For example:

<project name="my-utils">
  <srcdir>big-project/my-utils</srcdir>
  ...

The default CruiseControl behavior is to e-mail everyone who checked in changes since the last build. If you're pulling source code from a remote version-control repository, add the <remote-project/> element. This causes e-mail to be sent to the developers-address from the meta-config.xsl file.

Dependencies among projects

CruiseControl doesn't know anything about dependencies among projects. You can have one project that produces a JAR file containing a collection of utility classes that many of your other projects depend on, but CruiseControl remains blissfully unaware of this relationship unless you explain the relationship to it. You can make changes to the utility-classes project that cause it to be rebuilt, but the projects that depend on it will not be rebuilt and tested against the new version of the utility classes. This would limit the value of your integration testing, so some solutions to the problem are available.

The main tool that CruiseControl provides for this purpose is the <filesystem> element. You can include this element in a project's <modificationset> section so that the project will be rebuilt when some area of the filesystem is modified. The initial config.xml file in Listing 1 uses this approach to trigger a rebuild when a file in the force-build directory was modified. All projects will create or update artifacts in some location in the filesystem. (For example, the utility-classes project will update the JAR file that it produces as the result of a build.) You can use these locations in <filesystem> elements that will trigger building the projects that depend on those artifacts.

Ant allows a large amount of flexibility in the way that a project is built, so it is impossible to state with any certainty which areas of the filesystem a project will update when it has been rebuilt. The only approach to take here is to examine the build.xml file for each project and find out where it places its build artifacts. You can then add appropriate <filesystem> elements to the projects that depend on the artifacts. The simplified meta-config.xml file allows a <modificationset> element that can contain any CruiseControl elements. They will be copied across into the config.xml file. For example, a project that depends on XStream might include the following:

<project name="my-project">
  [...]
  <modificationset>
    <filesystem
          folder="/home/cruise/src/xstream/xstream-SNAPSHOT.jar"/>
  </modificationset>
</project>

Maven imposes a common build process on each project, so you can provide some common rules for specifying dependencies among Maven projects. A project can specify a dependency on artifacts in the Maven repository that are created by a specified group. More precisely, including <repo-dependency>classworlds</repo-dependency> causes a project to be rebuilt if any file under /home/cruise/.maven/repository/classworlds has changed. Assuming that your classworlds build is installing the produced JAR file in the local Maven repository, any projects that include this element will automatically be rebuilt.

A project can also specify a dependency on another project's build output. Including <srcdir-dependency>classworlds</srcdir-dependency> will cause a project to be rebuilt if any file under ${srcdir}/target has changed, where ${srcdir} is the source directory for the named project.

Adding a project to the build

Here are the steps for adding a new project to your continuous-integration build:

1. As the cruise user, check out the source code into the /home/cruise/src directory.
2. Check that you can build the source tree manually.
3. Add a suitable entry to meta-config.xml.
4. Run ./mkconfig.
5. Restart CruiseControl so that it can read the new project entries from config.xml. You can use the ps command to find the process ID of the JVM running CruiseControl, then kill the process using the kill command. Or you can run svc -t /service/cruisecontrol as the root user to have daemontools kill the process for you. Either way, supervise will make sure that CruiseControl is restarted.
6. Optionally, update the timestamp of /home/cruise/force-build/${project-name} so that CruiseControl triggers an automatic rebuild.

Back to top

The CruiseControl Web application

The CruiseControl installation that you now have running sends the results of each build as an e-mail message to your developers. But you probably have people involved in the development process who aren't recipients of these messages -- project managers or testers, for example. CruiseControl includes a simple Web application that lets these people monitor the continuous-integration builds.

The CruiseControl Web application runs in the Apache Tomcat application server, a copy of which is included in the Fedora Core 4 distribution. You need to install the tomcat5 and tomcat5-admin-webapps packages:

[root@fcvm ~]# yum install tomcat5 tomcat5-admin-webapps
[...]
Installed: tomcat5.i386 0:5.0.30-5jpp_6fc tomcat5-admin-webapps.i386 0:5.0.30-5jpp_6fc
Dependency Installed: tomcat5-jasper.i386 0:5.0.30-5jpp_6fc
Complete!
[root@fcvm ~]# 

You also need to install an implementation of the Java Transaction API (JTA). You can build your own JTA RPM using the RPM spec file from JPackage (see Resources), but the simplest option is to install geronimo-specs and geronimo-specs-compat from the Fedora development repository:

[root@fcvm ~]# rpm -Uvh http://download.fedora.redhat.com/\
pub/fedora/linux/core/development/i386/Fedora/RPMS/\
geronimo-specs-1.0-0.M2.2jpp_4fc.i386.rpm
Preparing...                ################################### [100%]
   1:geronimo-specs         ################################### [100%]
[root@fcvm ~]# rpm -Uvh http://download.fedora.redhat.com/\
pub/fedora/linux/core/development/i386/Fedora/RPMS/\
geronimo-specs-compat-1.0-0.M2.2jpp_4fc.i386.rpm
Preparing...                ################################### [100%]
   1:geronimo-specs-compat  ################################### [100%]
[root@fcvm ~]# 

The CruiseControl Web application can't find a suitable JAXP TransformerFactory implementation with the default Tomcat installation, so you need to add the default JAXP XML transformer to the endorsed classes directory:

[root@fcvm ~]# cd /usr/share/tomcat5/common/endorsed
[root@fcvm endorsed]# ln -s /usr/share/java/jaxp_transform_impl.jar \
\[jaxp_transform_impl\].jar
[root@fcvm endorsed]# ls -l
total 12
lrwxrwxrwx  1 root root 36 Sep 19 01:33 [jaxp_parser_impl].jar -> /usr
/share/java/jaxp_parser_impl.jar
lrwxrwxrwx  1 root root 39 Sep 19 01:47 [jaxp_transform_impl].jar -> /
usr/share/java/jaxp_transform_impl.jar
lrwxrwxrwx  1 root root 36 Sep 19 01:33 [xml-commons-apis].jar -> /usr
/share/java/xml-commons-apis.jar
[root@fcvm endorsed]# 

The CruiseControl Web application can draw graphs of important build statistics, such as the ratio of successful builds to failed builds. The libraries that draw the graphs use Java AWT, so you need to make sure the JVM runs in headless mode. To do this edit the /etc/tomcat5/tomcat5.conf file and insert a line saying JAVA_OPTS="-Djava.awt.headless=true" at around line 10.

Now add the CruiseControl Web application to Tomcat's configuration by creating a file called cruisecontrol.xml in /etc/tomcat5/Catalina/localhost. Listing 5 shows the contents of the file:

<Context path="/cruisecontrol"
         docBase="/home/cruise/pkg/cruisecontrol-2.2.1/reporting/jsp/d
ist/cruisecontrol.war">
  <Parameter name="logDir"
             value="/home/cruise/log/build"
             override="false"/>
  <Parameter name="cacheRoot"
             value="/var/cache/tomcat5/cruisecontrol"
             override="false"/>
</Context>

Note that the second line in Listing 5 has been wrapped for presentation purposes. The docBase attribute should be on a single line in the file you create.

You also need to create a directory for the CruiseControl Web application to store cached pages:

[root@fcvm ~]# cd /var/cache/tomcat5
[root@fcvm tomcat5]# mkdir cruisecontrol
[root@fcvm tomcat5]# chgrp tomcat cruisecontrol
[root@fcvm tomcat5]# chmod g+w cruisecontrol
[root@fcvm tomcat5]# ls -l
total 24
drwxrwxr-x  2 root tomcat 4096 Sep 16 09:32 cruisecontrol
drwxrwxr-x  2 root tomcat 4096 May 10 11:57 temp
drwxrwxr-x  3 root tomcat 4096 Sep 15 22:53 work
[root@fcvm tomcat5]# 

You can now start Tomcat and set it to restart when the system boots. The startup scripts currently generate some warning messages, but these can be ignored:

[root@fcvm ~]# service tomcat5 start
Starting tomcat5: find: warning: you have specified the -mindepth opti
on after a non-option argument -type, but options are not positional (
-mindepth affects tests specified before it as well as those specified
 after it).  Please specify options before other arguments.

find: warning: you have specified the -maxdepth option after a non-opt
ion argument -type, but options are not positional (-maxdepth affects 
tests specified before it as well as those specified after it).  Pleas
e specify options before other arguments.

Using CATALINA_BASE:   /usr/share/tomcat5
Using CATALINA_HOME:   /usr/share/tomcat5
Using CATALINA_TMPDIR: /usr/share/tomcat5/temp
Using JAVA_HOME:       /usr/lib/jvm/java
                                                           [  OK  ]
[root@fcvm ~]# chkconfig tomcat5 on
[root@fcvm ~]# chkconfig --list tomcat5
tomcat5         0:off   1:off   2:on    3:on    4:on    5:on    6:off
[root@fcvm ~]# 

You should now be able to use a Web browser to access the CruiseControl Web application at http://localhost:8080/cruisecontrol/. Figure 1 shows an example of the output you'll see:

Back to top

Security issues

Before I conclude, I should make a couple of points about the security issues that are involved in configuring and running your own continuous-integration server. First, I haven't tried to address the issues of securing access to the server that you build. You should consult other sources of information to ensure your system is secure, or run it on a private network that provides a suitable level of protection.

Second, you should consider how much you are prepared to trust the developers of external projects that you build on your continuous-integration server. A project's build process and unit tests can access your server's resources, including the network the server is connected to. The automated build process means that changes committed into a remote version-control repository will be downloaded and executed on your build server with no human intervention. This puts your build server at some risk from bugs and malicious code committed to source trees. You might want to limit the external projects you build on your build server, or put some effort into protecting your system and network from the projects that you are building.

Back to top

Conclusions

This article has guided you through the steps for setting up a continuous-integration server running CruiseControl. You've installed CruiseControl and learned what you need to do to keep it running all the time. You've gained an understanding of the day-to-day management of the continuous-integration server. And you've distilled the important elements of the configuration into a simpler XML document, including the choice of version-control and build tool and the targets or goals to be used to build each project.

You also now know how to specify the dependencies among projects. This is easier for Maven projects because they have a consistent build process and a shared repository for the artifacts produced. Ant leaves these mechanisms to each project, but if you have a number of Ant projects with a common build process you can use generated <filesystem> elements to extend the configuration to model the dependencies among these projects. CruiseControl has a number of other controls that can be used to enhance the continuous-integration process. You could easily take advantage of them by enhancing the XSLT stylesheet that I introduced.

I quickly covered the steps required to run the CruiseControl Web application, but you could improve your installation's security and reliability. A more secure configuration would use the Apache httpd to process requests and hand them to Tomcat. It might also be more reliable to have daemontools manage the Tomcat JVM in the same way that you configured it to manage CruiseControl itself. Beyond this, you should consider the security requirements of your build server and its network and explore some of the security tools that Linux provides.

This article aims to make your development process more agile and improve the quality of your software by adopting a continuous-integration approach. Creating a build server is a concrete and pragmatic step, and you can gain further improvements by adopting more practices from the agile-development methods. I encourage you to read more about these methods (see Resources for some starting points) and use their ideas to enhance and tune your development process.

Back to top

Download

Information about download methods

Resources

Learn

• Open Source Development with CVS, 3rd Edition and Version Control with Subversion: These books on version control are available online.
  
  
• Extreme Programming: A gentle introduction: Read about the Extreme Programming approach to software development.
  
  
• Integrate Often and Unit Tests: Extreme Programming advocates continuous integration and unit testing.
  
  
• Demystifying Extreme Programming: Roy Miller's former developerWorks column series can help you understand XP and its importance.
  
  
• "Automating the build and test process" (Erik Hatcher, developerWorks, August 2001): Find out how you can use Ant and the JUnit framework for customized automation of the build and test process.
  
  
• "Matching project and process" (Gary Pollice, developerWorks, May 2004): Read a discussion of the often overlooked necessity of tailoring a software development process to match the people, tools, and project type.
  
  
• Working with the SeaMonkey Tree and tinderbox: The Mozilla project documents its development practices. One of its important tools is the tinderbox system, which continually builds and tests the source tree on a number of different platforms.
  
  
• The Pragmatic Programmers: Great writing about agile software development practices.
  
  
• Apache Tomcat: Fedora Core 4 bundles the Apache Tomcat application server. Go here to find complete documentation.
  
  
• XStream: This article uses the XStream project's source tree as an example project.
  
  
• The Java technology zone: Hundreds of articles about every aspect of Java programming.
  
  

Get products and technologies

• CruiseControl: Find CruiseControl downloads and documentation at the project Web site.
  
  
• CVS and Subversion: Put your source code under version control.
  
  
• JUnit: JUnit is a popular unit-testing framework for Java programming.
  
  
• Maven: The Apache Maven project provides a build system that incorporates many best practices, including JUnit.
  
  
• Fedora Core 4: This Linux distribution is the basis of the build server described in this article.
  
  
• JPackage: The JPackage project originated the RPM packaging of Java projects that the Fedora project has adopted. The project can't redistribute the Sun JDK as RPMs, but you'll find instructions and RPM spec files for building them yourself.
  
  
• Fedora Core development repository: Go here to obtain updated Xerces packages. You need to upgrade to the xerces-j2-2.6.2-4jpp_8fc packages at a minimum; xerces-j2-2.6.2-5jpp_2fc is the most recent version available from the repository.
  
  
• XMLStarlet: This command-line program lets you perform useful operations on XML documents.
  
  
• daemontools: D. J. Bernstein's daemontools let you keep CruiseControl running all the time.
  
  

Discuss

• Participate in the discussion forum.
  
  

About the author

Comments (Undergoing maintenance)

Back to top

Trademarks  |  My developerWorks terms and conditions

Table of contents

• Why automatic builds?
• Configuring a build server
• Initial configuration
• Installing CruiseControl
• Making CruiseControl run all the time
• Simplifying the CruiseControl configuration
• The CruiseControl Web application
• Security issues
• Conclusions
• Download
• Resources
• About the author
• Comments

Next steps from IBM

Spring, JRuby, and Ajax development is easier with WebSphere Application Server - a smart Java 5 and J2EE Web services-based application server.

---

• Try: The no-charge WebSphere Application Server Community Edition is a pre-integrated, lightweight Java 5 application server built on Apache Tomcat and other best-of-breed open source software such as OpenEJB, Apache Axis, and Apache Derby.
• Article: The article leverage the Spring Framework and the WebSphere Application Server to improve your J2EE project productivity.
• Tutorial: See how the free WebSphere Application Server and XML can improve the efficiency of your JRuby on Rails and Ajax development.
• Buy: WebSphere Application Server - Express

My developerWorks community

Dig deeper into Java on developerWorks

Rate a product. Write a review.

Write a review for Rational products today!

Special offers