Learning next-generation build tools for software management

Using CruiseControl and SCons to simplify testing

If you are part of a software development team, you have probably had instances where your code ran well in isolation only to fail during integration testing. Discover two tools—CruiseControl and SCons—allowing continuous integration and builds respectively, thus making integration testing easier.

Arpan Sen (arpansen@gmail.com), Lead Engineer, electronic design automation industry

Arpan Sen is a lead engineer working on the development of software in the electronic design automation industry. He has worked on several flavors of UNIX, including Solaris, SunOS, HP-UX, and IRIX as well as Linux and Microsoft Windows for several years. He takes a keen interest in software performance-optimization techniques, graph theory, and parallel computing. Arpan holds a post-graduate degree in software systems.



29 June 2012

Also available in Russian

About a decade ago, when I started coding software for a living, my team and I used to write code, run some tests locally with the modified sources, and then check the code in. There were just too many tests, so it made sense that the developer ran only a required subset of the tests. Every evening at about 8:00 pm, a cron job would compile the sources and run integration testing. Every couple of days, it turned out that developer A's code and developer B's code ran fine in isolation, but integration testing would fail.

The rapid pace of software development over the past decade has meant that we need better tools for integration testing. This is where CruiseControl comes in (see Resources for a link). CruiseControl is a continuous-integration tool that meshes seamlessly with source code management systems, such as Apache Subversion. Another tool that you can learn about in this article is SCons (see Resources)—a a build tool, similar to Makefile, except that it is based on Python. So you are spared the ordeal of learning yet another build system-specific language such as make or jam.

Continuous integration

In continuous integration, the build system checks out the sources after every predefined time interval (say, 2 hours), compiles the code, and runs the integration tests. If a compile error occurs or an integration test fails, developers get a message containing the relevant errors. They might even have a web page that displays the build statistics, who broke the build, and so on.

Working with CruiseControl

CruiseControl works seamlessly with quite a few tools, including the Concurrent Versions System (CVS), Subversion, Git, Perforce, Microsoft® Visual SourceSafe, and IBM® Rational® ClearCase®. This article uses Subversion as the revision control tool of choice.

You can download CruiseControl from the project home page (see Resources for a link. The latest revision as of this writing is 2.8.4, and that is what this article refers to.

The core of the CruiseControl system is config.xml, which triggers builds and runs tests in response to code changes, and then notifies you of the results. CruiseControl defines several tags that the XML file typically uses: <listeners>, <bootstrappers>, <modificationset>, <schedule>, and <publishers>.

Creating a configuration file

The root element of the configuration is called cruisecontrol (aptly enough), and it takes no attributes. The root element typically contains one or more property elements. These property elements contain name-value pairs and are an obvious means of setting log directories, log files, and so on. Listing 1 shows a basic configuration.

Listing 1. Adding properties to the configuration
<?xml version="1.0"?>
<cruisecontrol>
  <property name="builddir" value=""/opt/logs/project1" />
  <property name="buildlog" value="${builddir}/build.log" />
</cruisecontrol>

The next and probably the most important tag in the configuration file is the <project> tag. A configuration file can contain multiple <project> tags, each defining a separate project. If there are multiple projects and you want to run them in parallel, you need to specify the <thread> tag and make the count attribute of the <thread> tag greater than 1. This is a good option to explore if you have a build or integration server with multiple cores at your disposal. Note that the <thread> tag is part of the <system> tag hierarchy. Listing 2 shows the <thread> tag being used.

Listing 2. Configuring for multicore servers
<?xml version="1.0"?>
<cruisecontrol>
  <property name="builddir" value="/opt/logs/project1" />
  <property name="buildlog" value="${builddir}/build.log" />
  <system>
     <configuration>
         <threads count="8" />
     </configuration>
  </system>
</cruisecontrol>

For every <project> tag, you need to define the name attribute. Listing 3 has multiple <project> tags.

Listing 3. Adding multiple projects to CruiseControl
<?xml version="1.0"?>
<cruisecontrol>
  <property name="builddir" value=""/opt/logs/project1" />
  <property name="buildlog" value="${builddir}/build.log" />
  <system>
     <configuration>
         <threads count="8" />
     </configuration>
  </system>
  <project name="project1">
    <! need to configure project child tags>
  </project>
  <project name="project2">
     <! need to configure project child tags>
  </project>
</cruisecontrol>

The <project> tag has a couple of interesting attributes. Of particular interest to us are the buildafterfailed and requireModification attributes. If buildafterfailed is set to True, then CruiseControl keeps trying to build your project, even though the last build has failed. When is this useful? It is worth a shot when your system has too many dependencies—such as sourcing files from a remote repository and connection fails or getting information from a database that may have a response timeout. By default, buildafterfailed is False and the requireModification attribute is set to True: It tells CruiseControl to rebuild sources only if there are check-ins in the code base. Set this attribute to False if you want to have builds after short intervals, irrespective of changes in the code. In the following example, these attributes are used.

  <project name="project1" buildafterfailed=true requireModification=no>
    <! need to configure project child tags>
  </project>

Next, you need to configure the hierarchy or child tags for each individual project. First, you need to see how to associate the source control with the <project> tag. Is there a tag for that? Of course—that is what the <modificationset> tag is for. Listing 4 shows how the Subversion source repository is being accessed from CruiseControl.

Listing 4. Configuring the source repository with CruiseControl
<?xml version="1.0"?>
<cruisecontrol>
  <property name="builddir" value="/opt/logs/project1" />
  <property name="buildlog" value="${builddir}/build.log" />
  <system>
     <configuration>
         <threads count="8" />
     </configuration>
  </system>
  <project name="project1">
    <modificationset>
       <svn RepositoryLocation="svn+ssh://dev/svn/project/trunk"
          username="integration" password="integration" >
    </modificationset>
  </project>
</cruisecontrol>

If you prefer building from a local copy of the repository, consider using the LocalWorkingCopy attribute, instead (see Listing 5).

Listing 5. Building from a locally checked-out copy of sources
  <project name="project1">
    <modificationset>
       <svn LocalWorkingCopy="/build/checkout/project1">
    </modificationset>
  </project>

Note that you need to provide either the RepositoryLocation or the LocalWorkingCopy attribute with the <svn> tag. The username and password attributes are clearly optional and necessary only if your authentication system demands it. For other source controls, you have similar tags, such as <cvs> and <vss>. The code in Listing 5 also brings up an interesting question: If you are specifying only the local working copy, then whose responsibility is it to keep this copy updated? The answer to this question brings us to the <bootstrappers> tag. The <bootstrappers> tag might have child tags such as <svnbootstrapper> or <clearcasebootstrapper> depending on the repository that you are using. Listing 6 shows the bootstrapping for the configuration in Listing 5.

Listing 6. Checking out sources using bootstrapper
  <project name="project1">
    <bootstrappers>
       <svnbootstrappers localWorkingCopy="/build/checkout/project1" 
          username="integration" password="integration" >
    </bootstrappers>
    <modificationset>
       <svn LocalWorkingCopy="/build/checkout/project1">
    </modificationset>
  </project>

The <svnbootstrapper> tag indicates to CruiseControl the local copy of the sources on which to run the update command before the build is fired.

So, you now have the source control set up. How about deciding when the build should run? Which command should you run to fire the build? From what you have seen so far, you need a new tag: Say hello to the <schedule>. Listing 7 provides an example of <schedule> in use.

Listing 7. Executing the build with the schedule tag
  <project name="project1">
    <bootstrappers>
       <svnbootstrappers localWorkingCopy="/build/checkout/project1" 
          username="integration" password="integration" >
    </bootstrappers>
    <modificationset>
       <svn LocalWorkingCopy="/build/checkout/project1">
    </modificationset>
    <schedule>
       <exec command="/opt/build-tools/mybuildscript.sh"
                  workingdir="/build/checkout/project1"
                  args="link=shared variant=debug j4"
                  errorstr="build failed" >
    </schedule>
  </project>

The <exec> tag (which is a child tag of <schedule>) runs a specified command or script as part of the build on the directory specified in workingdir. If the build fails, then there is also the option of a user-provided string in the error output using the errorstr attribute. If you want to schedule the build every hour, irrespective of code changes in the repository, <schedule> has an attribute called interval that specifies the build time interval in seconds, as shown in Listing 8.

Listing 8. Specifying the build time interval
  <project name="project1" requireModifications=no>
    <bootstrappers>
       <svnbootstrappers localWorkingCopy="/build/checkout/project1" 
          username="integration" password="integration" >
    </bootstrappers>
    <modificationset>
       <svn LocalWorkingCopy="/build/checkout/project1">
    </modificationset>
    <schedule interval=3600>
       <exec command="/opt/build-tools/mybuildscript.sh"
                  workingdir="/build/checkout/project1"
                  args="link=shared variant=debug j4"
                  errorstr="build failed" >
    </schedule>
  </project>

After the build has started, users will likely inquire about the status of the build. Having a web-based interface is probably one of the easiest things to provide for such cases. And this brings us to the listeners child element of the <project> tag. The tag with a particular interest to us is <currentbuildstatuslistener>, which writes a build status snippet to a file. If you want to explicitly specify the folder in which build logs should be maintained, then you need the <log> tag and its optional dir attribute to provide for the relevant folder. The <log> tag has a child tag aptly called <merge> that is used to merge miscellaneous data from the integration testing process into the CruiseControl log file. Using the command:

<merge dir=dir-name pattern="*.xml">

. . . merges all XML files in the folder dir-name with the CruiseControl log file—a neat feature if you prefer having an overall picture of both the build and the testing. Listing 9 shows the configuration.

Listing 9. The updated configuration
  <project name="project1" requireModifications=no>
    <bootstrappers>
       <svnbootstrappers localWorkingCopy="/build/checkout/project1" 
          username="integration" password="integration" >
    </bootstrappers>
    <modificationset>
       <svn LocalWorkingCopy="/build/checkout/project1">
    </modificationset>
    <schedule interval=3600>
       <exec command="/opt/build-tools/mybuildscript.sh"
                  workingdir="/build/checkout/project1"
                  args="link=shared variant=debug j4"
                  errorstr="build failed" >
    </schedule>
    <log dir="/build/logs/project1">
       <merge dir="/build/tests/project1" pattern="*.output" />
    </log>
    <listeners>
       <currentbuildstatuslistener file="/build/staus/latest" />
    </listeners>
  </project>

You are almost done here, except that you need to take a few post-build actions. For example, developers may need a message, or may be you need to build the .zip files out of the sources. This is where the <publishers> tag comes in handy. The <publishers> tag has a whole host of child tags that allows you to customize your post-build situation. For our purposes, try the most common approach: email. You can choose either the <email> or <htmlemail> child tag of <publishers>. Listing 10 shows a sample usage.

Listing 10. Adding the mailing feature to the configuration
  <project name="project1" requireModifications=no>
    <bootstrappers>
       <svnbootstrappers localWorkingCopy="/build/checkout/project1" 
          username="integration" password="integration" >
    </bootstrappers>
    <modificationset>
       <svn LocalWorkingCopy="/build/checkout/project1">
    </modificationset>
    <schedule interval=3600>
       <exec command="/opt/build-tools/mybuildscript.sh"
                  workingdir="/build/checkout/project1"
                  args="link=shared variant=debug j4"
                  errorstr="build failed" >
    </schedule>
    <log dir="/build/logs/project1">
       <merge dir="/build/tests/project1" pattern="*.output" />
    </log>
    <listeners>
       <currentbuildstatuslistener file="/build/staus/latest" />
    </listeners>
    <publishers>
        <email mailhost="smtp.myjob.com" subject="Build/Test Stats">
            <failure address="all@myjob.com" />
            <success address="developers@myjob.com" />
        </email> 
    </publishers>
  </project>

SCons

Yet another nifty software build management tool is SCons. SCons stands for software construction and is easily a replacement for make. Why bother to use SCons when you have make? For starters, make, automake, autoconf, and so on come with a syntax that is peculiar to the tool, while SCons is Python based. SCons syntax is also a lot easier on the head, and the SConstruct files (similar to Makefiles) are easier to maintain. SCons also comes with specialized support for C/C++ and Java™ files, and SCons accomplishes the make functionality with a lot less code.

SCons and Python

SCons version 2.1—the version used in this article—does not yet work with versions of Python later than 2.3.5.

Hello, World with SCons

Listing 11 shows a simple Hello, World program in C++.

Listing 11. Hello, World in C++
#include <iostream>
using namespace std;

int main( )
{
  cout << "Hello World" >> endl;
}

Here's the SConstruct file to create the executable:

env = Environment()
env.Program(hello.cpp)

That is it: Compare this with the make-based approach, and you can realize the benefits of SCons immediately. Program is called a builder_method—a Python application programming interface (API) that tells SCons that you want to build an executable file. It is possible to have multiple executables in the same file and also have SCons generating messages during the build process. Listing 12 provides a sample.

Listing 12. Building multiple executables with additional messaging
env = Environment()
env.Program('hello.cpp')

print "Done with building hello, on to hello2"       
env.Program('hello2.cpp')
print "Finished building hello2"

So, what is this env business? In SCons parlance, you are creating a construction environment, and env holds a lot of information about the compiler, linker flags, and the like.

Useful things to try with SCons

Here are some common tasks you can perform with SCons:

  • You want the executable to have a custom name. That means that hello2.cpp should generate an executable called newworld2. To do it, specify the name as the first argument to Program:

    env = Environment()
    print "Beginning building newworld2"
    env.Program('newworld2',  'hello2.cpp')
    print "Finished building newworld2"
  • You want to build an executable that depends on multiple sources. Say that newworld2 depends on hello2.cpp and world3.cpp. The SConstruct file would then look like this:

    env = Environment()
    print "Beginning building newworld2"
    env.Program('newworld2',  ['hello2.cpp', 'world3.cpp'])
    print "Finished building newworld2"
  • To clean your builds, just type scons c at a shell prompt.
  • It is possible that you want to build two executables that have a number of source files in common. It is prudent to have these files compiled into object files only once. Here is an example:

    env = Environment()
    
    common = ['common1.cpp', 'common2.cpp']
    hello1_sources = ['hello1.cpp'] + common
    hello2_sources = ['hello2.cpp', 'world2.cpp'] + common
           
    env.Program('hello1', hello1_sources)
    env.Program('hello2', hello2_sources)
  • Executables often need to link with specific libraries, such as pthread or math. Program can handle additional arguments to specify the LIBS and LIBPATH variables. Here is a sample usage to link against the pthread library:

    env = Environment()
    env.Program('new-exe',
                   'concurrent.cpp', 
                   LIBS = 'pthread',
            LIBPATH = ['/usr/lib', '/usr/local/lib'])
  • What if all files with a .cpp extension in a folder go toward the making of an executable? Thankfully, you do not need to specify the names of all the files. The Glob function makes your life easier:

    env = Environment()
    Program('new-exe',
                   Glob('*.cpp'),
                   LIBS = 'pthread',
            LIBPATH = ['/usr/lib', '/usr/local/lib'])
  • Building a static library is a breeze in SCons. Just use the Library function call:

    env = Environment()
    env.Library('arith', ['math1.cpp', 'math2.cpp', 'math3.cpp'])

    Note that internally, SCons creates math1.o, math2.o and math3.o, then follows it up with ar rc libarith.a math1.o math2.o math3.o and finally ranlib libarith.a.

  • Creating a shared library in SCons is almost as easy as creating a static library:

    env = Environment()
    env.SharedLibrary('arith', ['math1.cpp', 'math2.cpp', 'math3.cpp'])
  • Different source files may be required to be compiled with different compiler options. For example, if you want to pass DX86_64 to the sources, indicating some optimizations made for 64-bit platforms to be compiled, here is how you should tweak the SConstruct file:

    env = Environment()
    env.Object('hello.cpp', CCFLAGS='-DX86_64')
    env.Object('world.cpp', CCFLAGS='-DX86_64')
    env.Program('new-exe', ['hello.o', 'world.o']

    Note that you have also introduced a new builder function called Object. Program can be built using object files as well, so no need to pass in the sources as you have done earlier.

  • Throughout this discussion, I have mentioned the target as the first item. If you ever want to reverse or play around with the order, here is how you would go about it:

    common = ['common1.cpp', 'common2.cpp']
    hello1_sources = ['hello1.cpp'] + common
    hello2_sources = ['hello2.cpp', 'world2.cpp'] + common
           
    Program(source = hello1_sources, target = 'hello1')
    Program(target = 'hello2', source = hello2_sources)

    Note that source and target are SCons-specific keywords.

  • Although Program, SharedLibrary, and the rest are all members of the env object, it is okay to use them even without prefixing them with env. So, the following SConstruct file is just fine:

    common = ['common1.cpp', 'common2.cpp']
    hello1_sources = ['hello1.cpp'] + common
    hello2_sources = ['hello2.cpp', 'world2.cpp'] + common
           
    Program('hello1', hello1_sources)
    Program('hello2', hello2_sources)

Conclusion

In this article, you have learned about CruiseControl and continuous integration, including the more important tags, such as <bootstrappers>, <modificationset>, <schedule>, and <listeners>. You have also used SCons to build a small C++ project and learned how to build static and shared libraries. Both tools have a lot more to them than what was discussed in this article, so be sure to check out the Resources section for details.

Resources

Learn

Get products and technologies

  • Download the latest version of CruiseControl from the project's website.
  • Download SCons version 2.1 from SourceForge.
  • Learn more about and download Subversion.
  • Try out IBM software for free. Download a trial version, log into an online trial, work with a product in a sandbox environment, or access it through the cloud. Choose from over 100 IBM product trials.

Discuss

Comments

developerWorks: Sign in

Required fields are indicated with an asterisk (*).


Need an IBM ID?
Forgot your IBM ID?


Forgot your password?
Change your password

By clicking Submit, you agree to the developerWorks terms of use.

 


The first time you sign into developerWorks, a profile is created for you. Select information in your profile (name, country/region, and company) is displayed to the public and will accompany any content you post. You may update your IBM account at any time.

All information submitted is secure.

Choose your display name



The first time you sign in to developerWorks, a profile is created for you, so you need to choose a display name. Your display name accompanies the content you post on developerWorks.

Please choose a display name between 3-31 characters. Your display name must be unique in the developerWorks community and should not be your email address for privacy reasons.

Required fields are indicated with an asterisk (*).

(Must be between 3 – 31 characters.)

By clicking Submit, you agree to the developerWorks terms of use.

 


All information submitted is secure.

Dig deeper into AIX and Unix on developerWorks


static.content.url=http://www.ibm.com/developerworks/js/artrating/
SITE_ID=1
Zone=AIX and UNIX
ArticleID=822663
ArticleTitle=Learning next-generation build tools for software management
publish-date=06292012