Clean Rational Team Concert build workspaces

Automate the build workspaces cleaning process

When multiple builds run on multiple machines the process of maintaining the infrastructure can become a problem. In this article you will learn how an automated build cleaning process can improve the continuous integration environment and reduce manual maintenance.

Jesus Ivan Millan (jimillan@mx1.ibm.com), Build Engineer, IBM

author photoJesus is a build engineer with 5 years of experience in IBM. He has built software using different technologies such as Ant, Rational Build Forge and Rational Team Concert.



08 February 2014

The problem

When a team of developers request multiple builds on build machines in a continuous integration environment it is likely that multiple copies of the workspaces exist in different locations.

Having personal builds enabled can cause disk space issues, especially if you do not have a best practice to remove old content. The problem is that the current build infrastructure of the application does not provide anything to keep your build machines clean from old source code used in builds. You can improve this by having a cleaning script to detect old build content and then remove that content.

In this article I will show you how to solve this problem. I've provided a script for you to use to help you clean your workspaces.

Determine old content

The first step is to determine the workspace age. This is done in order to determine which content can be removed. Determining the workspace age is not a trivial task. The workspace content modification date might not be a good source of information if the build process changes the source code as part of its defined work. Additionally the build duration time along with any change made during the build could make some files look less older than they really are.

A reliable method to determine the entire build workspace age is through the use of a metadata file. The build interaction creates a metadata file when it receives source code. Build workspaces maintains this metadata file. The server uses this file to determine if and when the workspace is updated.

Knowing the workspace age makes it so you can add a cleaning policy to your build farm. Once you have a cleaning policy you can have an automated process to keep your build machines clean.

Proposed solution

Because the Ant-based build solution is the most commonly used, I based this on an Ant script. The script requires a working Ant environment with at least version 1.8.0.


Technical details of the solution

Workspace age

Workspace age is determined by the workspace metadata file, for example, .jazz5.

Dependencies

  • Ant 1.8.0 or higher
  • Ant-Contrib

Operating systems

  • Windows
  • UNIX

Get the script See the Downloads section.

The workspace metadata file

In any application workspace, including the build workspaces, you can find a hidden file named .jazz and then a number, for example .jazz5. This (in this example .jazz5) is the metadata file you will use to determine the workspace age. This file is modified every time your workspace changes.

Script functionality

The script depends on the Ant-Contrib library for the ability to test statements and to loop over values.

The script uses operating system (UNIX and Windows supported) native commands to determine the metadata file age instead of pure Ant statements. This means that there is an operating system family validation before the workspace age determination.

The script can be used to list old build workspaces or to clean them. You control this by passing optional parameters to the script.

Run the script

To list or remove old workspaces, enter the command shown in Listing 1.

Listing 1. Remove old workspaces command
ant -f cleanWs.xml -Dant-contribDir=antcontrib_directory
-DjazzWorkspaceRoot=workspace_path -DjazzWorkspaceOldDays=days [-DlistOnly]

Definitions from Listing 1:

-f cleanWs.xml
Specifies the script to run, in this case the cleaning script.
 
-Dant-contribDir antcontrib_directory
Specifies the Ant-Contrib directory.
 
-DjazzWorkspaceRoot workspace_path
Specifies the directory where the workspaces are stored.
 
-DjazzWorkspaceOldDays
Specifies the age of the workspaces to delete in days.
 
-DlistOnly
Lists the candidates to remove, but does not remove them.
 

Property definitions from Listing 1:

ant-contribDir
Directory where the ant-contrib.jar file is located.
 
jazzWorkspaceRoot
Root directory where the jazz workspaces are downloaded.
 
jazzWorkspaceOldDays
Expiration period in number of days. This will be your cleaning policy in days (any build workspace with a modification date equal to or older than this number will be removed).
 
listOnly
Optional property to list the workspaces without removing the workspace.
 

An example to clean up all the workspace older than 10 days is shown in Listing 2.
Note: The paths are not real.

Listing 2. Example of a script execution
ant -f cleanWs.xml -Dant-contribDir=/opt/ant/ant-contrib
 -DjazzWorkspaceRoot=/build/root/jazzWorkspaces
 -DjazzWorkspaceOldDays=10

Script output

The script results three values:

  • Workspace path.
  • Metadata file.
  • A message about the deletion of the workspace, for example "will be deleted."

The output of this example script execution is:

checkWorkspaces:

[echo] Os Family = UNIX

[echo] Only workspaces with a .jazz<number> metadata directory will be checked by the tool

[echo] Workspace = '/jazzWorkspaces/_AJppkRGxEeOQ6v_82l9IYQ '

[echo] MetadataDir = '/jazzWorkspaces/_AJppkRGxEeOQ6v_82l9IYQ/.jazz5 '

[echo] Workspace = '/jazzWorkspaces/_AJppkRGxEeOQ6v_82l9IYQ' will be deleted

Assumptions

In order to run the script you must have a working Ant environment version 1.8.0 or higher and the ant-contrib.jar file available in your file system.

Automate the cleaning process

You can automate the cleaning process by either having a scheduled process to run the script or by having a call to the script at the end of you regular build process. I have implemented this in three ways:

  1. Use a scheduled job in the local machines to run once a day.
  2. Add a cleaning script call at the end of my regular build script.
  3. Have a build script where the cleaning script is the main script. Use the IBM® Rational® Team Concert build scheduler to easily run this script on multiple machines.

Conclusion

A big part of continuous integration is the build service. A cleaning policy is required to keep the build infrastructure ready to build, and to avoid disk space issues that can cause delays in continuous integration. Once you have a cleaning policy in place, the next step is to automate the run. To do this, use the ant script provided in this article. After you have your cleaning policy automated as part of the build processes you have made your continuous integration environment more reliable and less dependent on human intervention.


Download

DescriptionNameSize
Clean workspaces scriptcleanWs.xml5kb

Resources

Learn

Get products and technologies

  • Download Rational Team Concert from Jazz.net and try it free on up to 10 developers for as long as you want (requires registration). If you'd prefer, you can try it in the sandbox instead, without installing it on your own system.
  • Evaluate IBM software in the way that suits you best: Download it for a trial, try it online, or use it in a cloud environment.

Discuss

Comments

developerWorks: Sign in

Required fields are indicated with an asterisk (*).


Need an IBM ID?
Forgot your IBM ID?


Forgot your password?
Change your password

By clicking Submit, you agree to the developerWorks terms of use.

 


The first time you sign into developerWorks, a profile is created for you. Information in your profile (your name, country/region, and company name) is displayed to the public and will accompany any content you post, unless you opt to hide your company name. You may update your IBM account at any time.

All information submitted is secure.

Choose your display name



The first time you sign in to developerWorks, a profile is created for you, so you need to choose a display name. Your display name accompanies the content you post on developerWorks.

Please choose a display name between 3-31 characters. Your display name must be unique in the developerWorks community and should not be your email address for privacy reasons.

Required fields are indicated with an asterisk (*).

(Must be between 3 – 31 characters.)

By clicking Submit, you agree to the developerWorks terms of use.

 


All information submitted is secure.

Dig deeper into Rational software on developerWorks


static.content.url=http://www.ibm.com/developerworks/js/artrating/
SITE_ID=1
Zone=Rational
ArticleID=962080
ArticleTitle=Clean Rational Team Concert build workspaces
publish-date=02082014