When a team of developers request multiple builds on build machines in a continuous integration environment it is likely that multiple copies of the workspaces exist in different locations.
Having personal builds enabled can cause disk space issues, especially if you do not have a best practice to remove old content. The problem is that the current build infrastructure of the application does not provide anything to keep your build machines clean from old source code used in builds. You can improve this by having a cleaning script to detect old build content and then remove that content.
In this article I will show you how to solve this problem. I've provided a script for you to use to help you clean your workspaces.
Determine old content
The first step is to determine the workspace age. This is done in order to determine which content can be removed. Determining the workspace age is not a trivial task. The workspace content modification date might not be a good source of information if the build process changes the source code as part of its defined work. Additionally the build duration time along with any change made during the build could make some files look less older than they really are.
A reliable method to determine the entire build workspace age is through the use of a metadata file. The build interaction creates a metadata file when it receives source code. Build workspaces maintains this metadata file. The server uses this file to determine if and when the workspace is updated.
Knowing the workspace age makes it so you can add a cleaning policy to your build farm. Once you have a cleaning policy you can have an automated process to keep your build machines clean.
Because the Ant-based build solution is the most commonly used, I based this on an Ant script. The script requires a working Ant environment with at least version 1.8.0.
Technical details of the solution
The workspace metadata file
In any application workspace, including the build workspaces, you can find a hidden file named .jazz and then a number, for example .jazz5. This (in this example .jazz5) is the metadata file you will use to determine the workspace age. This file is modified every time your workspace changes.
The script depends on the Ant-Contrib library for the ability to test statements and to loop over values.
The script uses operating system (UNIX and Windows supported) native commands to determine the metadata file age instead of pure Ant statements. This means that there is an operating system family validation before the workspace age determination.
The script can be used to list old build workspaces or to clean them. You control this by passing optional parameters to the script.
Run the script
To list or remove old workspaces, enter the command shown in Listing 1.
Listing 1. Remove old workspaces command
ant -f cleanWs.xml -Dant-contribDir=antcontrib_directory -DjazzWorkspaceRoot=workspace_path -DjazzWorkspaceOldDays=days [-DlistOnly]
Definitions from Listing 1:
- -f cleanWs.xml
- Specifies the script to run, in this case the cleaning script.
- -Dant-contribDir antcontrib_directory
- Specifies the Ant-Contrib directory.
- -DjazzWorkspaceRoot workspace_path
- Specifies the directory where the workspaces are stored.
- Specifies the age of the workspaces to delete in days.
- Lists the candidates to remove, but does not remove them.
Property definitions from Listing 1:
- Directory where the ant-contrib.jar file is located.
- Root directory where the jazz workspaces are downloaded.
- Expiration period in number of days. This will be your cleaning policy in days (any build workspace with a modification date equal to or older than this number will be removed).
- Optional property to list the workspaces without removing the workspace.
An example to clean up all the workspace older than 10 days is shown in Listing 2.
Note: The paths are not real.
Listing 2. Example of a script execution
ant -f cleanWs.xml -Dant-contribDir=/opt/ant/ant-contrib -DjazzWorkspaceRoot=/build/root/jazzWorkspaces -DjazzWorkspaceOldDays=10
The output of this example script execution is:
[echo] Os Family = UNIX
[echo] Only workspaces with a .jazz<number> metadata directory will be checked by the tool
[echo] Workspace = '/jazzWorkspaces/_AJppkRGxEeOQ6v_82l9IYQ '
[echo] MetadataDir = '/jazzWorkspaces/_AJppkRGxEeOQ6v_82l9IYQ/.jazz5 '
[echo] Workspace = '/jazzWorkspaces/_AJppkRGxEeOQ6v_82l9IYQ' will be deleted
In order to run the script you must have a working Ant environment version 1.8.0 or higher and the ant-contrib.jar file available in your file system.
Automate the cleaning process
You can automate the cleaning process by either having a scheduled process to run the script or by having a call to the script at the end of you regular build process. I have implemented this in three ways:
- Use a scheduled job in the local machines to run once a day.
- Add a cleaning script call at the end of my regular build script.
- Have a build script where the cleaning script is the main script. Use the IBM® Rational® Team Concert build scheduler to easily run this script on multiple machines.
A big part of continuous integration is the build service. A cleaning policy is required to keep the build infrastructure ready to build, and to avoid disk space issues that can cause delays in continuous integration. Once you have a cleaning policy in place, the next step is to automate the run. To do this, use the ant script provided in this article. After you have your cleaning policy automated as part of the build processes you have made your continuous integration environment more reliable and less dependent on human intervention.
|Clean workspaces script||cleanWs.xml||5kb|
- More about Rational Team Concert:
- Find Rational Team Concert articles and links to many other resources on IBM developerWorks, and check the product overview page, features and benefits, system requirements, and the user information center.
- Check the Rational Team Concert page on Jazz.net.
- Watch the Using Rational Team Concert in a globally distributed team webcast or a demonstration of the Dashboards and reports, or listen to the podcast about IBM Rational Team Concert and Jazz.
- Explore the Rational software area on developerWorks for technical resources, best practices, and information about Rational collaborative and integrated solutions for software and systems delivery.
- Stay current with developerWorks technical events and webcasts focused on a variety of IBM products and IT industry topics.
- Improve your skills. Check the Rational training and certification catalog, which includes many types of courses on a wide range of topics. You can take some of them anywhere, anytime, and many of the Getting Started ones are free.
Get products and technologies
- Download Rational Team Concert from Jazz.net and try it free on up to 10 developers for as long as you want (requires registration). If you'd prefer, you can try it in the sandbox instead, without installing it on your own system.
- Evaluate IBM software in the way that suits you best: Download it for a trial, try it online, or use it in a cloud environment.
- Get connected with your peers and keep up on the latest information in the Rational community.
- Rate or review Rational software. It's quick and easy.
- Share your knowledge and help others who use Rational software by writing a developerWorks article. Find out what makes a good developerWorks article and how to proceed.
- Follow Rational software on Facebook, Twitter (@ibmrational), and YouTube, and add your comments and requests.
- Ask and answer questions and increase your expertise when you get involved in the Rational forums, cafés, and wikis.
Dig deeper into Rational software on developerWorks
Get samples, articles, product docs, and community resources to help build, deploy, and manage your cloud apps.
Keep up with the best and latest technical info to help you tackle your development challenges.
Software development in the cloud. Register today to create a project.
Evaluate IBM software and solutions, and transform challenges into opportunities.