Automatically generate project builds using Ant

Creating, exporting, deploying, and testing your builds

Ahmed Abbass (aabbass@eg.ibm.com), IT Architect, IBM

Ahmed Abbas is an IT Architect at Cairo Technology Development Center. Through his involvement in lab services, Ahmed has been interested in bridging the gap between the theories behind technology and the practicality of implementations in order to serve customer business needs.

Summary: Get step-by-step instructions to automate your build procedure to improve productivity and quality using the built-in support of Ant for the IBM® Rational® Software Development platform. You can then deploy it using the IBM WebSphere® family of application servers.

Date: 30 Jan 2007
Level: Intermediate
Comments:

Introduction

For multi-release projects, it can be a challenge to provide new builds that keep up with the rate of additional features or defect reporting, while still maintaining an acceptable level of quality. Build automation solves part of this problem by ensuring accuracy and removing the possibility of human errors. Automation also increases team productivity by letting members focus on issues that require human intelligence rather than tasks that usually run faster and more efficiently when they are automated.

In this article learn how to automate your build procedure to obtain this higher level of productivity and quality. The example in the article uses the built-in support of Ant in the Rational Software Development Platform (SDP) (the build automation procedure is implemented as an Ant build file) as well as support in run time, such as WebSphere Application Server. The article wraps up with some optional features you can use to further streamline this process.

Project planning considerations

Although its execution does not start until the construction* phase, task automation should be planned during the elaboration* phase so that it is in place for the implementation team as soon as code is delivered for testing. After doing analysis and design, the vision of the components and how they should be built, tested, and deployed should be clear. Inspired by this vision, task automation begins to accomplish the tasks in a way that allows repeatability.

Rational Unified Process: project phases

* The terms construction and elaboration refer to two project phases according to Rational Unified Process.

To be repeatable, some tasks (such as test data) may need preparation before they start and cleanup after they finish. Changes to software configuration management (software repository), design, or target deployment environment should also be reflected in task automation. You should account for the overhead of implementing and maintaining this automation during effort estimation and project planning. You can best observe the impact of task automation with projects of multiple iterations, when the time to build, unit test, and deploy is significantly reduced for each iteration.

Automation environment

The automation procedure in this article works for Rational SDP-based tooling, such as:

For a deployment platform, it can be used with the WebSphere family of application servers that use WebSphere Application Server as its foundation, such as WebSphere Application Server, WebSphere Application Server Network Deployment, and WebSphere Process Server.

The same procedure can be applied to other Rational and WebSphere products. For simplicity, we'll refer to the tool used as Integrated Development Environment (IDE).

Preparing the environment

The procedure can run on a variety of environment configurations. Figure 1 shows an example environment.

Figure 1. Automation environment

This environment has one or more development workstations where developers write code. Changes are saved to a repository. Here we assume a Rational ClearCase repository with a snapshot view for each development workstation.

A development workstation should have a Rational SDP-based tool installed. The development workstation should have workspace that is configured with the software repository. The article "Using Rational Software Architect and ClearCase Remote Client" has more information about how to do such a configuration with Rational ClearCase. You can also use other repositories; for more information on using Ant with Concurrent Versions System (CVS), see Resources.

Any development workstation can be used as a build workstation. It's a good idea to segregate a workstation for builds, to avoid usage contention among developers. The build workstation should have the same configuration of any other development machine, and, at the same time, access the following servers:

Deployment servers (to deploy the build)
SMTP server (to notify testing team)
FTP server (to upload the build for backup or any other usage)

A typical development or build workstation can be implemented on Windows® or Linux®. For target deployment servers, WebSphere products have a wider range of supported operating systems.

Ant build file

All automation steps will be included in one Ant build file for ease of maintenance. If you are not already familiar with Rational tools:

From the File menu, select New > Project > Java > Java Project.
Enter ar-autotaskcode.zip for the Project name field and click Finish.

XML Capability
It is assumed that XML capability is enabled for the workspace. See "Author developerWorks content using IBM Rational Web Developer" for more information. After enabling XML capability, if the XML node does not appear under the Wizards tree in the New dialog, check Show All Wizards.
Right-click the AutomationScripts project and select New > Other from the context menu. From the New dialog, select XML > XML and click Next.
From the Create XML file dialog, select Create XML file from scratch, then click Next.
Enter build.xml as the File name. Click Finish. (This is where we are going to include all of our steps.)

Outline the ant build file, as shown in Listing 1.

Listing 1. Initial outline of the Ant build file

						
<?xml version="1.0" encoding="UTF-8"?>
<project name="Automation Project" default="init" basedir=".">
<!-- Read project properties from a properties file. -->
<property file="build.properties" />
<!-- A stub for the default target -->
 <target name="init" description="Initialize automation environment">
 <!-- Set the standard DSTAMP, TSTAMP, and TODAY properties. -->
 <tstamp />
 <!-- Set a variable to host environment variables. -->
 <property environment="env" />
 <!-- Print the current WORKSPACE value. -->
 <echo message="Workspace:
 ${env.WORKSPACE}" />
 <!-- Delete old export directory. -->
 <delete dir="${destination.dir}" />
 <!-- Create the target export directory for the build. -->
 <mkdir dir="${destination.dir}" />
 </target>
</project>

You can download the complete Ant build file, ar-autotaskcode.zip, now.

In the same directory that has the Ant build project, create a properties file that holds its properties, build.properties. The complete properties file is also part of ar-autotaskcode.zip.
Show the Ant view by selecting Window > Show View > Other > Ant > Ant. Drag and drop your build.xml into the Ant view. Using the Ant view you can browse and run the different tasks of your Ant build file. You can also edit your build file using the Ant editor by selecting Open With Ant Editor from the context menu that appears when you right-click the name of the Ant project Automation Project in the Ant view.

Now the Ant build file is ready for adding new Ant targets that represent the automation procedure.

Automating the build procedure

Figure 2 shows a summary of the build automation procedure.

Figure 2. Automation procedure

This section describes each step of the procedure in detail.

Get latest code from repository

The first step is to get the latest code from the ClearCase repository, as shown in Listing 2.

Listing 2. Updating the workspace from ClearCase repository

						
				
<target name="updateview" depends="init" description="Update ClearCase snapshot view">
						
<!-- update snapshot view -->

<ccupdate viewpath="${workspace.dir}" graphical="false" overwrite="true" 

currenttime="true" />
</target>

Since the workspace is configured to point to the ClearCase view, updating the view actually updates the workspace content.

Run code reviews

Code review can be automated. Although it doesn't eliminate the need for manual code reviews, it's good practice to have the automated reviews to know where the development team stands regarding coding quality standards. There is built-in code review support within Rational SDP. (For more information on how to invoke it from Ant, see Code review headless mode reference.)

There are also several tools that perform automated code reviews, such as PMD and CheckStyle. The example in Listing 3 shows how to invoke PMD as an Ant task and generate the result in HTML format. To run this task, you need to place pmd-3.8.jar, jakarta-oro-2.0.8.jar, and jaxen-1.1-beta-10.jar under the directory specified by the lib.dir property.

Listing 3. Automated code review using PMD

						
<target name="reviewcode" description="Review code using PMD">
  <taskdef name="pmd" classname="net.sourceforge.pmd.ant.PMDTask" 
  classpath="${lib.dir}/pmd-3.8.jar" />
    <pmd shortFilenames="true">
	<!-- Determine the ruleset to be used -->
	<ruleset>rulesets/favorites.xml</ruleset>
	<ruleset>basic</ruleset>
	<!-- Generate and HTML report into the designated directory -->
	<formatter type="html" 
toFile="${report.dir}/pmd_automated_code_review_report.html" />
	<!-- Files to be configured for review -->
	<fileset dir="${workspace.dir}/">
	<!-- Include all .java files except those under directories that are 
	   automatically generated -->
	<include name="**/*.java" />
	<!-- A sample exlusion directory that has generated java source code -->
	<exclude name="**/generated/**/*.java" />
	</fileset>
	</pmd>
</target>

After this step gets executed, you will find the report pmd_automated_code_review_report.html in HTML format under the directory specified by report.dir property. (For more information on automated code reviews, see Resources.)

Build code

Now it's time to build the different projects to be included in our build using the built-in functions available in the IDE.

For SDP-provided Ant tasks to run within the IDE, the Ant build file needs to run in the same Java™ Runtime Environment (JRE) as the workspace. Otherwise, if run from the command line, runAnt batch file does the required initialization to include the definition of these tasks. This is discussed in Options when running the procedure. The code in Listing 4 builds one of the projects that exist in the workspace using IDE built-in "projectBuild" Ant task.

Listing 4. Building a single project

						
<target name="buildproject" description="Build a project">
	
<!-- Build the project with some options -->

<projectBuild projectName="${project.name}" BuildType="full" failonerror="false" 
   DebugCompilation="false" />

<!-- Print build errors -->

<echo message="projectBuild: projectName=${project.name}

project Error Count=${ProjectErrorCount}

project Error Messages=${ProjectErrorMessages}" />

</target>

Execution time errors that might occur get printed to the console, and the build does not terminate. To change this behavior, you can set the failonerror attribute to true.

The same target can be used with different projects by setting the project.name property to the name of the project to be built, as the code shows in Listing 5.

Listing 5. Building a set of projects

						
<target name="buildall">
 <!-- Invoke the "buildproject" target for a project from the workspace -->
  	<antcall target="buildproject">
	<!-- Pass the name of the project to be built -->
	<param name="project.name" value="MyFirstProject" />
	</antcall>
	<!-- Invoke the "buildproject" target to build another project -->
	<antcall target="buildproject">
	<!-- The project name is different -->
	<param name="project.name" value="MySecondProject" />
	</antcall>
	</target>

Record the build number

Sometimes this step is overlooked. It is useful to have a build number for each build you release for easier tracking of defects and newly added features. Ant provides a simple buildnumber task for that purpose. Listing 6 shows another way of logging more information on the build, as needed.

Listing 6. Logging build information

						
<target name="baseline" description="Record build information">
<!-- The name of the file that holds the build information. If no such file exists, a new 
   one gets created. -->
<propertyfile file="${destination.dir}/build.info">
<!-- Initial build number is 0001. Then, any subsequent build increments

 this number by one  each time. -->
<entry default="0001" key="build.number" operation="+" pattern="0000" type="int" />
<!-- Records the current time to the same file. -->
<entry default="now" key="build.time" pattern="yyyy.MM.dd-HH.mm" type="date" />
</propertyfile>
</target>

The "baseline" target creates a file named build.info that has the build number and time. If the file exists, it just updates it with the new build number and time. You may need to add more information in this file, such as the name of the user who ran the build or the name of the machine on which the build was created. This information can be extracted from the environment the same way we extracted the WORKSPACE environment variable in the "init" target, as shown in Ant build file.

Run pre-deployment unit tests

This step invokes JUnit test cases that do not require run-time resources, such as data sources. It is assumed that you already have such test cases in order to run this step. Listing 7 shows how to invoke all the JUnit test cases in the workspace and assumes that they all end with the word "Test." You can change that if you have a different naming convention, of course.

Listing 7. Invoking JUnit test cases

						
<target name="predeptests" description="Invoke pre-deployment JUnit test cases">
 <!-- Run JUnit and print a summary when done. -->
  <junit printsummary="yes" haltonfailure="no">
  <classpath>
  <!--
  Set the class path to include all needed classes that are used by the JUnit test cases.
  -->
  </classpath>
  <!-- Generate reports in an XML format -->
  <formatter type="xml" />
  <!-- Run a batch of JUnit test cases and generate reports to the report directory. -->
  <batchtest fork="yes" todir="${report.dir}">
  <fileset dir="${workspace.dir}/">
  <!-- Include all .java files that end with "Test" -->
  <include name="**/*Test.java" />
  </fileset>
  </batchtest>
  </junit>
  </target>

The JUnit Ant task requires junit.jar. You can put it under the directory specified by the lib.dir property in the properties file of the Ant build project mentioned in the "init" target. Reports generated by the JUnit task can be transformed to other formats using JUnitReport Ant task. In our case, the reports will be generated in the directory specified by the report.dir property.

Export deployable units

In this step you package the deployable units to be ready for deployment in the next step. Listing 8 shows how to export an enterprise application.

Listing 8. Exporting an enterprise application

						
<target name="exportear" description="Export an Enterprise Application">
 <!-- Export EAR file-->
 <earExport earprojectname="${project.name}" 
 
 earexportfile="${destination.dir}/${project.name}.ear" 
 exportsource="false" IncludeProjectMetaFiles="false" 

 overwrite="true" />
 </target>

Similar to dealing with a set of projects during step 3 (building code), we can do the same while exporting enterprise applications. The code in Listing 9 shows how it is done.

Listing 9. Exporting an enterprise application

						
<target name="exportall" depends="buildall" 

description="Export a set of Enterprise Applications">
<!-- Export an Enterprise Application -->
<antcall target="exportear">
<!-- Pass the name of the project to be exported -->
<param name="project.name" value="MyFirstEAR" />
</antcall>
<!-- Export another Enterprise Application -->
<antcall target="exportear">
<!-- The name of the other Enterprise Application -->
<param name="project.name" value="MySecondEAR" />
</antcall>
</target>

For other project types, there are warExport, ejbExport and jar tasks that export Web modules, EnterpriseJavaBeans (EJB) modules, and Java libraries, respectively. These are all SDP-provided Ant tasks.

Deploy to target servers

In this step you'll deploy the units that were exported in the previous step. For our automation procedure we'll be using built-in Ant tasks that wrap wsadmin. Listing 10 shows wsListApps, wsUninstallApp, and wsInstallApp Ant tasks that provide administration functions to list installed applications, uninstall an enterprise application, and install an enterprise application, respectively. Remember to start the target application server before running this step.

About wsadmin and Ant
WebSphere Application Server Ant support is based on its command line administration tools, wsadmin. As an alternative to the example in this article, you can invoke wsadmin using the Exec Ant task and pass program parameters the same way you invoke any other executable. WebSphere Application Server also provides an Ant task WsAdmin that wraps wsadmin. There are more Ant tasks that wrap the WsAdmin task for specific operations, such as application installation InstallApplication and StartApplication. The latter option is used in our example.

Listing 10. Administering enterprise applications

						
<target name="listapps" description="List installed Enterprise Applications">
<!-- Define the wsListApps task that lists installed Enterprise Applications. -->
<taskdef name="wsListApps" classname="com.ibm.websphere.ant.tasks.ListApplications">
<!-- Include all JAR files under WebSphere Application Server lib directory. -->
<classpath>
<fileset dir="${was.home}/lib/" includes="*.jar" />
</classpath>
</taskdef>
<!-- List all installed Enterprise Application the profile specified. -->
<wsListApps profilename="${was.profilename}" wasHome="${was.home}/" conntype="SOAP"
   host="${was.hostname}" port="${was.hostport}" user="${was.username}" 
    password="${was.userpassword}" />
</target>
<target name="uninstallapp" depends="listapps" 

description="Uninstall an Enterprise Application">
<!-- Define the wsUninstallApp task that uninstalls an Enterprise Application. -->
<taskdef name="wsUninstallApp" 

classname="com.ibm.websphere.ant.tasks.UninstallApplication">
<!-- Include all JAR files under WebSphere Application Server lib directory. -->
<classpath>
<fileset dir="${was.home}/lib/" includes="*.jar" />
</classpath>
</taskdef>
<!-- Uninstall an Enterprise Application under the profile specified. -->
<wsUninstallApp profilename="${was.profilename}" wasHome="${was.home}/"  
   application="${project.name}"  conntype="SOAP" 
   host="${was.hostname}" 

   port="${was.hostport}" 
   user="${was.username}" password="${was.userpassword}" 

   failonerror="false" />
<!-- Invoke listapps target to list installed applications. -->
<antcall target="listapps" />
</target>
<target name="installapp" depends="uninstallapp">
<!-- Define the wsInstallApp task that installs an Enterprise Application. -->
<taskdef name="wsInstallApp" classname="com.ibm.websphere.ant.tasks.InstallApplication">
<!-- Include all JAR files under WebSphere Application Server lib directory. -->
<classpath>
<fileset dir="${was.home}/lib/" includes="*.jar" />
</classpath>
</taskdef>
<!-- Install an Enterprise Application under the profile specified. -->
<wsInstallApp profilename="${was.profilename}" wasHome="${was.home}/" 

ear="${destination.dir}
   /${project.name}.ear" conntype="SOAP" 

   host="${was.hostname}" port="${was.hostport}" 
      user="${was.username}" 

      
      password="${was.userpassword}" failonerror="false" />
<!-- Invoke listapps target to list installed applications. -->
<antcall target="listapps" />
</target>

Depending on the number of applications you need to deploy, you can invoke installapp target several times with different values of the project.name parameter. The installapp target invokes uninstallapp target to ensure that the enterprise application is uninstalled, if already installed. The two targets invoke listapps several times to list the application before and after uninstallation and installation. Listing 11 shows how this can this be done.

Listing 11. Installing multiple enterprise applications

						
<target name="installallapps" description="Install all Enterprise Applications">
<!-- Invoke installapp target to install an Enterprise Application. -->
<antcall target="installapp">
<!-- Pass the name of the Enterprise Application to be installed. -->
<param name="project.name" value="MyAppEAR" />
</antcall>
<!-- Another invocation to the installapp target can be done
 for further installations. -->
</target>

Run post-deployment unit tests
This step is similar to running pre-deployment unit tests, except for the nature of the JUnit test cases to run. Post-deployment test cases are expected to exploit code that is already deployed on run time and is functioning. These test cases can take advantage of server-side resources and test components, such as Cactus. As far as the build automation procedure is concerned, this step is just an invocation of a set of JUnit test cases.

Upload build

With the build all set, the automation procedure uploads the deployable units along with the generated reports to an FTP server. This step is useful both for backup purposes and to keep the identity of the build as a whole. The target shown in Listing 12 shows how to use FTP to upload build files.

Listing 12. Uploading build files to an FTP server

				
<target name="uploadbuild" description="Upload build to an FTP server">
<!-- Upload everything under the destination.dir to the FTP server. -->
<ftp server="${ftp.hostname}" remotedir="/" userid="${ftp.username}" 
   password="${ftp.userpassword}" separator="\" verbose="yes" binary="yes">
<fileset dir="${destination.dir}">
<include name="**/*.*" />
</fileset>
</ftp>
</target>

Notify testing team

At the end, the procedure sends e-mails, shown in Listing 13 to the testers to start testing. The following example assumes no authentication on the SMTP server.

Listing 13. Sending e-mails to the testing team

				
<target name="notifyteam" description="Notify testing team of the new build">
<!-- Read build information from the build.info file. -->
<property file="${destination.dir}/build.info" />
<!-- Send a mail to the testing team. -->
<mail mailhost="${smtp.hostname}" mailport="${smtp.hostport}"
 subject="Test build #${build.number}" 
   from="${smtp.from}" tolist="${smtp.tolist}">
<message>The build #${build.number} is now available for testing.</message>
</mail>
</target>

If your SMTP server requires authentication, you will need to download JavaMail .jar files.

Options when running the procedure

This section briefly describes a few options available when running the build procedure.

Conditional flows

The automation procedure described above does not impose any conditions that control the flow of task. If such a control is needed, you can use the attribute that allows failure on errors in some tasks. For example, the wsInstallApp task has a failonerror attribute that serves this purpose.

Using conditions is another way to control the flow. For more information on conditions in Ant, see Conditions task. Ant also provides dependencies to relate targets to each other to ensure the sequence of targets that get executed.

Running from the IDE

Running the Ant build file from the IDE is straightforward. As shown in Figure 3, from the Ant view, you can select the Run Ant option on one of the Ant targets that appear under the tree view.

Figure 3. Select Run Ant from the context menu

From the Modify attributes and launch dialog, select Run in the same JRE as the workspace from the JRE tab, as shown in Figure 4. Click Apply and Run.

Figure 4. Modify used JRE

After the execution, the output is printed to the Console view, as shown in Figure 5.

Figure 5. Execution output

Running from command line

Information about customizing Ant to run from the command line with Rational SDP resources is in the <IDE installation directory>\rwd\eclipse\plugins\com.ibm.etools.j2ee.ant_6.0.1.XXX. Under that directory there is a batch file, runAnt, that enables you to run your Ant build files. This batch file requires the WORKSPACE environment variable to be set. This variable points to the workspace that contains your automation resources. You can set it from the command line, or add a line to the runAnt batch file, as shown in Listing 14.

Listing 14. Running targets from command line

				
set WORKSPACE=C:\workspaces\MyWorkSpace
runAnt.bat -buildfile <path to the Ant build file>\build.xml init

Instead of init target in the preceding example, you can similarly invoke other targets from the Ant build file.

Scheduled runs

Since this build procedure is automated, the development team might decide to execute it daily, depending how often they need to provide builds for testing. This execution can be partial (running a subset of the Ant targets) or full. In both cases, the procedure can be scheduled to run at the desired frequency. For more information about how to schedule tasks on the Windows platform, see "Performing unattended daily builds with WebSphere Studio and Ant, Part 2."

Summary

After going through the steps in this article, hopefully you've improved your productivity and quality using a procedure for automating repetitive build tasks for software development teams. The build automation procedure used only a subset of automation capabilities offered by Rational and WebSphere software family of products. You will need to adjust this procedure to match the context you work in, the tools and run time you are using, the knowledge of your team, and the type of solution under development.

Acknowledgement

The author wishes to thank Nouran Abdel-Hamid, Rosaline Makar, Amr Ali, and Ahmed Mamdouh: Your efforts made this automation solution work out.

Download