Contents


Continuous delivery with Rational Team Concert and UrbanCode Deploy

Part 2. How to extend the build process

Comments

Content series:

This content is part # of # in the series: Continuous delivery with Rational Team Concert and UrbanCode Deploy

Stay tuned for additional content in this series.

This content is part of the series:Continuous delivery with Rational Team Concert and UrbanCode Deploy

Stay tuned for additional content in this series.

This series of articles presents two mechanisms for integrating IBM Rational Team Concert ™ and IBM® UrbanCode Deploy to create a continuous delivery process. The first approach included in this article, Part 1, is a packaged out-of-the-box implementation that is easy to set up. The second approach, presented here in Part 2 and in Part 3, uses extensions to the Ant build.xml file.

Note:
The use of the Ant build process does not limit you to just building Java applications. Rational Team Concert users have used the Ant build process, and Rational Team Concert extensions in the build process to build applications in other languages such as C and ADA.

The example application

The JKE Banking sample project supplied with Rational Team Concert is used as the example in this article. With this, you can quickly create the application code, if required, then apply the changes in this article to create the complete solution. Alternatively, you can extend your own build.xml files with the additions included in the download file.

Note:
It is not recommended to create the sample project in a production Rational Team Concert environment because the project creates a number of users that most people do not want within their production system. Creating the sample project in a test Rational Team Concert server is recommended.

Build definition

A build definition is in the JKE Banking sample project (jke.dev). The build definition used in this article is named jke.dev.build_and_deploy. The changes to make to the build definition are highlighted in the sections that follow.

Note:
Because you are making changes to this build definition it is appropriate to take a copy and leave the original alone.

Build.xml

The build.xml file is a text file that instructs the Ant build process which operations to perform to carry out the build process. As supplied with the sample project it contains instructions to Ant to:

  1. Build a jar file
  2. Build a war file
  3. Zip the war file into a zip file.

The zip file is then stored as a built artifact in the build record in Rational Team Concert. A log file of the operations performed by Ant is also stored as a build artifact. The original build.xml file is also included in the zip file as build-Original.xml.

Build extensions for continuous delivery

Phase 1. Build security

In the first phase of extensions to the standard build.xml process a change is made to the management of the password that the Ant process uses to connect to the Rational Team Concert build process. Many other properties are also added to support later phases of the build process.

The standard build.xml file supplied with the sample JKE banking project of Rational Team Concert includes details of the build user ID and password. These are presented as properties and shown in Listing 1.

Listing 1. User ID and password
<property name="userId" value="build" /> 
<property name="password" value="build" />

Many users of build and automation systems do not want passwords to be visible in the build.xml file. For those people, you can replace the code in Listing 1 with a reference to a file that contains an encrypted password, shown in Listing 2.

Listing 2. Encrypted password
<property name="userId" value="build" />    <property name="passwordFile" 
value="${basedir}/EncryptedBuildUserPassword.txt"/>

A file containing the encrypted password is stored within the same directory as the build.xml file. It is referenced within the build.xml file with the variable ${basedir}. Given that the build.xml file is two directories down from where the content was loaded by the Rational Team Concert build process (as shown in Figure 1), it might be useful to store a variable that identifies the 'root' of the loaded content: <property name="loadDir" value="${basedir}/../.." />.

You can use the property loadDir as the base point to reference any file within the loaded content managed by the Rational Team Concert build process.

Figure 1. The build.xml file within the project directory structure
Build.xml file within the directory structure
Build.xml file within the directory structure

The jbe executable program found within the [Rational Team Concert build installation location] > buildsystem > buildengine > eclipse directory creates the encrypted password. A readme file is in the buildsystem directory. That file details how to create a password file using the JBE program.

To complete the security enhancement, any reference to password is replaced with a reference to the password file. Replace password="${password}" with passwordFile="${passwordFile}".

The second section to add includes many new property definitions near the beginning of the file. Properties are added to the build.xml file to identify a working directory for a number of the commands that follow. The target war file to be produced is identified by a parameter too.

Add the lines of code shown in Listing 3 to the build.xml file below other property definitions that define other destination directories.

Listing 3. Properties to add to the build.xml file
<property name="destdir.war" value="${loadDir}/build/war" />
<property name="workingDir" value="${basedir}" />
<property name="target.war" value="jke.war" />

Ant build extension tasks from Rational Team Concert

There are two tasks to complete to use the Ant tasks supplied with Rational Team Concert. The first task is to include the location of the Jazz Build Toolkit tasks on the Ant library path. To do this, select the checkbox to Include the Jazz build toolkit tasks on the Ant library path as shown in Figure 2. You'll find this option on the Ant tab of the Rational Team Concert build definition.

Figure 2. Reference the Jazz build toolkit within a build definition
Select the Jazz Build Toolkit option on the build
Select the Jazz Build Toolkit option on the build

The second thing to do is to add the task definition to the Ant build.xml file. An example of the task definition for the start build activity task is shown in Listing 4.

Note:
All required task definitions are already in the example build.xml file that is used as the starting point for this work. The task definition entries only need to be manually added if the user is augmenting an existing build.xml file with the steps described in this article. If that is the case then simply copy the task definition block from the build.xml file given in the download files link at the bottom of the article.

Listing 4. Example task definition for the start build activity function.
<taskdef name="startBuildActivity"
classname="com.ibm.team.build.ant.task.StartBuildActivityTask" />

The build definition, the build result and the related artifacts are shown in Figure 3. Three builds are shown and are identified using the date and time build label.

Build log files
Textual log files produced during the build. Within the extensions created in this document a number of new log files are produced for help with debugging.
 
Snapshot
A snapshot of the source content held within the Team Concert source code management facility is taken at the time of the build.
 
Built artifact
The war file produced in the build is stored within the build record.
Note:
If you have build processes that produce many large objects you might want to copy the files to a specific network share and then publish a link to the artifacts in the build result.

A list of work items recently delivered and referenced by the build result is included. This helps to show the traceability between the different elements of the software development lifecycle, as shown in Figure 3.

Figure 3. Hierarchy of content produced by a build
Hierarchical representation of built artifacts
Hierarchical representation of built artifacts

Extending the information available from the build record, Figure 4 shows the details of a change set that is associated with a task. The sequence of operations that generate this linked data is:

  1. A developer is assigned the Rational Team Concert task number 1239.
  2. The developer begins to work on the activity and modifies (or creates) three java source code files and a project build.xml file.
  3. The four file changes are grouped into a single change set which is then associated with the developers' task.
  4. The task is then delivered to the shared work stream the developer is using.
  5. A scheduled build process starts a short time after the delivery and identifies the newly delivered change set as a new change since the last build. The task is added to the list of work items in the build record.
Figure 4. Hierarchy of extended build artifacts – 1
Extended representation of built artifacts – 1
Extended representation of built artifacts – 1

Figure 5 extends this image further to show the traceability that is available from the developers' task into other areas of the software development lifecycle, including:

  • The story from which the task was derived
  • The requirement that elaborates on the story
  • The test case associated with the story.

Figure 5 also shows the results of executing a test that is deemed to have failed and a defect has been created. The pale lines show the links created from the defect to the story, requirement and test case automatically. Finally a change set is shown associated with the defect which shows that the defect has been fixed and the specified files have been modified. The defect will therefore also show up on the next build record.

Figure 5: Hierarchy of extended build artifacts - 2
Extended representation of built artifacts – 2
Extended representation of built artifacts – 2

Phase 1. Summary

In phase one a simple change is made to the security of the build process and a number of properties are added. Phase one has also explored the extensive traceability across the development lifecycle that is available in Rational Team Concert and associated products.

Phase 2. Load the WAR file into UrbanCode Deploy CodeStation

In this phase of the build.xml extensions, a command line client connects to UrbanCode Deploy and loads the generated WAR file into the versioned storage area of the UrbanCode Deploy product (known as CodeStation).

Note:
CodeStation is not a separate product it is simply a name given to the versioned storage area provided in UrbanCode Deploy.

Each UrbanCode Deploy component can store multiple versions of files to deploy to targets. You can select different versions for deployment to different environments and progress a version of a component through a series of environments performing different test and verification activities in the route to production. Figure 6 shows the result of loading the WAR file into UrbanCode Deploy.

Figure 6. Loading built content to UrbanCode Deploy
Loading a deployable file in UrbanCode Deploy
Loading a deployable file in UrbanCode Deploy

Properties

It is useful to store properties used within the build process as a series of definitions at the top of the build.xml file. If the properties need to change (and they will if you want to take the files and make use of them in your own environment) group them together. It'll make the changes easier.

The properties shown in Listing 5 establish the connection to the UrbanCode Deploy server and identify the component into which new versions are created. The use of an authentication token avoids the necessity to store a plain text password in the build.xml file. The server name and authentication token for UrbanCode deploy should both be specific to your environment.

Listing 5. Properties
<!-- UrbanCode Deploy Connection Properties -->
<property name="Deploy_WebURL" value="https://rational-srv-02:8443" />
<property name="Deploy_AuthToken" value="6b29ed7f-956b-46ed-97ac-f3cbd4e6e876" />
<property name="Deploy_Component" value="Sample Component" />

The properties in Listing 6 enable the UrbanCode Deploy component version to store a reference to the Rational Team Concert build record.

Listing 6. Store a reference to the build record
<!-- RTC information for UrbanCode to use to create a link to the RTC build record --> 
<property name="Deploy_Component_Version_Backlink" value = "Team Concert Build Record" />
<property name="CCM_URL" value="https://rational-srv-02:9443/ccm" />
<property name="BackLink_Base_URL"
	value="${CCM_URL}/web/projects/…#action=com.ibm.team.build.viewResult&amp;id=" />

The person responsible for the deployment activity can quickly find the appropriate build record and then easily find the work items that were delivered for that build. Further traceability links can be followed to requirements and test cases if the requirements and test management capabilities of the Rational product set have been used.

Note:
The value of the last item is shortened in the Listing 6. To find out what the build record URL is for your site and Rational Team Concert project load a build record in a web browser and copy the URL from the project name up to, and including, the &id=. The build ID that follows the &id= is added as part of the URL construction process that is described in Phase 4 of this article.

Note:
&amp; is used in the URL shown in Listing 6.

Listing 7 is a log file used to gather information about the component version creation process. Should any error occur this file is a useful source of debug information.

Listing 7. Log file to gather information
<!-- Log files used by the connectivity to UrbanCode Deploy --> 
<property name="DeployComponentVersionCreate"
		value="${workingDir}/DeployComponentVersionCreate.log" />

Deploy build target

A new target is added to the build.xml file to contain the tasks that interact with UrbanCode Deploy. This allows you to select when the deployment interaction operations run and allows the fast turnaround of the compile and unit test activity to continue without necessarily interacting with UrbanCode Deploy.

The target to add is contained within the build.xml lines in Listing 8.

Listing 8. Add a target
<target name="deploy" depends="compile">
</target>

The new target depends on the compile process which requires a new WAR file to be produced before attempting to add it to a new component version in UrbanCode Deploy.

To perform the deploy operation every time the build runs it is necessary to include the Deploy target on the main build target as shown in Listing 9.

Listing 9. Deploy target
<target name="all"
depends="compile, deploy" />

UrbanCode Deploy command line interface

Prior to completing the next steps he UrbanCode Deploy command line interface must be installed. This is very easy to do and it can be downloaded from the tools menu on the right hand side of the UrbanCode Deploy web interface. Extract the zip file to a location on the build server and add the location to the path for the build server. Stop and restart the running build engine from a new command window. If you don't stop and restart the build engine process cannot access the UrbanCode command line interface.

UrbanCode Deploy login

The first step of the Deploy target is to login to UrbanCode Deploy. This is the first instance of using the udclient command line interface with the argument of login and with sub arguments for the authentication token and the url to connect to. Both are taken from properties defined earlier. The build.xml command to login is shown in Listing 10.

Listing 10. Login command
<exec executable="cmd" dir="${workingDir}">
    <arg value="/c"/> 
    <arg value="udclient"/> 
    <arg value="login" /> 
    <arg value="-authtoken"/> 
    <arg value="${Deploy_AuthToken}"/> 
    <arg value="-weburl"/> 
    <arg value="${Deploy_WebURL}"/> 
</exec>

Reporting build progress with StartBuildActivity

The StartBuildActivity build command is the first part of many operations in this article. It reports the command being executed to the user watching the build in the Eclipse or the web interface. It also records a stage of the build in the build record. The build record is located in the Rational Team Concert database. Figure 7 shows an example of the reporting of build progress for the Eclipse user interface.

Figure 7. Reporting build stage progress
Informative text to show the stage of the build
Informative text to show the stage of the build

The label field of the StartBuildActivity statement includes a reference to the name of the component that was defined in the properties section at the start of the file.

Listing 11. StartBuildActivity
<startBuildActivity activityIdProperty="CreateDeployComponentVersion"
    label="Create new component version - ${Deploy_Component}"
    buildResultUUID="${buildResultUUID}" 
    repositoryAddress="${repositoryAddress}"
    userId="${userId}" passwordFile="${passwordFile}" 
    autoComplete="true" />

Create component version in UrbanCode Deploy

After the login has completed the next step is to create the new version of the component in UrbanCode Deploy. After creating the new version it initially has no stored content since this has to be added afterwards as a separate step. The first arguments of the command are the same as the login command and then further options are given which include the createVersion option and the identifiers for the component and the name of the new version to be created. The name of the version, given by ${buildLabel} is passed to the ant execution by the Rational Team Concert build process. The build label is a date and time by default.

Listing 12. Argument options
<exec executable="cmd" dir="${workingDir}"
    output="${DeployComponentVersionCreate}" >
    <arg value="/c"/>
    <arg value="udclient"/>
    <arg value="-authtoken"/> 
    <arg value="${Deploy_AuthToken}"/> 
    <arg value="-weburl"/> 
    <arg value="${Deploy_WebURL}"/> 
    <arg value="createVersion"/> 
    <arg value="-component"/>  
    <arg value="&quot;${Deploy_Component}&quot;"/> 
    <arg value="-name"/> 
    <arg value="${buildLabel}"/> 
</exec>

The final command in the version creation process is the storage of the log file generated by the execution of the createVersion command shown in Listing 12. The output argument references a parameter defined earlier in the build.xml file that is a file held within the directory structure of the build machine. The log file is published into the Rational Team Concert build record using the command shown in Listing 11. The textual description of the log file includes the component name so in larger builds for customers who have many components it will be possible to identify which log is associated with which component.

Listing 13. Log file storage
<logPublisher filePath="${DeployComponentVersionCreate}"
    label="Create new component version - ${Deploy_Component} / ${buildLabel}"
    buildResultUUID="${buildResultUUID}" repositoryAddress="${repositoryAddress}" 
    userId="${userId}" passwordFile="${passwordFile}" verbose="true"/>

Add content to a new component version

After the component version is created in UrbanCode Deploy it is necessary to load the deployable content into the version reference. The first part of the version content creation is the use of the StartBuildActivity build command.

The second part of the version content creation is the use of the udclient command to load the new content into the component version in UrbanCode Deploy. The first arguments of the command are the same as the login command. The addVersionFiles command, the identifiers for the component, and the location of the files to be loaded are also included.

Listing 14. Update the version reference
<startBuildActivity activityIdProperty="AddDeployComponentVersion"
    label="Add component version to CodeStation - ${Deploy_Component}"
    buildResultUUID="${buildResultUUID}" repositoryAddress="${repositoryAddress}"
    userId="${userId}" passwordFile="${passwordFile}" autoComplete="true" /> 

<exec executable="cmd" dir="${workingDir}" >
    <arg value="/c"/> 
    <arg value="udclient"/> 
    <arg value="-authtoken"/> 
    <arg value="${Deploy_AuthToken}"/> 
    <arg value="-weburl"/> 
    <arg value="${Deploy_WebURL}"/> 
    <arg Value="addVersionFiles"/> 
    <arg Value="-component"/> 
    <arg Value="&quot;${Deploy_Component}&quot;"/> 
    <arg Value="-version"/> 
    <arg Value="${buildLabel}"/> 
    <arg Value="-base"/> 
    <arg Value="${destdir.distro}"/> 
    <arg Value="-include"/> 
    <arg Value="${target.war}"/> 
</exec>

Phase 2. Summary

After phase two the continuous delivery process pushes content from the Rational Team Concert build into UrbanCode Deploy as a new component version that contains the newly built war file.

Phase 3. Create a link to the component version on the RTC build record

Since the purpose of this work is to create a seamless flow of information from the development team to the deployment environment it is useful to create a link from the Team Concert build record to the newly created component version. This enables a developer to easily view the deployable content and from there to see into which environments it has been deployed.

The format of the link is: https://<UrbanCode server name>:<UrbanCode secure port>/#version/<component version ID>

The purpose of the next sections is to get the component version ID, construct the URL and store it in the Team Concert build record.

Get the component version ID

The component version ID can be extracted from the JSON format output result of the createVersion command shown above. In order to do this a Java application is executed from within the build script to process the output of the createVersion command and extract the ID. The Java application is supplied in source code format with the other files of the article (file getJSONPropertyValue.zip). It includes a build.xml file to enable you to build it. You can build the executable by creating a build definition in Rational Team Concert. Once built, add the generated jar file to the directory: JKEBuildScripts \ sample.jke.build \ Support \ get-JSON-property-value.jar then deliver it to the stream. Follow the same steps as you would to deliver the build.xml file when changes have been made.

To execute the application from within the build.xml file many additional properties are added to the existing properties section at the top of the build.xml file. The properties to add are shown in Listing 15.

Listing 15. Additional properties
<!-- Properties for a JSON processing application that will extract the component version ID --> 
<property name="getJSONPropertyValue"
    value="${workingDir}/Support/get-JSON-property-value.jar"/>
<property name="getJSONPropertyValue-Log" value="${workingDir}/getJSONPropertyValuelog.txt"/>
<property name="DeployComponentVersionCreationIDFile"
    value="${workingDir}/DeployComponentVersionCreationIDFile.txt"/>
<property name="VersionCreationJSONTitle" value="id"/>

A path section is added to the Deploy target under the commands added in Phase 2. This enables the java execution to find the compiled jar file.

Listing 16. Add a path section
<path id="getJSONPropertyValuePaths">
    <pathelement path="."/> 
    <pathelement path="${getJSONPropertyValue}" /> 
</path>

The java command is then executed to process the data from the createVersion command. The arguments to the command are:

  • Input file: The output file generated from the createVersion command above.
  • Title: The JSON format title for which the value is to be extracted.
  • Output file: The name of a file to create. The file contains the extracted data.

An example of the input file is shown below. The information required is the id field.

Listing 17. Input file
{
    "id": "1044c181-b77c-4468-b1e8-0fc72cc370e0",
    "name": "I20140506-2120",
    "type": "FULL",
    "created": 1399407705637,
    "active": true,
    "archived": false
}

The section of xml required to execute the java application with the required properties is shown in Listing 18.

Listing 18. XML with required properties
<java jar="${getJSONPropertyValue}" output="${getJSONPropertyValue-Log}" fork="true">
    <classpath refid="getJSONPropertyValuePaths" />
    <arg value="inputFile=${DeployComponentVersionCreate}" />
    <arg value="Title=${VersionCreationJSONTitle}" />
    <arg value="outputFile=${DeployComponentVersionCreationIDFile}" />
</java> 

The output file is created after the command runs. The output file contains the component version ID: 1044c181-b77c-4468-b1e8-0fc72cc370e0.

The next step is to read the file content and get the ID into a property value so that it can be used in later steps. The command to load the file into a property is shown in Listing 19. It also includes an echo command so that the ID is displayed in the build log file.

Listing 19. Load file command
<loadfile property="DeployComponentVersionCreationID"
srcFile="${DeployComponentVersionCreationIDFile}"/> 
<echo message="ID of Component version =${DeployComponentVersionCreationID}" />

Publish a link to the component version in the build record

Use the Rational Team Concert link publisher command to publish a URL constructed from a header URL and defined as a property at the top of the file and with the ID extracted from the file in the step above. Add the header URL property needs to the build.xml file as shown in Listing 20.

Listing 20. Header URL property
<property name="DeployVersionLinkHeader" value="${Deploy_WebURL}/#version/"/> 
Listing 21. Publish the link
<!-- Publish Link to Deploy Component Version on RTC Build Result -->
<linkPublisher label="UrbanCode Deploy Component - ${Deploy_Component}"
url="${DeployVersionLinkHeader}${DeployComponentVersionCreationID}"
componentName="Component Versions" buildResultUUID="${buildResultUUID}"
repositoryAddress="${repositoryAddress}" userId="${userId}"
passwordFile="${passwordFile}" failOnError="false" />

Phase 3. Summary

After phase three the continuous delivery process can push content from the Rational Team Concert build into UrbanCode Deploy. A link is established from the build record to the component version, too.

Phase 4. Create a link to the build record on the component version

In addition to the links created in Phase 3, it is also useful to create a link from the component version back to the build record in Rational Team Concert. This allows the person responsible primarily for deployment to easily find the Rational Team Concert work items included in the build used to create the component version they are using. From the work items you can then follow other traceability links to test cases and requirements as indicated in Figure 5.

Properties

A new property is added above the exec command shown in Listing 22 for the build backlink URL from values previously defined in the build.xml file.

Listing 22. Add a new property
<property name = "Build_Backlink" value = "${BackLink_Base_URL}${buildResultUUID}" />

The command in the build.xml file to create and publish the link is shown in Listing 23.

Listing 23.Create and publish the link
<exec executable="cmd">
    <arg value="/c"/>
    <arg value="udclient"/>
    <arg value="-authtoken"/>
    <arg value="${Deploy_AuthToken}"/>
    <arg value="-weburl"/>
    <arg value="${Deploy_WebURL}"/>
    <arg value="addVersionLink"/>
    <arg value="-component"/>
    <arg value="&quot;${Deploy_Component}&quot;"/>
    <arg value="-version"/>
    <arg value="${buildLabel}"/>
    <arg value="-linkName"/>
    <arg value="&quot;${Deploy_Component_Version_Backlink}&quot;"/>
    <arg value="-link"/>
    <arg value="&quot;${Build_Backlink}&quot;"/>
</exec> 

The result of the links created in Phase 3 is shown in Figure 8.

Figure 8. Links between UrbanCode Deploy and Rational Team Concert
Links between RTC build and UrbanCode Components
Links between RTC build and UrbanCode Components

Phase 4. Summary

In phase 4 you directed the continuous delivery process to push content from the Rational Team Concert build into UrbanCode Deploy, and a link is created from the component version to the build record.

Summary

This article presented the first part of a continuous delivery process that is appropriate if you have complex build processes that generate content to put in multiple UrbanCode components. The final article of this series shows you how to initiate a deployment to complete the continuous delivery process.


Downloadable resources


Related topics


Comments

Sign in or register to add and subscribe to comments.

static.content.url=http://www.ibm.com/developerworks/js/artrating/
SITE_ID=1
Zone=DevOps, Rational
ArticleID=996968
ArticleTitle=Continuous delivery with Rational Team Concert and UrbanCode Deploy: Part 2. How to extend the build process
publish-date=02062015