Simplify Rational Build Forge projects with an external control file

IBM® Rational® Build Forge® is a web-based tool for automating build and deployment tasks. Build Forge projects that require too many steps, checkpoints, and exceptions can become tedious to create and difficult to maintain. This article explains how to simplify those types of Build Forge projects by using an external control file. We begin with a brief explanation of how Build Forge projects work and then present this solution as an easy way to simplify things.

Li Luo (liluo@ca.ibm.com), IBM Certified IT Specialist, IBM

author photoLi Luo has a bachelor’s degree and a master’s degree in Engineering from Harbin Institute of Technology, in China. She also has an MS degree in Computer Engineering from the University of Waterloo, in Canada. Li is working on different projects in the IBM Software Group as a system administrator or infrastructure consultant.



José De Jesús (jdejesus@us.ibm.com), Senior Certified Architect, IBM

Photo of Jose De JesusJosé De Jesús is a Senior Certified Architect on the IBM Software Services for WebSphere (ISSW) for IBM (I4I) team. José is the Non-SAP Infrastructure team lead for the Blue Harmony internal IBM project, as well as the ISSW Certification lead. He is also leading the I4I Cloud Initiative and the I4I Technical Vitality Initiative and is a member of the Distributed Management Task Force. José has a BS degree in Computer Science from Fordham University and an MS degree in Management of Technology from the University of Miami.


developerWorks Contributing author
        level

Derrick Wong (derrickw@ca.ibm.com), Senior Consultant, IBM

author photoDerrick Wong is a senior consultant in the Software Group for IBM Canada. He has a BMath degree from the University of Waterloo and started to work at IBM during his co-op terms as a student.



18 December 2012

Overview of Rational Build Forge components

IBM® Rational® Build Forge® is a web-based tool that is used for running, tracking, scheduling, and distributing processes to various computers. Build Forge helps with automating change and release management activities.

As Figure 1 illustrates, Build Forge consists of a server component and agents that are running on target systems. The server communicates with the agents and oversees the activities on the managed target systems, organizes the projects, and manages user access, permissions, logging, and audit trails. Any system that takes part in a build, deployment, and test process managed by Build Forge requires a Build Forge agent.

Figure 1. Build Forge components
General Layout: Console, Engine, Database, Clients

The Build Forge management console is the main graphical user interface (GUI) for the system. It is the interface that Build Forge administrators use to manage users, configure resources, organize commands into projects, and manage the server and environment resources.

The Build Forge agent is a small process that does three things:

  • Listens for instructions from the management server
  • Executes those instructions (project steps)
  • Sends results from the instructions back to the management server in the form of output or logs.

Build Forge also includes adapters for communicating with version control and defect-tracking systems, as well as with test systems. It supports the Rational Automation Framework for WebSphere, which can help encapsulate the configuration and deployment of applications.


Build Forge projects

When a Build Forge agent is installed and running on a target system, the Build Forge server can connect to the target system and execute the requested commands by running a Build Forge project.

You can create, schedule for execution, and execute a Build Forge project through the management console. Figures 2 through 7 explain the general steps.

A target system runs a Build Forge agent that can execute the requested commands and return results to the Build Forge server. Configuring the target system's server authentication, as shown in Figure 2, is the first step. For this, use the credentials with which you normally log in. The connection can be tested, and it must pass the tests before you continue to the next step.

Figure 2. Configure server authentication
Screen capture shows how to authenticate servers

A selector is configured so that a Build Forge project will run on the target system.

The next step, shown in Figure 3, is to configure a selector. A selector is configured so that a Build Forge project will run on the target system. Selectors are dynamically chosen servers or systems on which the project or part of the project will run. In Figure 3, BF_NAME is an example of a selector variable with a value that is DS_DEV2_BHD2DS02_Portsmouth.

Figure 3. Configure a selector
Screen capture shows how to configure selectors

Figure 4 shows the third step, which involves defining your environment. An environment is a set of variables that are defined for a project or part of a project before its execution. You can use environment variables to abstract common parts of code, such as paths to binary files and version numbers.

Figure 4. Define Build Forge environment variables
Screen capture shows how to add new variables

In Step 4, you create the Build Forge project and add the related steps. Each project step consists of one or more commands that you can include in the Details section for the step. Figure 5 shows an example. A project contains a sequence of steps to run on specified selectors and with specified environments. Each step is composed of one or more commands that can be run from a command line. You can add, edit, or delete the steps and change their sequences. You can also specify the selector and the environment for every step or for the whole project.

Screen capture: creating a project and its steps

You can execute a Build Forge project after it is created.

Step 5, shown in Figure 6, is where you can execute the project. You can do so by clicking the blue Play button next to the project name or by clicking the Execute button on the project page. You can view the log and the results of the run from the Jobs menu on the left.

Figure 6. Execute the Build Forge project
Screen capture shows how to execute a project

You can also create a schedule of execution for your projects through the Schedules menu (see Figure 7). You can use that menu to plan the execution of a project that is based on a specific time and other criteria.

Figure 7. Schedule the Build Forge project
Screen capture: scheduling execution of a project

In general, this is the effort that is required to create and run a Build Forge project.


A more complex example

Now, assume that we have a more complex example project, with the following characteristics:

  • An application that is Microsoft Windows-based and consists of multiple application servers and a database server.
  • The Build Forge project needs to incorporate dozens of application-specific projects, each of which consists of several packages.
  • Each package, in turn, consists of several deployment artifacts.
  • Both packages and artifacts must be built and executed in a specific order and on a specified server.
  • The deployment artifacts can be .msi files, .sql files, .dll files, and application configuration files.
  • Depending on the build and deployment, certain files must be deployed in a certain way and to specific servers.
  • Before deployment, in some cases, special strings in the configuration files must be replaced before running the corresponding application.
  • The project is programmed to know what to do, depending on the artifact type.
  • This behavior should be extensible for new artifact types introduced in future builds.
  • The list of artifacts to be deployed and their order will change frequently.

Using an external control file

These requirements are much more elaborate and would make it difficult to implement a solution with just Rational Build Forge. It would involve hundreds of Build Forge steps and either too many scripts or a very complex script to handle the different possibilities. Maintenance would also be problematic.

The solution that we present here uses a generic Java-based script that can process a .properties file (which we call a control file) that contains "instructions" on what to do and when. The Build Forge project would call this Java-based script as one of its steps.

These instructions are merely a list of package names and artifacts listed as key-value pairs in the order in which they should be deployed (a typical .properties file). The Java program that processes these instructions is the key. It recognizes exactly how to handle each type of artifact listed. The complete source code for the Java-based script and a sample control file is provided with this article in the Download section.

Let's take a look at the general flow. Figure 8 shows what happens at the level of the Build Forge project. The steps in the project perform the initial setup work, such as gathering all of the necessary artifacts from IBM® Rational® ClearCase® software and moving them to the correct target systems. The Java-based script then reads the control file to perform the corresponding build and deploy operations.

Figure 8. General flow of the Build Forge project
General flow for using a control file with Build Forge

When the Java-based script runs, it reads the control file to determine which packages and files to need to run and in what order. The script then executes them one by one according to their file types. Figure 9 shows a sample control file. The key PackageDeploymentOrder specifies two packages, Package1 and Package2, to run in the order listed. The dynamically created keys, Package1.RunOrder and Package2.RunOrder, specify the files to build for each package. This script is also capable of editing a specific configuration file according to the find-and-replace key-value pairs in the control file.

Figure 9 shows two keys that are used to replace a string in a configuration file:

  • PackageName.FileName_Find_Lines
  • PackageName.FileName_Replace_Lines
Figure 9. Sample control file
Shows the main structure of a sample control file

In some cases, the build or deployment process might need to switch between different target systems. In that case, the Java-based script uses Java Remote Method Invocation (RMI) to reach the remote environment and deploy the necessary code onto it. For example, as Figure 8 shows, some of the artifacts that get built or extracted should be executed on the application servers, but others run on the database server. SQL files in the control file and all other data related deployments go to the database server, while the application servers receive other types of artifacts.


Benefits

By abstracting your build and deployment tasks with an external control file and a generic script that can act upon its content, you can simplify what would otherwise be an incredibly complex Build Forge project. Using a control file decouples some of the invocations from Build Forge and makes creating and maintaining the application much easier. Of course, what is presented here is only an initial iteration of the solution. It can be improved in many ways. Here are a few ideas:

  • You can configure the control file either to update what changed over the previous deployment or to run a new deployment.
  • A check-before-deploy feature or flag can be implemented that would involve the user as little as possible.
  • The deployment operations could be generalized and encapsulated in a plug-in class that would define generic operations for new plug-ins. This would make the code more reusable and make it easier to address the specific build and deployment processes of other applications.

Using other tools

With this type of approach, you would still use Build Forge as the main platform to automate and streamline your software build and deployment process.

The control file approach works well with other tools too. For example, it does not intend to replace the functions that are provided by Apache Ant, which is an open source, Java-based build tool to automate software builds, similar in many ways to a "make" utility. In this case, you can use Ant, for example, to transfer the artifacts from ClearCase to the target system and use the control file and the Java-based script to the build and deployment parts.

Ant is based on Java and can be easily extended with Java classes. So it is also possible to replace some of the script's functionality with Ant. In some cases, however, because Ant works from an XML file, users might find it difficult to use for very complex build and deployment processes, as in our example. One reason is that Ant and XML is not as convenient to use to depict the complex logic.

In this case, although XML is human-readable, it is still more difficult to maintain and validate than a simple .properties file. For example, defining common XML elements and reusing them for other applications can be tedious. Also, as mentioned earlier, you can easily extend the functionality of the control file with separate Java classes that implement specific interfaces. A JSON interface might also be a viable alternative.

Ultimately, it depends on design philosophies, but the idea is to simplify things and make your build and deployment tasks more reliable and much easier to maintain. Knowing which tools to use and when depends on the specific needs of your project.


Real-life results

We used an external control file to complement our Build Forge projects on an internal IBM project. This approach has worked quite well and enabled our Build Forge projects to handle very complex processes for some specific applications. We hope that sharing the ideas presented in this article with you will shed some light and give you ideas for your own projects.


Acknowledgement

The authors would like to thank Paul Gordon for his technical review of this article.


Download

DescriptionNameSize
Source codeSimplifyingBuildForge-generalizedsourcecode.zip8KB

Resources

Learn

Get products and technologies

  • Check the Trials and Demos page for Rational software.
  • Evaluate IBM software in the way that suits you best: Download it for a trial, try it online, use it in a cloud environment, or spend a few hours in the SOA Sandbox learning how to implement service-oriented architecture efficiently.

Discuss

Comments

developerWorks: Sign in

Required fields are indicated with an asterisk (*).


Need an IBM ID?
Forgot your IBM ID?


Forgot your password?
Change your password

By clicking Submit, you agree to the developerWorks terms of use.

 


The first time you sign into developerWorks, a profile is created for you. Information in your profile (your name, country/region, and company name) is displayed to the public and will accompany any content you post, unless you opt to hide your company name. You may update your IBM account at any time.

All information submitted is secure.

Choose your display name



The first time you sign in to developerWorks, a profile is created for you, so you need to choose a display name. Your display name accompanies the content you post on developerWorks.

Please choose a display name between 3-31 characters. Your display name must be unique in the developerWorks community and should not be your email address for privacy reasons.

Required fields are indicated with an asterisk (*).

(Must be between 3 – 31 characters.)

By clicking Submit, you agree to the developerWorks terms of use.

 


All information submitted is secure.

Dig deeper into Rational software on developerWorks


static.content.url=http://www.ibm.com/developerworks/js/artrating/
SITE_ID=1
Zone=Rational
ArticleID=852442
ArticleTitle=Simplify Rational Build Forge projects with an external control file
publish-date=12182012