Overview and motivation
IBM® Rational® Build Forge® is often used to orchestrate complex processes, such as building, packaging, or testing software. In most nontrivial situations, these processes involve multiple servers. To avoid preventable failures, release engineers often perform a series of checks to make sure that all of the servers that are going to be involved in the process are ready. With increasing the number of servers, this can be both tedious and error-prone.
A solution that works well in practice is to use a dashboard, which is a tool that regularly checks on the servers and presents the aggregate information in an easy-to-read manner. It enables the user to review the health of all the relevant servers at a glance.
This article explains how to create such a dashboard for Build Forge. By the end of the article, you will end up with a running dashboard similar to the one shown in Figure 1.
Figure 1. Simple dashboard showing 5 servers, one of which is low on disk space
This article is based on the assumption that the following are available:
- Build Forge Version 7.0.1 or later: A version of Build Forge that provides the API used in this article.
- Java™ 5 or later (or Perl): The Build Forge API is offered in both Java™ and Perl. In this article Java is used, but all the concepts can be equally implemented in Perl.
- (Recommended) Eclipse Version 3.3 or later: Eclipse project files are provided, along with the source code.
An implicit assumption that we make in this article is that, in your Build Forge Management Console, you have defined at least one server and installed the Build Forge agent on it.
The application presented in this article consists of these components:
- Build Forge server
- Dashboard application
OSPerfMeter, a cross-platform performance measurement tool (see the Downloads section)
By default, the dashboard application polls the Build Forge server and displays the following information about each of the monitored servers:
- Number of Build Forge jobs currently running
- Current CPU load
- Free memory available
- Disk space available on the current partition
The list of parameters can be extended to meet your needs. See the section dedicated to custom parameters for details on how to achieve that.
OSPerfMeter can be used to provide alternative
measurements to the default list. It is a cross-platform Java application. For more details see the OSPerfMeter section.
Step 1. Create and configure the collector
- In the left menu of the Build Forge Management Console, select Servers and then Collectors.
- In the lower half of the main page, in the input box asking for a name, type the name for the new collector (for example,
mydashboard) and click Save.
- Use the form in the lower half of the main window to add the ON_DASHBOARD variable with Set Value as the type, and the value set to true.
- Click Save. The upper-half of the main page will update and display the newly created variable.
- In the upper half of the main page, click Add Collector Variable button. In the lower-half, select the Built-in type, and then select
CPU_LOAD. Click Save.
- Repeat the previous step for built-in variables: CPU_LOAD1, MEM_FREE,
DISK_FREE, and OS
Figure 2. The collector configured
The newly created collector should look like Figure 2. Notice that order of variables is irrelevant.
Step 2. Assign the collector to servers
Now that the collector is ready, you can assign it to the interesting servers (servers that will be monitored).
- In the Build Forge Management Console left menu, select Servers.
- In the upper half of the main page, click one of the servers. In the form in the lower half, in the Collector field, select the name of the collector that you created in the previous section, and then click Save.
- Repeat the previous step for all of the servers that are to be monitored and displayed on the dashboard.
Verify that all interesting servers have the correct collector specified in the Collector column in the server list.
Figure 3. Configured servers
After the collector is assigned, the servers will start displaying additional information in their manifests. In our case the manifest will include the following fields:
CPU_LOAD, CPU_LOAD1, MEM_FREE,
ON_DASHBOARD. In other words, by assigning a collector, you demand that the server starts reporting on additional parameters.
Here is how you can display the manifest of a server:
- In the Build Forge Management Console menu, on the left, select Servers.
- In the upper half of the main page, click one of the servers that has the dashboard collector assigned.
- In the lower half, click the Manifest tab.
Figure 4. Manifest described
What this server reports is that it has a 0% CPU load, 1760MB of free memory, and about 5.8GB of free disk space (on the partition specified for this particular server).
If the manifest is empty, it means that the server is in the process of updating it. Wait 20–30 seconds and redo the steps to display the manifest.
Notice that, in addition to the requested variables, there are a few others (
BF_AGENT_VERSION, BF_JOBS, and others) These parameters, prefixed by
BF_, are the default parameters and are found in the manifests of all servers.
Step 3. Generate the HTML files
So far, you have set up the Build Forge Management Console to keep track of several parameters for your servers, namely: CPU load, available memory, and free disk space. Keep in mind that the list of parameters can be extended to include any parameter, as long as there is a command that can measure it.
Now you can run the dashboard application and generate the HTML files that comprise the dashboard. The following steps should be done on a computer that has Java 5 or later installed:
- Download the dashboard application and the accompanying files from the Downloads section of this article.
- Download the Build Forge API from your Build Forge server:
- Go to
http://<yourBFserver>/clients/and log in, if necessary (use your Build Forge Management Console credentials).
- Select the Services Layer Java Client JAR file.
- Save the JAR file in the same directory as the dashboard application
- Go to
- Edit the
buildforge.propertiesfiles to match the settings on your server. For now, use your own credentials. Later you can create a separate user ID for this task.
- Run the dashboard application:
update_dashboard.bat(on Microsoft® Windows® platforms)
update_dashboard.sh(on UNIX® or similar platforms)
If the execution is successful, the dashboard will generate several HTML files. In case of failure, analyze the error messages to pinpoint the issue: network problems, authentication problems, and so forth.
An HTML file will be generated for each server that uses the collector defined previously. The name of the files is of the form
<FULL_HOSTNAME>.html (for example:
An additional file called
lastupdate.html is generated. It contains the date and time when the dashboard was generated.
Step 4. Edit the dashboard index.html file
The index.html file is structured similarly to a template with placeholders. It includes the HTML files generated by the dashboard application by using
<iframe> HTML elements. This way, the
index.html file needs to be edited only once. in the beginning (or when servers are added or removed), but the server HTML files can be generated often without causing conflicts.
For example, assume that you have five servers with the characteristics described in Table 1.
Table 1. Description of 5 hypothetical servers used as example
|Server name||Operating System||Generated file|
By editing the
index.html file to include the appropriate files, you get what the following code listing shows.
Listing 1. The modifications to be made to index.html file in to make it display the servers in Table 1
<tr> <td> <span class="stitle">Windows</span> </td> <td> <div class="boxes"> <iframe class="sframe" frameborder="0" scrolling="no" src="servers/myserv2.mydomain.com.html"> </iframe> </div> </td> </tr> <tr> <td> <span class="stitle">Linux</span> </td> <td> <div class="boxes"> <iframe class="sframe" frameborder="0" scrolling="no" src="servers/myserv1.mydomain.com.html"> </iframe> <iframe class="sframe" frameborder="0" scrolling="no" src="servers/myserv4.mydomain.com.html"> </iframe> </div> </td> </tr> <tr> <td> <span class="stitle">AIX</span> </td> <td> <div class="boxes"> <iframe class="sframe" frameborder="0" scrolling="no" src="servers/myserv3.mydomain.com.html"> </iframe> <iframe class="sframe" frameborder="0" scrolling="no" src="servers/myserv5.mydomain.com.html"> </iframe> </div> </td> </tr>
Now the dashboard view is ready to be displayed. If you open the
index.html file in your browser, the dashboard should look like Figure 1.
MYSERV1 is low on disk space and has been color-coded accordingly. On the dashboard, the server can be in one of three states: OK (green), Warning (yellow), and Critical (red). To find more details read the help page of the dashboard.
Step 5. Copy the dashboard files to a Web server
Now that the dashboard displays information about the servers, it is the time to publish it, meaning to copy the files to a Web server. There are two ways to do this:
- Use the Build Forge built-in Web server
- Use a dedicated Web server for the dashboard
The first method is simple:
- On the server where Build Forge Management Console is installed, create a directory:
- Move the content of the
web/directory (the updated dashboard files) to the newly created directory.
- Point the browser to
http://<MY_BUILDFORGE_SERVER>/dashbard/to see dashboard.
Notice that this page is public and anyone in your network will be able to access it.
For the second method, you must follow these steps:
- Install and configure a Web server on a separate server.
- Move the content of the directory
web/(the updated dashboard files) to a directory under the Web root of the server.
- Point the browser to
http://<DASHBOARD_WEB_SERVER>/<PATH_TO_DASHBOARD>/to see the dashboard.
Step 6. Embed the dashboard in Build Forge (optional)
This step is optional and requires Build Forge V7.1 or later and administrative privileges.
- Log in to the Build Forge Management Console as an administrator.
- In the upper-right part, click the UI Config tab.
Figure 5. UI Config tab
- Click Add Tab, and complete the fields in the Details section (see Figure 6):
- Title: Dashboard
- Enabled: Enabled
- Visible: Visible
- Root Only: False
- Target: Internal
Figure 6. Dashboard tab settings
- Click Save to apply the changes.
Now all users will see a new tab called "Dashboard." When clicked it will load the dashboard page. Clicking on the "Console" tab takes the user back to the default Build Forge page.
Update the dashboard automatically
To keep the dashboard up to date, a job can be scheduled to generate the HTML files and copy them to the appropriate location. On UNIX-like systems this can be done using
cron, while on Windows the equivalent tool is Task Scheduler. An update interval of 5 minutes is suitable for most situations.
Scripts that perform this task (both for Windows and UNIX-like platforms) are included with the dashboard application. For example on a UNIX-like system the script is called
update_dashboard.sh and has the following content:
Listing 2. The update_dashboard.sh script
#!/bin/sh # add Java to PATH if not already done # export PATH=$PATH:/opt/java1.5/jre/bin #path to the directory that is under the web serer root export WEBDIR=web # put timestamp on top of log file date > log.txt # run dashboard updater to generate HTML files java -cp dashboard.jar:rbf-services-client-java.jar \ com.ibm.rational.dashboard.Main >> log.txt 2>&1 # move files to directory under web server root mv *.html $WEBDIR/servers/ mv log.txt $WEBDIR/servers/
So far, you have used built-in variables provided by Build Forge (
MEM_FREE, etc.). It is also possible to define custom parameters in the collectors. To do so, follow these steps:
- In the Build Forge Management Console left menu, select Servers and thenCollectors.
- Select an existing collector or create a new one.
- Use the form in the lower part of the screen to add a custom variable (see Figure 7).
For example you could add a
JAVA_VERSION variable in the way that Figure 7 shows:
- Type: Run Command
- Regular Expression:
^java version "([^"]+)
Figure 7. Adding custom command variable
As you can see, the command
java -version is executed and the output is captured by a regular expression. After adding this variable to the collector, the servers will include this new parameter in the manifest, as Figure 8 shows.
Figure 8. Updated manifest
This approach can be used to get any server parameter that can be obtained by running a command and capturing the output.
In order to display this nonstandard parameter on the dashboard, relatively simple modifications should be made to the source code and templates.
OSPerfMeter and alternative parameter measurements
This approach (mentioned previously) is useful in the situation when you need alternative measurements for the basic parameters: CPU load, free memory, free disk space, and so on. This can occur on systems where the Build Forge agent fails to read these parameters properly (older versions of HPUX, for example).
For this case, specifically, the dashboard application supports alternative parameter measurements:
CPU_LOAD_ALT variable appears in the collector, the dashboard will use it rather than the standard
CPU_LOAD. The same thing applies for
MEM_FREE_ALT and DISK_FREE_ALT.
If you are not sure how to make alternative measurements on your servers, you can use
OSPerfMeter, which is included with this article, in the Downloads section (source code and binary). The tool works on the following platforms:
- Microsoft® Windows® (XP, 2003 and later)
- Linux® (2.4 kernel and later)
- Solaris (8 and later)
- IBM® AIX® (Version 5.2 and later)
- HPUX (11.11 and later)
Listing 3 shows how it is used.
Listing 3. Running OSPerfMeter from the command line (requesting all measurements)
$ java -jar OSPerfMeter.jar CPU Usage: 1.6 % RAM Available: 953 megabytes Swap Usage: 0.5 % HDD Space Available: 8109 megabytes
Alternatively, it can print individual values, as Listing 4 shows.
Listing 4. Running OSPerfMeter from the command line (requesting measurements separately)
$ java -jar OSPerfMeter.jar -cpu CPU Usage: 1.6 % $ java -jar OSPerfMeter.jar -mem RAM Available: 953 megabytes $ java -jar OSPerfMeter.jar -swap Swap Usage: 0.5 % $ java -jar OSPerfMeter.jar -hdd HDD Space Available: 8109 megabytes
OSPerfMeter measurements in a Build Forge collector, use the information in Table 2.
Table 2. Build Forge collector parameters for alternative performance measurements with OSPerfMeter
|Variable name||Command||Regular expression|
|CPU_LOAD_ALT||java -jar OSPerfMeter.jar -cpu||^CPU Usage:(.*) %|
|MEM_FREE_ALT||java -jar OSPerfMeter.jar -mem||^RAM Available:(.*) megabytes|
|DISK_FREE_ALT||java -jar OSPerfMeter.jar -hdd||^HDD Space Available:(.*) megabytes|
This article describes a method of using the Build Forge API to extract, process, and display performance parameters of servers. The information thus obtained is used to create a dashboard, a tool that allows release engineers and developers to check the status of multiple servers at a glance before launching lengthy processes. From the author's experience having a dashboard in multiserver environments reduces the number of unexpected failures and makes the process more predictable.
|Dashboard application binary and required files||dashboard.zip||50KB|
|Dashboard application source code||dashboard_src.zip||42KB|
|OSPerfMeter application binary||osperfmeter.zip||12KB|
|OSPerfMeter application source code||osperfmeter_src.zip||42KB|
More about Build Forge:
- IBM Rational Build Forge Java API guide (developerWorks, May 2008) is the Build Forge Java API reference
- IBM Rational Build Forge Perl API guide (developerWorks, Jun 2008) is the Build Forge Perl API reference
- Build Forge examples is a collection of Build Forge documentation, examples, and tools
- developerWorks Build Forge page, a portal to many other resources
- Build Forge demos show you how to Accelerate, standardize, and automate software assembly processes
- Learn about other applications in the IBM Rational Software Delivery Platform, including collaboration tools for parallel development and geographically dispersed teams, plus specialized software for architecture management, asset management, change and release management, integrated requirements management, process and portfolio management, and quality management.
- Visit the Rational software area on developerWorks for technical resources and best practices for Rational Software Delivery Platform products.
- Explore Rational computer-based, Web-based, and instructor-led online courses. Hone your skills and learn more about Rational tools with these courses, which range from introductory to advanced. The courses on this catalog are available for purchase through computer-based training or Web-based training. Additionally, some "Getting Started" courses are available free of charge.
- Subscribe to the Rational Edge newsletter for articles on the concepts behind effective software development.
- Subscribe to the IBM developerWorks newsletter, a weekly update on the best of developerWorks tutorials, articles, downloads, community activities, webcasts and events.
- Browse the technology bookstore for books on these and other technical topics.
Get products and technologies
- Download trial versions of IBM Rational software.
- Download these IBM product evaluation versions and get your hands on application development tools and middleware products from DB2®, Lotus®, Tivoli®, and WebSphere®.
- Participate in the IBM discussion forum for Build and Release Management and Rational Build Forge. Follow the forum discussion about Build Forge dashboard abilities.
- Check out developerWorks blogs and get involved in the developerWorks community.