Steps to scale load balancing in Rational DOORS Web Access

Configure any environment, and scale the installation to handle larger loads

After you have installed IBM® Rational® DOORS® Web Access requirements management software, this article will guide you through configuring the multiple tiers of the application. Then it explains how to scale the components to balance larger loads as your business needs increase.

Share:

Lee J. Byrnes (byrnesle@uk.ibm.com), Load, Performance, and Security Tester, IBM

author photoLee is a load tester with more than 10 years of experience, encompassing HP Loadrunner and Rational Performance Tester. He is often the first person to investigate scaling product installations to handle larger loads.



13 September 2011

Also available in Chinese Spanish

The two key topics of this article

The first part of this article defines the files and parameters that can be altered to optimize the performance of the DOORS Web Access product and allow for the components to be distributed across several servers. The regular load testing that is performed on this application has led to suggestions about use, and these are included, too.

The second part provides a set up guide for clustering the Rational DOORS Web Access server layer of the application. This is necessary to scale the product so that it can meet the needs of enterprise customers.

The clustering section describes the steps required to set up a load balancer in front of Rational DOORS Web Access servers, talking to a single Rational DOORS Web Access broker, with each element on a separate server.

All of these components could feasibly co-exist on a suitably large server, but that would not have made for a clear explanation of the process. We usually suggest distributing the components across a group of servers to prevent over-utilization of any particular resource.

Note:
This configuration is a round-robin load balancer controlled by percentages and makes no attempt to intelligently balance loads by server capacity or availability.


Post-installation configuration

A note about the version

It should be noted all the file path examples in this article use the installation pathway for Rational DOORS Web Access Version 1.4.0.1, if you are installing a later version this will obviously be different.

Basic Rational DOORS Web Access architecture

The diagram in Figure 1 shows the conceptual layout of a the Rational DOORS Web Access architecture, the image does includes the Redirector Service the use of which is not covered in this article.

Figure 1. Rational DOORS Web Access architectural layout
Standard layout of the four application elements

Rational DOORS Web Access server configuration

The Rational DOORS Web Access server is constructed on the Apache Tomcat web server, which is a Java Virtual Machine (JVM). This means that memory allocation to the Rational DOORS Web Access Server can be critical. To prevent the initial installation taking over a significant proportion of RAM on a host server the default setting is for a 512 megabyte JVM. This can be changed in this file:
C:\Program Files \IBM\Rational\Doors Web Access\1.4.0.1\server.start.bat

On line 36 of this file, you will find this entry:

set JAVA_OPTS=%JAVA_OPTS% -Xmx512M -XX:+HeapDumpOnOutOfMemoryError

The value after -Xmx defines the upper limit of RAM allowed to the JVM. If you are installing on a server with sufficient RAM, we suggest raising this to 1536 megabytes (1.5 gigabytes). This will support 70 users without any degradation in performance and can support 100 users with only slight degradation in performance. The users in load testing should be considered power users, because they are performing a transaction once a minute, which means they are maintaining approximately five transactions per second in the steady state of any load test.

If you want to monitor the behavior of the DOORS Web Access server with a tool such as Jconsole, there is a line of code that needs uncommented in the server.start.bat file. On line 26, remove the remprefix so that appropriate monitoring tools can connect to the server that server is installed on port 9004.

Important:
This has been set up as an unsecure port, so it should not be left available if not in use.

The connection from the DOORS Web Access server to the DOORS Web Access broker, the DOORS server ID, and the licensing server are handled in the same file:
C:\Program Files \IBM\Rational\Doors Web Access\1.4.0.1\server\festival \config\FESTIVAL.XML

Therefore, you will deal with these at the same time.

There are several entries for local host in the festival.xml file (127.0.0.1), because the initial installation setup is based on the assumption that the Rational DOORS Web Access broker is on the same physical server as the Rational DOORS Web Access server. If this is not true in your installation, the local host values need to be changed to the IP address of the physical server where the Rational DOORS Web Access broker is hosted.

Line 26 of the festival.xml file contains a urn value defined as:
urn:telelogic:ers-

This needs to be changed from the default to the Rational DOORS repository urn of your Rational DOORS server installation.

  1. To get this value, connect a thick client to the Rational DOORS server, and select Tools > Edit DXL.
  2. Run the following piece of code in the pop-up dialog window that appears:
    print getDatabaseIdentifier
  3. The lower part of the dialog window now contains the urn value to use to replace the value in the festival.xml file.
  4. Also, change the license server entry on line 31 of the festival.xml file to the port and server of your license server (for example, 19353@myLicenseServer).

Rational DOORS Web Access broker configuration

The Rational DOORS Web Access broker is based on the Apache AMQ message broker. There are no adjustments needed for the DOORS Web Access broker, because it is a passive message broker. The default JVM memory allocation of 128 megabytes has always proven to be sufficient.

If, for security reasons, you require ports to be closed when they are not secure, be aware that the monitoring port on Rational DOORS Web Access broker is open by default. To close the port, open this file:
C:\Program Files \IBM\Rational\Doors Web Access\1.4.0.1\broker.start.bat

Find Lines 29 and 30 and put a rem statement at the beginning of each of those lines while the broker is not running.

Interop server configuration

The Interop servers are the layer between the Rational DOORS Web Access broker and the Rational DOORS server. We suggest that you have one Interop server for every 25 users that you expect to use the system, with no more than one Interop server per available CPU core. It is good practice to have four Interop servers initialized even at low user volumes. This means that a sufficiently large physical server can have multiple Interop instances on it; in our testing, we have eight instances on one server.

By installing a normal thick client, you will have all the necessary code in place and be connected to the Rational DOORS server installation.

Interop servers can be started with a desktop shortcut, the following is example code for the two fields that need to be completed in any shortcut:

Target:

"C:\Program Files\IBM\Rational\DOORS\9.3\bin\doors.exe" -interop -brokerHost"
Doors Web Access Broker Server" -maxMemory 640

Start in:

"C:\Program Files\IBM\Rational\DOORS\9.3\bin"

Adjustments to ensure that doors.exe launches an Interop server instance and connects successfully to the rest of the Rational DOORS Web Access architecture are handled in the Target line of the shortcut properties.

  • The -interop command tells the doors.exe that it is required to start up in Interop server mode.
  • The -brokerhost command designates the location of the Rational DOORS Web Access broker so that the Interop server can pick up requests.
  • The -maxMemory statement prevents the Interop server from consuming memory. Without adding –maxMemory, the Interop server has the potential to consume the entire available RAM on its host server. In the example above, the Interop server is set to perform a lossless restart when it reaches a cached memory size of 640 megabytes.

How to cluster Rational DOORS Web Access server

Important:

  • The one external software component that needs to be added to the environment is an instance of the Apache web server
  • The proofing for this article was performed on Version 2.2.11 of the Apache web server. The latest Apache Software Foundation version available when this article was written (August 2011) was Version 2.2.17, and we have not tested to determine whether the latest version invalidates anything in this article.
  • We have proved the clustering of the Rational DOORS Web Access Server layer using the Apache approach only. There are several other ways that you might choose to handle your load balancing, but we are unable to offer support or advice on any of these.

A typical clustered installation will look like the illustration in Figure 2.

Figure 2. Conceptual clustered environment
Diagram of application tiers normally clustered

Rational DOORS Web Access server changes

For each instance of the DOORS Web Access server that you have installed, you need to make adjustments to the FESTIVAL.XML and SERVER.XML files. FESTIVAL.XML is the file where the license and DOORS server adjustments are made after installation. If the default installation path was accepted during installation, you can find that file here:
C:\Program Files \IBM\Rational\Doors Web Access\1.4.0.1\server\festival \config\FESTIVAL.XML

Lines 17, 19, and 21 of this file contain a local host IP address that lets the Rational DOORS Web Access server find the DOORS Web Access broker. These need to be changed to the IP address of the server hosting the broker.

When the Web Access servers can find the broker location, you need to set them up to accept requests from the Apache web server. If the default installation path was accepted during installation, you'll find the SERVER.XML file here:
C:\Program Files \IBM\Rational\Doors Web Access\1.4.0.1\server\conf\SERVER.XML

Line 144 of this file contains an Engine name entry:
<Engine name="Catalina" defaultHost="localhost">

This needs to be changed for each separate Rational DOORS Web Access server installation to (the two references that follow tie into the BalanceMember lines in the example in the next section):

  • For Doors Web Access Server One (where that is the name of your server), it becomes:
    <Engine name="Catalina" defaultHost="localhost" jvmRoute="DWAbalance1">
  • For Doors Web Access Server Two, it becomes:
    <Engine name="Catalina" defaultHost="localhost" jvmRoute="DWAbalance2">

Where the jvmRoute designation is unique for each of the Rational DOORS Web Access server instances, this will correspond to a routing assignment in the Apache web server to allow the sticky session to work correctly.

These unique route names are assigned in Apache (see the section that follows), and the IP of the address and the route name must be the same.

Tip:
We strongly suggest that you make this change in an editor that shows HTML and XML formatting, because there are other lines in the code nearby that resemble this but are, in fact, comments.

Apache server setup

From a plain vanilla installation of Apache 2.2.11, there is only one file that needs to be edited to enable the load balancing functionality.

The installation location for Apache 2.2, the conf folder contains a file called HTTPD.CONF, which should be here:
C:\java\Apache2.2\conf\httpd.conf

In the HTTPD.CONF file, you need to remove hash marks from the following lines for Apache to load the required modules at startup:

  • LoadModule proxy_module modules/mod_proxy.so
  • LoadModule proxy_ajp_module modules/mod_proxy_ajp.so
  • LoadModule proxy_balancer_module modules/mod_proxy_balancer.so
  • LoadModule proxy_http_module modules/mod_proxy_http.so
  • LoadModule status_module modules/mod_status.so

After doing that, we added the following code to the end of the file to align the routing set up in the DOORS Web Access servers earlier. For clarity in this example, we have left the IP addresses of the servers where the proof was performed.

Server one has an IP address of Doors Web Access Server One, and we set the jvmRoute to be Doors Web Access balance1 (as shown in previous section).

Server two has an IP address of Doors Web Access Server Two, and we set the jvmRoute to be Rational DOORS Web Access balance2 (as shown in previous section).

The load factor is the percentage statement for the request handling. So in this simple example, each server receives 50% of the requests (request size is not a consideration, so this might not equate to 50% of the load):

ProxyPass / balancer://mycluster/ stickysession=JSESSIONID|jsessionid 
nofailover=On 
ProxyPassReverse / http://DOORS Web Access Server One:8080/ 
ProxyPassReverse / http://DOORS Web Access Server Two:8080/ 
<Proxy balancer://mycluster> 

BalancerMember http://DOORS Web Access Server One:8080 route=DWAbalance1 loadfactor=50
BalancerMember http://DOORS Web Access Server Two:8080 route=DWAbalance2 loadfactor=50

</Proxy>

Monitoring changes

To observe the behavior of your configuration, put the following code segment under the ServerRoot statement:

<Location /balancer-manager> SetHandler balancer-manager </Location>

Now, in your browser, on the server that you installed the Apache web server on, launch http://localhost/balancer-manger. This web page will allow you to monitor the load balancing in real time.


Startup

After you have made all of these changes, start the Rational DOORS Web Access broker followed by the Rational DOORS Web Access servers, and then start the Apache web server. During startup in any configuration, there is a dependency that requires the DOORS Web Access broker to be started before the DOORS Web Access servers.

From the browser client, address Rational DOORS Web Access to this location:
http://ApacheServerLocation/Doors Web Access

Failover

There are four layers in the Rational DOORS Web Access architecture. In this section, we look at the consequences of any of them being restarted.

Doorsd
A restart of the Rational DOORS server should be considered as a full system restart and will require the Interop servers to be restarted.
 
Interop server
The Interop servers actually restart as part of their memory management process, so they can be stopped and started at will. If this is performed by the application, it is lossless. A forced restart might cause the loss of a single request.
 
Rational DOORS Web Access broker
The DOORS Web Access broker is a passive element of the architecture and stateless. However, there is an issue in connecting to the DOORS Web Access server. Currently, if the Web Access broker is restarted, the server will not be able to re-establish the connection and will have to be restarted, tool. There is an enhancement request pending to correct this.
 
Rational DOORS Web Access server
The server can be stopped and started without loss of connectivity for any component, although any request that a component is servicing will be lost at the time of the restart.

Summary

This article includes all of the steps necessary to install and scale Rational DOORS Web Access to meet the expected requirements of the installation, regardless of size. It also gives the administrator a clear understanding of the consequences if any of these architectural components need to be restarted.

Resources

Learn

Get products and technologies

  • Get the free trial download for Rational DOORS Web Access.
  • Evaluate IBM software in the way that suits you best: Download it for a trial, try it online, use it in a cloud environment, or spend a few hours in the SOA Sandbox learning how to implement service-oriented architecture efficiently.

Discuss

Comments

developerWorks: Sign in

Required fields are indicated with an asterisk (*).


Need an IBM ID?
Forgot your IBM ID?


Forgot your password?
Change your password

By clicking Submit, you agree to the developerWorks terms of use.

 


The first time you sign into developerWorks, a profile is created for you. Information in your profile (your name, country/region, and company name) is displayed to the public and will accompany any content you post, unless you opt to hide your company name. You may update your IBM account at any time.

All information submitted is secure.

Choose your display name



The first time you sign in to developerWorks, a profile is created for you, so you need to choose a display name. Your display name accompanies the content you post on developerWorks.

Please choose a display name between 3-31 characters. Your display name must be unique in the developerWorks community and should not be your email address for privacy reasons.

Required fields are indicated with an asterisk (*).

(Must be between 3 – 31 characters.)

By clicking Submit, you agree to the developerWorks terms of use.

 


All information submitted is secure.

Dig deeper into Rational software on developerWorks


static.content.url=http://www.ibm.com/developerworks/js/artrating/
SITE_ID=1
Zone=Rational
ArticleID=756565
ArticleTitle=Steps to scale load balancing in Rational DOORS Web Access
publish-date=09132011