Debugging techniques for IBM Rational Change runtime problems

When you know some of the different levels of debugging techniques to analyze any runtime problems in IBM Rational Change enterprise change management. software, you might resolve the issue yourself without calling tech support. Even if you do have to call, you’ll be better prepared and save considerable time, because Level 3 support engineers often ask you to add or edit configuration settings. So either way, you’re likely to save time and avoid frustration.

Share:

Ritesh Nigam (ritesh.nigam@in.ibm.com), System Software Engineer, IBM

author photoRitesh has been working with the Rational Change development team for the last four years. A major part of his work has been on core development of this software.



09 August 2011

This article covers some of the IBM® Rational® Change initial debugging techniques that developers or technical support technicians can use before contacting L3 support. This will save a considerable amount of time. For example, for any performance issue related to database calls or any LDAP calls (Rational Change calls the Rational Directory Server), the general log does not give any helpful information. Rational Change has a few configuration keys, and using them will log some useful data in application log.

This article provides detailed information about the runtime errors, so that you can do the initial debugging. You might avoid having to call tech support. Even if you do end up calling, you’ll save considerable time when you know about the techniques and tools that this article covers:

  • Enabling the debug level in application log
  • Using application-specific keys to enable communication logs with supporting products
  • Connecting the Your Kit tool for memory analysis
  • Enabling SSL debugging
  • Using JConsole and Verbose GC
  • Other profiling tools or utilities

Enabling a debug level in the application log

For every web application, there are some primary logs available in case of any issue. These logs can be referred, and they include the application’s log and the application/web server’s log. The Rational Change application log file is called event.log, and you can find it at CHANGE_APP_HOME/logs.

Rational Change supports two modes of logging

  • Normal mode
  • Debug mode

Normal mode of logging

This mode is the default mode for logging by Rational Change. In this mode, Rational Change logs very minimal and add only necessary statements to the event.log file.

Debug mode of logging

Rational Change can be configured to use Debug mode of logging, which logs the statements that Normal mode does, plus more detailed statements. In this mode, Rational Change does not delete some of the temporary files, such as the XML data file that supplies the data to Perl triggers.

To enable the Debug mode for Rational Change, follow these steps:

  1. Log in as Administrator in the Rational Change administration interface.
  2. Under the Home link, check the menu options, and click the Event Log link.
  3. In this interface, check the box labeled Log Debug Messages (see Figure 1).
Figure 1. Enabling the Debug mode of logging in Rational Change
Configuring debug mode in Versions 5.2 and 5.3

Note:
Rational Change prints lots of statements in the log file, so the size of the log file increases rapidly.


Application-specific keys

Rational Change depends on two other products to store its data:

  • Rational® Synergy (all change request and task data)
  • Rational Directory Server (for storing user-related data, such as profiles and preferences)

Whenever required, Rational Change opens communication with these products and gets the required data. Sometimes, because of these dependencies, the application encounters issues that can degrade performance. Rational Change provides a way to track these communications in terms of time they take. There are few configuration settings that can be enabled to track the time taken by any call to either the Rational Synergy database or to Rational Directory Server. These configuration settings can be added or edited in the app.user.properties file of the Rational Change enterprise change management application. You can find this file at this location:
CHANGE_ HOME/WEB-INF/wsconfig/system

The following configuration settings or keys need to be modified or added:

ldap.call_time_threshold_ms
This key is used to track all of the calls made to RDS from Rational Change. Set an integer value to this key, such as 10000. In this case, every call to RDS that takes more than 10000 ms will be logged in the event.log file. The default value of this key is 5000 ms.

For example: ldap.call_time_threshold_ms = 10000
 
session.accent.call_time_threshold_ms
This key is used to track all the calls to synergy database. It takes a value in integer, all the calls to synergy database which tales more than the configured time will be logged in the event.log. The default value of this key is set to 20000 ms.

For example: session.accent.call_time_threshold_ms = 10000
 

Connecting the Your Kit tool for memory profiling

Rational Change is a web application based on Java 2 Platform, Enterprise Edition. J2EE applications are prone to memory leaks if not developed properly. Memory profiling can be possible for Rational Change also. Your Kit is one tool that you can use with Rational Change to analyze memory. Similarly, other available tools can be configured to Rational Change for memory profiling.

Common steps for all platforms:

  1. Install the Your Kit Java profiler if it is not installed already. You need a license to use it.
  2. Let’s say that the Your Kit profile is installed at this location:
    PROFILER_HOME

Windows (Jetty web server)

  1. Go to this directory: CHANGE_HOME/jetty/win32
  2. In the text editor of Rational Change 5.3, open the service.cfg file (or for Rational Change 5.2, open the change.cfg file).
  3. Add the Your Kit agent as a Java option:
    agentpath:PATH_TO_YOURKIT_AGENT
  4. Run the service.bat /remove command, and then the service.bat /install command to reinstall the service.
  5. Start the Your Kit profiler, and click the Connect to remote application link to enter the host where Rational Change is running (by default, the Your Kit service runs on the 10001 port).
  6. Follow the Your Kit interface for profiling the Rational Change application.

Linux and UNIX platforms

  1. Go to the CHANGE_HOME/jetty/bin directory.
  2. Open the jetty.sh script file in a text editor.
  3. Locate this line:
    JETTY_CONSOLE=null/logs/stdout.log; export JETTY_CONSOLE
  4. Add this line just after the line mentioned above:
    - JAVA_OPTIONS“-agentpath:PATH_TO_YOURKIT_AGENT; export JAVA_OPTIONS
  5. Restart the Rational Change application.

To run the Your Kit profiler on an Apache Tomcat server, IBM® WebSphere® Application Server Community Edition, or WebSphere Application Server 7.0, see the documentation provided with the server. Consult the Your Kit product manual to learn more about the Your Kit agent.

Enabling SSL debugging

Rational Change also supports secure communication using the HTTPS protocol. Any issue at the SSL layer becomes hard to find, and normal application logs are not sufficient to debug an SSL issue.

SSL debugging can be enabled at the Java Virtual Machine (JVM) level by using this JVM option:

-Djavax.net.debug=true

However, this option can differ between JVMs. This option shown above is applicable only to IBM JVM, which is what Rational Change supports. Enabling this option prints useful logging information related to SSL communication that takes place for Rational Change.

Follow these steps to enable SSL debugging for Rational Change on the Jetty web server:

Microsoft Windows

  1. Stop the Rational Change service.
  2. Browse to this directory: CHANGE_HOME/jetty/win32
  3. In a text editor, open the service.cfg configuration file for Rational Change 5.3 (or the change.cfg file for Rational Change 5.2).
  4. Add another JVM option to the list (replace ? by the next integer in the list:
    jvmOption?=-Djavax.net.debug=true
  5. Open a command prompt, and change to this directory:
    CHANGE_HOME/jetty
  6. Run the service.bat /remove command to remove the service.
  7. Now install and start the Rational Change service (the SSL logs will be available on the command prompt):
    service.bat /console.

Linux and UNIX platforms

  1. Browse to this directory: CHANGE_HOME/jetty/bin
  2. Open the jetty.sh script file in a text editor.
  3. Locate this line:
    JETTY_CONSOLE=null/logs/stdout.log; export JETTY_CONSOLE
  4. Add this just after the line mentioned above:
    - JAVA_OPTIONS=-Djavax.net.debug=true; export JAVA_OPTIONS
  5. Restart Rational Change.

Rational Change 5.3 supports other application servers, including Apache Tomcat, WebSphere Application Server Community Edition, and WebSphere Application Server 7.0. To enable SSL debugging on these application servers, see the documentation provided for the server.


Using JConsole and Verbose GC (bundled with IBM JRE)

You can use the Java Monitoring and Management Console (JConsole) to monitor and manage the behavior of any Java application, and it comes with a very good graphical user interface (GUI). When JConsole is configured with Rational Change, it gives useful information about memory use, the running threads, and the loaded classes. Later, this information can be handy in debugging performance problems, memory use issues, hangs, or deadlocks.

Before you start

Rational Change should be running in command line mode before setting it up with JConsole to make it accessible to JConsole. Follow these steps to run Rational Change in command line mode for the Jetty web server:

Windows

  1. Open the command prompt, and change to this directory:
    CHANGE_HOME/jetty
  2. Type service.bat /console to start Rational Change in command line mode.

Setting up JConsole to monitor Rational Change

Windows (Jetty web server)

  1. Before starting Rational Change, edit the change.cfg file in Version 5.2 (or the service.cfg in Version 5.3), which is located in this directory:
    CHANGE_HOME/jetty/win32
  2. Add these JVM options to the list:
    • jvmOption?=-Dcom.sun.management.jmxremote.port=available_port_number
    • jvmOption??= -Dcom.sun.management.jmxremote.authenticate=false
    • jvmOption???=-Dcom.sun.management.jmxremote.ssl=false
    where available_port_number is a free port on your server. The second and third options deactivate both password authentication and encryption using SSL (Secure Sockets Layer). This means that any instance of JConsole, or any other JMX agent, can connect to your Java application as long as it has access to the specified port.

    Important:
    This nonsecure setup should only be used in a test environment.
  3. Open a command prompt, and change to this directory:
    CHANGE_HOME/jetty
  4. Run the service.bat /remove command to remove the service.
  5. Now use the service.bat /install command to install and start the Rational Change service.

    Reminder:
    To start the Rational Change service from the console, stop it first, and start it in command line mode.
  6. Start the JConsole by typing jconsole at a command prompt.

    Note:
    Because this is an SDK utility, the bin directory of the SDK should be set in the path.
  7. When the JConsole New Connection dialog window appears, enter the host name and port of the workstation where Rational Change is running. If you are running JConsole on the same workstation as Rational Change, use localhost for the host name. You can leave the Username and Password fields blank if you used the options specified in step 1.
  8. Click Connect. JConsole starts with the Summary tab view.
  9. Follow the UI for profiling options.

Linux and UNIX platforms

In the Linux and UNIX platforms, the Java options related to enabling JConsole can be added by editing the jetty.sh script file located in the CHANGE_HOME/jetty/bin directory.

To use JConsole with Apache Tomcat, WebSphere Application Server Community Edition and WebSphere Application Server 7.0, please refer the documentation provided with respective servers.

Verbose GC

This JVM utility is used to track garbage collection (GC), which can assist in debugging performance problems. Enabling this provides detailed information about the garbage collection process controlled by JVM. Follow these steps to enable verbose GC for Rational Change:

Windows (Jetty web server)

  1. Stop the Rational Change service.
  2. Browse to this directory:
    CHANGE_HOME/jetty/win32
  3. In a text editor, open the service.cfg configuration file for Rational Change 5.3 (or change.cfg for Version 5.2.
  4. Add another JVM option to the list, replacing ? by the next integer in the list:
    jvmOption?=-verbose:gc
  5. Open a command prompt, and change to this directory:
    CHANGE_HOME/jetty
  6. Run the service.bat /remove command to remove the service.
  7. Now install and start the Rational Change service by using this command:
    service.bat /console

Linux and UNIX platforms

For Linux and UNIX platforms, the Java options related to enabling JConsole can be added by editing the jetty.sh script file in this directory:
CHANGE_HOME/jetty/bin

Running that setup prints the GC information on the standard, or you have the option to redirect the information to a separate file (see the verbose GC utility help for details).

To enable verbose GC on Apache Tomcat, WebSphere Application Server Community Edition, or WebSphere Application Server 7.0, see the documentation provided with the server.

Other profiling tools and utilities

Apart from the profiling tools mentioned already, there are several other utilities or tools such as JProfiler, JStack, Java interactive profiler, and so on that you can for profiling Rational Change. To use or configure those tools, consult their manuals or documentation.


Using the IBM JVM dump analyzer

IBM provides a suite of memory and dump analyzers in form of the IBM Support Assistant Workbench. Using this suite, you can analyze memory dumps or dumps created by a JVM crash. This can generate relevant reports about the JVM, which can be analyzed to pinpoint issues with the Rational Change installation. See the product manual for details.

Resources

Learn

Get products and technologies

Discuss

Comments

developerWorks: Sign in

Required fields are indicated with an asterisk (*).


Need an IBM ID?
Forgot your IBM ID?


Forgot your password?
Change your password

By clicking Submit, you agree to the developerWorks terms of use.

 


The first time you sign into developerWorks, a profile is created for you. Information in your profile (your name, country/region, and company name) is displayed to the public and will accompany any content you post, unless you opt to hide your company name. You may update your IBM account at any time.

All information submitted is secure.

Choose your display name



The first time you sign in to developerWorks, a profile is created for you, so you need to choose a display name. Your display name accompanies the content you post on developerWorks.

Please choose a display name between 3-31 characters. Your display name must be unique in the developerWorks community and should not be your email address for privacy reasons.

Required fields are indicated with an asterisk (*).

(Must be between 3 – 31 characters.)

By clicking Submit, you agree to the developerWorks terms of use.

 


All information submitted is secure.

Dig deeper into Rational software on developerWorks


static.content.url=http://www.ibm.com/developerworks/js/artrating/
SITE_ID=1
Zone=Rational
ArticleID=750982
ArticleTitle=Debugging techniques for IBM Rational Change runtime problems
publish-date=08092011