IBM Sterling B2B Integrator Must Gather: Troubleshooting high CPU issues

Common data points needed:

1. Logs needed: All the logs from <SBI_INSTALL>/logs directory
2. Dump Info output: How to Run a dump_info for Sterling B2B Integrator
3. Properties files -- The properties files can be found at <SBI_INSTALL>/properties directory:
   • tuning.properties
   • noapp.properties
   • customer_overrides.properties
4. Performance tuning wizard screenshot -- to capture it, navigate to dashboard > Operations > System > Performance > tuning > View Performance configuration and capture the screenshot.

Operating System (OS) specific data points:

• • Linux -- Capturing the data:
    
    1. The top command output for the noapp child process id during high CPU. Follow the steps below to find the child PID and to capture the command output.
     
    • Navigate to <SBI_INSTALL> directory.
    • In the command line, enter cat noapp.pid (This lists the parent thread process ID)
    • Enter ps -ef | grep noapp.pid (This returns the child process IDs associated with the parent process ID)
    • Enter top -H -p child.pid
    
    Monitor the above command output and capture the screen shot.

    2. gatherstats.sh output during high CPU. Follow the below steps before running the script.

    • Navigate to <SBI_INSTALL>/bin directory.
    • Download and unzip gatherstats.sh.zip. Place the gatherstats.sh.in at <SBI_INSTALL>/bin directory
    • Run setupfiles.sh (This creates the gatherstats.sh)
    • Restart Sterling B2B Integrator
    • Run gatherstats.sh when you see the high CPU issue and attach the report to the PMR.

AIX -- Capturing the data:

• • 1. The tprof command output during high CPU. To capture run the following command at the command line:
    • tprof -skex sleep 60
    
    Monitor and capture the command output.

    2. Take few thread dumps during high CPU issue. Follow these steps to take thread dumps:

     
    • Navigate to <SBI_INSTALL> directory.
    • In the command line, enter cat noapp.pid (This lists the parent thread process ID)
    • Enter ps -ef | grep noapp.pid(This returns the child process ID associated with the parent PID)
    • Enter kill -QUIT noapp.pid child.pid(You must include both the parent PID and the child PID.)
    
    The thread dump is placed in the javacore*.txt file in the <SBI_INSTALL>/noapp/bin directory.

Windows -- Capturing the data:

• • 1. Enable JMX connection by following the steps below.
    • Navigate to the <SBI_Install>/bin directory and modify the InstallNoappWindowsService.cmd.in file:
    
    
    •  
      • • Add the following lines (NOTE: port should match DEBUG_PORT in sandbox.cfg or any free port available) :
          •  
            rem *************ADDED BY IBM Support***********************
            echo -Dcom.sun.management.jmxremote.port=12000>> %PARAM_FILE%
            echo -Dcom.sun.management.jmxremote.authenticate=false >> %PARAM_FILE%
            echo -Dcom.sun.management.jmxremote.ssl=false >> %PARAM_FILE%
            rem ********************************************************
             
          immediately after this line:
          •  
            echo -Dcom.certicom.tls.record.maximumPaddingLength=0 >> %PARAM_FILE%
       
    • Save your changes, then run setupfiles from the same directory.
    • Verify your changes were added to the InstallNoappWindowsService.cmd by setupfiles.
    • Run UninstallWindowsService.cmd followed by InstallWindowsService.cmd
    • Start Sterling B2B Integrator

    2. Find any machine where JDK is installed and you can access the SBI server.

    Run the jTop.jar pointing to the right location as per your install location and share the screen shot to support team.

    
    

    The above command should return all the thread associated with the noapp process along with the CPU usage as below.

    
    
    
    
    Note: By enabling JMX connection you should be able to monitor JVM and resource usage remotely or locally.
    
    See this blog post for more details.
    Enabling JMX connection in IBM Sterling B2B Integrator and monitoring IBM Sterling B2B Integrator from remote machine or locally