Why collecting the Websphere "MustGather" data for opening a performance PMR really is beneficial

• Application Server Hardware Configuration
• Operating System (Name, Version, 32-bit / 64-bit)
• Websphere Application Server Version (32-bit / 64-bit)
• Hardware Manufacturer/Model
• # of CPU’s
• CPU Speed
• Physical Memory
• Disk Size
• Is this server an LPAR? If so,
• How many LPARs per physical server?
• How much physical memory is allocated?
• How many physical processors are allocated?

• Load Balancing Configuration
  • Do you use hardware or software load balancing? (Explain)
  • Is IBM HTTP Server used?
  • If so, what is the load balancing policy (weight based, round-robin, random)?
    

•  Authentication Configuration
  • Do you use Single Sign-on?
  • Do you use LDAP synchronization?
    

•  Cluster Architecture
  • Do you have separate clusters for:
    • User Interface (UI) processing?
    • Cron Tasks?
    • Integrations?
    • Schedules / Report Jobs?
  • For each cluster implemented, please provide the following data:
    • Are there other applications deployed to the same server?
    • How many load balancers do you have?
    • How many servers?
    • How many clusters?
    • How many members in each cluster?
    • How many JVM’s per Server?
    • Memory allocation per JVM (minimum and maximum heap size)?
      

•  WAS and IHS Plug-in Configuration
  
  • Please send the output from the following command:
    <WAS_HOME>/java/jre/bin/java –fullversion
  • For each cluster member, please send the following file:
    <WAS_HOME>/profiles/<profilename>/config/cells/<cellname>/nodes/<nodename>/servers/<servername>/server.xml
  • For each cluster, please send the following file:
    <WAS_HOME>/profiles/<profilename>/config/cells/<cell name>/clusters/<clustername>/cluster.xml
  • For each IHS server, please send the following file:
    <IHS_HOME>/plugins/config/<webserver_name>/plugin-cfg.xml
    

•  Websphere Logs
  • Please send copies of the following logs:
    SystemOut.log, SystemErr.log, native_std_err.log

•  Server Hardware Configuration

Different operating systems can be tuned differently, so it's helpful to know whether the application server is running on AIX, Windows, Linux, etc. Knowing if the OS and WAS are both 64-bit determines whether or not the JVM heap size can be larger than what's available on 32-bit operating systems. And, some folks prefer to run the 32-bit version of WAS even if the OS is 64-bit. Also, understanding the version of WAS / IHS installed can help rule out known issues with back-level releases of the middle-ware. Similarly, knowing the CPU, memory, and disk info can indicate whether or not the hardware has the capability to handle the application server load. Lastly, there are special considerations that must be taken into account when the environment is virtualized.

•  Load Balancing Configuration

Many performance issues can be traced to the system not properly load balancing across JVMs or servers. There are many ways to do load balancing, so understanding whether hardware or software is used can be important. Likewise, knowing if IBM HTTP Server with it's WAS plug-in is being used and what policy for load balancing has been chosen can be important factors in diagnosing load balancing issues.

•  Authentication Configuration

Bottlenecks can occur at any point in the solution, so knowing if single sign-on or LDAP group synchronization is used can be important to rule out potential issues in those areas.

•  Cluster Architecture

The Best Practices for System Performance white paper highly recommends that you setup multiple clusters to handle the various types of processing that come into the system. The main goal is to ensure that the JVMs servicing the user interface requests aren't being bogged down by log running escalations or reports. Similarly, you want integration processes to be free from the UI when performing their transactions. Knowing the cluster architecture decisions that have been made can help pinpoint issues that may be related to various types of processes conflicting with one another. Once the overall architecture is understood, it is important to know how each cluster is configured, the number of servers involved in each cluster, the number of JVMs used, etc.

•  WAS and IHS Configuration

•  Websphere Logs

Analyzing the SystemOut.log and SystemErr.log can uncover many issues at the application server level that could affect performance. With verbose garbage collection enabled, the native_std_err.log shows any potential garbage collection issues that can have a negative effect on performance.