The Support Authority: Real time problem determination with WebSphere diagnostic providers

New diagnostic provider technology enables you to probe configuration and state information of IBM® WebSphere® Application Server components (and even components of your own custom applications) in real time, and also enables these components to execute self tests to check if any known problem conditions exist. This content is part of the IBM WebSphere Developer Technical Journal.

Don Bourne (dbourne@ca.ibm.com), WebSphere Serviceability Architect, IBM

Don Bourne is a WebSphere Serviceability Architect at the IBM Toronto Lab. Don joined IBM in 1996 and has been specializing in the area of serviceability since joining the WebSphere Application Server team in 2003. Currently, Don is working with the IBM Support Assistant team architecting future solutions to help users resolve problems quickly.



Daniel Julin (dpj@us.ibm.com), WebSphere Serviceability Technical Area Lead, EMC

Daniel Julin has 20 years experience developing and troubleshooting complex online systems. As Technical Area Lead for the WebSphere Serviceability Team, he currently focuses on helping the team define and implement a collection of tools and techniques to assist in problem determination for WebSphere Application Server, and to maximize the efficiency of IBM Support. He also occasionally assists directly in various critical customer support situations.



18 July 2007

Also available in Chinese

In each column, The Support Authority discusses resources, tools, and other elements of IBM Technical Support that are available for WebSphere products, plus techniques and new ideas that can further enhance your IBM support experience.

This just in...

As always, we begin with some new items of interest for the WebSphere community at large:

  1. Two new tools have become available in the IBM Support Assistant:

    • A new version of the Log and Trace Analyzer, which has been mentioned a few times previously in this column and has been available for some time in various forms as a standalone tool, is now available through the IBM Support Assistant, and is now called simply Log Analyzer. Log Analyzer scans log files from a wide variety of products, combines these log files into a single view, highlights events of particular interest, and matches these events against a database of symptoms for known problems.

    • The Visual Configuration Explorer, making its first public appearance as a Technology Preview, collects configuration information from many products (including WebSphere Application Server) and presents this information in a convenient, consolidated, graphical view that makes it easy to find and correlate configuration information in one place when diagnosing problems. In addition, the tool can compare different configurations to flag potentially significant differences, and it can run a variety of validation rules to flag potential problems.

  2. If you've downloaded fixes recently for IBM WebSphere Portal, you may have noticed that many of these fixes are now managed through a new centralized Web site, called Fix Central. This new Web site makes it easy to find, download, and install fixes for selected IBM products, plus keep track of the dependencies and prerequisites between fixes. This facility continues to evolve, and we expect that many more WebSphere products will leverage this powerful site soon. Check back regularly for new added features and products.

  3. Take note of the upcoming technical conferences for the WebSphere community. New items of interest worldwide are added frequently to this list, so check back often!

Continue to monitor the various support-related Web sites, as well as this column, for news about other tools as we encounter them.

And now, on to our main topic...


What are diagnostic providers?

An earlier article provided an overview of many problem determination facilities included in the WebSphere Application Server runtime. This article will give you a closer look at diagnostic providers, a new technology that was introduced with WebSphere Application Server V6.1. With this technology, each WebSphere Application Server component (for example, Web container, connection manager, system management, and so on) -- or even components in the application -- can be instrumented to export its own diagnostic provider object. Each diagnostic provider object provides a uniform mechanism to perform three key functions:

  • Dump the configuration of a component.
  • Dump the state of a component.
  • Run self tests on a component.

Each individual diagnostic provider can implement all three or just a subset of these functions, based on the diagnostic needs of the component with which a diagnostic provider is associated.

Terminology
The term component is used loosely here; a diagnostic provider can be created for something as small as a single class, or as large as the entire application server in which it runs.

Diagnostic providers differ from other troubleshooting facilities in the application server. Logging facilities provide a means for recording events that occur in the application server as they happen, and provide severity information to help administrators understand the importance of those events. Trace facilities provide a means for recording details about activities happening in the application server as they occur; due to their verbose and technical nature, these details can be of more diagnostic value to the original author of the code being traced than to administrators. Similarly, the first failure data capture (FFDC) system in the application server captures information about the system when exceptions occur. Diagnostic providers, on the other hand, provide a means to request current information about the state of the application server, on demand, as it is known at the time of the request.

In some cases, diagnostic providers expose information that cannot be obtained through any other mechanisms. In other cases, particularly for most configuration information, there might be other ways to obtain similar information from particular components. The key, though, is that diagnostic providers offer a uniform mechanism to access diagnostic information across all components that take advantage of the diagnostic provider technology, providing a simpler interface to enable human troubleshooters to find all the information that they need in one place. Diagnostic providers also open the door to various forms of automated tools and scripts for problem determination, as you will see in some of the examples later in this article.

Diagnostic providers are particularly well suited to exposing details about the stateful parts of a system. The application server is full of stateful components, for example: the dynamic cache facility holds cached page fragments in memory, a connection pool maintains a set of connections, Web services rely on a set of SOAP servlet threads, the EJB container has a set of initialized EJB components, the logging subsystem has a set of configured loggers, and so on. All of these entities maintain a set of resources which, under normal conditions, don't warrant much consideration. When something is wrong, such as a thread pool being depleted, it is of significant value to be able to directly interrogate the components under consideration. As problems become known to component owners, self tests can be designed to check if these problems exist. This benefits those with less experience diagnosing problems related to the component, and has the added benefit that it can help monitoring systems to identify issues. Self tests can also perform simple experiments, such as attemping to connect to a database to confirm whether everything has been properly set up and configured.


Diagnostic providers in WebSphere Application Server V6.1

The infrastructure for implementing diagnostic providers was first introduced in WebSphere Application Server V6.1. Based on that infrastructure, each individual component of the system can implement its own diagnostic provider, tailored to that particular component. WebSphere Application Server V6.1 started with this initial set of components:

  • Connection manager (ConnMgrDP) provides configuration and state information about data sources and connection factories. This includes information about managed connections, active transactions, connection pool information, and other useful data.

  • Node synchronization (NodeSyncDP) provides state information about the most recent attempt to perform configuration synchronization, and includes node synchronization configuration details, such as information about auto synchronization. There is also a general self test that checks for connectivity, file transfer capability, and whether or not the system is in sync.

  • Node agent (NodeAgentDP) provides information about the node agent. This includes state information about which servers are known to the node agent, and a self test that checks the status of known servers.

  • Deployment manager (DeploymentManagerDP) provides information about the deployment manager. This includes state information about which node agents are known to the agent manager, and a self test that checks the status of known node agents.

  • Web container (WebContainerDP) provides configuration and state information about the application server's Web container, including information about cached servlets, virtual hosts, and general configuration information about the Web container.

  • Runtime advisor (AdvisorDP) provides state information about previously generated heap dumps, and configuration information about heap dump creation settings for the application server. It also provides a self test that detects if the application server is leaking memory.

Expect additional components to add their own diagnostic provider in future versions of the system.

Diagnostic providers are fully integrated with the WebSphere Application Server systems management facilities, and can thus be accessed through the same mechanisms as other WebSphere administrative functions, which include:

  • administration console
  • wsadmin scripting tool
  • a JMX client that directly invokes the relevant MBeans.

The method chosen to interact with a diagnostic provider depends on the situation. This article illustrates this with a few examples that cover the various diagnostic provider functions: self test (Example 1), state dumps (Example 2), and configuration (Example 3).


Example 1: Using the administration console to check if an application server is experiencing a memory leak

This example illustrates the runtime advisor diagnostic provider self-test function.

You can use the administration console when quickly checking the data from a diagnostic provider is the key objective. In this example, the runtime advisor diagnostic provider is used to demonstrate how to check for a Java™ memory leak in an application server using the administration console. To find the diagnostic providers in the administration console:

  1. Launch the administration console. (If security is enabled, then log in.)
  2. Select Troubleshooting from the left navigation bar.
  3. Expand Diagnostic Provider (Figure 1) to list the three main diagnostic provider functions.
    Figure 1. Navigation bar in administration console showing diagnostic provider section
    Figure 1. Navigation bar in administration console showing diagnostic provider section
  4. Select Tests in the left navigation bar to find the appropriate diagnostic provider.
  5. Expand cell and server in the right content pane, and click on the appropriate server name. A list of available diagnostic providers that have a test function are displayed (Figure 2).
    Figure 2. Available diagnostic providers in an unfederated application server
    Figure 2. Available diagnostic providers in an unfederated application server
  6. The memory leak check is a part of the runtime advisor (AdvisorDP) diagnostic provider. To run this provider, select AdvisorDP. A list of available tests for this diagnostic provider display (Figure 3).
    Figure 3. Available self-diagnostics for the runtime advisor
    Figure 3. Available self-diagnostics for the runtime advisor
  7. To run the test, select ServerRule11. The result will be the output of the test, as shown in Figure 4.
    Figure 4. Results from the memory leak self test
    Figure 4. Results from the memory leak self test
  8. If you receive a message that states
    The Performance and Diagnostic Advisor is disabled. In order to perform a Self Diagnostic
    of this component you should enable it.

    then the run time functions necessary for this advisor are currently disabled. To enable them:
    1. Select Servers in the left navigation bar.
    2. Select Application Servers.
    3. Select the appropriate server in the right content pane.
    4. Select Performance and Diagnostic Advisor Configuration.
    5. Select the Runtime tab.
    6. Check Enable Performance and Diagnostic Advisor Framework.
    7. Click OK.

Certain self tests and state data are only interesting to view when the system has collected data for a period of some time. For example, the algorithms used to detect potential memory leaks need time to watch for trends in Java heap utilization to make a proper determination.


Example 2: Using wsadmin scripts to diagnose common connection manager usage issues

This example illustrates using the connection manager diagnostic provider state dump function.

For a more easily repeatable interaction with diagnostic providers, wsadmin scripts are a good approach. In this example, Connection Manager diagnostic providers will be accessed using wsadmin scripts to investigate problems related to connection usage. The first step is to use wsadmin to find the appropriate diagnostic provider MBean. As a convenience, all diagnostic provider MBeans contain diagnosticProvider=true in their JMX object names, so the following script fragment lists all the available diagnostic providers:

Listing 1. Jython script listObjectNames.py to print out all MBeans with diagnosticProvider=true in their name
dpObjectNames = AdminControl.queryNames('diagnosticProvider=true,*')

print dpObjectNames

The script in Listing 1 can be executed using wsadmin using this command:

wsadmin -lang jython -f listObjectNames.py

and will produce output like this:

Listing 2. Output from DiagnosticProvider=true script run on an unfederated server
WASX7209I: Connected to process "server1" on node don7Node03 using SOAP connector;
The type of process is: UnManagedProcess

WebSphere:name=Default Datasource,process=server1,platform=dynamicproxy,node=don7Node03,
JDBCProvider=Derby JDBC Provider,diagnosticProvider=true,j2eeType=JDBCDataSource,
J2EEServer=server1,Server=server1,version=6.1.0.0,type=DataSource,mbeanIdentifier=cells
/don7Node03Cell/nodes/don7Node03/servers/server1/resources.xml#DataSource_1156718666915,
JDBCResource=Derby JDBC Provider,cell=don7Node03Cell,spec=1.0

WebSphere:name=DefaultEJBTimerDataSource,process=server1,platform=dynamicproxy,node=
don7Node03,JDBCProvider=Derby JDBC Provider (XA),diagnosticProvider=true,j2eeType=
JDBCDataSource,J2EEServer=server1,Server=server1,version=6.1.0.0,type=DataSource,
mbeanIdentifier=cells/don7Node03Cell/nodes/don7Node03/servers/server1/resources.xml
#DataSource_1000001,JDBCResource=Derby JDBC Provider (XA),cell=don7Node03Cell,spec=1.0

WebSphere:name=PLANTSDB,process=server1,platform=dynamicproxy,node=don7Node03,JDBCProvider
=Samples Derby JDBC Provider (XA),diagnosticProvider=true,j2eeType=JDBCDataSource,
J2EEServer=server1,Server=server1,version=6.1.0.0,type=DataSource,mbeanIdentifier=cells
/don7Node03Cell/nodes/don7Node03/servers/server1/resources.xml#DataSource_1156718677751,
JDBCResource=Samples Derby JDBC Provider (XA),cell=don7Node03Cell,spec=1.0


WebSphere:name=Runtime Advisor,process=server1,platform=dynamicproxy,node=don7Node03,
diagnosticProvider=true,reInit="Des:reinitialize the Runtime Performance Advisor.
#DesLookup:perfTuningAdmin.operation.reInit.des",version=6.1.0.0,type=ServerRule
DriverMBean,mbeanIdentifier=ServerRuleDriverMBean2,cell=don7Node03Cell,spec=1.0,
perfTuningAdmin.operation.takeHeapDump="Des:Triggers IBM JDK to take a multiple heap
dumps based upon downward trends in memory#DesLookup:perfTuningAdmin.operation.
takeHeapDump.des"

WebSphere:name=WebcontainerDiagnosticProvider,process=server1,platform=dynamicproxy,node=
don7Node03,diagnosticProvider=true,version=6.1.0.0,type=WebcontainerDiagnosticProvider,
mbeanIdentifier=null,cell=don7Node03Cell,spec=1.0

In Listing 2, multiple object names can be seen with j2eeType=JDBCDataSource. These object names correspond to the names of connection manager datasource MBeans. Listing 3 shows Jython code to execute a formatted state dump on the PLANTSDB data source diagnostic provider.

Listing 3. Jython script to generate a formatted state dump from the PLANTSDB data source diagnostic provider
import javax.management.ObjectName
import java.util.Locale

# find JMX object name for DiagnosticService MBean -- provides formatting helper methods
dsObjectName = AdminControl.queryNames('type=DiagnosticService,*')

# find JMX object name for specific diagnosticProvider MBean we're interested in
dpObjectName = AdminControl.queryNames('name=PLANTSDB,*')

# turn on all state collection for connection manager diagnostic providers
result = AdminControl.invoke_jmx(javax.management.ObjectName(dsObjectName),
 'setStateCollectionSpec', ['ConnMgrDP.*:.*=1'],['java.lang.String'])

# invoke the stateDump on the diagnostic provider
result = AdminControl.invoke_jmx(javax.management.ObjectName(dsObjectName),
 'stateDumpFormattedById', [dpObjectName, '.*', java.lang.Boolean(0),
 java.util.Locale('en_US')], ['java.lang.String', 'java.lang.String',
 'boolean', 'java.util.Locale'])

print result[0]

The output might look like this:

Listing 4. Output of formatted state dump from the PLANTSDB data source diagnostic provider
WASX7209I: Connected to process "server1" on node don7Node03 using SOAP connector;
 The type of process is: UnManagedProcess
DiagnosticProviderId: ConnMgrDP_jdbc/PlantsByWebSphereDataSource
EventType: stateDump  MessageKey: null
ResourceBundleName: null
Source Class: ConnectionFactoryMbeanImpl  Source Method: stateDump  ContentType: 1
Cell: don7Node03Cell   Node: don7Node03  Server: server1  SrvDetail:

Item Concatenated Name                   Value

 approximateNumberFreeConnections                 = 1
 approximateNumberSharedConnections               = 0
 approximateNumberUnSharedConnections             = 0
 approximateNumberWaitingConnections              = 0
 connections-7e3a7e3a-destroyState                = false
 connections-7e3a7e3a-handleCount                 = 0
 connections-7e3a7e3a-isParkedWrapper             = false
 connections-7e3a7e3a-isStale                     = false
 connections-7e3a7e3a-managedConnection           = WSRdbManagedConnectionImpl@6c3a6c3a
 connections-7e3a7e3a-mcWrapperState              = STATE_ACTIVE_FREE
 connections-7e3a7e3a-poolState                   = STATE_ACTIVE_FREE
 connections-7e3a7e3a-tranWrapperInUse            = NONE
 connections-7e3a7e3a-transactionId               = null
 inStuckConnectionMode                            = false
 poolManagerObject                                = 526655332
 quiesced                                         = false

The output in Listing 4 shows some global properties of the data source, and then some properties associated with one particular connection that is currently active in the data source (connections-7e3a7e3a).

DiagnosticService MBean

In Listing 4, notice that rather than accessing the connection manager's diagnostic provider stateDump method directly, the DiagnosticService MBean's stateDumpFormattedById method was invoked. The DiagnosticService provides a number of methods to help format results returned from diagnostic provider methods. The main roles of the DiagnosticService MBean are to format diagnostic provider output and to serve as a registry from which to locate any diagnostic providers in the JVM on which the DiagnosticService is running.

You might find the following DiagnosticService methods useful to obtain formatted results from diagnostic providers. Each takes the diagnostic provider's JMX object name, an attribute list (a regular expression indicating which attributes are of interest; for example connections-.*-transactionId), and the locale (used to select the appropriate localization for formatting) as parameters:

  • configDumpFormattedById(java.lang.String, java.lang.String, boolean, java.util.Locale)
  • stateDumpFormattedById(java.lang.String, java.lang.String, boolean, java.util.Locale)
  • selfDiagnosticFormattedById(java.lang.String, java.lang.String, boolean, java.util.Locale)

Using this information

All that information generated by the diagnostic providers can appear daunting. What should you do with it? Here are two typical situations that can be identified from this information:

  • Many connections used by a single transaction

    Connection pool exhaustion is a commonly reported problem in applications. Each data source and connection factory maintains its own pool of connections to an underlying resource. Once all of the resources from a connection pool are allocated to threads, the next thread that requests a connection from the connection pool will be forced to wait. The datasource or connection factory can provide the same managed connection to multiple callers, if connections are configured to be shared and an application is properly coded to make sharing possible. Conversely, poorly coded applications can consume many physical managed connections from the same connection pool for the same resource. In some cases, the blocked thread requesting the connection from the pool may actually be the same thread that is holding on to other connections to the same resource, which can cause a deadlock condition. By studying the state output of the Connection Manager diagnostic providers, it is possible to diagnose some common application issues.

    1. To begin, you might want to check to see if there is contention for the connections in a connection pool for a particular data source or J2C connection factory. A distinct Connection Manager diagnostic provider is created for each data source or J2C connection factory, so it is easy to focus on the data source or connection factory in question by selecting the appropriate diagnostic provider. The state information returned from the data source or connection factory's diagnostic provider will indicate the number of threads that are waiting to get a connection via the approximateNumberWaitingConnections metric. If the number of waiting connections is greater than 0 across multiple samples, you may have contention for that resource.

    2. The next step is to check if the same transactions are utilizing multiple managed connections. Transactions which use multiple managed connections might be doing so inadvertently. The connections-(connection-reference)-transactionId metric will show you the transactionId with which a particular managed connection is associated. If multiple managed connections are associated with the same transactionId, then further investigation is warranted.

    When applications cause each transaction to hold references to multiple managed connections for the same resource, the pool of available connections for that resource can be quickly depleted when the application is under load. There are many ways to remedy this type of situation, via reconfiguration or application redesign, but those remedies are beyond the scope of this discussion (see Resources).

  • Large number of connection handles

    Connection pools can also be quickly depleted when connections are not closed and returned to the pool when they are no longer needed. This can occur in situations where application code makes use of the UserTransaction class to explicitly demarcate transactions, and then mistakenly does not always commit or rollback transactions when complete.

    In this situation, you would expect to find a number of connections with a non-null connections-(connection-reference)-transactionId value since the transactions were not properly closed. You would also expect to see that the connections-(connection-reference)-handleCount value for these connections is greater than 0.


Example 3: Using wsadmin scripts to diagnose common Web container issues

This example illustrates using the Web container diagnostic provider configuration dump function.

Another diagnostic provider that can be useful in finding common problems is WebContainerDP. To begin with, a number of problems can be diagnosed by looking at the configuration data available from the WebContainerDP's configDump. Invoking a configuration dump is very similar to the process described earlier for invoking a state dump.

Listing 5. Jython script to generate a formatted configuration dump from the WebContainer diagnostic provider
import javax.management.ObjectName
import java.util.Locale

# find JMX object name for DiagnosticService MBean -- provides formatting helper methods
dsObjectName = AdminControl.queryNames('type=DiagnosticService,*')

# find JMX object name for the specific diagnosticProvider MBean we're interested in
dpObjectName = AdminControl.queryNames('name=WebcontainerDiagnosticProvider,*')

# invoke the config dump and show results containing PlantsByWebSphere
result = AdminControl.invoke_jmx(javax.management.ObjectName(dsObjectName), 
'configDumpFormattedById', [dpObjectName, '.*PlantsByWebSphere.*', 
java.lang.Boolean(0), java.util.Locale('en_US')], ['java.lang.String', 'java.lang.String',
'boolean', 'java.util.Locale'])

print result[0]

Notice in Listing 5 that the configuration dump was limited to only those attributes containing "PlantsByWebSphere." Executing the above script without this restriction would typically provide a lot of output, as the Web container's diagnostic provider reports configuration information for every installed Web archive. Similar constraints can be used to look at the content for other Web archives.

The output might look like this:

Listing 6. Output of formatted configuration dump from the WebContainer diagnostic provider
WASX7209I: Connected to process "server1" on node don7Node03 using SOAP connector;
  The type of process is: UnManagedProcess
DiagnosticProviderId: WebSphere:name=WebcontainerDiagnosticProvider,process=server1,
platform=dynamicproxy,node=don7Node03,diagnosticProvider=true,version=6.1.0.0,type=
WebcontainerDiagnosticProvider,mbeanIdentifier=null,cell=don7Node03Cell,spec=1.0
EventType: configDump  MessageKey: null
ResourceBundleName: null
Source Class: com.ibm.ws.webcontainer.diag.WebcontainerDiagnosticProviderMBean  
Source Method: configDump  ContentType: 1
Cell: don7Node03Cell   Node: don7Node03  Server: server1  SrvDetail:

Item Concatenated Name                   Value

 startup-vhosts-default_host-webapps-PlantsByWebSphere#PlantsByWebSphere.war-
additionalClasspath = Null
 startup-vhosts-default_host-webapps-PlantsByWebSphere#PlantsByWebSphere.war-
applicationName = PlantsByWebSphere
 startup-vhosts-default_host-webapps-PlantsByWebSphere#PlantsByWebSphere.war-
applicationStartupWeight = 1
 startup-vhosts-default_host-webapps-PlantsByWebSphere#PlantsByWebSphere.war-
autoLoadFiltersEnabled = false
 startup-vhosts-default_host-webapps-PlantsByWebSphere#PlantsByWebSphere.war-
autoRequestEncoding = true
 startup-vhosts-default_host-webapps-PlantsByWebSphere#PlantsByWebSphere.war-
autoResponseEncoding = true
 startup-vhosts-default_host-webapps-PlantsByWebSphere#PlantsByWebSphere.war-
codeErrorPages = 500=com.ibm.ws.container.ErrorPage@63dc63dc;404=com.ibm.ws.container.
ErrorPage@63c263c2
 startup-vhosts-default_host-webapps-PlantsByWebSphere#PlantsByWebSphere.war-
contextParams = password=wsdemo1;userID=WSDEMO;dataSourceName=jdbc/PlantsByWebSphereData
Source
 startup-vhosts-default_host-webapps-PlantsByWebSphere#PlantsByWebSphere.war-
contextRoot = /PlantsByWebSphere
 startup-vhosts-default_host-webapps-PlantsByWebSphere#PlantsByWebSphere.war-
defaultErrorPage = Null
 startup-vhosts-default_host-webapps-PlantsByWebSphere#PlantsByWebSphere.war-
defaultErrorPageDescription = PlantsByWebSphere Web App
 startup-vhosts-default_host-webapps-PlantsByWebSphere#PlantsByWebSphere.war-
directoryBrowingEnabled = false
 startup-vhosts-default_host-webapps-PlantsByWebSphere#PlantsByWebSphere.war-
displayName = PlantsByWebSphere Web Application
 startup-vhosts-default_host-webapps-PlantsByWebSphere#PlantsByWebSphere.war-
documentRoot = D:WebSphere610profilesAppSrv01installedAppsdon7Node03CellPlantsByWebSphere.
earPlantsByWebSphere.war
 startup-vhosts-default_host-webapps-PlantsByWebSphere#PlantsByWebSphere.war-
exceptionErrorPages = Null
 startup-vhosts-default_host-webapps-PlantsByWebSphere#PlantsByWebSphere.war-
fileServingAttributes = Null
 startup-vhosts-default_host-webapps-PlantsByWebSphere#PlantsByWebSphere.war-
fileServingEnabled = true
 startup-vhosts-default_host-webapps-PlantsByWebSphere#PlantsByWebSphere.war-
invokerAttributes = Null
 startup-vhosts-default_host-webapps-PlantsByWebSphere#PlantsByWebSphere.war-
mimeFilteringEnabled = false
 startup-vhosts-default_host-webapps-PlantsByWebSphere#PlantsByWebSphere.war-
mimeMappings = Null
 startup-vhosts-default_host-webapps-PlantsByWebSphere#PlantsByWebSphere.war-
moduleID = Null
 startup-vhosts-default_host-webapps-PlantsByWebSphere#PlantsByWebSphere.war-
moduleName = PlantsByWebSphere.war
 startup-vhosts-default_host-webapps-PlantsByWebSphere#PlantsByWebSphere.war-
moduleStartupWeight = 10000
 startup-vhosts-default_host-webapps-PlantsByWebSphere#PlantsByWebSphere.war-
reloadInterval = 3
 startup-vhosts-default_host-webapps-PlantsByWebSphere#PlantsByWebSphere.war-
reloadingEnabled = true
 startup-vhosts-default_host-webapps-PlantsByWebSphere#PlantsByWebSphere.war-
servletServingByClassnameEnabled = false
 startup-vhosts-default_host-webapps-PlantsByWebSphere#PlantsByWebSphere.war-
syncToThreadEnabled = false
 startup-vhosts-default_host-webapps-PlantsByWebSphere#PlantsByWebSphere.war-
version = 24
 startup-vhosts-default_host-webapps-PlantsByWebSphere#PlantsByWebSphere.war-
welcomeFiles = index.html
 startup-vhosts-default_host-webapps-PlantsByWebSphere#PlantsGallery.war-
additionalClasspath = Null
 startup-vhosts-default_host-webapps-PlantsByWebSphere#PlantsGallery.war-
applicationName = PlantsByWebSphere
 startup-vhosts-default_host-webapps-PlantsByWebSphere#PlantsGallery.war-
applicationStartupWeight = 1
 startup-vhosts-default_host-webapps-PlantsByWebSphere#PlantsGallery.war-
autoLoadFiltersEnabled = false
 startup-vhosts-default_host-webapps-PlantsByWebSphere#PlantsGallery.war-
autoRequestEncoding = false
 startup-vhosts-default_host-webapps-PlantsByWebSphere#PlantsGallery.war-
autoResponseEncoding = false
 startup-vhosts-default_host-webapps-PlantsByWebSphere#PlantsGallery.war-
codeErrorPages = Null
 startup-vhosts-default_host-webapps-PlantsByWebSphere#PlantsGallery.war-
contextParams = Null
 startup-vhosts-default_host-webapps-PlantsByWebSphere#PlantsGallery.war-
contextRoot = /PlantsByWebSphere/docs
 startup-vhosts-default_host-webapps-PlantsByWebSphere#PlantsGallery.war-
defaultErrorPage = Null
 startup-vhosts-default_host-webapps-PlantsByWebSphere#PlantsGallery.war-
defaultErrorPageDescription = Contains the Technology PlantsByWebSphere Sample Gallery 
files
 startup-vhosts-default_host-webapps-PlantsByWebSphere#PlantsGallery.war-
directoryBrowingEnabled = false
 startup-vhosts-default_host-webapps-PlantsByWebSphere#PlantsGallery.war-
displayName = PlantsByWebSphere Sample Gallery Web Application
 startup-vhosts-default_host-webapps-PlantsByWebSphere#PlantsGallery.war-
documentRoot = D:WebSphere610profilesAppSrv01installedAppsdon7Node03CellPlantsByWebSphere.
earPlantsGallery.war
 startup-vhosts-default_host-webapps-PlantsByWebSphere#PlantsGallery.war-
exceptionErrorPages = Null
 startup-vhosts-default_host-webapps-PlantsByWebSphere#PlantsGallery.war-
fileServingAttributes = Null
 startup-vhosts-default_host-webapps-PlantsByWebSphere#PlantsGallery.war-
fileServingEnabled = true
 startup-vhosts-default_host-webapps-PlantsByWebSphere#PlantsGallery.war-
invokerAttributes = Null
 startup-vhosts-default_host-webapps-PlantsByWebSphere#PlantsGallery.war-
mimeFilteringEnabled = false
 startup-vhosts-default_host-webapps-PlantsByWebSphere#PlantsGallery.war-
mimeMappings = Null
 startup-vhosts-default_host-webapps-PlantsByWebSphere#PlantsGallery.war-
moduleID = Null
 startup-vhosts-default_host-webapps-PlantsByWebSphere#PlantsGallery.war-
moduleName = PlantsGallery.war
 startup-vhosts-default_host-webapps-PlantsByWebSphere#PlantsGallery.war-
moduleStartupWeight = 10000
 startup-vhosts-default_host-webapps-PlantsByWebSphere#PlantsGallery.war-
reloadInterval = 3
 startup-vhosts-default_host-webapps-PlantsByWebSphere#PlantsGallery.war-
reloadingEnabled = true
 startup-vhosts-default_host-webapps-PlantsByWebSphere#PlantsGallery.war-
servletServingByClassnameEnabled = true
 startup-vhosts-default_host-webapps-PlantsByWebSphere#PlantsGallery.war-
syncToThreadEnabled = false
 startup-vhosts-default_host-webapps-PlantsByWebSphere#PlantsGallery.war-
version = 24
 startup-vhosts-default_host-webapps-PlantsByWebSphere#PlantsGallery.war-
welcomeFiles = Null

Using this information

A number of problems can be diagnosed from this example with the WebContainer diagnostic provider:

  1. Symptom: Static content and HTML files do not display properly.
    What to look for: configDump: Verify that fileServingEnabled is set to true for the application in question.
    Example:

    startup-vhosts-admin_host-webapps-filetransfer#filetransfer.war-fileServingEnabled = true
  2. Symptom: User attempts to invoke a JSP and receives Error 404: SRVE0190E: File not found.
    What to look for: configDump: Verify that the correct context root is specified for the application.
    Example:

    startup-vhosts-default_host-webapps-DefaultApplication#DefaultWebApplication.war-
    contextRoot = /
  3. Symptom: Welcome files are not served.
    What to look for: configDump: Verify that the welcome file is specified correctly and, if the welcome file is static, that fileServingEnabled is set to true.
    Example:

    startup-vhosts-default_host-webapps-DefaultApplication#DefaultWebApplication.war-
    fileServingEnabled = false
  4. Symptom: Some types of resources display correctly, but you cannot display a servlet by its class name.
    What to look for: configDump: Verify that servletServingByClassname Enabled is true.
    Example:

    startup-vhosts-default_host-webapps-DefaultApplication#DefaultWebApplication.war-
    servletServingByClassnameEnabled = true
  5. Symptom: As you test your console modules, you discover that changes you make to JSPs are not rendered when you log in to the application server.
    What to look for: configDump: Verify reloadingEnabled is set to true for the application in question to enable the JSP to reload each time you deploy a new change.
    Example:

    startup-vhosts-default_host-webapps-DefaultApplication#DefaultWebApplication.war-
    reloadingEnabled = true
  6. Symptom: An installed application cannot be found via the correct context root.
    What to look for: configDump: Check for overlapping virtual host aliases.
    Example:

    startup-vhosts-admin_host-aliases = :9062;:9045;
  7. Symptom: Filters are not invoked correctly.
    What to look for: stateDump: Verify that filters are named and mapped correctly.
    Example:

    vhosts-default_host-webapps-DefaultApplication#DefaultWebApplication.war-filters-
    AsyncFilter-filterClassName = com.ibm.ws.ard.filter.AsyncFilter
    
    vhosts-default_host-webapps-DefaultApplication#DefaultWebApplication.war-filters-
    AsyncFilter-filterMapping-349705432-dispatchMode = REQUEST
    
    vhosts-default_host-webapps-DefaultApplication#DefaultWebApplication.war-filters-
    AsyncFilter-filterMapping-349705432-servletName = Null
    
    vhosts-default_host-webapps-DefaultApplication#DefaultWebApplication.war-filters-
    AsyncFilter-filterMapping-349705432-urlPattern = /*
  8. Symptom: The same output is displayed when a servlet/JSP is repeatedly refreshed.
    What to look for: stateDump: The page is likely being cached. Verify that caching enabled is set to false.
    Example:

    vhosts-default_host-webapps-DefaultApplication#DefaultWebApplication.war-servlets-Snoop 
    Servlet-cachingEnabled = true
  9. Symptom: A servlet is not initialized fully before its server is considered started.
    What to look for: stateDump: Verify that loadOnStartup is true for the application in question.
    Example:

    vhosts-default_host-webapps-DefaultApplication#DefaultWebApplication.war-servlets-Hello 
    Pervasive Servlet-loadOnStartup = true

Conclusion

This article explored the diagnostic provider facility in WebSphere Application Server V6.1. Components that have diagnostic providers can be probed in real time to see their configuration information and state information, and to execute self diagnostic tests. All the information provided by diagnostic providers can be conveniently found in one place, rather than requiring you to work through many different mechanisms to get different pieces of information, plus the collection and analysis of this information can be easily scripted.

Several of key WebSphere Application Server components have diagnostic providers that provide useful information to help solve common problems. Diagnostic providers can be accessed via the administrative console, via wsadmin scripts, or via JMX client code. The runtime advisor, connection manager, and Web container diagnostic providers were used here to illustrate some of their uses.

While not discussed in this article, a System Programming Interface (SPI) is also provided with WebSphere Application Server V6.1 to enable you to create new diagnostic providers. In particular, it is possible to create new diagnostic providers for other products built on top of WebSphere Application Server, and even for components within a complex custom application. Information about this SPI is included in the WebSphere Application Server Information Center.


Acknowledgements

The authors thank Erinn Koonce, Jim Stephens and Kevin Kelle for their assistance in putting this information together.

Resources

Learn

Get products and technologies

Discuss

Comments

developerWorks: Sign in

Required fields are indicated with an asterisk (*).


Need an IBM ID?
Forgot your IBM ID?


Forgot your password?
Change your password

By clicking Submit, you agree to the developerWorks terms of use.

 


The first time you sign into developerWorks, a profile is created for you. Information in your profile (your name, country/region, and company name) is displayed to the public and will accompany any content you post, unless you opt to hide your company name. You may update your IBM account at any time.

All information submitted is secure.

Choose your display name



The first time you sign in to developerWorks, a profile is created for you, so you need to choose a display name. Your display name accompanies the content you post on developerWorks.

Please choose a display name between 3-31 characters. Your display name must be unique in the developerWorks community and should not be your email address for privacy reasons.

Required fields are indicated with an asterisk (*).

(Must be between 3 – 31 characters.)

By clicking Submit, you agree to the developerWorks terms of use.

 


All information submitted is secure.

Dig deeper into WebSphere on developerWorks


static.content.url=http://www.ibm.com/developerworks/js/artrating/
SITE_ID=1
Zone=WebSphere
ArticleID=240601
ArticleTitle=The Support Authority: Real time problem determination with WebSphere diagnostic providers
publish-date=07182007