Troubleshooting
Problem
Use this document to set up trace strings and request dumps for WebSphere Application Server Liberty since there is no administrative console available.
Resolving The Problem
For tracing components in traditional WebSphere Application Server refer to MustGather: Read first for WebSphere Application Server.
To request dumps for Liberty running with IBM SDK:
Run the Liberty dump command on the target Liberty server to request the desired dump files. The dump type can be any combination of thread, heap, or system. It is specified by:
- --include=thread
- --include=heap
- --include=system
- --include=thread,system,heap (or any combination)
NOTE: For Oracle SDKs the only supported type is "heap".
The location of the dump file will be returned when the command completes:
- For Windows platforms, run:
<LIBERTY_HOME>\bin\server.bat dump <serverName> --include=<desired dump type>
For UNIX platforms, run:
<LIBERTY_HOME>/bin/server dump <serverName> --include=<desired dump type>
- Collect the resulting dump zip file.
Note: The current Liberty dump command also gathers deployed applications.
http://www-01.ibm.com/support/docview.wss?rs=180&uid=swg21469413
http://www.ibm.com/support/docview.wss?rs=180&uid=swg21052642
** Steps 1 - 3 ONLY
http://www.ibm.com/support/docview.wss?rs=180&uid=swg21115658
** Steps 1 - 3 ONLY
http://www.ibm.com/support/docview.wss?rs=180&uid=swg21053924
** Section "Configuring Full Core Dumps"
Enabling Trace on Liberty:
There are 2 formats for trace output - binary or HPEL, which uses fewer resources and disk space but has to be formatted to be readable and BASIC, which is readable. Binary HPEL format requires an addition to the bootstrap.properties file before starting servers and includes binary trace in the trace specifications. Once the bootstrap.properties file is edited, trace is picked up dynamically for both formats. . The trace strings are specific to a module or component within WebSphere Liberty that you want to troubleshoot. Examples are for the BASIC format.
- To enable tracing on the target Liberty server:
- To enable binary logging, first add this line to the bootstrap.properties file in the LIBERTY_HOME/usr/servers/<server> location:
websphere.log.provider=binaryLogging-1.0
- Open the server.xml file in the root directory of the target server:
<LIBERTY_HOME>/usr/servers/<server>
Where LIBERTY_HOME is the location of the extracted binary or is <AppServer>/wlp if Liberty is installed with WebSphere Application Server Version 8.5
Add the <logging> tag to the server.xml file with the following elements:
For example, to trace for the web container component, you would add the following snippet to the server.xml file, with the appropriate traceSpecification not this example uses binaryTrace format (explanation to each of these elements is provided in the product documentation)
<logging traceSpecification="com.ibm.ws.webcontainer*=all:com.ibm.wsspi.webcontainer*=all:HTTPChannel=all:GenericBNF=all:HTTPDispatcher=all"
traceFileName="trace.log"
maxFileSize="20"
maxFiles="10"
traceFormat="BASIC" />
- To enable binary logging, first add this line to the bootstrap.properties file in the LIBERTY_HOME/usr/servers/<server> location:
- If possible, stop the server and clear the logs under the following directory:
<LIBERTY_HOME>/usr/servers/<serverName>/logs
- Recreate the problem and collect the information.
- Follow instructions to send diagnostic information to IBM Support.
Trace specifications available
The following trace strings can be used as the trace specification with the instructions above to diagnose an issue in a specific component:
- Aries / OSGi
org.apache.aries.jndi.*=all
- Classloading
ClassLoadingService=all:SharedLibrary=all
Also, so that native JVM classloader tracing is captured, add the following JVM argument to <LIBERTY_HOME>/usr/servers/<server name>/jvm.options Add the argument, save, and stop-restart the server:
-verbose:class
Note: osgi classloading issues require the use of the OSGI console see section on OSGI console
- Configuration
com.ibm.ws.config.*=all
- Connections/Connection Pooling/Transaction RRA=all:WAS.j2c=all:com.ibm.ws.jdbc.*=all:Transaction=all:ConnLeakLogic=all
- Deploy (EAR/WAR/EBA)
app.manager*=all
- EJB
EJBContainer=all:Injection=all
- Java Batch
com.ibm.jbatch.*=all:com.ibm.ws.jbatch.*=all
- Java Management Extensions (JMX) or JMX client API (Server)
jmx.rest.server.connector=all:com.ibm.ws.jmx.connector.server.rest.*=all
- Java Persistence API (JPA)
JPA=all
- Java Transaction Service (JTS)
Transaction=all:com.ibm.ws.transaction.services.WebAppTransactionCollaboratorImpl=all
- JavaServer Pages (JSP)
com.ibm.ws.jsp=all
- JAX-RS (RESTful Web Services)
com.ibm.ws.jaxrs*=all:com.ibm.websphere.jaxrs*=all:org.apache.wink.*=all:org.apache.cxf.*=all
- JAX-WS (web services)
org.apache.cxf.*=all:com.ibm.ws.jaxws.*=all
- JNDI and Naming
Naming=all:NamingService=all
- JSF
com.ibm.ws.jsf*=all:org.apache.myfaces*=all:com.ibm.ws.webcontainer*=all:com.ibm.ws.jsp=all
- Managed Executors and Thread Context Propagation
concurrent=all
If an issue with concurrency constraints (for example, max async or max queued),
concurrent=all:concurrencyPolicy=all
If an issue with context propagation to managed executor tasks,
concurrent=all:context=all
If an issue with the underlying thread pool,
concurrent=all:concurrencyPolicy=all:Threading=all
- MQJMS:
If the IBM® MQ resource adapter is installed using either the wmqJmsClient-1.1 or wmqJmsClient-2.0 feature,
enable the following trace:
*=info:JMSApi=all:Transaction=all:WAS.j2c=all:EJBContainer=all:Injection=all
If the IBM® MQ resource adapter is installed using generic JCA support, trace can be enabled by adding the following entry to the WebSphere Liberty file jvm.options:
-Dcom.ibm.msg.client.commonservices.trace.status=ON
-Dcom.ibm.msg.client.commonservices.trace.outputName=C:\Trace\MQRA-WLP_%PID%.trcThis results in trace being written to a trace file called MQRA-WLP_<process identifier>.trc in the directory C:\Trace.
- ORB:
For IIOP and Yoko ORB issues:
IIOP=all:org.apache.yoko.*=all:yoko.verbose.*=fine
For JNDI and Naming issues:
Naming=all
CosNaming issues can involve both IIOP and JNDI, so enable both sets:
Naming=all:IIOP=all:org.apache.yoko.*=all:yoko.verbose.*=fine
- Plug-in Generation
com.ibm.ws.webcontainer.osgi.mbeans.*=all
- PMI/Performance Tools
General:
com.ibm.websphere.monitor.*=all:com.ibm.ws.monitor.*=all:com.ibm.websphere.pmi.*=all:com.ibm.websphere.pmi.*=all:com.ibm.ws.monitors.*=all
In case the ConnectionPoolStats MBean is required, use the following instead of the above PMI trace string:
*=info:com.ibm.websphere.monitor.jmx.*=all:com.ibm.websphere.monitor.meters.Meter*=all:com.ibm.websphere.monitor.meters.Counter*=all:com.ibm.websphere.monitor.meters.Gauge*=all:com.ibm.websphere.monitor.meters.StatisticsMeter=all:com.ibm.websphere.monitor.meters.StatisticsReading=all:com.ibm.ws.monitor.*=all:com.ibm.websphere.pmi.*=all:com.ibm.ws.monitors.*=all
- Security
MustGather: Security problems for WebSphere traditional and Liberty
MustGather: SSL problems on WebSphere Liberty
MustGather: Web Single Sign-on problems with WebSphere Application Server
MustGather: Web Services Security (WS-Security) problems with WebSphere Application Server
MustGather: SPNEGO problems on WebSphere Liberty
- Servlet Engine/Web Container
com.ibm.ws.webcontainer*=all:com.ibm.wsspi.webcontainer*=all:HTTPChannel=all:GenericBNF=all:HTTPDispatcher=all
- Sessions and Session Management
com.ibm.ws.session.*=all:com.ibm.ws.webcontainer*=all:com.ibm.wsspi.webcontainer*=all
In addition,:
RRA=all (for database persistence issues)
Additional trace for scheduler problems:persistentExecutor=all:concurrent=all:EJBContainer=all:RRA=all:WAS.j2c=all:com.ibm.ws.jdbc.*=all:Transaction=all
- SIBJMS connection problems
SIBTrm=all
- SIBJMS Communications and TCP/IP
SIBCommunications=all:SIBJFapChannel=all:TCPChannel=fine:com.ibm.io.async.*=all
- SIBJMS Client Applications
SIBJms*=all:SIBCommunications=all:SIBJFapChannel=all:
SIBMessageTrace=all:SIBTrm=all:SIBJmsRa=all:SIBRa=all
- SIBJMS locked messages
SIBProcessor=all:SIBMessageTrace=all
- SIBJMS MDB
SIBMessageTrace=all:SIBJmsRa=all:SIBRa=all
- SIBJMS message format and schema
SIBMfp=all:SIBCommunications=all
- SIBJMS message processor
SIBProcessor=all:SIBMessageTrace=all
- SIBJMS message store
SIBMessageStore=all
- SIBJMS performance
SIBMessageTrace=all
- SIBJMS publish/subscribe
SIBMatchSpace=all:SIBProcessor=all
- SIBJMS security
SIBSecurity=all
- WebSocket
websockets=all:HTTPChannel=all:TCPChannel=all:com.ibm.ws.webcontainer.*=all:com.ibm.wsspi.webcontainer.*=all:com.ibm.ws.webcontainer31.*=all:HTTPDispatcher=all
Please see the full trace specifications in the related information URL.
Related Information
Was this topic helpful?
Document Information
Modified date:
24 January 2022
UID
swg21596714