Troubleshooting
Problem
Collecting diagnostic data (MustGather) aids in problem determination and saves time when resolving Problem Management Records (PMRs) for WebSphere Service Registry and Repository.
Resolving The Problem
Learning more about MustGather |
The term "MustGather" represents the diagnostic data (system information, symptoms, log files, traces, and so on) that is required to resolve a problem. By collecting MustGather data early, even before you open a PMR, you help IBM Support quickly determine the following:
- If symptoms match known problems (rediscovery)
- If you have a non-defect problem that can be identified and resolved
- If there is a defect that identifies a workaround to reduce severity
- If locating the root cause can speed development of a code fix
Collect MustGather data |
Manually collect MustGather data |
Collect the following information for MustGather data:
- WebSphere Service Registry and Repository version details
It is important to provide details of exactly which version of WSRR is being used. This includes Fix Pack and iFix details. The simplest way to determine the exact version of WSRR is to look at the welcome page in the WSRR Web User Interface.
Look for the section titled About your Service Registry and Repository.
This shows the version number and build number. For example:
8.5.0.1
Build Number: 20140814-1023
Both these pieces of information should be included when raising a problem relating to WSRR. - Business Space version details (WSRR V7.0.0 to V8.0.0 only)
WAS_HOME/BusinessSpace/bspace.versions.txt |
WAS_HOME/profiles/profile_name/logs/server_name |
The support teams might ask for more logs during the problem diagnosis. However, by default include the contents of this directory. It is recommended that you compress the logs directory using compression tools such as zip, or tar and gzip. In addition to this directory, the /ffdc is often useful. This is located at the same level as the server_name directory. For example: |
WAS_HOME/profiles/profile_name/logs/ffdc |
Trace files are not generated by default. When trace is switched on, output is sent to the logs/server_name/trace.log. In the case where the maximum rollover log file size is reached, this file is renamed to a file with a name of the form: |
logs/server_name/trace_<TIMESTAMP>.log |
- Log and trace files
Often the most important of the user-supplied information to assist with the diagnosis of problems. All log and trace files are stored on your machine within the /profiles directory structure.
- Switching trace on for WebSphere Service Registry and Repository
The simplest way to turn on trace is by using the administrative console. To do this, start a session to the administrative console and log in. You might need to alter the filter at the bottom of the main panel in order that you can see the Troubleshooting in the left column. - 1. Select Troubleshooting > Logs and Trace.
2. Select your server from the list.
3. Change Log Detail Levels. You are presented with a free-form text box, which contains (by default) the string *=info. To trace general WebSphere Service Registry and Repository problems, this trace string should be replaced with the following string.
Note: There are no spaces or new line characters in this string. It should appear as one line of text when you enter it in the administrative console:
*=info:com.ibm.sr.api*=all:com.ibm.serviceregistry*=all
5. Restart the server to initiate the production of trace.
6. At this point, re-create the problem with the minimum set of steps and send the trace log files into the support group.
Note: If to re-create the problem you have to perform a number of steps or if some of the steps take a long time to complete a lot of trace can be produced and the trace files might be overwritten. The default trace log size is 20MB, and one historical file is kept, this is often not sufficient. It is recommended to use 100MB trace files and to retain 10 historical trace log files. Ensure you have adequate free disk space before changing the trace settings.
- Configuration profiles
WSRR stores all of the configuration information within a configuration profile. There can be multiple configuration profiles loaded but only one active profile. It is often useful to provide the Technical Support with the entire active configuration profile.
To export the configuration profile navigate to the configuration perspective in the WSRR UI.
1. Select Manage Profiles -> Configuration Profiles
2. Check the box next to the configuration profile marked 'Active'.
3. Press the Export button
4. Save the zip to your local machine and send it to the support team.
- Problem Description
Provide a detailed description of the problem, including in detail, the steps needed to re-create. If the problem is related to a particular artifact like a WSDL file or an XSD then if possible provide that artifact and any of the files that it depends on both directly and indirectly.
- Operating system and hardware architecture
A brief description of the OS and architecture type is useful to determine which version of the Java™ SDK you are using. For example: Solaris 10 (SPARC), or Linux RHEL AS 4 (zSeries). - Deployment topology
Indicate if you are using the product deployed in a cluster, and if so indicate if all the machines in the cluster are of the same OS type and version.
Gather data relating to problems with Service Registry Dashboard |
For problems where the Dashboard is not accessible or where widgets are appearing but their contents is blank, the following trace should be collected. The trace should be enabled and then the Application Server restarted before attempting to access the dashboard.
Note: There are no spaces or new line characters in this string. It should appear as one line of text when you enter it in the administrative console:
*=info:com.ibm.cre*=all:com.ibm.sr.dashboard.debug*=all: com.ibm.sr.dashboard.cre*=all: com.ibm.sr.dashboard.client=all: com.ibm.sr.api*=all:com.ibm.serviceregistry*=all:com.ibm.sr.dashboard.noserver*=all:com.ibm.mm.proxy.connection*=all |
If the Dashboard is accessible but there is a problem occurring while it is being used, the following trace should be captured. There is no need to restart the Application Server after applying this runtime trace, just enable the trace and re-create the problem.
Note: There are no spaces or new line characters in this string. It should appear as one line of text when you enter it in the administrative console:
*=info:com.ibm.sr.api*=all:com.ibm.serviceregistry*=all:com.ibm.sr.dashboard.client=all:com.ibm.sr.ui*=all |
Gather data relating to problems with Promotion |
Promotion will always involve the promotion of data from one system to at least on other. Trace should be collected from ALL the systems involved in the promotion. It is important to identify these systems clearly when providing trace. Typically the source system is referred to as the "Governance Master" and each of the target systems as a "Runtime".
The following trace string should be used when gathering data for a promotion problem.
Note: There are no spaces or new line characters in this string. It should appear as one line of text when you enter it in the administrative console:
*=info:com.ibm.sr.api*=all:com.ibm.serviceregistry*=all:com.ibm.sr.promotion*=all |
Gather data relating to SQL performance |
Note: There are no spaces or new line characters in this string. It should appear as one line of text when you enter it in the administrative console:
*=info:com.ibm.sr.api*=all:com.ibm.serviceregistry*=all:com.ibm.athene.stats.Performance=all:com.ibm.athene.util.SQLLogger=all |
Gather data relating to the DB2 database |
To aid the collection of such information, the commands and queries to be run are listed on this page so the DBA can collect most of the important information in one go.
Substitute DBNAME for the WSRR database name, USERNAME for the database user and PASSWORD for the database user's password.
- db2look -d DBNAME -e -o db2look.ddl -a -l -f
- db2 get dbm cfg > dbmcfg.out
- db2 get db cfg for DBNAME > dbcfg.out
- db2 connect to DBNAME user USERNAME using PASSWORD
- db2 -tvf table.sql > table.out
- db2 -tvf index.sql > index.out
- db2 -tvf buffer.sql > buffer.out
- db2level > db2level.out
Gather data relating to problems with WSRR Business Space performance |
Enable WebSphere Application Server tracing with the following trace string:
*=info:com.ibm.serviceregistry.feeds.api=finer:com.ibm.sr.perf=all:com.ibm.sr.api.*=finer |
Then using IE9 (or higher):
- Start IE9
- Go to the login page for Business Space, but do not log in yet
- Press F12 to bring up the debugger and go to the Network tab and click the Start capturing button
- Log in to Business Space and perform the action which has poor performance Wait for all rendering to finish
- Click the Stop capturing button in the IE debug panel and then click the Export capture traffic button to save the NetworkData.xml file
- Stop the WebSphere Application Server trace
Additionally send output from the following:
- From a command line on the machine running the browser run a traceroute to the WSRR server machine, for example:
- Full specification of the machine running the browser (such as that provided by Control Panel > System)
Collect MustGather data for email notification problems |
For any problems with the email notification system the support team will require an export of the active configuration profile running on the WSRR instance that should be sending emails.
In addition to the profile, the following trace string should be used to gather trace showing the email notification process. Send the entire logs and ffdc directories for the support team to review.
Note: There are no spaces or new line characters in this string. It should appear as one line of text when you enter it in the administrative console:
*=info:com.ibm.sr.api*=all:com.ibm.serviceregistry*=all:com.ibm.sr.subscriptionnotifier.plugin*=all |
Lastly, to rule out problems with the WebSphere Application Server mail session, enable debug mode for the WSRRMailSession.
1. Open the administrative console.
2. Click Resources > Mail > Mail sessions > WSRRMailSession.
3. Click Enable debug mode. Debugging is enabled for the WSRRMailSession only.
4. Click Apply or OK.
With the trace enabled and mail debug enabled, perform the action in WSRR that should trigger an email to be sent. Note the time that the action was performed. Remember that WSRR uses an asynchronous notification mechanism so the email might not be sent for several minutes after the action was performed; this is dictated by the interval setting of the SubscriptionNotifierPluginScheduler scheduled task.
Collect MustGather data for SLAAttachment API problems |
Note: There are no spaces or new line characters in this string. It should appear as one line of text when you enter it in the administrative console:
*=info:com.ibm.sr.api*=all:com.ibm.serviceregistry*=all:com.ibm.sr.persistence.create*=all:com.ibm.sr.management*=all:com.ibm.sr.policy*=all |
Collect MustGather data for Access Control problems |
Note: There are no spaces or new line characters in this string. It should appear as one line of text when you enter it in the administrative console. Note that this trace string includes the WebSphere Application Server security MustGather trace.
*=info:com.ibm.sr.api*=all:com.ibm.serviceregistry*=all:com.ibm.sr.authz*=all:com.ibm.ws.security.*=all:com.ibm.websphere.security.*=all:com.ibm.websphere.wim.*=all:com.ibm.wsspi.wim.*=all:com.ibm.ws.wim.*=all |
In addition to the trace, provide an export of the current active configuration profile and a description of the users or groups who demonstrate the problem. For a user, describe which WSRR roles they belong to as well as which User Registry (typically LDAP) groups they belong to.
Collect MustGather data for related products |
MustGather: Read first for WebSphere Application Server
MustGather: Read first for WebSphere Process Server
Submit MustGather data to IBM Technical Support |
To diagnose or identify a problem, it is sometimes necessary to provide Technical Support with data and information from your system. In addition, Technical Support might also need to provide you with tools or utilities to be used in problem determination. You can submit files using one of following methods to help speed problem diagnosis:
- IBM Support Assistant (ISA)
- Service Request (SR)
- FTP to the Enhanced Customer Data Repository (ECuRep)
For information about using each of these services, see Exchanging information with IBM Technical Support.
Related information |
WebSphere Enterprise Service Bus Support page
Recommended fixes for WebSphere Enterprise Service Bus
[{"Business Unit":{"code":"BU059","label":"IBM Software w\/o TPS"},"Product":{"code":"SSWLGF","label":"WebSphere Service Registry and Repository"},"Component":"Documentation","Platform":[{"code":"PF002","label":"AIX"},{"code":"PF016","label":"Linux"},{"code":"PF027","label":"Solaris"},{"code":"PF033","label":"Windows"},{"code":"PF035","label":"z\/OS"}],"Version":"8.5;8.0","Edition":"","Line of Business":{"code":"LOB45","label":"Automation"}}]
Product Synonym
WSRR
Was this topic helpful?
Document Information
Modified date:
21 March 2022
UID
swg21258610