IBM Support

MustGather: Web Services Security (WS-Security) problems with WebSphere Application Server

Troubleshooting


Problem

Collecting data for problems with IBM WebSphere® Application Server versions 9.0, 8.5, 8.0, 7.0, and 6.1 and WebSphere Liberty using Web Services Security. Gathering this MustGather information before calling IBM support will help you understand the problem and save time analyzing the data.

Resolving The Problem


 
  • Read first and MustGathers

    MustGather: Read first for WebSphere Application Server

    EJB container problem Servlet engine and Web container problem Core security problem JAX-WS/JAX-RPC Web services engine and tooling problem JAX-WS PolicySet and Bindings problem

    For a listing of all technotes, downloads, and educational materials specific to the Web Services Security component, search the WebSphere Application Server support site.

  • Exchange data with IBM Support

    To diagnose or identify a problem, it is sometimes necessary to provide Technical Support with data and information from your system. In addition, Technical Support might also need to provide you with tools or utilities to be used in problem determination. You can submit files using one of following methods to help speed problem diagnosis:

    • Service Request (SR)
    • E-mail
    • FTP to the Enhanced Customer Data Repository (ECuRep)

    Instructions for exchanging information with IBM Support

  • WS-Security trace specifications
    Traces that you are obtaining to send to IBM support must be gathered from server startup because WS-Security configuration data is emitted in the trace only during application server startup.

    • WebSphere traditional

      Trace strings for WebSphere traditional must be entered as one line with no breaks or spaces.

      *=info:com.ibm.websphere.wssecurity.*=all:com.ibm.ws.webservices.wssecurity.*=all:com.ibm.wsspi.wssecurity.*=all:com.ibm.ws.wssecurity.*=all:com.ibm.xml.soapsec.*=all:com.ibm.ws.webservices.trace.*=all:com.ibm.ws.websvcs.trace.*=all:com.ibm.ws.webservices.multiprotocol.AgnosticService=all:com.ibm.ws.websvcs.utils.SecurityContextMigrator=all:com.ibm.ws.wssecurity.platform.audit.*=off

      Avoid delay: Do not add com.ibm.ws.security.*, com.ibm.ws.webservices.*, com.ibm.ws.websvcs.* or SSL=all to your trace specification unless IBM support asks you to do so. Adding these trace strings makes a WS-Security trace very difficult to read. IBM support may ask you to re-gather the trace if you do this.
       
    • Liberty

      org.apache.xml.security.*=all:com.ibm.ws.security.*=all:com.ibm.ws.wssecurity.*=all:com.ibm.ws.webcontainer.*=all:com.ibm.ws.http.*=all:com.ibm.ws.ssl.*=all:com.ibm.ws.channel.ssl.*=all:com.ibm.ws.transport.http.*=all:com.ibm.ws.http.internal.*=all:com.ibm.ws.http.dispatcher.*=all:org.apache.cxf.*=all:org.apache.ws.security.*=all:org.opensaml.*=all:com.ibm.websphere.channelfw.ChannelUtils=all
  • Collect data for WebSphere traditional
    This section is for collecting data for WEBSPHERE TRADITIONAL. If you want to collect data for Liberty, see the Collect data for Liberty section below.


    To troubleshoot a WS-Security problem in WebSphere traditional, collect the information listed in the step-by-step section below.

    When all the information for your issue is ready, follow the instructions on Exchanging information with IBM Technical Support for problem determination to send the information and files that you collected.

    • Trace specification
      WS-Security trace specification for WebSphere traditional:
      *=info:com.ibm.websphere.wssecurity.*=all:com.ibm.ws.webservices.wssecurity.*=all:com.ibm.wsspi.wssecurity.*=all:com.ibm.ws.wssecurity.*=all:com.ibm.xml.soapsec.*=all:com.ibm.ws.webservices.trace.*=all:com.ibm.ws.websvcs.trace.*=all:com.ibm.ws.webservices.multiprotocol.AgnosticService=all:com.ibm.ws.websvcs.utils.SecurityContextMigrator=all

      Trace strings for WebSphere traditional must be entered as one line with no breaks or spaces.
       
    • Step-by-step
      Avoid delay: Traces that you are obtaining to send to IBM support must be gathered from server startup because WS-Security configuration data is emitted in the trace only during application server startup.
      Item to collect
      Comments / Instructions
      Problem description Provide a clear, specific problem description, including specific usage information and error scenario.
      Diagnostic questions
      1. Did this work at one time before changes were made? Please explain.
      2. What is the Web service client (for example, a servlet running on WebSphere Application Server, a standalone Java™ application, Microsoft® .NET client, and so on)?
      3. What is the Web service provider (WebSphere Application Server, .NET, unknown third party)?
      4. Is the failure reported in the logs from the Web service client or the Web service provider?
      5. When does the problem occur?
      6. How often does the problem occur?
      7. Is SSL being used?
      Simplified test case, EAR file or RAD project interchange file Provide a simplified test case which demonstrates the problem. Include step-by-step instructions for running the test case. Due to the complex nature of Web services security problems, the fastest way for us to resolve your issue is through a test case.

      If a test case is not available, provide an EAR file from both the Web service provider and Web service client or provide a Project Interchange file exported from Rational® Application Developer.
       
      JAX-WS WS-Security policy and binding data

      -or-

      JAX-RPC deployment descriptors      		 If a problem requires inspection of the WS-Security configuration, failure to send the complete set of PolicySet and bindings files from the server will delay resolution of the problem.
      For JAX-WS Applications:
      Avoid delay: If at all possible, for this step, use the zip utility to zip the directories. rar and tar files cannot be used with the automated tools for extracting policy/bindings from the zips.

      Generate separate recursive zip files from each of the following directories:
      • (cellRoot)/PolicySets
      • (cellRoot)/bindings
      • (cellRoot)/applications/(earName)/deployments/(appName)/META-INF

      For JAX-RPC Applications:
      Gather the following files:
      • (serverRoot)/ws-security.xml
      • (cellRoot)/ws-security.xml, if it exists
      • (cellRoot)/sibws-wssecurity.xml, if it exists
      • For Servlets, zip the contents of
        • (cellRoot)/applications/(earName)/deployments/(appname)/(warName)/WEB-INF
      • For EJBs, zip the contents of
        • (cellRoot)/applications/(earName)/deployments/(appname)/(jarName)/META-INF
      WS-Security trace Enable Web services security tracing and reproduce the problem. If possible, enable tracing on both the Web service client and the Web service provider.

      Avoid delay: WS-Security traces must be gathered from client or application server startup in order to confirm that the components are initialized without error.

      1. Determine your trace specification
      1. Expand the Trace specification section above.
      2. Copy the trace specification.

      2. Enable trace
      • To enable trace on a MANAGED or UNMANAGED CLIENT (launchClient or thinclient):
        1. Follow the instructions in the Tracing web services topic in the Knowledge Center.
        2. Use the WS-Security trace string that you chose in the Determine your trace specification step above.
        3. After enabling trace on your client, proceed to 'Reproduce the problem'

        To enable trace on an APPLICATION SERVER:
        1. In the Administrative Console, expand Troubleshooting > Logs and Trace > server_name > Diagnostic trace service
        2. Configure the trace file specifications
          1. Under Trace Output, select File
          2. (Optional) Set Maximum File Size and Maximum Number of Historical Files
            • The default values for Maximum File Size and Maximum Number of Historical Files are sufficient if the problem can be recreated with one request. However, if the problem is intermittent, it is necessary to increase the File Size to 50 MB and set an appropriate number of historical files.
          3. Click Apply
        3. Set the trace specification
          1. Click Change log level details
          2. If there is a trace specification in the box, delete it
          3. Enter the WS-Security trace string that you chose in the Determine your trace specification step above.
          4. Click Apply
        4. Click Save
        5. Proceed to 'Reproduce the problem'

      3. Reproduce the problem
      Avoid delay: WS-Security traces must be gathered from client or application server startup.
      On the client or application server on which you are obtaining a trace do the following:
      1. Stop the client or application server
      2. Restart the client or application server
      3. Reproduce the problem

      4. Locate the trace file
      In WEBSPHERE TRADITIONAL, the trace is found in the following location:
      • (was_profile_root)/logs/(server_name)/trace*.log

      Follow instructions to send diagnostic information to IBM support to send the files mentioned in the preceding steps.
       
    • Video
      The following video shows how to gather data for WebSphere Application Server traditional to send to IBM support for problem analysis. This video is a companion to the ' Step-by-step' section above.

  • Collect data for Liberty
    This section is for collecting data for LIBERTY. If you want to collect data for WebSphere traditional, see the Collect data for WebSphere traditional section above.


    To troubleshoot a WS-Security problem in Liberty, collect the information listed in the step-by-step section below.

    When all the information for your issue is ready, follow the instructions on Exchanging information with IBM Technical Support for problem determination to send the information and files that you collected.

    • Trace specification
      WS-Security trace specification for Liberty:
      org.apache.xml.security.*=all:com.ibm.ws.security.*=all:com.ibm.ws.wssecurity.*=all:com.ibm.ws.webcontainer.*=all:com.ibm.ws.http.*=all:com.ibm.ws.ssl.*=all:com.ibm.ws.channel.ssl.*=all:com.ibm.ws.transport.http.*=all:com.ibm.ws.http.internal.*=all:com.ibm.ws.http.dispatcher.*=all:org.apache.cxf.*=all:org.apache.ws.security.*=all:org.opensaml.*=all:com.ibm.websphere.channelfw.ChannelUtils=all
    • Step-by-step
      Avoid delay: Traces that you are obtaining to send to IBM support must be gathered from server startup because WS-Security configuration data is emitted in the trace only during Liberty server startup.
      Item to collect
      Comments / Instructions
      Problem description Provide a clear, specific problem description, including specific usage information and error scenario.
      Diagnostic questions
      1. Did this work at one time before changes were made? Please explain.
      2. When does the problem occur?
      3. How often does the problem occur?
      WS-Security configuration
      information      		 Gather the following files:
      • At a minimum, send in the following files:
        • server.xml
        • the wsdl for your application
      • If you can recursive zip your Liberty installation and that zip file is 500mb or smaller, send a recursive zip of your Liberty installation directory.
      WS-Security trace Enable the WS-Security tracing and reproduce the problem.

      Avoid delay: WS-Security traces must be gathered from Liberty server startup in order to confirm that the component is initialized without error.

      1. Determine your trace specification
      1. Expand the Trace specification section above.
      2. Copy the trace specification.

      2. Enable trace
      To enable trace on Liberty:
      1. Follow the instructions in the Enabling Trace on Liberty Profile section in Set up trace and get a full dump for WebSphere Liberty.
      2. Use the trace string that you chose in the Determine your trace specification step above.
      3. Proceed to 'Reproduce the problem'

      3. Reproduce the problem
      Avoid delay: WS-Security traces must be gathered from Liberty server startup.
      On the Liberty server on which the feature is configured, do the following:
      1. Stop the Liberty server
      2. Restart the Liberty server
      3. Reproduce the problem, taking note of any relevant user/group names used, exact URL strings accessed and general time stamps.

      4. Locate the trace and log files
      On LIBERTY, by default, the trace is found in the following location:
       
      • (wlp.install.dir)/usr/servers/(server_name)/logs
      If you do not see your trace in that directory, you will find the log directory configured on the logDirectory attribute in your server.xml file.

      5. Recursive zip the logs directory
      Perform a recursive zip in the directory identified in the step above and send in the file. This zip will gather the following files:
          * console.log
          * messages.log
          * trace.log
          * ffdc/*

      Follow instructions to send diagnostic information to IBM support to send the files mentioned in the preceding steps.
       
Note:
This document uses the term WebSphere traditional to refer to WebSphere Application Server v9.0 traditional, WebSphere Application Server v8.5 full profile, WebSphere Application Server v8.0 and earlier, WebSphere classic, traditional WebSphere, traditional WAS and tWAS.
 

[{"Product":{"code":"SSEQTP","label":"WebSphere Application Server"},"Business Unit":{"code":"BU053","label":"Cloud & Data Platform"},"Component":"Web Services Security","Platform":[{"code":"PF002","label":"AIX"},{"code":"PF010","label":"HP-UX"},{"code":"PF016","label":"Linux"},{"code":"PF027","label":"Solaris"},{"code":"PF033","label":"Windows"}],"Version":"9.0;8.5.5;8.5;8.0;7.0","Edition":"Base;Developer;Express;Liberty;Network Deployment","Line of Business":{"code":"LOB45","label":"Automation"}}]

Document Information

Modified date:
30 April 2021

UID

swg21199335

Page Feedback