Troubleshooting tips for Liberty

Check that your JDKs are at a supported level

If you experience problems that are not readily explained, check that the JDKs you are using are compliant with Java Version 7 or later, and are at a current service level. See Minimum supported Java levels.

Troubleshooting security

This section describes some common security problems and solutions you can choose.

SESN0008E: A user authenticated as anonymous has attempted to access a session owned by user:LdapRegistry/cn=steven,o=myCompany,c=US.

This error happens when an unauthenticated user tries to access a session that is created by an authenticated user. Session security that is enabled by default prevents unauthorized access of the sessions. Only the user who created a session can access it. For more information, see session security.

This error can happen when you use a JSP (login.jsp for example) for your form-login and the SSO token sent by the browser is expired. Because the SSO token is expired, the user is prompted to log in again using the login.jsp page that is configured for the form-login. The jsp page, by default, tries to get a session. This session was originally created by the user whose token is expired. Because the token is expired and the user is not authenticated, no credentials are established when you access this session, which results in this error.

To avoid this error, restart the browser that starts a new session, or configure the login.jsp file to not create the session by default. If you choose to update the login.jsp file, set <%@ page session="false" %>.

CWWKS9104A: Authorization failed for user {0} while invoking {1} on {2}. The user is not granted access to any of the required roles: {3}.

This error occurs when you do not have authorization to the roles required by the application. Make sure that the user or the group it belongs to is mapped to at least one of the roles that are mentioned in the error message. A user-to-role mapping that is defined in the ibm-application-bnd.xmi/xml file takes precedence over a mapping that is defined in the server.xml file. Check both resources to ensure that the correct mapping is defined. See Configuring authorization for applications in Liberty.

CWWKS9104A: Authorization failed for user {0}.

This error can occur if you specify both an application and webApplication for the same context root. If a conflict happens the latest configuration that is defined is ignored and causes an unexpected error, such as CWWKS9104A.

CWWKZ0013E: It is not possible to start two applications called {0} followed by unexpected security behavior and error messages such as CWWKS9104A.

This error occurs when you specify your application in both the server.xml by using the application element and in the dropins folder. As a result, the application is attempted to be installed twice and the security configuration in the server.xml file might or might not take effect. To fix this problem, you must remove your application from the dropins folder and copy it to another directory. If you must leave it in the dropins folder, you must disable the application monitoring by using the following code in your server.xml file:

<applicationMonitor dropinsEnabled="false"/>

An unauthenticated user was able to access my servlet or JSP.

A user with a principal of UNAUTHENTICATED (or the unauthenticated SAF user on z/OS®) is created when authentication fails or when your servlet or JSP is unprotected. An unauthenticated user can access your servlet or JSP if you do not define any security constraints or if you map the EVERYONE special subject to the role required by your application. Review the user-to-role mappings in the ibm-application-bnd.xmi/xml and server.xml files. Take one of the following options:

• If your servlet or JSP is unprotected, add security constraints to the deployment descriptor of your application. See Authentication.
• If you do not want any unauthenticated user to access your application, remove the EVERYONE special subject from the mapping for that role. See Configuring authorization for applications in Liberty.
• If a certain user cannot be authorized to your servlet or JSP, review who is mapped to that role by examining the ibm-application-bnd.xmi/xml and server.xml files. You can map a role to a user, group, or special subject. If your role is mapped to the EVERYONE special subject, any user is granted access. If your role is mapped to the ALL_AUTHENTICATED special subject, any authenticated user is granted access. Remove these special subjects if you want to further limit who can access your servlet or JSP. Finally, check what group the user belongs to. Although the user might not have explicit access, the group might be mapped to the role. In this case, remove the user from the authorized group or remove the group from role mapping. See Configuring authorization for applications in Liberty.

Single sign-on (SSO) does not work.

If SSO does not work, make sure that the different Liberty servers that use the same LTPA keys, password, and ssoCookieName attribute of webAppSecurity element have the same Universal Time (UTC) and share the user registry. Also, if the token expires or if the cookie is sent from a web browser after you change the user registry in a manner that is inconsistent, such as modifying the realm or removing the user that the cookie represents, the SSO fails, and the user is prompted to enter the credential information again. See Customizing SSO configuration using LTPA cookies in Liberty.

Debugging security public APIs.

WSSecurityHelper and RegistryHelper are loaded even if security is not enabled, for example, if a security feature - appSecurity-1.0, appSecurity or zosSecurity-1.0 - is not specified. If security is not enabled, then WSSecurityHelper.isServerSecurityEnabled() and WSSecurityHelper.isGlobalSecurityEnabled() methods both return false, and RegistryHelper.getUserRegistry() method returns null.

Other security public API classes might not be loaded if security is not enabled. If you try to access these classes and call a method on one of these classes, you might get a java.lang.NoClassDefFoundError exception.

To avoid getting java.lang.NoClassDefFoundError exceptions, you must first test to see whether security is enabled by calling the WSSecurityHelper.isServerSecurityEnabled() or WSSecurityHelper.isGlobalSecurityEnabled() class, and then call other security public API classes only when security is enabled. See Security public APIs for examples of this coding technique.

Note: This behavior is different from the WebSphere Application Server traditional. In WebSphere Application Server traditional, all classes are always loaded regardless of whether security is enabled or not.

Cannot authenticate users with Unicode characters

To authenticate users whose names contain Unicode characters, you must set the character encoding type that is used by the Liberty server to UTF-8 by adding the following jvm option to the server start command:

-Dclient.encoding.override=UTF-8

You must also specify the charset and pageEncoding in your login page. Here is an example for specifying these parameters on a login JSP page:

<%@page contentType="text/html; charset=UTF-8" pageEncoding="UTF-8"%>

java.lang.annotation.AnnotationFormatError: java.lang.IllegalArgumentException:Wrong type at constant pool index at sun.reflect.annotation.AnnotationParser.parseAnnotations(AnnotationParser.java:87)

This error can occur when an OpenID Connect or OAuth provider uses a Database Store for client registration with some JDK 7 versions.

To avoid this problem, upgrade to JDK version 7.1.

Troubleshooting LDAP

This section describes some common LDAP problems and solutions you can choose.

FFDC1015I: An FFDC Incident has been created: javax.naming.ServiceUnavailableException: myldapserver.mycompany.com:636; socket closed com.ibm.ws.security.registry.ldap.internal.LdapRegistry 298

This message in messages.log indicates that the configured LDAP server cannot be reached. Check your LDAP server to verify that it is running and that its IP address can be accessed from the Liberty server.

The javax.naming.CommunicationException: simple bind failed: myldapserver.mycompany.com:636 [Root exception is javax.net.ssl.SSLHandshakeException: com.ibm.jsse2.util.g: PKIX path building failed: java.security.cert.CertPathBuilderException: unable to find valid certification path to requested target]

If you enable SSL on your LDAP server without copying the signer of the LDAP server into the truststore that is referenced in the LDAPSSLSettings element, a connection with the LDAP server fails with an SSL handshake error. Make sure that you copy the signer of the LDAP server into your truststore.

The javax.naming.CommunicationException: myldapserver.mycompany.com:389 [Root exception is java.net.BindException: Address already in use: connect]

This message might appear in the FFDC logs and indicates that the usable sockets on the local server are exhausted, which can result in a failure when you connect to your LDAP server. Make sure that the socket is not used and try again.

CWWKS1100A: Authentication did not succeed for user ID xxxxx. An invalid user ID or password was specified

This FFDC exception might happen on the server even though the user who is mentioned in the message is a valid user on the LDAP server under heavy load. With the LDAP configuration, you can add the reuseConnection=false property or disable it by using the developer tools. To fix the problem, set this property to the default value of true.

Login, authentication, or authorization is slow for LDAP or a federated repository.

Several reasons can exist for a slow login. Review the following list for some common causes.

• If the LDAP configuration for referrals is set to follow, then any referrals to other LDAP instances are followed on LDAP requests. This situation can result in contacting one or more additional LDAP servers, depending on your LDAP configuration, and thus, additional time is spent. For example, the LDAP1 server could follow a referral to the LDAP2 server, which could follow a referral to the LDAP3 server. If you do not need to acquire more information from other LDAP servers, set referrals to ignore. Additionally, one LDAP server could be experiencing a problem and error that might not display in the JVM logs. Examples include a TCP read timeout or a DNS issue when the LDAP server talks to a referred LDAP server. To diagnose these situations, you can capture packets to see how many calls are being made and if any delays or errors exist due to an LDAP server that is following a referral.
• A firewall or other software closes connections to LDAP. By default, a pool of LDAP connections is maintained to improve performance. If a cached connection is closed remotely, a new connection is made and put back in the context pool. This process can cause a delay and can cause errors to be logged in the JVM logs. To avoid this situation, set the context pool timeout to less than the remote connection closure time. For example, if a firewall closes the connection at 10 minutes, you could set the connection pool timeout for 9 minutes. Instead of encountering a failed connection and creating a new connection, the expiration is checked on a connection from the pool and a new connection is created, skipping the failure step. Example errors include the following one:

  java.net.SocketException: Connection reset

  For more information on setting the context pool timeout, see the contextPool element on the ldapRegistry element.
• With federated repositories, all repositories and registries are checked to ensure that a unique user is in the realm. If a repository or registry is responding slowly, every call is slow even if the user is not in the slow registry. Ensure that all participating base entries are responding promptly.

Occasional connection exceptions accessing the LDAP server. For example, java.net.SocketException: Connection reset

A firewall or other software closes connections to LDAP. By default, a pool of LDAP connections is maintained to improve performance. If a cached connection is closed remotely, a new connection is made and put back in the context pool. This process can cause a delay and can cause errors to be logged in the JVM logs. To avoid this situation, set the context pool timeout to less than the remote connection closure time. For example, if a firewall closes the connection at 10 minutes, the connection pool timeout could be set for 9 minutes. Instead of encountering a failed connection and creating a new connection, the expiration is checked on a connection from the pool and a new connection is created, skipping the failure step.

Troubleshooting SSL

This section describes some common SSL problems and solutions you can choose.

CWWKS9105E: Could not determine the SSL port for automatic redirection.

This error occurs when you try to access an application that redirects to an SSL port and the SSL port is not available. The port might not be available because of a missing SSL configuration or some error in the SSL configuration definition. Check the SSL configuration in the server.xml file to make sure that it exists and is correct. See Securing communications with Liberty.

FFDC1015I: An FFDC Incident has been created: java.lang.IllegalArgumentException: Unknown, incomplete configuration: missing id field com.ibm.ws.config.internal.cm.ManagedServiceFactoryTracker aSyncReadNupdate. Exception thrown while trying to read configuration and update ManagedServiceFactory. Exception = java.lang.IllegalArgumentException: Unknown, incomplete configuration: missing id field at ffdc_12.04.18_16.09.42.0.log

This error occurs when a keystore element exists in the configuration without an ID field. If you use a minimal SSL configuration, set the ID field to defaultKeyStore.

You might get an exception if you use an LDAP user registry with sslEnabled and a sslRef value is not specified.

For example, a configuration has sslEnabled set to true but there is not a sslRef value, as shown in the following example:

<ltldapRegistry id="ldap" realm="SampleLdapIDSRealm" 
host="ccwin12.austin.ibm.com" port="636" ignoreCase="true"
baseDN="o=ibm,c=us"
bindDN="cn=root"
bindPassword="rootpwd"
ldapType="IBM Tivoli Directory Server"
idsFilters="ibm_dir_server"
sslEnabled="true"
searchTimeout="8m" />

You must enter a sslRef value. If you are using a minimal SSL configuration that is similar to the following configuration, the sslRef field must be set to defaultSSLConfig.

If you are using the PKCS12 keystore, your minimal configuration would be similar to the following one.

<ltkeyStore id="defaultKeyStore" location="key.p12"
password="mypassword" />

If you are using the JKS keystore, your minimal configuration would be similar to the following one.

<ltkeyStore id="defaultKeyStore" location="key.jks"
password="mypassword" />

If a custom SSL configuration is configured, the name of that configuration must be placed in the sslRef field.

If you use a JDK from the WebSphere Application Server, you might see the following error if SSL is enabled on your Liberty Server.

java.net.SocketException: java.lang.ClassNotFoundException: Cannot find the specified class com.ibm.websphere.ssl.protocol.SSLSocketFactory 
      at javax.net.ssl.DefaultSSLSocketFactory.a(SSLSocketFactory.java:11) 
      at javax.net.ssl.DefaultSSLSocketFactory.createSocket(SSLSocketFactory.java:6) 
      at com.ibm.net.ssl.www2.protocol.https.c.afterConnect(c.java:161) 
      at com.ibm.net.ssl.www2.protocol.https.d.connect(d.java:36) 
      at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1184) 
      at java.net.HttpURLConnection.getResponseCode(HttpURLConnection.java:390) 
      at com.ibm.net.ssl.www2.protocol.https.b.getResponseCode(b.java:75) 
      at com.ibm.ws.jmx.connector.client.rest.internal.RESTMBeanServerConnection.loadJMXServerInfo(RESTMBeanServerConnection.java:142) 
      at com.ibm.ws.jmx.connector.client.rest.internal.RESTMBeanServerConnection.<init>(RESTMBeanServerConnection.java:114) 
      at com.ibm.ws.jmx.connector.client.rest.internal.Connector.connect(Connector.java:315) 
      at com.ibm.ws.jmx.connector.client.rest.internal.Connector.connect(Connector.java:103) 

This error occurs because the WebSphere Application Server SSL socket factories are not supported by Liberty. You can get past this problem by taking the following steps:

• Create a text file, such as my.java.security with the following two empty entries

  ssl.SocketFactory.provider=
  ssl.ServerSocketFactory.provider=

• Create a jvm.options file for your Liberty server
• Add the following property to your jvm.options file, that includes the full path to your text file that you just created

  -Djava.security.properties=fullPathTo/my.java.security 

• If you want to make this more reusable, you can put the my.java.security file in your server directory, and then you are able to use a relative path like this:

  -Djava.security.properties=./my.java.security 

For more information, see Customizing the Liberty environment.

Troubleshooting CORBA/IIOP

This section describes some common CORBA problems and solutions you can choose.

If you use a JDK from the WebSphere Application Server, you might see the following error if your application uses CORBA/IIOP communications.

15:21:58.096 com.ibm.rmi.pi.InterceptorManager runPreInit:178 Init Process ORBRas [default]  java.lang.ClassNotFoundException:
com.ibm.ISecurityLocalObjectBaseL13Impl.CSIClientRI
        at com.ibm.CORBA.iiop.UtilDelegateImpl.loadClass(UtilDelegateImpl.java:685)
        at javax.rmi.CORBA.Util.loadClass(Util.java:246)
        at com.ibm.rmi.pi.InterceptorManager.runPreInit(InterceptorManager.java:172)
        at com.ibm.rmi.corba.ORB.initializeInterceptors(ORB.java:664)
        at com.ibm.CORBA.iiop.ORB.initializeInterceptors(ORB.java:1084)
        at com.ibm.rmi.corba.ORB.orbParameters(ORB.java:1359)
        at com.ibm.rmi.corba.ORB.set_parameters(ORB.java:1271)
        at com.ibm.CORBA.iiop.ORB.set_parameters(ORB.java:1694)
        at org.omg.CORBA.ORB.init(ORB.java:371) 
        ...

This error occurs because the WebSphere Application Server Object Request Broker (ORB) interceptors are not supported by Liberty. You can resolve this problem by editing the orb.properties file from the JDK to not use these interceptors. This file is found in the WebSphere <JAVA_HOME>/jre/lib directory, though it might be overwritten with a copy in the user's <USER_HOME> directory. The following example shows the lines in the orb.properties file that must be commented out:

# WS Interceptors
#org.omg.PortableInterceptor.ORBInitializerClass.com.ibm.ws.Transaction.JTS.TxInterceptorInitializer
#org.omg.PortableInterceptor.ORBInitializerClass.com.ibm.ejs.ras.RasContextSupport
#org.omg.PortableInterceptor.ORBInitializerClass.com.ibm.ISecurityLocalObjectBaseL13Impl.ClientRIWrapper
#org.omg.PortableInterceptor.ORBInitializerClass.com.ibm.ws.activity.remote.cos.ActivityServiceClientInterceptor
#org.omg.PortableInterceptor.ORBInitializerClass.com.ibm.ISecurityLocalObjectBaseL13Impl.CSIClientRI 
#org.omg.PortableInterceptor.ORBInitializerClass.com.ibm.debug.olt.ivbtrjrt.OLT_RI
#org.omg.PortableInterceptor.ORBInitializerClass.com.ibm.ws.wlm.client.WLMClientInitializer

# WS ORB & Plugins properties
#com.ibm.ws.orb.transport.ConnectionInterceptorName=com.ibm.ISecurityLocalObjectBaseL13Impl.SecurityConnectionInterceptor

Troubleshooting logging and tracing

This section describes some common problems with logging and tracing.

The java.util.logging -- Java logging programming interface.

Liberty does not support using the logging.properties file to configure java.util.logging. Use Java code, for example in a deployed application or user feature to create and add java.util.logging handlers, filters, or formatters.

Since the Liberty server manages the java.util.logging logger levels in accordance with the traceSpecification attribute of the logging configuration element, avoid using the Logger.setLevel method.

Troubleshooting JAX-WS

This section describes some common JAX-WS problems and solutions you can choose.

The org.apache.cxf.bus.extension.ExtensionException occurred when you deploy web services application on Liberty.

If your application has CXF libraries and configurations that are embedded already and you want to deploy the application to Liberty, you must make sure that the jaxws-2.2 server feature is disabled by removing the feature from the server.xml file.

The java.lang.NoClassDefFoundError exception occurred when you run IBM fastpath with the Oracle JVM.

To use IBM fastpath Java Architecture for XML Binding (JAXB), you can configure the com.ibm.xml.xlxp.jaxb.opti.level custom property to control whether optimization methods are enabled for JAXB unmarshalling (deserialization) and marshalling (serialization). If you are experiencing the java.lang.NoClassDefFoundError exception when you run IBM fastpath JAXB with the Oracle JVM, you can change the value of com.ibm.xml.xlxp.jaxb.opti.level property to 0 to turn off the optimization. For example, you can use the -Dcom.ibm.xml.xlxp.jaxb.opti.level=0 property in the command line, or add the following line to the jvm.options file:

# Turn off the JAXB optimization
-Dcom.ibm.xml.xlxp.jaxb.opti.level=0

See more information of the com.ibm.xml.xlxp.jaxb.opti.level property on the Java virtual machine custom properties page.

The java.lang.NoClassDefFoundError : com.ibm.CORBA.iiop.ORB exception occurred when you run the WebSphere Application Server traditional thin client with the Oracle JVM.

This error occurs when you try to run the WebSphere Application Server traditional thin client with the Oracle JVM and the jaxws-2.2 feature is already enabled on Liberty. To resolve this problem, you can use the -Dcom.ibm.websphere.thinclient=true property when you run the WebSphere Application Server traditional thin client as follows:

java -Dcom.ibm.websphere.thinclient=true 
     -cp <classpath_entries_including_tWAS_thinclient_jar> <WebServicesClientEntryClass> <any_more_parameters>

See the similar information of the com.ibm.websphere.thinclient property on the PM39777: ADMINISTRATIVE THIN CLIENT USING SOAP CONNECTOR AND SUN JDK DOES NOT WORK page.

An Empty file that is returned when you retrieve binary files from an MTOM service to a MTOM client.

This scenario happens when a MTOM client sent binary files successfully to an MTOM service, but an empty file is returned when you retrieve the same binary files back from the MTOM service.

As a workaround, you can create a new DataHandler and initialize the DataHandler by filling the content of the binary file. For example, use a FileDataStore object to read from I/O and return the newly created DataHandler back, and then pass the DataHandler to other web services.

...
File f = new File("received_image");
	if (f.exists()) {
		f.delete();
	}

	FileOutputStream fos = new FileOutputStream(f);
	img_in.writeTo(fos);

	FileDataSource fos_out = new FileDataSource(f);
	DataHandler img_out = new DataHandler(fos_out);
	return img_out;
...

The javax.xml.ws.soap.SOAPFaultException: Unmarshalling Error occurred when you use XMLGregorianCalendar in xsd:gMonth format with the Oracle JVM.

This error occurs when you use the XMLGregorianCalendar class to store gMonth format constants as part of your SOAP request on client side and the jaxws-2.2 feature is enabled on Liberty with the Oracle JVM. The root cause is that the Oracle JVM supports the new standard format --MM for the xsd:gMonth type while the latest JAXB RI (Reference Implementation) supports the format of --MM-- only.

To resolve this problem, you can change the Oracle JVM to the IBM JVM for both the client side and the server-side applications.

The java.io.FileNotFoundException occurred when you resolve WSDL files that are specified by the wsdlLocation attribute.

This error occurs when you specify the WSDL file as in the code wsdlLocation = "file:/WEB-INF/wsdl/a.wsdl" and the WSDL file is packaged within your application. If you want to refer to the WSDL file that is packaged in your application, you must use relative URIs for the wsdlLocation attribute that is defined in the @WebService, @WebServiceProvider, @WebServiceClient, or @WebServieRef annotation.

The relative URI for the wsdlLocation attribute must be one of the following values:

• wsdlLocation = "/WEB-INF/wsdl/a.wsdl"
• wsdlLocation = "WEB-INF/wsdl/a.wsdl"

Many Creating service messages occur in the messages.log file.

This scenario happens when you call the getPort or related methods in the generated client stub classes. Liberty is configured with the default logging configuration, and all the information messages are written into the messages.log file. There are lots of occurrences of messages that might be similar to the following one:

00000037 org.apache.cxf.service.factory.ReflectionServiceFactoryBean I Creating Service {http://www.echo.org/}EchoService from WSDL: 
    wsjar:file:/wlp/usr/servers/default/workarea/org.eclipse.osgi/bundles/100/data/cache/
    com.ibm.ws.app.manager_gen_15a42559-f979-4ee6-b488-9fa1fb212c96/.cache/Echo.war!/WEB-INF/wsdl/Echo.wsdl

To suppress these information messages, you can change the logging configuration in the server.xml file as follows:

<logging traceSpecification="org.apache.cxf.*=warning=enabled"/>

The SOAPFaultException: The given SOAPAction test does not match an operation occurred when a client application sent a SOAP Action.

Note: test is the value of the soapAction attribute in the request HTTP header.

Each operation in a SOAP web service can be associated with a SOAP Action String, such as in the WSDL binding or via an annotation. The web service client sends the SOAP Action string as a header with the request to match the operation of the web service provider. The error message is displayed in either of the following scenarios:

• When the SOAP action that is sent by the client does not match the SOAP action of the operation.
• When you use WebSphere Application Server traditional as the JAX-WS client and Liberty as the JAX-WS service and the soapAction defined in the WSDL file equals empty value "".

To resolve this problem:

• Make sure that you specify the same SOAP action for both the web service client and service provider.
• Provide a valid value to the soapAction attribute that is defined in the WSDL file or do not use soapAction in the WSDL file.

The javax.xml.ws.WebServiceException occurred when you use the Service.create() method to create service.

This error occurs when you use the Service.create() method to create a service while the WSDL document is not provided. If you want to use the Service.create() to create a service and getPort() method to get the port number, you must use the addPort() method to provide the binding information.

A sample example code is provided as follows on how to use the addPort() method:

QName serviceName = new QName("http://test.com/wssvt/acme/InsBusiness/", "MTOM11Service");
QName portName = new QName("http://test.com/wssvt/acme/InsBusiness/", "MTOM11Port");

// Setup the necessary JAX-WS artifacts
Service svc = Service.create(serviceName);
// Add the port in the service instance
svc.addPort(portName, SOAPBinding.SOAP11HTTP_MTOM_BINDING, mtom11URL); 

port = svc.getPort(portName, MTOMInterface.class);

The Null Response returned when the name attribute in @WebResult annotation is different from the name element in the WSDL document.

This problem occurs when you use a SEI class to define the name attribute for @WebResult annotation, and the SEI class supplies the WSDL location as follows:

@WebService(wsdlLocation="WEB-INF/wsdl/WebServiceIfc.wsdl")
public interface WebServiceRuntimeIfc {
        @WebMethod
        @WebResult(name="nononoreturn") 
        public String echo (String parm);
}

But the XML element declaration in the supplied WSDL document is as follows:

<xs:element name="echoResponse">
	<xs:complexType>
		<xs:sequence>
			<xs:element name="return" type="xs:string" minOccurs="0"/>
		</xs:sequence>
	</xs:complexType>
</xs:element>

Then the web service client gets a NULL response when the echo() method is called.

To resolve this problem, ensure that the name attribute of the @WebResult annotation matches the value of the name element in the WSDL document.

The JAX-WS client application failed to get the WSDL document changes from the server side.

The problem occurs when the web service client and service provider are in two different applications on Liberty, and the client needs to retrieve the WSDL document from the service provider dynamically. Because Liberty caches the WSDL definition when the WSDL document is accessed for the first time, which is different from the behavior in WebSphere Application Server traditional. By caching the WSDL definition in Liberty, applications can avoid connecting and parsing the WSDL document frequently.

To solve this problem, you must restart the client application so that the updated WSDL definitions can be reloaded.

JAXB warning during WSDL import

The following warning message can be encountered while you import a WSDL file.

[WARNING] unknown extensibility element or attribute "wsdl" (in namespace "http://www.w3.org/2000/xmlns/" (http://www.w3.org/2000/xmlns/%27) )

Troubleshooting WS-Security

Enabling WS-Security trace

If the cause of the problem cannot be determined by the information in the error message, you can use the Liberty trace and logging facility to collect trace for the WS-Security component. See Liberty: Trace and logging.

org.apache.ws.security.*=all:
org.apache.cxf.ws.security.*=all:
org.apache.cxf.ws.policy.*=all:
org.apache.xml.security.*=all:
com.ibm.ws.wssecurity.*=all

This section described some common WS-Security problems and solutions you can choose.

org.apache.cxf.ws.policy.PolicyException: None of the policy alternatives can be satisfied.

This error usually occurs when the WS-Security feature wsSecurity-1.1 is not defined in the server.xml file. To avoid this error, you must define the wsSecurity-1.1 feature in the server.xml file as follows:

<featureManager>
  <feature>usr:wsseccbh-1.0</feature>
  <feature>servlet-3.0</feature>
  <feature>appSecurity-2.0</feature>
  <feature>jaxws-2.2</feature>
  <feature>wsSecurity-1.1</feature>
</featureManager>

org.apache.cxf.ws.policy.PolicyException: No callback handler and no password available.

This error occurs when the WS-Security run time cannot retrieve a password that is required to generate a UsernameToken or to access a private key from the keystore. To avoid this error, check the following configuration:

1. Ensure that the password callback handler feature wsseccbh-1.0 is defined as a user feature in the server.xml file:

   <featureManager>
     <feature>usr:wsseccbh-1.0</feature>
     <feature>servlet-3.0</feature>
     <feature>appSecurity-2.0</feature>
     <feature>jaxws-2.2</feature>
     <feature>wsSecurity-1.1</feature>
   </featureManager>

2. Ensure that the callback handler property ws-security.callback-handler is defined in the <wsSecurityClient> or <wsSecurityProvider> elements of the server.xml file:

   ws-security.callback-handler="<full class name of the callback handler>"
   
   example:
   ws-security.callback-handler="com.ibm.ws.wssecurity.example.cbh.CommonPasswordCallback"

3. Ensure that the password callback handler returns the correct password for the user name or the key alias that is specified in the server.xml file. The password must be clear text and not encoded or encrypted.

org.apache.ws.security.WSSecurityException: The signature does not cover the required elements (soap:Body).

This error occurs when a web service provider application expects the SOAP body in the request message to be signed, but the received SOAP message does not have the SOAP body signed. To avoid this error, ensure that your web service client is configured with the correct WS-Security policy that matches the web service provider's policy.

org.apache.ws.security.WSSecurityException: The signature or decryption was invalid.

This error occurs when the WS-Security run time fails to validate the signature or decrypt an encrypted message part in the received SOAP message. To avoid this error, verify that the correct keys are used when you configure WS-Security in the <wsSecurityClient> and <wsSecurityProvider> elements of the server.xml file. If a web service client uses Bob's public key to encrypt a message part, then the web service provider must have access to Bob's private key to decrypt the message. Similarly, if a web service client signs a message part by using Alice's private key, then the web service provider must use Alice's public key to validate the signature.

org.apache.cxf.ws.policy.PolicyException: These policy alternatives cannot be satisfied.

This error occurs when the WS-Security run time cannot process the incoming SOAP message successfully by using the WS-Security policy that is configured to process this request. To avoid this error, look at the messages that follow this exception to determine the specific WS-Security policy assertions that are causing this exception. For example, the log file might contain the following messages:

Caused by: org.apache.cxf.ws.policy.PolicyException: These policy alternatives can not be satisfied: 
{http://docs.oasis-open.org/ws-sx/ws-securitypolicy/200702}AsymmetricBinding: The encryption algorithm does not match the requirement
{http://docs.oasis-open.org/ws-sx/ws-securitypolicy/200702}InitiatorEncryptionToken
{http://docs.oasis-open.org/ws-sx/ws-securitypolicy/200702}RecipientEncryptionToken
at org.apache.cxf.ws.policy.AssertionInfoMap.checkEffectivePolicy(AssertionInfoMap.java:167)
at org.apache.cxf.ws.policy.PolicyVerificationInInterceptor.handle(PolicyVerificationInInterceptor.java:101)

In this case, the error occurred because the encryption algorithm that is used by the web service client does not match the encryption algorithm that is specified by the AlgorithmSuite assertion. The algorithm suites that are specified in the WS-Security policy of both the web service client and the web service provider must specify the same encryption algorithm.

javax.xml.ws.soap.SOAPFaultException: The message has expired (WSSecurityEngine: Invalid timestamp The security semantics of the message have expired).

This message occurs when the message timestamp is expired or the message was created with a timestamp in the future.

Applying fix packs and interim fixes to an archive install

If you installed your Liberty runtime environment from an archive file, rather than by using Installation Manager, you must take special measures when you apply service updates. For more information, see Applying a fix pack to a Liberty Java archive installation and Applying an interim fix to a Liberty archive installation.

Troubleshooting performance

This section describes some common performance problems and solutions you can choose.

High CPU usage by your application monitor.

This error can occur if your application monitor has many files under the apps/ directory and is polling too frequently.

To avoid this problem there are a number of things you can change.

1. Increase the value of the pollingRate attribute in the applicationMonitor element.
2. Update the server.xml to include an applicationMonitor element with an updateTrigger that is not polled.

   server.xml                                                                      
   <applicationMonitor updateTrigger="mbean" /> 
   

3. Reduce the number of files under the apps/ directory.

For more information about the applicationMonitor element, see Controlling dynamic updates.

Troubleshooting collectives

This section describes a problem with collectives and the solution you must apply.

java.lang.IllegalArgumentException: CWWKX0219E: The MBean 'WebSphere:feature=collectiveController,type=CollectiveRepository,name=CollectiveRepository' does not have an operation by the name 'retrieveMemberRootCertificate'

A server with feature collectiveMember-1.0, a member, cannot register with a server with feature collectiveController-1.0, a controller, that is at an earlier level of Liberty. For example, a member at this beta level of Liberty cannot register itself with a controller at V8.5.5.2 Liberty.

With default logging, this problem is reported as a First Failure Data Capture (FFDC) in the FFDC logs of the member.

To fix the problem, you must move the controller to the same or higher level of Liberty as the member.

Troubleshooting SAML

This section describes a problem with SAML and the solution you must apply.

java.lang.ArrayIndexOutOfBoundsException: Array index out of range: 0

This exception can occur when you attempt multiple logins through an unsolicited Service Provider (SP) initiated request without removing the Identity Provider token (IdP).

To avoid this, add <httpSession invalidateOnUnauthorizedSessionRequestException="true" /> in the relevant unsolicited server.xml file.

java.lang.IllegalStateException: CWWKS0010E: While getting the caller principal, the caller subject was found to have more than one principal of type WSPrincipal. Only one WSPrincipal can exist in the subject. The names of the WSPrincipals are: {0}

This exception can occur if a SAML user has previously logged directly into an on-premises user registry. To avoid this problem, a SAML user must not directly login to an on-premises user registry.

Troubleshooting REST API Discovery

If the IBM REST API Discovery Explorer Try it out selection is invoked remotely and fails with a response code equal to 0, ensure that the fully qualified hostname or the IP address is returned to the IBM REST API Discovery Explorer in the Curl and Request URL associated with the GET, PUT, POST, or DELETE operation. If the fully qualified hostname or IP address is not in the Curl and Request URL, complete the following actions:

1. Add the variable element in server.xml and specify the fully qualified hostname. Here is the example of adding the variable element that specifies the fully qualified hostname in the server.xml file:

    <httpEndpoint host="*" httpPort="9080" httpsPort="9443" id="defaultHttpEndpoint"/>          
   <variable name="defaultHostName" value="developer.rtp.raleigh.ibm.com"/>

2. Specify the full computer name for your operating system as the fully qualified host name.
3. Verify that the fully qualified hostname is returned in the Curl and Request URL associated with the GET, PUT, POST, or DELETE operations in the IBM REST API Discovery Explorer. For more information, see Setting the default host name of a Liberty server and read about configuring your network in the IBM InfoSphere® Information Server, Version 11.3.1 product documentation.

Troubleshooting applications not starting in an embedded Liberty server

Ensure that the Java process that starts the embedded Liberty server was started with the -javaagent JVM argument that pointed to the libertyInstallDir/bin/tools/ws-javaagent.jar. If the -javaagent JVM argument is not used the server runtime starts, but applications fail to start with no obvious exceptions.