SPNEGO troubleshooting tips

You can securely negotiate and authenticate HTTP requests for secured resources in WebSphere® Application Server by using the Simple and Protected GSS-API Negotiation Mechanism (SPNEGO). You might encounter issues using Simple and Protected GSS-API Negotiation Mechanism (SPNEGO) as the web authentication service for WebSphere Application Server.

SPNEGO issues and their possible solutions

Note: This topic references one or more of the application server log files. As a recommended alternative, you can configure the server to use the High Performance Extensible Logging (HPEL) log and trace infrastructure instead of using SystemOut.log , SystemErr.log, trace.log, and activity.log files on distributed and IBM® i systems. You can also use HPEL in conjunction with your native z/OS® logging facilities. If you are using HPEL, you can access all of your log and trace information using the LogViewer command-line tool from your server profile bin directory. See the information about using HPEL to troubleshoot applications for more information on using HPEL.
You might encounter the following issues when you use SPNEGO as the web authentication service for WebSphere Application Server. Possible solutions are provided.

Unable to resolve the Kerberos principal name

If you are unable to resolve the Kerberos principal name, as shown in the following trace example:

[11/11/03 1:42:29:795 EST] 1d01b21e GetKrbToken   > Negotiation (GSS): Begin handshake
[11/11/03 1:42:29:795 EST] 1d01b21e Context       > GSS Context init, servername:HTTP@johnwang5.jwcmd.com
[11/11/03 1:42:29:866 EST] 1d01b21e TraceNLS      u No message text associated with key Error.getting.the.Token,
.GSS.Exception:org.ietf.jgss.GSSException,.major.code:.13,.minor.code:.0
	major.string:.Invalid.credentials
	minor.string:.Cannot.get.credential.from.JAAS.Subject.for.principal:.HTTP/192.168.0.4@168.0.4 in bundle 
com.ibm.ejs.resources.security
[11/11/03 1:42:29:866 EST] 1d01b21e GetKrbToken   E Error getting the Token, GSS Exception:org.ietf.jgss.GSSException, 
major code: 13, minor code: 0
	major string: Invalid credentials
	minor string: Cannot get credential from JAAS Subject for principal: HTTP/192.168.0.4@168.0.4
[11/11/03 1:42:29:876 EST] 1d01b21e TraceNLS      u No message text associated with key SpnegoTAI.exits.due.to.an.exception. 
in bundle com.ibm.ejs.resources.security
[11/11/03 1:42:29:876 EST] 1d01b21e SpnegoTAI     E SpnegoTAI exits due to an exception.

Add the IP address of the server in its host file. You must also recycle the application server to load the new host file.

WebSphere Application Server and the time on the Active Directory (AD) domain controller are not synchronized within 5 minutes

The trace.log file for this issue is similar to the following:
[11/11/03 1:44:09:499 EST] 1d01b21e GetKrbToken   > Negotiation (GSS): Begin handshake
[11/11/03 1:44:09:499 EST] 1d01b21e Context       > GSS Context init, servername:HTTP@backendrc4.ibm.net
[11/11/03 1:44:09:499 EST] 1d01b21e Context       > GSS Context init, done.
[11/11/03 1:44:09:679 EST] 1d01b21e SpnegoTAI     > Server response token as follows...
0000:  6082014f 06062b06 01050502 a1820143     `?.O..+.....¡?.C
0010:  3082013f a0030a01 01a10b06 092a8648     0?.? ....¡...*?H
0020:  82f71201 0202a282 01290482 01256082     ?÷....¢?.).?.%`?
0030:  01210609 2a864886 f7120102 0203007e     .!..*?H?÷......~
0040:  82011030 82010ca0 03020105 a1030201     ?..0?.. ....¡...
0050:  1ea41118 0f323030 33313131 31303634     .¤...20031111064
0060:  3430395a a5050203 0a3548a6 03020125     409Z¥....5H¦...%
0070:  a90b1b09 4a57434d 442e434f 4daa2630     ©.....IBM.NETª&0
0080:  24a00302 0100a11d 301b1b04 48545450     $ ....¡.0...HTTP
0090:  1b136a6f 686e7761 6e67352e 6a77636d     ..backendrc4.ibm
00a0:  642e636f 6dab81ab 1b81a86f 72672e69     .net.«?«.?¨org.i
00b0:  6574662e 6a677373 2e475353 45786365     etf.jgss.GSSExce
00c0:  7074696f 6e2c206d 616a6f72 20636f64     ption, major cod
00d0:  653a2031 302c206d 696e6f72 20636f64     e: 10, minor cod
00e0:  653a2033 370a096d 616a6f72 20737472     e: 37..major str
00f0:  696e673a 20446566 65637469 76652074     ing: Defective t
0100:  6f6b656e 0a096d69 6e6f7220 73747269     oken..minor stri
0110:  6e673a20 436c6965 6e742074 696d6520     ng: Client time 
0120:  54756573 6461792c 204e6f76 656d6265     Tuesday, Novembe
0130:  72203131 2c203230 30332061 7420313a     r 11, 2003 at 1:
0140:  33353a30 3120414d 20746f6f 20736b65     35:01 AM too ske
0150:  776564                                  wed

You can fix this issue in one of two ways. The preferred method is to synchronize the WebSphere system time to within 5 minutes of the time of the AD server. A best practice is to use a time server to keep all of the systems synchronized. Alternatively, you can also add or adjust the clockskew parameter in the Kerberos configuration file. Note that the default is 300 seconds (5 minutes).

No factory is available to create name for mechanism 1.3.6.1.5.5.2

If the systemout.log file contains an exception error similar to the following:
[4/8/05 22:51:24:542 EDT] 5003e481 SystemOut     O [JGSS_DBG_PROV] Provider IBMJGSSProvider version 1.01 
does not support mech 1.3.6.1.5.5.2
[4/8/05 22:51:24:582 EDT] 5003e481 ServerCredent E com.ibm.issw.spnegoTAI.ServerCredential initialize() SPNEGO014: 
Kerberos initialization Failure: org.ietf.jgss.GSSException, major code: 2, minor code: 0
	major string: Unsupported mechanism
	minor string: No factory available to create name for mechanism 1.3.6.1.5.5.2
	at com.ibm.security.jgss.i18n.I18NException.throwGSSException(I18NException.java:30)
	at com.ibm.security.jgss.GSSManagerImpl.a(GSSManagerImpl.java:36)
	at com.ibm.security.jgss.GSSCredentialImpl.add(GSSCredentialImpl.java:217)
	at com.ibm.security.jgss.GSSCredentialImpl.<init>(GSSCredentialImpl.java:264) 
.
.
Make sure that the java.security file contains the IBMSPNEGO security provider and is defined correctly. It should contain a line similar to the following:
security.provider.6=com.ibm.security.jgss.mech.spnego.IBMSPNEGO

A Kerberos error is received while decoding and verifying the SPNEGO token

You might receive the following exception error as the Java™ Generic Security Service (JGSS) library attempts to process the SPNEGO token:
Error authenticating request. Reporting to client
Major code = 11, Minor code = 31
org.ietf.jgss.GSSException, major code: 11, minor code: 31
	major string: General failure, unspecified at GSSAPI level
	minor string: Kerberos error while decoding and verifying token: com.ibm.security.krb5.internal.KrbException, status code: 31
	message: Integrity check on decrypted field failed
This error is caused when the ticket is encoded by using one key and then an attempt is made to decode the ticket by using another key. There are number of possible explanations for this:
  • The keytab file has not been copied to the server machine after it has been regenerated.
  • The Kerberos configuration points to the wrong keytab file.
  • The SPN was defined to Active Directory more than once. This is also caused by another userid with a similarly defined SPN (either the same name or it might differ by having a port defined as part of the SPN).
  • If the encryption type is DES, the password associated with the Service userid might only exist for RC4-HMAC encryption. This occurs when a new userid is created, the SPN is defined, and the keytab is generated with the +DesOnly option. The service ticket generated for this SPN is encrypted with one secret that does not match that found in the keytab.
  • An older version of the Microsoft ktpass tool is being used. Older versions of the tool create keytab files that are incorrect and might result in this error. If you are using Windows Server 2003 as your Domain controller, use the version of ktpass.exe that is part of Windows Server 2003 SP 2 (specifically, version 5.2.3790.2825).

If the problem is with the keytab file, then fix it. If the problem is with multiple SPN definitions, remove the extra or conflicting SPN, confirm that the SPN is no longer registered with AD, and then add the SPN again. Read about Creating a Kerberos service principal name and keytab file for more information. The Active Directory might need to be searched for other entries with SPNs defined that clash with the SPN using an LDAP browser.

To confirm that the SPN is not registered, the following command:
setspn -l userid
should return with:
Cannot find account userid

After you create the userid, change the password for the userid and then create the keytab file. If you are using Windows Server 2003 upgrade to the latest version of ktpass.

Single sign-on does not occur

When trace is turned on, the following error message might appear:
Client sent back a non-SPNEGO authentication header, SpnegoTAI exits
A possible reason for this error is that the client is returning an NT LAN manager (NTLM) response to the authorize challenge, not an SPNEGO token. This can occur due to one or more of the following issues:
  • The client has not been properly configured.
  • The client is not using a supported browser.
  • The user has not logged into the AD domain or into a trusted domain, or the client used does not support Integrated Authentication with Windows. In this case, the TAI is working properly.
  • The user accesses a service defined on the same machine as the client is running (the localhost). A browser might resolve the hostname of the URL to http://localhost<someURL> instead of to the fully-qualified name that is provided.
  • The SPN is not found in the Active Directory. The SPN must be of the format HTTP/server.realm.com. The command to add the SPN is: 
    setspn -s HTTP/server.realm.com userid

    If the SPN is defined incorrectly as HTTP/server.realm.com@REALM.COM with the addition of @REALM.COM, then delete the user, redefine it, and then redefine the SPN.

  • The hostname is resolved as a DNS Alias, not as a HOST record. Change the hostname to a HOST record.
  • The account in AD that holds the ServicePrincipalName is in an AD domain that is remote from the AD domain that the user has logged into, and these domains are not Windows 2003 domains. Migrate the domains to Windows 2003, or limit SSO to users within the same domain as the ServicePrincipalName userid.

Unable to use sign-on (SSO) with RC4-HMAC encryption

Important: RC4-HMAC is now considered a weak cipher and no longer supported by Windows by default. Use stronger ciphers such as aes128-cts-hmac-sha1-96 and aes256-cts-hmac-sha1-96.
When trace is turned on you might receive the following error message:
com.ibm.security.krb5.internal.crypto.KrbCryptoException, status code: 0
message: Checksum error; received checksum does not match computed checksum
Some possible reasons for this error include the following
  • RC4-HMAC encryption is not supported with a Windows version prior to 2003 KDC. To confirm that this is a problem, examine the previous trace where the exception is thrown. The content of the incoming ticket should be visible in the trace. While it is encrypted, the SPN name for the service is readable. If a Windows version prior to 2003 KDC is used, and the system is configured to use RC4-HMAC, the string representing the ticket for userid@REALMinstead of the expected HTTP/hostname.realm@REALM is shown. For example, this is beginning of the ticket received from a Windows version prior to 2003 KDC:
    0000: 01 00 6e 82 04 7f 30 82  04 7b a0 03 02 01 05 a1  ..n...0.........
0010: 03 02 01 0e a2 07 03 05  00 20 00 00 00 a3 82 03  ................
0020: a5 61 82 03 a1 30 82 03  9d a0 03 02 01 05 a1 0a  .a...0..........
0030: 1b 08 45 50 46 44 2e 4e  45 54 a2 18 30 16 a0 03  ...REALM.COM.0..
0040: 02 01 01 a1 0f 30 0d 1b  0b 65 70 66 64 77 61 73  .....0...userid
0050: 75 6e 69 74 a3 82 03 6e  30 82 03 6a a0 03 02 01  .a.f...n0..j....
    The realm is REALM.COM. The service name is userid. A correctly formed ticket for the same SPN is:
    0000: 01 00 6e 82 04 56 30 82  04 52 a0 03 02 01 05 a1  ..n..V0..R......
0010: 03 02 01 0e a2 07 03 05  00 20 00 00 00 a3 82 03  ................
0020: 82 61 82 03 7e 30 82 03  7a a0 03 02 01 05 a1 0a  .a...0..z.......
0030: 1b 08 45 50 46 44 2e 4e  45 54 a2 2a 30 28 a0 03  ..REALM.COM.0...
0040: 02 01 02 a1 21 30 1f 1b  04 48 54 54 50 1b 17 75  .....0...HTTP..u
0050: 73 31 30 6b 65 70 66 77  61 73 73 30 31 2e 65 70  serid.realm.com.
0060: 66 64 2e 6e 65 74 a3 82  03 39 30 82 03 35 a0 03  ...n.....90..5..

    To correct the problem, either use single DES encryption or use a Windows Server 2003 for a KDC. Remember to regenerate the SPN and the keytab file.

  • RC-HMAC encryption does not work when the credential delegation feature is used. To determine if you have this problem, enable JGSS and Krb5 tracing. If the SPN name is correct, messages such as the following might appear:
    [JGSS_DBG_CTX] Successfully decrypted ticket
[JGSS_DBG_CTX] Put authz info in cache
[JGSS_DBG_CTX] Session key type = rc4-hmac
…
[JGSS_DBG_CTX] Successfully decrypted authenticator
[JGSS_DBG_CTX] Error authenticating request. Reporting to client
…
Major code = 11, Minor code = 0
org.ietf.jgss.GSSException, major code: 11, minor code: 0
	major string: General failure, unspecified at GSSAPI level
	minor string: Kerberos error converting KRBCred: com.ibm.security.krb5.internal.crypto.KrbCryptoException, status code: 0
	message: Checksum error; received checksum does not match computed checksum

    This indicates that the delegated credential contained in the SPNEGO token was not encrypted with the proper key.

    Obtain APAR IY76826. This replaces ibmjgssprovider.jar with a version that can accept the Microsoft defined RC4 encrypted delegated credential.

  • The password used when generating the keytab file with ktpass does not match the password assigned to the service account. When the password changes you should regenerate and redistribute the keys., even if it is reset to the same password.
    In addition, the ktpass tool might generate a keytab file with a non-matching password as in the following cases:
    • If the password entered to ktpass matches the password for the service account, then the produced keytab file does work.
    • If the password entered to ktpass does not match the password for the service account, and is less than 7 characters in length, ktpass stops and does not produce a keytab file.
    • If the password entered to ktpass does not match the password for the service account, and is greater than 6 characters in length, ktpass does not stop. Instead, it produces a keytab file containing an invalid key. Use of this key to decrypt a SPNEGO token produces the checksum error previously listed.

    Use a non-null password for the service account, and then use that password when invoking ktpass.

  • The ktpass version 1830 (in Support Tools SP1) can produce the error in some Windows 2003 Server environments. Use the SP2 version of the tool to avoid the error.

    Use the Support Tools SP2 version of ktpass to generate the keytab file.

Credential delegation might not work due to an invalid option in the ticket request

When trace is turned on, if the following error message appears:
com.ibm.security.krb5.KrbException, status code: 101 message: Invalid option in ticket request

the Kerberos configuration file is not properly configured. Ensure that neither renewable nor proxiable are set to true.

Problems when accessing a protected URL through the SPNEGO single sign-on (SSO)

You might receive an error similar to the following when accessing a protected URL through the SPNEGO SSO:
Bad Request

Your browser sent a request that this server could not understand.
Size of request header field exceeds server limit.

Authorization: Negotiate YII……

This message is generated by the Apache/IBM HTTP Server, and indicates that the authorization header that your browser has returned is too large. The long string that follows the word Negotiate is the SPNEGO token. This SPNEGO token is a wrapper of the Windows Kerberos token. Windows includes the PAC information of the user in the Kerberos token. The more security groups that the user belongs to, the more PAC information is inserted in the Kerberos token, and the larger SPNEGO becomes. IBM HTTP Server 2.0 (as well as Apache 2.0 and IBM HTTP Server 6.0) limit the size of any acceptable HTTP header to be 8K. In Windows domains with many groups, and with user membership in many groups, the size of the user's SPNEGO token can exceed the 8K limit.

If possible, reduce the number of security groups that the user is a member of. IBM HTTP Server 2.0.47 cumulative fix PK01070 allows for HTTP header sizes up to and beyond the Microsoft limit of 12K.

After applying the fix you must specify the LimitRequestFieldSize parameter in the httpd.conf file to increase the size of allowable headers from the default of 8192.

Even with JGSS tracing disabled, some KRB_DBG_KDC messages appear in the SystemOut.log

While most of the JGSS tracing is controlled by the com.ibm.security.jgss.debug property, a small set of messages are controlled by the com.ibm.security.krb5.Krb5Debug property. The default value of the krb5 property is to emit some messages to SystemOut.log.

To remove all KRB_DBG_KDC messages from the SystemOut.log, set the JVM property to -Dcom.ibm.security.krb5.Krb5Debug=none.

ktpass is unable to find the userid

When using ktpass, you might receive an error message similar to the following:
DsCrackNames returned 0x2 in the name entry for server3
Failed getting target domain for specified user.

In an Active Directory forest, the userid lookup used by the ktpass.exe does not have a default domain name to be used. This does not occur when the domain controller is not in a forest.

To fix this problem, instead of specifying option -mapUser userid, use -mapUser userid@domain instead. For example, specify -mapUser server3@WIBM.NET.

Credential delegation does not work for any userid

If in the trace.log, an error exception similar to the following appears:
> com.ibm.issw.spnegoTAI.Context getDelegateCred() Entry
d com.ibm.issw.spnegoTAI.Context getDelegateCred() unable to get Delegate Credential
< com.ibm.issw.spnegoTAI.Context getDelegateCred() Exit
W com.ibm.issw.spnegoTAI.SpnegoHandler handleRequest() SPNEGO021: No delegated credentials were found for user: nauser@NA.IBM.NET

the domain account on which the SPN is attached does not have the Account is trusted for Delegation property defined.

To address this issue, ensure that the domain account does define the Account is trusted for Delegation property.

A user is challenged for credentials even though the browser is properly configured

A user might be challenged for credentials even though the browser is configured properly. The TAI might have obtained the user's credentials from the SPNEGO token, and the user might have failed to log in. In the trace.log an exception error similar to the following appears:
< com.ibm.issw.spnegoTAI.SpnegoTAI getAuthenticatedUsername(): lansche Exit
d com.ibm.issw.spnegoTAI.SpnegoTAI negotiateValidateandEstablishTrust(): Handshake finished, sending 200 :SC_OK
< com.ibm.issw.spnegoTAI.SpnegoTAI negotiateAndValidateEstablishedTrust Exit
A SECJ0222E: An unexpected exception occurred when trying to create a LoginContext. The LoginModule alias is system.WEB_INBOUND 
and the exception is...
The userid (which is lansche in the previous example) does not exist in the registry in use by WebSphere. This problem can be caused when:
  • The registry used by WebSphere is not the Active Directory domain LDAP, or Global Catalogue, but is some other virtual registry (for example, a file-based custom user registry).
  • A custom IClientToServerUseridMapper implementation modifies the username such that the name it is mapped to does not exist in the registry.
  • The attribute mapped to by the WebSphere LDAP User Filter property is incorrect.

To fix this problem, ensure that the user that is being asserted to WebSphere Application Server by the TAI is the configured WebSphere registry.

A user using the Novell client cannot authenticate using SPNEGO

If a user using the Novell client cannot authenticate using SPNEGO they might receive a An NTLM token is received. message.

The user might have logged into the Novell Client but did not perform a Windows Kerberos login (this can be confirmed using the Kerbtray utility). If a user has logged on to the Windows domain and has a Kerberos ticket, the user cannot utilize SPNEGO authentication.

To fix this problem, remove the Novell client and use the default Windows domain login.

Accessing SPNEGO sites via some caching proxy servers can cause SPNEGO authentication issues

If you access SPNEGO sites via some caching proxy servers you might not be able to authenticate using SPNEGO. The message SPNEGO authentication not supported on this client might be displayed.

It is possible that the caching proxy is changing the hostname that returns on the HTTP 401 Authenticate Negotiate response.

If you have this issue, contact your proxy vendor for a possible solution.

Virtual Private Networks (VPN) software and firewalls might interfere with SPNEGO operations

You might experience problems with VPN software and firewalls that might interfere with SPNEGO operations.

To resolve these issues, contact your VPN and or firewall vendors for any configuration changes that might be necessary.

Possible browser issue when accessing a SPNEGO protected application

There might be a browser issue if you log on to a domain machine using one password (for example, passwordA) and then log on to a second domain machine by changing your original password (for example, you might change your password on the second domain machine to passwordB).

Once you return to the original domain machine, you might not be able to obtain either a SPNEGO/Kerberos or an NTLM response to the Negotiate challenge. After two attempts, the browser displays an HTTP 404 error message.

To resolve this issue, log off the original domain machine and log back on with the new password (passwordB).

Error pages defined for the NTLMTokenReceivedPage or the SpnegoNotSupportedPage properties do load from an http:// URL

The error pages defined for the NTLMTokenReceivedPage or the SpnegoNotSupportedPage properties do load from an http:// URL. The following trace message might appear:
Could not load the SPNEGO not supported content, going with the default content. 
Exception received: java.net.ProtocolException: Server redirected too many  times (20)

This issue occurs when the loaded file performs an automatic redirect. It is not possible to both load the file from a web server and also use an automatic redirection

To resolve this issue, load the content from a file:/// URL, not an http:// URL.

A client browser single sign-on (SSO) attempt fails to authenticate

An error can occur when a client browser single sign-on (SSO) attempt fails to authenticate with WebSphere Application Server when you use a SPNEGO token with Microsoft Internet Security Acceleration Server

When tracing is enabled, the following messages exist:
com.ibm.ws.security.spnego.SpnegoHandler isAuthHeaderNotSPNEGO 
ENTRY Negotiate 
com.ibm.ws.security.spnego.SpnegoHandler isAuthHeaderNotSPNEGO 
Client sent back a non-SPNEGO authentication header

When a Microsoft Internet Security Acceleration Server (ISA) exists between a client browser and WebSphere Application Server, ISA might intercept the SPNEGO authentication header from the client browser request. ISA converts the SPNEGO object identifier (OID) to a Kerberos OID. The authentication attempt with WebSphere Application Server fails because the SPNEGO OID has been converted and is now missing.

For information about how to fix this issue, see the Users cannot access a web site that is published in ISA Server 2006 if the web site accepts only the SPNEGO authentication package topic on the Microsoft Corporation Support site.

Establishing an unrestricted policy then using AES256 encryption

You can use AES256 encryption after first establishing an unrestricted policy. Follow these steps:
  1. Stop the application server.
  2. Download and install the new policy files.
    Important: Your country of origin might have restrictions on the import, possession, use, or re-export to another country, of encryption software. Before downloading or using the unrestricted policy files, you must check the laws of your country, its regulations, and its policies concerning the import, possession, use, and re-export of encryption software to ensure compliance.
    1. Click on the appropriate SDK level.
    2. Scroll down the page then click IBM SDK policy files. The unrestricted JCE policy files for SDK web site displays.
    3. Click Sign in and provide your IBM.com ID and password.
    4. Select unrestricted JCE policy files for SDK and click Continue.
    5. View the license and click I Agree to continue.
    6. Extract the unlimited jurisdiction policy files that are packaged in the ZIP file. The ZIP file contains a US_export_policy.jar file and a local_policy.jar file.
    7. In your WebSphere Application Server installation, go to the $JAVA_HOME/lib/security directory and back up your existing US_export_policy.jar and local_policy.jar files.
    8. Replace your US_export_policy.jar and local_policy.jar files with the two files that you downloaded from the IBM.com web site.
    Note: Take a backup before replacing these files. An example of a path that would be used is WAS_Install/java/jre/lib/security .
  3. Start the application server.