PI70402: SAML WEB SSO OUTOFMEMORY IN KEYSTOREMANAGER

Fixes are available

APAR status

Closed as program error.

Error description

In SAML Web SSO, the customer has trustStore set to
"CellDefaultTrustStore".  After some time, an OutOfMemory
condition occurs.

The class
"com.ibm.ws.wssecurity.wssapi.token.impl.KeyStoreManager",
occupies 1,257,927,688 (81.79%) bytes.

3707 [9/28/16 13:53:19:625 UTC] 0000027a WasKeyStoreUt >
getKeyStore(String keyStoreRef[CellDefaultTrustStore]) Entry
[9/28/16 13:54:40:024 UTC] 0000027a webapp        E
com.ibm.ws.webcontainer.webapp.WebApp logServletError SRVE0293E:
[Servlet Error]-[IBMWebSphereSamlACSListenerServlet]:
java.lang.OutOfMemoryError: Java heap space
at java.lang.Class.getConstructorsImpl(Native Method)
at java.lang.Class.getConstructors(Class.java:568)
at com.ibm.crypto.provider.bd.newInstance(Unknown Source)
...

Local fix

Please ensure following property can be set as workaround.

sso_<id>.sp.keyStore="name=CellDefaultTrustStore
managementScope=(cell):myCellName"

Problem summary

****************************************************************
* USERS AFFECTED:  All users of IBM WebSphere Application      *
*                  Server                                      *
****************************************************************
* PROBLEM DESCRIPTION: OutOfMemory may occur in SAML Web SSO   *
*                      when trustStore property has no         *
*                      management scope                        *
****************************************************************
* RECOMMENDATION:  Install a fix pack that contains this       *
*                  APAR.                                       *
****************************************************************
If the SAML Web SSO TAI sso_<id>.sp.trustStore custom property
is configured either without specifying a management scope or
with a management scope that does not match the current
management scope of the keystore that WebSphere security finds
in its cache, a memory leak in the
com.ibm.ws.wssecurity.wssapi.token.impl.KeyStoreManager class
will occur.

Problem conclusion

The SAML code relies on the base security code to cache
managed keystores. The SAML code will cache private keys
retrieved from the keystores and the cache key for these keys
includes a hash of the keystore object from which it was
obtained.

When SAML attempts to retrieve a keystore that either has no
management scope or has the wrong management scope, instead of
returning the keystore that is in its cache, base security is
reading the keystore from the disk and returning a new
keystore object. Although we technically have the same
keystore from the disk and the same key from the keystore,
since the Java object for the keystore is different, so the
cache key is different.

Since SAML keeps generating unique cache keys for the same
keystore/key, the same private key keeps getting cached over
and over using different cache keys each time. Given enough
requests, an OutOfMemory condition will occur.

The base security code is updated to return the keystore from
the cache if there is no management scope specified instead
of creating a new object using the keystore on the disk.

If the management scope that is specified is not correct, then
the OutOfMemory condition may still occur.  The resolution to
this problem is to either remove or correct the management
scope.  Example:

name=myKeyStoreRef managementScope=(cell):myCell:(node):myNode


The fix for this APAR is currently targeted for inclusion in
fix packs 7.0.0.43, 8.0.0.13, 8.5.5.12 and 9.0.0.2.  Please
refer to the Recommended Updates page for delivery
information:
http://www.ibm.com/support/docview.wss?rs=180&uid=swg27004980

Temporary fix

Comments

APAR Information

APAR number
PI70402
Reported component name
WEBS APP SERV N
Reported component ID
5724H8800
Reported release
850
Status
CLOSED PER
PE
NoPE
HIPER
NoHIPER
Special Attention
NoSpecatt / Xsystem
Submitted date
2016-10-07
Closed date
2016-11-15
Last modified date
2016-11-15

APAR is sysrouted FROM one or more of the following:
APAR is sysrouted TO one or more of the following:

Fix information

Fixed component name
WEBS APP SERV N
Fixed component ID
5724H8800

Applicable component levels

R700 PSY
UP
R800 PSY
UP
R850 PSY
UP
R900 PSY
UP

[{"Business Unit":{"code":"BU059","label":"IBM Software w\/o TPS"},"Product":{"code":"SSEQTP","label":"WebSphere Application Server"},"Platform":[{"code":"PF025","label":"Platform Independent"}],"Version":"8.5","Line of Business":{"code":"LOB45","label":"Automation"}}]

Document Information

Modified date:
04 May 2022

Tips

PI70402: SAML WEB SSO OUTOFMEMORY IN KEYSTOREMANAGER

Fixes are available

Subscribe

APAR status

Closed as program error.

Error description

Local fix

Problem summary

Problem conclusion

Temporary fix

Comments

APAR Information

APAR number

Reported component name

Reported component ID

Reported release

Status

PE

HIPER

Special Attention

Submitted date

Closed date

Last modified date

APAR is sysrouted FROM one or more of the following:

APAR is sysrouted TO one or more of the following:

Fix information

Fixed component name

Fixed component ID

Applicable component levels

R700 PSY

R800 PSY

R850 PSY

R900 PSY

Document Information

Share your feedback

Need support?