Troubleshooting
Problem
WebSphere Liberty using feature openidConnectClient may cause high CPU in releases after 20.0.0.6.
Symptom
Higher CPU after fixpack upgrade with WebSphere Liberty z/OS
Review of a core dump or SVC dump for the WebSphere Liberty z/OS address space at the time of the high CPU showed that high CPU threads are running in method:
com/ibm/ws/security/registry/saf/internal/SAFRegistry.getGroupsForUser
Cause
The problem is caused by changes in how WebSphere Liberty manages the Custom Cache Key.
The changes were introduced in WebSphere Liberty 20.0.0.6.
Prior to the change, a custom cache key was never added for JWT authentication using OIDC.
Environment
WebSphere Liberty version 20.0.0.6 or later
z/OS Connect 3.0.34.0 or later
Diagnosing The Problem
Collecting a core dump or SVC dump for the WebSphere Liberty address space at the time of the high CPU shows that the high CPU threads looked similar to this:
com/ibm/ws/security/registry/saf/internal/SAFRegistry.ntv_getGroupsForUser(byte[], java.util.List)
com/ibm/ws/security/registry/saf/internal/SAFRegistry.getGroupsForUser(String)
com/ibm/ws/security/registry/saf/internal/SAFAuthorizedRegistry.getGroupsForUser(String)
com/ibm/ws/security/registry/saf/internal/SAFDelegatingUserRegistry.getGroupsForUser(String)
com/ibm/ws/security/wim/adapter/urbridge/URBridge.get(com.ibm.wsspi.security.wim.model.Root)
com/ibm/ws/security/wim/registry/util/MembershipBridge.getUniqueGroupIds(String)
com/ibm/ws/security/wim/registry/WIMUserRegistry.getUniqueGroupIdsForUser(String)
com/ibm/ws/security/credentials/wscred/internal/WSCredentialProvider.getUniqueGroupAccessIds(com.ibm.ws.security.registry.UserRegistry, String, String)
com/ibm/ws/security/credentials/wscred/internal/WSCredentialProvider.createUserWSCredential(javax.security.auth.Subject, String, String, String, String, String)
com/ibm/ws/security/credentials/wscred/internal/WSCredentialProvider.setCredential
com/ibm/ws/security/authentication/jaas/modules/HashtableLoginModule.handleUserId(String)
com/ibm/ws/security/authentication/jaas/modules/HashtableLoginModule.login()
Resolving The Problem
Set allowCustomCacheKey="false" in the openidConnectClient configuration
Name: allowCustomCacheKey
Type: boolean Default: true Description: Specifies whether a custom cache key is used to store users in an authentication cache. If this property is set to true and the cache key for a user is not found in the authentication cache, the user will be required to log in again. Set this property to false when you use the jwtSso feature to allow the security subject to be constructed directly from the jwtSso cookie. Set this property to false when you do not use the jwtSso feature to allow the security subject to be constructed directly from the user registry. If the security subject is reconstructed from the user registry, there will be no SSO components in the subject. If your LTPA cookie is used by more than one server, consider setting this property to false. If your application always requires the SSO components to be present in the subject, you must either set this property to true or use the jwtSso feature.Related Information
Document Location
Worldwide
[{"Type":"MASTER","Line of Business":{"code":"LOB45","label":"Automation"},"Business Unit":{"code":"BU059","label":"IBM Software w\/o TPS"},"Product":{"code":"SS7K4U","label":"WebSphere Application Server for z\/OS"},"ARM Category":[{"code":"a8m3p000000F7yhAAC","label":"IBM WebSphere Liberty-All Platforms-\u003EHigh CPU"}],"ARM Case Number":"TS011487412","Platform":[{"code":"PF035","label":"z\/OS"}],"Version":"All Versions"}]
Was this topic helpful?
Document Information
Modified date:
27 September 2023
UID
ibm17037339