Fixes are available
8.5.5.2: WebSphere Application Server V8.5.5 Fix Pack 2
8.5.5.3: WebSphere Application Server V8.5.5 Fix Pack 3
8.5.5.4: WebSphere Application Server V8.5.5 Fix Pack 4
8.5.5.5: WebSphere Application Server V8.5.5 Fix Pack 5
8.5.5.6: WebSphere Application Server V8.5.5 Fix Pack 6
8.5.5.7: WebSphere Application Server V8.5.5 Fix Pack 7
8.5.5.8: WebSphere Application Server V8.5.5 Fix Pack 8
8.5.5.9: WebSphere Application Server V8.5.5 Fix Pack 9
8.5.5.10: WebSphere Application Server V8.5.5 Fix Pack 10
8.5.5.11: WebSphere Application Server V8.5.5 Fix Pack 11
8.5.5.12: WebSphere Application Server V8.5.5 Fix Pack 12
8.5.5.13: WebSphere Application Server V8.5.5 Fix Pack 13
8.5.5.14: WebSphere Application Server V8.5.5 Fix Pack 14
8.5.5.15: WebSphere Application Server V8.5.5 Fix Pack 15
8.5.5.14: WebSphere Application Server V8.5.5 Fix Pack 14
8.5.5.17: WebSphere Application Server V8.5.5 Fix Pack 17
8.5.5.20: WebSphere Application Server V8.5.5.20
8.5.5.18: WebSphere Application Server V8.5.5 Fix Pack 18
8.5.5.19: WebSphere Application Server V8.5.5 Fix Pack 19
8.5.5.16: WebSphere Application Server V8.5.5 Fix Pack 16
8.5.5.21: WebSphere Application Server V8.5.5.21
APAR status
Closed as program error.
Error description
When running Websphere Application Server zOS, the com/ibm/ws/asynchbeans/services/wlm/EnclaveManager goes into a deadlock with two threads causing the server to hang and/or slowdown. The Servant could timeout and abend with an ABENDEC3 RC=0413000x . Looking at a javacore dump from the servant region you may see the following deadlock contention. . 1LKDEADLOCK Deadlock detected !!! NULL --------------------- NULL 2LKDEADLOCKTHR Thread "WebSphere WLM Dispatch Thread t=008b3620" (0x0000000040F03A00) 3LKDEADLOCKWTR is waiting for: 4LKDEADLOCKMON sys_mon_t:0x00000049A1DE4738 infl_mon_t: 0x00000049A1DE47B0: 4LKDEADLOCKOBJ com/ibm/ws/asynchbeans/services/wlm/ Enclave@0x000000009E60F370 3LKDEADLOCKOWN which is owned by: 2LKDEADLOCKTHR Thread "java.lang.ThreadGroup■name=xx.zzzzzzzzzz :xxxxxxxx.0, 3LKDEADLOCKWTR which is waiting for: 4LKDEADLOCKMON sys_mon_t:0x00000049A1DE46B0 infl_mon_t: 0x00000049A1DE4728: 4LKDEADLOCKOBJ java/util/HashMap@0x0000000082724078 3LKDEADLOCKOWN which is owned by: 2LKDEADLOCKTHR Thread "WebSphere WLM Dispatch Thread t=008b3620" (0x0000000040F03A00)
Local fix
Problem summary
**************************************************************** * USERS AFFECTED: All users of IBM WebSphere Application * * Server V8.5 * **************************************************************** * PROBLEM DESCRIPTION: Deadlock between asynchbeans * * registerEnclave and deregisterEnclave * * threads * **************************************************************** * RECOMMENDATION: * **************************************************************** Under extreme load due to timing issues a deadlock can occur between the following two thread stacks involving asynchbeans EnclaveManager. WorkManagerImpl.schedule(): "WebSphere WLM Dispatch Thread t=008b6528" J9VMThread:0x00000000400C3000, j9thread_t:0x0000004808FC3260, java/lang/Thread:0x00000000992C5F38, state:B, prio=5 (native thread ID:0x4899B100, native priority:0x5, native policy:UNKNOWN) Blocked on: com/ibm/ws/asynchbeans/services/wlm/Enclave@0x000000009D74F590 Owned by: "java.lang.ThreadGroup[name=wm.XXXName: XXXX04.0,maxpri=10]" (J9VMThread:0x0000000041030C00, java/lang/Thread:0x000000009D6F17E8) Java callstack: at com/ibm/ws/asynchbeans/services/wlm/EnclaveManager.registerEncla ve(EnclaveManager.java:214) (entered lock: java/util/HashMap@0x000000008273FD30, entry count: 1) at com/ibm/ws/asynchbeans/services/wlm/ClassificationService.peek(C lassificationService.java:248) at com/ibm/ws/asynchbeans/J2EEServiceManager.peek(J2EEServiceManage r.java:361(Compiled Code)) at com/ibm/ws/asynchbeans/WorkManagerImpl.create(WorkManagerImpl.ja va:842) at com/ibm/ws/asynchbeans/WLMWorkManagerImpl.create(WLMWorkManagerI mpl.java:180) at com/ibm/ws/asynchbeans/WorkManagerImpl.schedule(WorkManagerImpl. java:921) Work.run(): "java.lang.ThreadGroup[name=wm.XXXX: XXXX4.0,maxpri=10]" J9VMThread:0x0000000041030C00, j9thread_t:0x0000004A6FA48560, java/lang/Thread:0x000000009D6F17E8, state:B, prio=5 (native thread ID:0x444EA400, native priority:0x5, native policy:UNKNOWN) Blocked on: java/util/HashMap@0x000000008273FD30 Owned by: "WebSphere WLM Dispatch Thread t=008b6528" (J9VMThread:0x00000000400C3000, java/lang/Thread:0x00000000992C5F38) Java callstack: at com/ibm/ws/asynchbeans/services/wlm/EnclaveManager.deregisterEnc lave(EnclaveManager.java:247) at com/ibm/ws/asynchbeans/services/wlm/EnclaveManager.leaveEnclave( EnclaveManager.java:326) at com/ibm/ws/asynchbeans/services/wlm/EnclaveManager.unpinEnclave( EnclaveManager.java:346) (entered lock: com/ibm/ws/asynchbeans/services/wlm/Enclave@0x000000009D74F590, entry count: 1) at com/ibm/ws/asynchbeans/services/wlm/ClassificationService.push(C lassificationService.java:282) at com/ibm/ws/asynchbeans/J2EEServiceManager.pushContext(J2EEServic eManager.java:484(Compiled Code)) at com/ibm/ws/asynchbeans/J2EEContext.push(J2EEContext.java:448) at com/ibm/ws/asynchbeans/J2EEContext.run(J2EEContext.java:1066) at com/ibm/ws/asynchbeans/WorkWithExecutionContextImpl.go(WorkWithE xecutionContextImpl.java:199) at com/ibm/ws/asynchbeans/CJWorkItemImpl.run(CJWorkItemImpl.java:23 6(Compiled Code)) at com/ibm/ws/asynchbeans/WLMCJWorkItemImpl.run(WLMCJWorkItemImpl.j ava:69)
Problem conclusion
The WebSphere asynchbeans component has been corrected to handle the deadlock by using proper synchronization blocks. APAR PM91608 is currently targeted for inclusion in Fix Pack 8.5.5.2 of WebSphere Application Server V8.5. Please refer to the Recommended Updates page for delivery information: http://www.ibm.com/support/docview.wss?rs=180&uid=swg27004980 In addition, please refer to URL: http://www.ibm.com/support/docview.wss?rs=404&uid=swg27006970 for Fix Pack PTF information.
Temporary fix
Comments
APAR Information
APAR number
PM91608
Reported component name
WEBSPHERE FOR Z
Reported component ID
5655I3500
Reported release
850
Status
CLOSED PER
PE
NoPE
HIPER
NoHIPER
Special Attention
NoSpecatt
Submitted date
2013-06-21
Closed date
2013-09-06
Last modified date
2013-09-06
APAR is sysrouted FROM one or more of the following:
APAR is sysrouted TO one or more of the following:
Fix information
Fixed component name
WEBSPHERE FOR Z
Fixed component ID
5655I3500
Applicable component levels
R850 PSY
UP
Document Information
Modified date:
28 April 2022