APAR status
Closed as program error.
Error description
MXserver got hung due to the deadlock and "Internal Server Error" is displayed on the client browser. SystemOut_13.09.09_18.59.21.log shows many threads got hung as follows. ----------------------------------------------------------- [13/09/09 18:17:35:546 JST] 0000001f ThreadMonitor W WSVR0605W: Thread "WebContainer : 11" (000000ce) has been active for 675828 milliseconds and may be hung. There is/are 108 thread(s) in total in the server that may be hung. ----------------------------------------------------------- At the same time, javacore shows the deadlock as follows. ----------------------------------------------------------- 1LKDEADLOCK Deadlock detected !!! Page 28 of 37 NULL --------------------- NULL 2LKDEADLOCKTHR Thread "WebContainer : 6" (0x0000000003CA9100) 3LKDEADLOCKWTR is waiting for: 4LKDEADLOCKMON sys_mon_t:0x000000001964CEE0 infl_mon_t: 0x000000001964CF50: 4LKDEADLOCKOBJ psdi/webclient/system/session/WebClientSessionFactory@0x00000000 8B3AFD70 /0x000000008B3AFD7C: 3LKDEADLOCKOWN which is owned by: 2LKDEADLOCKTHR Thread "WebContainer : 5" (0x0000000003CB1300) 3LKDEADLOCKWTR which is waiting for: 4LKDEADLOCKMON sys_mon_t:0x000000001964CF80 infl_mon_t: 0x000000001964CFF0: 4LKDEADLOCKOBJ com/ibm/ws/session/store/memory/MemorySession@0x000000026DCA4840 /0x00000 Page 29 of 37 0026DCA484C: 3LKDEADLOCKOWN which is owned by: 2LKDEADLOCKTHR Thread "WebContainer : 6" (0x0000000003CA9100) ------------------------------- WebContainer : 6 stuck 3XMTHREADINFO "WebContainer : 6" J9VMThread:0x0000000003CA9 100,j9thread_t:0x00000000191E1A40, java/lang/Thread:0x00000 0008CC077C8,state:B, prio=5 3XMTHREADINFO1 (native thread ID:0xDBC, native priority:0x5, nativepolicy:UNKNOWN) 3XMTHREADINFO3 Java callstack: 4XESTACKTRACE at psdi/webclient/system/session/WebClientSessionManager.value Unbound(WebClientSessionManager.java:335) 4XESTACKTRACE at com/ibm/ws/session/http/HttpSessionObserver.sessionDestroyed (HttpSessionObserver.java:193) 4XESTACKTRACE at ... WebContainer : 5 3XMTHREADINFO "WebContainer : 5" J9VMThread:0x0000000003CB1 300,j9thread_t:0x00000000191E2A50, java/lang/Thread:0x00000 0008CC07850,state:B, prio=5 3XMTHREADINFO1 (native thread ID:0x18B8, native priority: 0x5, nativepolicy:UNKNOWN) 3XMTHREADINFO3 Java callstack: 4XESTACKTRACE at com/ibm/ws/session/http/HttpSessionImpl.getAttribute(Http SessionImpl.java:189(Compiled Code)) 4XESTACKTRACE at com/ibm/ws/session/SessionData.getSessionValue(SessionData. java:300(CompPage 32 of 37 ------------------------------------------------------------ The problem is reported in the following environment. ENVIRONMENT: Product information: ISCCD Product version: 7.5.0.1 Application within Product: TPAE 7.5.0.3 OS: Windows 2008 R2 DB: DB2 v9.7 Application Server: WebSphere Application Server ND 7.0.0.17
Local fix
Problem summary
**************************************************************** * USERS AFFECTED: * * All Maximo Users * **************************************************************** * PROBLEM DESCRIPTION: * * MXserver got hung due to the deadlock and "Internal Server * * Error" is displayed on the client browser. * **************************************************************** * RECOMMENDATION: * * Upgrade to latest release * **************************************************************** SystemOut_13.09.09_18.59.21.log shows many threads got hung as follows. ----------------------------------------------------------- [13/09/09 18:17:35:546 JST] 0000001f ThreadMonitor WWSVR0605W: Thread "WebContainer : 11" (000000ce) has been active for 675828 milliseconds and may be hung. There is/are 108 thread(s) in total in the server that may be hung. ----------------------------------------------------------- At the same time, javacore shows the deadlock as follows. ----------------------------------------------------------- 1LKDEADLOCK Deadlock detected !!! Page 28 of 37 NULL --------------------- NULL 2LKDEADLOCKTHR Thread "WebContainer : 6" (0x0000000003CA9100) 3LKDEADLOCKWTR is waiting for: 4LKDEADLOCKMON sys_mon_t:0x000000001964CEE0 infl_mon_t:0x000000001964CF50: 4LKDEADLOCKOBJ psdi/webclient/system/session/WebClientSessionFactory@0x00000000 8B3AFD70/0x000000008B3AFD7C: 3LKDEADLOCKOWN which is owned by: 2LKDEADLOCKTHR Thread "WebContainer : 5" (0x0000000003CB1300) 3LKDEADLOCKWTR which is waiting for: 4LKDEADLOCKMON sys_mon_t:0x000000001964CF80 infl_mon_t:0x000000001964CFF0: 4LKDEADLOCKOBJ com/ibm/ws/session/store/memory/MemorySession@0x000000026DCA4840 /0x00000 Page 29 of 37 0026DCA484C: 3LKDEADLOCKOWN which is owned by: 2LKDEADLOCKTHR Thread "WebContainer : 6" (0x0000000003CA9100) ------------------------------- WebContainer : 6 stuck 3XMTHREADINFO "WebContainer : 6" J9VMThread:0x0000000003CA9100,j9thread_t:0x00000000191E1A40, java/lang/Thread:0x000000008CC077C8,state:B, prio=5 3XMTHREADINFO1 (native thread ID:0xDBC, native priority:0x5,nativepolicy:UNKNOWN) 3XMTHREADINFO3 Java callstack: 4XESTACKTRACE at psdi/webclient/system/session/WebClientSessionManager.valueUnbou nd(WebClientSessionManager.java:335) 4XESTACKTRACE at com/ibm/ws/session/http/HttpSessionObserver.sessionDestroyed(Htt pSessionObserver.java:193) 4XESTACKTRACE at ... WebContainer : 5 3XMTHREADINFO "WebContainer : 5" J9VMThread:0x0000000003CB1300,j9thread_t:0x00000000191E2A50, java/lang/Thread:0x000000008CC07850,state:B, prio=5 3XMTHREADINFO1 (native thread ID:0x18B8, native priority:0x5, nativepolicy:UNKNOWN) 3XMTHREADINFO3 Java callstack: 4XESTACKTRACE at com/ibm/ws/session/http/HttpSessionImpl.getAttribute(HttpSession Impl.java:189(Compiled Code)) 4XESTACKTRACE at com/ibm/ws/session/SessionData.getSessionValue(SessionData.java: 300(Comp Page 32 of 37 ------------------------------------------------------------ The problem is reported in the following environment. ENVIRONMENT: Product information: ISCCD Product version: 7.5.0.1 Application within Product: TPAE 7.5.0.3 OS: Windows 2008 R2 DB: DB2 v9.7 Application Server: WebSphere Application Server ND 7.0.0.17
Problem conclusion
Avoid deadlocks due to multiple simultaneous login from a single client. The fix for this APAR is contained in the following maintenance package: | release\fix pack | Interim Fix for Release 7.5.0.4 TPAE
Temporary fix
Comments
APAR Information
APAR number
IV50706
Reported component name
ADMINISTRATION
Reported component ID
5724R46A1
Reported release
750
Status
CLOSED PER
PE
NoPE
HIPER
NoHIPER
Special Attention
NoSpecatt / Xsystem
Submitted date
2013-10-10
Closed date
2013-10-13
Last modified date
2013-10-13
APAR is sysrouted FROM one or more of the following:
APAR is sysrouted TO one or more of the following:
Modules/Macros
MAXIMO
Fix information
Fixed component name
ADMINISTRATION
Fixed component ID
5724R46A1
Applicable component levels
R750 PSY
UP
[{"Business Unit":{"code":"BU048","label":"IBM Software"},"Product":{"code":"SSCHPP5","label":"System Related"},"Component":"","ARM Category":[],"Platform":[{"code":"PF025","label":"Platform Independent"}],"Version":"750","Edition":"","Line of Business":{"code":"","label":""}}]
Document Information
Modified date:
13 October 2013