IBM Support

PI97241: SERVER HANG ON SERVER SHUTDOWN DUE TO DEADLOCK

Subscribe

You can track all active APARs for this component.

 

APAR status

  • Closed as program error.

Error description

  • Running WebSphere Application Server Network Deployment version
    8.0.0.14, experiencing intermittent, unpredictable,
    non-reproducible server hangs on server shutdown.
    
    Javacore file generated with linperf.sh script reveals deadlock:
    
    1LKDEADLOCK Deadlock detected !!! NULL ---------------------
    NULL 2LKDEADLOCKTHR Thread "WorkManager.IB_RETAIL_WM : 17"
    (0x000000008D29DE00) 3LKDEADLOCKWTR is waiting for:
    4LKDEADLOCKMON sys_mon_t:0x00007F65F15B7F88 infl_mon_t:
    0x00007F65F15B8000: 4LKDEADLOCKOBJ
    java/lang/Object@0x0000000001C10410 3LKDEADLOCKOWN which is
    owned by: 2LKDEADLOCKTHR Thread "WorkManager.IB_RETAIL_WM : 3"
    (0x000000008CB7D600) 3LKDEADLOCKWTR which is waiting for:
    4LKDEADLOCKMON sys_mon_t:0x00007F65ED9B94F8 infl_mon_t:
    0x00007F65ED9B9570: 4LKDEADLOCKOBJ
    com/ibm/ejs/j2c/J2CGlobalConfigProperties@0x000000001A5EC5E0
    3LKDEADLOCKOWN which is owned by: 2LKDEADLOCKTHR Thread
    "WorkManager.IB_RETAIL_WM : 17" (0x000000008D29DE00)
    
    while executing these threads:
    3XMTHREADBLOCK Blocked on:
    com/ibm/ejs/j2c/J2CGlobalConfigProperties@0x000000001A5EC5E0
    Owned by: "WorkManager.IB_RETAIL_WM : 17"
    (J9VMThread:0x000000008D29DE00,
    java/lang/Thread:0x000000001B4A8C88) 3XMHEAPALLOC Heap bytes
    allocated since last GC cycle=0 (0x0) 3XMTHREADINFO3 Java
    callstack: 4XESTACKTRACE at
    com/ibm/ejs/j2c/J2CDiagnosticAlertHelperImpl.propertyChange
    (J2CDiagnosticAlertHelperImpl.java:969) 4XESTACKTRACE at
    com/ibm/ejs/j2c/J2CGlobalConfigProperties.propertyChange
    (J2CGlobalConfigProperties.java:2162) 4XESTACKTRACE at
    com/ibm/ws/rasdiag/AlertConfig.
    notifyListeners(AlertConfig.java:388)
    
    and
    
    3XMTHREADINFO "WorkManager.IB_RETAIL_WM : 17" J9VMThread:
    0x000000008D29DE00, j9thread_t:0x00007F65F4064D10,
    java/lang/Thread: 0x000000001B4A8C88, state:B, prio=5
    3XMJAVALTHREAD (java/lang/Thread getId:0x13D, isDaemon:true)
    3XMTHREADINFO1 (native thread ID:0xB4B3, native priority: 0x5,
    native policy:UNKNOWN, vmstate:B, vm thread flags:0x00000201)
    3XMTHREADINFO2 (native stack address range from:
    0x00007F65D6A73000, to:0x00007F65D6AB4000, size:0x41000)
    3XMCPUTIME CPU usage total: 4.011094347 secs 3XMTHREADBLOCK
    Blocked on: java/lang/Object@0x0000000001C10410 Owned by:
    "WorkManager.IB_RETAIL_WM : 3" (J9VMThread: 0x000000008CB7D600,
    java/lang/Thread:0x000000001B436F20) 3XMHEAPALLOC Heap bytes
    allocated since last GC cycle=0 (0x0) 3XMTHREADINFO3 Java
    callstack: 4XESTACKTRACE at
    com/ibm/ws/performance/tuning/serverAlert/ServerRuleDriver.compl
    eteInit (ServerRuleDriver.java:1147) 4XESTACKTRACE at
    com/ibm/ws/performance/tuning/serverAlert/ServerRuleDriverMBean.
    getRPAAttributeValue(ServerRuleDriverMBean.java:1116)
    4XESTACKTRACE at
    com/ibm/ws/performance/tuning/serverAlert/ServerRuleDriverMBean.
    getRPAAttributeValues(ServerRuleDriverMBean.java:1086)
    

Local fix

  • n/a
    

Problem summary

  • ****************************************************************
    * USERS AFFECTED:  All users of IBM WebSphere Application      *
    *                  Server using performance monitoring and     *
    *                  datasources                                 *
    ****************************************************************
    * PROBLEM DESCRIPTION: A intermittent deadlock may occur       *
    *                      between the ServerRullDriverMBean and   *
    *                      the J2CDiagnosticAlertHelperImpl code.  *
    ****************************************************************
    * RECOMMENDATION:                                              *
    ****************************************************************
    A java core will have the following threads deadlocked, while
    executing these threads:
    3XMTHREADBLOCK Blocked on:
    com/ibm/ejs/j2c/J2CGlobalConfigProperties@0x000000001A5EC5E0
    Owned by: "WorkManager.IB_RETAIL_WM : 17"
    (J9VMThread:0x000000008D29DE00,
    java/lang/Thread:0x000000001B4A8C88) 3XMHEAPALLOC Heap bytes
    allocated since last GC cycle=0 (0x0) 3XMTHREADINFO3 Java
    callstack: 4XESTACKTRACE at
    com/ibm/ejs/j2c/J2CDiagnosticAlertHelperImpl.propertyChange
    (J2CDiagnosticAlertHelperImpl.java:969) 4XESTACKTRACE at
    com/ibm/ejs/j2c/J2CGlobalConfigProperties.propertyChange
    (J2CGlobalConfigProperties.java:2162) 4XESTACKTRACE at
    com/ibm/ws/rasdiag/AlertConfig.
    notifyListeners(AlertConfig.java:388)
    and
    3XMTHREADINFO "WorkManager.IB_RETAIL_WM : 17" J9VMThread:
    0x000000008D29DE00, j9thread_t:0x00007F65F4064D10,
    java/lang/Thread: 0x000000001B4A8C88, state:B, prio=5
    3XMJAVALTHREAD (java/lang/Thread getId:0x13D, isDaemon:true)
    3XMTHREADINFO1 (native thread ID:0xB4B3, native priority: 0x5,
    native policy:UNKNOWN, vmstate:B, vm thread flags:0x00000201)
    3XMTHREADINFO2 (native stack address range from:
    0x00007F65D6A73000, to:0x00007F65D6AB4000, size:0x41000)
    3XMCPUTIME CPU usage total: 4.011094347 secs 3XMTHREADBLOCK
    Blocked on: java/lang/Object@0x0000000001C10410 Owned by:
    "WorkManager.IB_RETAIL_WM : 3" (J9VMThread: 0x000000008CB7D600,
    java/lang/Thread:0x000000001B436F20) 3XMHEAPALLOC Heap bytes
    allocated since last GC cycle=0 (0x0) 3XMTHREADINFO3 Java
    callstack: 4XESTACKTRACE at
    com/ibm/ws/performance/tuning/serverAlert/ServerRuleDriver.compl
    eteInit (ServerRuleDriver.java:1147) 4XESTACKTRACE at
    com/ibm/ws/performance/tuning/serverAlert/ServerRuleDriverMBean.
    getRPAAttributeValue(ServerRuleDriverMBean.java:1116)
    

Problem conclusion

Temporary fix

Comments

APAR Information

  • APAR number

    PI97241

  • Reported component name

    WEBS APP SERV N

  • Reported component ID

    5724H8800

  • Reported release

    800

  • Status

    CLOSED PER

  • PE

    NoPE

  • HIPER

    NoHIPER

  • Special Attention

    NoSpecatt / Xsystem

  • Submitted date

    2018-04-25

  • Closed date

    2018-12-06

  • Last modified date

    2018-12-06

  • APAR is sysrouted FROM one or more of the following:

  • APAR is sysrouted TO one or more of the following:

Fix information

  • Fixed component name

    WEBS APP SERV N

  • Fixed component ID

    5724H8800

Applicable component levels

  • R800 PSY

       UP

  • R850 PSY

       UP

  • R900 PSY

       UP

[{"Business Unit":{"code":"BU053","label":"Cloud \u0026 Data Platform"},"Product":{"code":"SSEQTP","label":"WebSphere Application Server"},"Platform":[{"code":"PF025","label":"Platform Independent"}],"Version":"8.0","Line of Business":{"code":"LOB36","label":"IBM Automation"}}]

Document Information

Modified date:
18 October 2021