IBM Support

PI79261: Deadlock with persistent EJB timers for Singleton beans

Fixes are available

17.0.0.2: WebSphere Application Server Liberty 17.0.0.2
17.0.0.3: WebSphere Application Server Liberty 17.0.0.3
17.0.0.4: WebSphere Application Server Liberty 17.0.0.4
18.0.0.1: WebSphere Application Server Liberty 18.0.0.1
18.0.0.2: WebSphere Application Server Liberty 18.0.0.2
18.0.0.3: WebSphere Application Server Liberty 18.0.0.3
18.0.0.4: WebSphere Application Server Liberty 18.0.0.4
19.0.0.1: WebSphere Application Server Liberty 19.0.0.1
19.0.0.2: WebSphere Application Server Liberty 19.0.0.2
19.0.0.3: WebSphere Application Server Liberty 19.0.0.3
19.0.0.4: WebSphere Application Server Liberty 19.0.0.4
19.0.0.5: WebSphere Application Server Liberty 19.0.0.5
19.0.0.6: WebSphere Application Server Liberty 19.0.0.6
19.0.0.7: WebSphere Application Server Liberty 19.0.0.7
19.0.0.8: WebSphere Application Server Liberty 19.0.0.8
19.0.0.9: WebSphere Application Server Liberty 19.0.0.9
19.0.0.10: WebSphere Application Server Liberty 19.0.0.10
19.0.0.11: WebSphere Application Server Liberty 19.0.0.11
19.0.0.12: WebSphere Application Server Liberty 19.0.0.12
20.0.0.1: WebSphere Application Server Liberty 20.0.0.1
20.0.0.2: WebSphere Application Server Liberty 20.0.0.2
20.0.0.3: WebSphere Application Server Liberty 20.0.0.3
20.0.0.4: WebSphere Application Server Liberty 20.0.0.4
20.0.0.5: WebSphere Application Server Liberty 20.0.0.5
20.0.0.6: WebSphere Application Server Liberty 20.0.0.6
20.0.0.7: WebSphere Application Server Liberty 20.0.0.7
20.0.0.8: WebSphere Application Server Liberty 20.0.0.8
20.0.0.9: WebSphere Application Server Liberty 20.0.0.9
20.0.0.10: WebSphere Application Server Liberty 20.0.0.10
20.0.0.11: WebSphere Application Server Liberty 20.0.0.11
20.0.0.12: WebSphere Application Server Liberty 20.0.0.12

Subscribe

You can track all active APARs for this component.

 

APAR status

  • Closed as program error.

Error description

  • If a persistent EJB timer for a Singleton EJB expires at the
    same time that a local or remote method is being called on
    that Singleton and the method attempts to access EJB timers
    by calling a method on the EJB TimerService, then it is
    possible the two threads deadlock.  This occurs because the
    thread that is running the persistent EJB timer will obtain
    a lock on the timer database and then a lock on the
    Singleton bean, whereas the thread running the local or
    remote method will first obtain the lock on the Singleton
    bean and then the lock on the timer database.
    
    When this occurs, the stack of the EJB timer thread will
    contain the following:
    
    com.ibm.ejs.container.SingletonBeanO.preInvoke(SingletonBean
    O.java:526)
    com.ibm.ejs.container.EJSContainer.preInvokeActivate(EJSCont
    ainer.java:3264)
    com.ibm.ejs.container.EJSContainer.EjbPreInvoke(EJSContainer
    .java:2576)
    com.ibm.ejs.container.TimedObjectWrapper.invokeCallback(Time
    dObjectWrapper.java:84)
    com.ibm.ejs.container.PersistentTimerTaskHandler.run(Persist
    entTimerTaskHandler.java:290)
    com.ibm.ws.concurrent.persistent.internal.InvokerTask.run(In
    vokerTask.java:426)
    
    And the stack of the local or remote method call will look
    something like :
    
    com.ibm.ws.concurrent.persistent.internal.PersistentExecutor
    Impl.remove(PersistentExecutorImpl.java:1243)
    com.ibm.ws.ejbcontainer.timer.persistent.osgi.internal.EJBPe
    rsistentTimerRuntimeImpl.remove(EJBPersistentTimerRuntimeImp
    l.java:352)
    com.ibm.ws.ejbcontainer.timer.persistent.osgi.internal.Persi
    stentTimerImpl.cancel(PersistentTimerImpl.java:114)
    ejb.AutoTimer.cancel(AutoTimer.java:29)
    ejb.EJSLocalNSGAutoTimer_0535901b.cancel(EJSLocalNSGAutoTime
    r_0535901b.java)
    

Local fix

Problem summary

  • ****************************************************************
    * USERS AFFECTED:  All users of IBM WebSphere Application      *
    *                  Server Liberty - Enterprise Java Beans      *
    *                  (EJB)                                       *
    ****************************************************************
    * PROBLEM DESCRIPTION: Deadlock with persistent EJB timers for *
    *                      Singleton beans                         *
    ****************************************************************
    * RECOMMENDATION:                                              *
    ****************************************************************
    If a persistent EJB timer for a Singleton EJB expires at the
    same time that a local or remote method is being called on that
    Singleton and the method attempts to access EJB timers by
    calling a method on the EJB TimerService, then it is possible
    the two threads deadlock.  This occurs because the thread that
    is running the persistent EJB timer will obtain a lock on the
    timer database and then a lock on the Singleton bean, whereas
    the thread running the local or remote method will first obtain
    the lock on the Singleton bean and then the lock on the timer
    database.
    
    When this occurs, the stack of the EJB timer thread will contain
    the following:
    
    com.ibm.ejs.container.SingletonBeanO.preInvoke(SingletonBeanO.ja
    va:526)
    com.ibm.ejs.container.EJSContainer.preInvokeActivate(EJSContaine
    r.java:3264)
    com.ibm.ejs.container.EJSContainer.EjbPreInvoke(EJSContainer.jav
    a:2576)
    com.ibm.ejs.container.TimedObjectWrapper.invokeCallback(TimedObj
    ectWrapper.java:84)
    com.ibm.ejs.container.PersistentTimerTaskHandler.run(PersistentT
    imerTaskHandler.java:290)
    com.ibm.ws.concurrent.persistent.internal.InvokerTask.run(Invoke
    rTask.java:426)
    
    And the stack of the local or remote method call will look
    something like :
    
    com.ibm.ws.concurrent.persistent.internal.PersistentExecutorImpl
    .remove(PersistentExecutorImpl.java:1243)
    com.ibm.ws.ejbcontainer.timer.persistent.osgi.internal.EJBPersis
    tentTimerRuntimeImpl.remove(EJBPersistentTimerRuntimeImpl.java:3
    52)
    com.ibm.ws.ejbcontainer.timer.persistent.osgi.internal.Persisten
    tTimerImpl.cancel(PersistentTimerImpl.java:114)
    ejb.AutoTimer.cancel(AutoTimer.java:29)
    ejb.EJSLocalNSGAutoTimer_0535901b.cancel(EJSLocalNSGAutoTimer_05
    35901b.java)
    

Problem conclusion

  • The deadlock scenario has been resolved by changing the order
    that locks are obtained for the timer expiration.  When an EJB
    timer for a singleton bean expires, the lock on the Singleton
    bean will now be acquired before the lock on the timer database.
    Since the order the locks are obtained is now the same as when a
    local or remote method is called, the deadlock can no longer
    occur.
    
    The fix for this APAR is currently targeted for inclusion in fix
    pack 17.0.0.2.  Please refer to the Recommended Updates page for
    delivery information:
    http://www.ibm.com/support/docview.wss?rs=180&uid=swg27004980
    

Temporary fix

Comments

APAR Information

  • APAR number

    PI79261

  • Reported component name

    WAS LIBERTY COR

  • Reported component ID

    5725L2900

  • Reported release

    CD0

  • Status

    CLOSED PER

  • PE

    NoPE

  • HIPER

    NoHIPER

  • Special Attention

    NoSpecatt / Xsystem

  • Submitted date

    2017-04-03

  • Closed date

    2017-04-12

  • Last modified date

    2017-04-12

  • APAR is sysrouted FROM one or more of the following:

  • APAR is sysrouted TO one or more of the following:

Fix information

  • Fixed component name

    WAS LIBERTY COR

  • Fixed component ID

    5725L2900

Applicable component levels

  • RCD0 PSY

       UP

[{"Business Unit":{"code":"BU053","label":"Cloud \u0026 Data Platform"},"Product":{"code":"SSD28V","label":"WebSphere Application Server Liberty Core"},"Platform":[{"code":"PF025","label":"Platform Independent"}],"Version":"CD0","Line of Business":{"code":"LOB36","label":"IBM Automation"}}]

Document Information

Modified date:
18 October 2021