IBM Support

PI92182: SERVER STOP PROCESSING GETS HUNG UP IN SIB COMPONENT

Subscribe

You can track all active APARs for this component.

 

APAR status

  • Closed as program error.

Error description

  • Rarely, when the customer tries to stop the application server,
    the stop processing gets hang.
    When we took javacore of the hang process, we observed that the
    SIBus components are hanging.
    The following thread was trying to stop the application server
    and it was waiting for the completion of work on
    "pool-8-thread-1".
    "Non-deferrable Alarm : 0"  state:P, prio=5
    -----------------------------
    Java callstack:
    at sun/misc/Unsafe.park(Native Method)
    at
    java/util/concurrent/locks/LockSupport.parkNanos(LockSupport.jav
    a:222)
    at
    java/util/concurrent/locks/AbstractQueuedSynchronizer$ConditionO
    bject.awaitNanos(AbstractQueuedSynchronizer.java:2036)
    at
    java/util/concurrent/ThreadPoolExecutor.awaitTermination(ThreadP
    oolExecutor.java:1263)
    at com/ibm/ws/sib/trm/wlm/server/Manager.stop(Manager.java:251)
    at com/ibm/ws/sib/trm/TrmMeMainImpl.stop(TrmMeMainImpl.java:301)
    at
    com/ibm/ws/sib/admin/impl/JsMessagingEngineImpl.stop(JsMessaging
    EngineImpl.java:996)
    at
    com/ibm/ws/sib/admin/impl/HAManagerMessagingEngineImpl.leave(HAM
    anagerMessagingEngineImpl.java:1700)
    at
    com/ibm/ws/sib/admin/impl/HAManagerMessagingEngineImpl.stopCondi
    tional(HAManagerMessagingEngineImpl.java:1206)
    at
    com/ibm/ws/sib/admin/impl/JsMainImpl.stop(JsMainImpl.java:975)
    :
    at
    com/ibm/ws/runtime/component/ApplicationServerImpl.stop(Applicat
    ionServerImpl.java:257)
    -----------------------------
    
    "pool-8-thread-1" ... state:B
    -----------------------------
    Java callstack:
    at
    com/ibm/ws/sib/trm/contact/Neighbourhood.makeDirectConnection(Ne
    ighbourhood.java:492)
    at
    com/ibm/ws/sib/trm/topology/RoutingManagerImpl.connectToME(Routi
    ngManagerImpl.java:120)
    at com/ibm/ws/sib/processor/io/MPIO.forceConnect(MPIO.java:688)
    at
    com/ibm/ws/sib/processor/impl/AnycastInputHandler$ReachabilityCh
    angeListener.destinationLocationChange(AnycastInputHandler.java:
    5145)
    at
    com/ibm/ws/sib/processor/impl/DestinationChangeListener.destinat
    ionLocationChange(DestinationChangeListener.java:179)
    at
    com/ibm/ws/sib/trm/wlm/server/Manager$NotificationHandlerTask.ru
    n(Manager.java:1547)
    at
    java/util/concurrent/ThreadPoolExecutor$Worker.runTask(ThreadPoo
    lExecutor.java:895)
    at
    java/util/concurrent/ThreadPoolExecutor$Worker.run(ThreadPoolExe
    cutor.java:918)
    at java/lang/Thread.run(Thread.java:761)
    -----------------------------
    "pool-8-thread-1" was waiting for the completion of work on
    "TRM Service Thread"
    and it never ends with the following stack.
    Thus, the stop server processing is never completed.
    "TRM Service Thread" ... state:CW
    -----------------------------
    Java callstack:
    at java/lang/Object.wait(Native Method)
    at java/lang/Object.wait(Object.java:167)
    at com/ibm/ws/sib/utils/Semaphore.waitOn(Semaphore.java:79)
    at
    com/ibm/ws/sib/utils/Semaphore.waitOnIgnoringInterruptions(Semap
    hore.java:122)
    at
    com/ibm/ws/sib/jfapchannel/impl/ConversationImpl.close(Conversat
    ionImpl.java:474)
    at
    com/ibm/ws/sib/comms/server/mesupport/MEConnectionImpl.sendClose
    RequestCompletes(MEConnectionImpl.java:687)
    at
    com/ibm/ws/sib/comms/server/mesupport/MEConnectionImpl.access$20
    0(MEConnectionImpl.java:150)
    at
    com/ibm/ws/sib/comms/server/mesupport/MEConnectionImpl$CloseSend
    Listener.dataSent(MEConnectionImpl.java:932)
    at
    com/ibm/ws/sib/jfapchannel/impl/ConnectionWriteCompletedCallback
    .complete(ConnectionWriteCompletedCallback.java:278)
    at
    com/ibm/ws/sib/jfapchannel/impl/ConnectionWriteCompletedCallback
    .proddle(ConnectionWriteCompletedCallback.java:222)
    at
    com/ibm/ws/sib/jfapchannel/impl/Connection.send(Connection.java:
    695)
    at
    com/ibm/ws/sib/jfapchannel/impl/ConversationImpl.sendInternal(Co
    nversationImpl.java:1157)
    at
    com/ibm/ws/sib/jfapchannel/impl/ConversationImpl.send(Conversati
    onImpl.java:1024)
    at
    com/ibm/ws/sib/comms/server/mesupport/MEConnectionImpl.close(MEC
    onnectionImpl.java:599)
    at com/ibm/ws/sib/trm/contact/Connect.contact(Connect.java:269)
    at
    com/ibm/ws/sib/trm/contact/Neighbourhood._scan(Neighbourhood.jav
    a:405)
    :
    at
    com/ibm/ws/sib/trm/contact/Neighbourhood.direct(Neighbourhood.ja
    va:956)
    at
    com/ibm/ws/sib/trm/general/ServiceThread.run(ServiceThread.java:
    85)
    -----------------------------
    In the other case, "Non-deferrable Alarm : 0" was blocked by
    "pool-8-thread-1" directly. At that time, the stack of
    "pool-8-thread-1" was the following.
    "pool-8-thread-1" ... state:CW
    -----------------------------
    Java callstack:
    at java/lang/Object.wait(Native Method)
    at java/lang/Object.wait(Object.java:167)
    at com/ibm/ws/sib/utils/Semaphore.waitOn(Semaphore.java:79)
    at
    com/ibm/ws/sib/utils/Semaphore.waitOnIgnoringInterruptions(Semap
    hore.java:122)
    at
    com/ibm/ws/sib/jfapchannel/impl/ConversationImpl.close(Conversat
    ionImpl.java:474)
    -----------------------------
    

Local fix

Problem summary

  • ****************************************************************
    * USERS AFFECTED:  Users of IBM WebSphere Application Server   *
    *                  messaging, particularly those using HA      *
    *                  failover.                                   *
    ****************************************************************
    * PROBLEM DESCRIPTION: After restart of communications between *
    *                      Messaging Engines, establishment of the *
    *                      new connection becomes hung.            *
    ****************************************************************
    * RECOMMENDATION:  Apply the Apar fix.                         *
    ****************************************************************
    Connection closing callbacks can cause additional
    communication across the connection. Multiple threads
    attempting to close the same connection could lead to a thread
    tasked with writing messages to the connection to incorrectly
    wait, thinking that another thread had been given this task,
    resulting in a hang.
    Resolution was to defer connection closing callback
    notification to beyond the point where the thread tasked with
    writing to the connection has relinquished this responsiblity.
    

Problem conclusion

Temporary fix

Comments

APAR Information

  • APAR number

    PI92182

  • Reported component name

    WEBS APP SERV N

  • Reported component ID

    5724H8800

  • Reported release

    700

  • Status

    CLOSED PER

  • PE

    NoPE

  • HIPER

    NoHIPER

  • Special Attention

    NoSpecatt / Xsystem

  • Submitted date

    2018-01-08

  • Closed date

    2018-04-25

  • Last modified date

    2018-04-25

  • APAR is sysrouted FROM one or more of the following:

  • APAR is sysrouted TO one or more of the following:

Fix information

  • Fixed component name

    WEBS APP SERV N

  • Fixed component ID

    5724H8800

Applicable component levels

  • R700 PSY

       UP

  • R800 PSY

       UP

  • R850 PSY

       UP

  • R900 PSY

       UP

[{"Business Unit":{"code":"BU053","label":"Cloud \u0026 Data Platform"},"Product":{"code":"SSEQTP","label":"WebSphere Application Server"},"Platform":[{"code":"PF025","label":"Platform Independent"}],"Version":"7.0","Line of Business":{"code":"LOB36","label":"IBM Automation"}}]

Document Information

Modified date:
19 October 2021