PI92182: SERVER STOP PROCESSING GETS HUNG UP IN SIB COMPONENT

Fixes are available

APAR status

Closed as program error.

Error description

Rarely, when the customer tries to stop the application server,
the stop processing gets hang.
When we took javacore of the hang process, we observed that the
SIBus components are hanging.
The following thread was trying to stop the application server
and it was waiting for the completion of work on
"pool-8-thread-1".
"Non-deferrable Alarm : 0"  state:P, prio=5
-----------------------------
Java callstack:
at sun/misc/Unsafe.park(Native Method)
at
java/util/concurrent/locks/LockSupport.parkNanos(LockSupport.jav
a:222)
at
java/util/concurrent/locks/AbstractQueuedSynchronizer$ConditionO
bject.awaitNanos(AbstractQueuedSynchronizer.java:2036)
at
java/util/concurrent/ThreadPoolExecutor.awaitTermination(ThreadP
oolExecutor.java:1263)
at com/ibm/ws/sib/trm/wlm/server/Manager.stop(Manager.java:251)
at com/ibm/ws/sib/trm/TrmMeMainImpl.stop(TrmMeMainImpl.java:301)
at
com/ibm/ws/sib/admin/impl/JsMessagingEngineImpl.stop(JsMessaging
EngineImpl.java:996)
at
com/ibm/ws/sib/admin/impl/HAManagerMessagingEngineImpl.leave(HAM
anagerMessagingEngineImpl.java:1700)
at
com/ibm/ws/sib/admin/impl/HAManagerMessagingEngineImpl.stopCondi
tional(HAManagerMessagingEngineImpl.java:1206)
at
com/ibm/ws/sib/admin/impl/JsMainImpl.stop(JsMainImpl.java:975)
:
at
com/ibm/ws/runtime/component/ApplicationServerImpl.stop(Applicat
ionServerImpl.java:257)
-----------------------------

"pool-8-thread-1" ... state:B
-----------------------------
Java callstack:
at
com/ibm/ws/sib/trm/contact/Neighbourhood.makeDirectConnection(Ne
ighbourhood.java:492)
at
com/ibm/ws/sib/trm/topology/RoutingManagerImpl.connectToME(Routi
ngManagerImpl.java:120)
at com/ibm/ws/sib/processor/io/MPIO.forceConnect(MPIO.java:688)
at
com/ibm/ws/sib/processor/impl/AnycastInputHandler$ReachabilityCh
angeListener.destinationLocationChange(AnycastInputHandler.java:
5145)
at
com/ibm/ws/sib/processor/impl/DestinationChangeListener.destinat
ionLocationChange(DestinationChangeListener.java:179)
at
com/ibm/ws/sib/trm/wlm/server/Manager$NotificationHandlerTask.ru
n(Manager.java:1547)
at
java/util/concurrent/ThreadPoolExecutor$Worker.runTask(ThreadPoo
lExecutor.java:895)
at
java/util/concurrent/ThreadPoolExecutor$Worker.run(ThreadPoolExe
cutor.java:918)
at java/lang/Thread.run(Thread.java:761)
-----------------------------
"pool-8-thread-1" was waiting for the completion of work on
"TRM Service Thread"
and it never ends with the following stack.
Thus, the stop server processing is never completed.
"TRM Service Thread" ... state:CW
-----------------------------
Java callstack:
at java/lang/Object.wait(Native Method)
at java/lang/Object.wait(Object.java:167)
at com/ibm/ws/sib/utils/Semaphore.waitOn(Semaphore.java:79)
at
com/ibm/ws/sib/utils/Semaphore.waitOnIgnoringInterruptions(Semap
hore.java:122)
at
com/ibm/ws/sib/jfapchannel/impl/ConversationImpl.close(Conversat
ionImpl.java:474)
at
com/ibm/ws/sib/comms/server/mesupport/MEConnectionImpl.sendClose
RequestCompletes(MEConnectionImpl.java:687)
at
com/ibm/ws/sib/comms/server/mesupport/MEConnectionImpl.access$20
0(MEConnectionImpl.java:150)
at
com/ibm/ws/sib/comms/server/mesupport/MEConnectionImpl$CloseSend
Listener.dataSent(MEConnectionImpl.java:932)
at
com/ibm/ws/sib/jfapchannel/impl/ConnectionWriteCompletedCallback
.complete(ConnectionWriteCompletedCallback.java:278)
at
com/ibm/ws/sib/jfapchannel/impl/ConnectionWriteCompletedCallback
.proddle(ConnectionWriteCompletedCallback.java:222)
at
com/ibm/ws/sib/jfapchannel/impl/Connection.send(Connection.java:
695)
at
com/ibm/ws/sib/jfapchannel/impl/ConversationImpl.sendInternal(Co
nversationImpl.java:1157)
at
com/ibm/ws/sib/jfapchannel/impl/ConversationImpl.send(Conversati
onImpl.java:1024)
at
com/ibm/ws/sib/comms/server/mesupport/MEConnectionImpl.close(MEC
onnectionImpl.java:599)
at com/ibm/ws/sib/trm/contact/Connect.contact(Connect.java:269)
at
com/ibm/ws/sib/trm/contact/Neighbourhood._scan(Neighbourhood.jav
a:405)
:
at
com/ibm/ws/sib/trm/contact/Neighbourhood.direct(Neighbourhood.ja
va:956)
at
com/ibm/ws/sib/trm/general/ServiceThread.run(ServiceThread.java:
85)
-----------------------------
In the other case, "Non-deferrable Alarm : 0" was blocked by
"pool-8-thread-1" directly. At that time, the stack of
"pool-8-thread-1" was the following.
"pool-8-thread-1" ... state:CW
-----------------------------
Java callstack:
at java/lang/Object.wait(Native Method)
at java/lang/Object.wait(Object.java:167)
at com/ibm/ws/sib/utils/Semaphore.waitOn(Semaphore.java:79)
at
com/ibm/ws/sib/utils/Semaphore.waitOnIgnoringInterruptions(Semap
hore.java:122)
at
com/ibm/ws/sib/jfapchannel/impl/ConversationImpl.close(Conversat
ionImpl.java:474)
-----------------------------

Local fix

Problem summary

****************************************************************
* USERS AFFECTED:  Users of IBM WebSphere Application Server   *
*                  messaging, particularly those using HA      *
*                  failover.                                   *
****************************************************************
* PROBLEM DESCRIPTION: After restart of communications between *
*                      Messaging Engines, establishment of the *
*                      new connection becomes hung.            *
****************************************************************
* RECOMMENDATION:  Apply the Apar fix.                         *
****************************************************************
Connection closing callbacks can cause additional
communication across the connection. Multiple threads
attempting to close the same connection could lead to a thread
tasked with writing messages to the connection to incorrectly
wait, thinking that another thread had been given this task,
resulting in a hang.
Resolution was to defer connection closing callback
notification to beyond the point where the thread tasked with
writing to the connection has relinquished this responsiblity.

Problem conclusion

The fix for this APAR is currently targeted for inclusion in
fix pack 8.5.5.14 and 9.0.0.8.  Please refer to the Recommended
Updates page for delivery information:
http://www.ibm.com/support/docview.wss?rs=180&uid=swg27004980

Temporary fix

Comments

APAR Information

APAR number
PI92182
Reported component name
WEBS APP SERV N
Reported component ID
5724H8800
Reported release
700
Status
CLOSED PER
PE
NoPE
HIPER
NoHIPER
Special Attention
NoSpecatt / Xsystem
Submitted date
2018-01-08
Closed date
2018-04-25
Last modified date
2018-04-25

APAR is sysrouted FROM one or more of the following:
APAR is sysrouted TO one or more of the following:

Fix information

Fixed component name
WEBS APP SERV N
Fixed component ID
5724H8800

Applicable component levels

R700 PSY
UP

[{"Business Unit":{"code":"BU059","label":"IBM Software w\/o TPS"},"Product":{"code":"SSEQTP","label":"WebSphere Application Server"},"Platform":[{"code":"PF025","label":"Platform Independent"}],"Version":"7.0","Line of Business":{"code":"LOB45","label":"Automation"}}]

Document Information

Modified date:
04 May 2022

Tips

PI92182: SERVER STOP PROCESSING GETS HUNG UP IN SIB COMPONENT

Fixes are available

Subscribe

APAR status

Closed as program error.

Error description

Local fix

Problem summary

Problem conclusion

Temporary fix

Comments

APAR Information

APAR number

Reported component name

Reported component ID

Reported release

Status

PE

HIPER

Special Attention

Submitted date

Closed date

Last modified date

APAR is sysrouted FROM one or more of the following:

APAR is sysrouted TO one or more of the following:

Fix information

Fixed component name

Fixed component ID

Applicable component levels

R700 PSY

Document Information

Share your feedback

Need support?