APAR status
Closed as program error.
Error description
Control region experiences hang during shutdown of server. The server has Intelligent Management enabled. Formatting out the threads from a console dump will show many threads with stacktrace: com/ibm/son/mesh/CfwTCPImpl.complete com/ibm/ws/ssl/channel/impl/SSLReadServiceContext$SSLReadComplet edCallback.complete com/ibm/ws/tcp/channel/impl/ZAioTCPReadRequestContextImpl.readCo mpleted com/ibm/ws/tcp/channel/impl/ZAioTCPConnLink.readCompleted com/ibm/ws/tcp/channel/impl/ZAioTCPChannel.readCompleted com/ibm/ws/tcp/channel/impl/ZAioTCPChannelCPPUtilities.readCompl eted Java deadlock: thread: com.ibm.son.mesh.Peer-tcp-port-11091 id: 0x45c52000 (owns monitor for com/ibm/son/mesh/Peer object : 0x81c0fe78) waiting for => thread: WebSphere t=009af250 id: 0x45c6c000 (owns monitor for java/lang/Object object : 0x82b2b038) waiting for => thread: com.ibm.son.mesh.Peer-tcp-port-11091 id: 0x45c52000 (owns monitor for com/ibm/son/mesh/Peer object : 0x81c0fe78)
Local fix
Disable Intelligent Management if none of its capabilities are required. Review description for LargeTopologyOptimization property for more details. https://www.ibm.com/docs/en/was-zos/9.0.5?topic=cells-cell-custo m-properties#LargeTopologyOptimization
Problem summary
**************************************************************** * USERS AFFECTED: All users of IBM WebSphere Application * * Server V9.0 for z/OS * **************************************************************** * PROBLEM DESCRIPTION: Controller region with Intelligent * * Management enabled hangs during * * server stop. * **************************************************************** * RECOMMENDATION: * **************************************************************** Controller region with Intelligent Management enabled hangs during server stop. The stop does not complete because two threads are in a deadlock. The first thread has the ZAioTCPConnLink read lock and wants the IM Mesh lock. The second thread has the IM Mesh lock and wants the ZAioTCPConnLink read lock. First thread com/ibm/son/mesh/CfwTCPImpl.complete com/ibm/ws/ssl/channel/impl/ SSLReadServiceContext$SSLReadCompletedCallback.complete com/ibm/ws/tcp/channel/impl/ ZAioTCPReadRequestContextImpl.readCompleted com/ibm/ws/tcp/channel/impl/ZAioTCPConnLink.readCompleted com/ibm/ws/tcp/channel/impl/ZAioTCPChannel.readCompleted com/ibm/ws/tcp/channel/impl/ ZAioTCPChannelCPPUtilities.readCompleted Second thread com/ibm/ws/tcp/channel/impl/ZAioTCPConnLink.destroyCommon com/ibm/ws/tcp/channel/impl/ZAioTCPConnLink.destroy com/ibm/wsspi/channel/base/OutboundConnectorLink.close com/ibm/ws/ssl/channel/impl/SSLConnectionLink.close com/ibm/wsspi/channel/base/InboundApplicationLink.close com/ibm/son/channelfw/SonTCPLink.close com/ibm/son/mesh/CfwTCPImpl.closeLinks com/ibm/son/mesh/CfwTCPImpl.hardClose com/ibm/son/mesh/Neighbor.hardClose com/ibm/son/mesh/MemberMgr.handleIOExceptionGlobalCleanup com/ibm/son/mesh/MemberMgr.handleIOException com/ibm/son/mesh/RandNeighborMaintenanceJob.dropNbrs com/ibm/son/mesh/RandNeighborMaintenanceJob.dropNbrs() com/ibm/son/mesh/RandNeighborMaintenanceJob.run com/ibm/son/mesh/JobQueue.run() com/ibm/son/mesh/Peer.run()
Problem conclusion
Updated the code to avoid the deadlock when custom property com.ibm.ws390.tcpchannel.skipConLinkReadLockOnReadComplete is set to true in a controller region. The following shows how to set the custom property for server1. servers > Server Types > WebSphere application servers > server1 > Server Infrastructure > Java and Process Management > Process definition > Control > Java Virtual Machine > Custom properties Do not use this custom property without the direction of the WebSphere Support team. The fix for this APAR is targeted for inclusion in fix pack 9.0.5.8. For more information, see 'Recommended Updates for WebSphere Application Server': https://www.ibm.com/support/pages/node/715553
Temporary fix
Comments
APAR Information
APAR number
PH34576
Reported component name
WEBSPHERE FOR Z
Reported component ID
5655I3500
Reported release
900
Status
CLOSED PER
PE
NoPE
HIPER
NoHIPER
Special Attention
NoSpecatt / Xsystem
Submitted date
2021-02-17
Closed date
2021-03-10
Last modified date
2024-09-24
APAR is sysrouted FROM one or more of the following:
APAR is sysrouted TO one or more of the following:
PH63378
Fix information
Fixed component name
WEBSPHERE FOR Z
Fixed component ID
5655I3500
Applicable component levels
[{"Business Unit":{"code":"BU048","label":"IBM Software"},"Product":{"code":"SS7K4U","label":"WebSphere Application Server for z\/OS"},"Platform":[{"code":"PF025","label":"Platform Independent"}],"Version":"900","Line of Business":{"code":"LOB67","label":"IT Automation \u0026 App Modernization"}}]
Document Information
Modified date:
24 September 2024