IBM Support

PH34576: DURING SHUTDOWN OF SERVER, CONTROL REGION EXPERIENCES HANG IN COM/IBM/SON/MESH/CFWTCPIMPL.COMPLETE

Subscribe to this APAR

By subscribing, you receive periodic emails alerting you to the status of the APAR, along with a link to the fix after it becomes available. You can track this item individually or track all items by product.

Notify me when this APAR changes.

Notify me when an APAR for this component changes.

 

APAR status

  • Closed as program error.

Error description

  • Control region experiences hang during shutdown of server.
    The server has Intelligent Management enabled.
    
    Formatting out the threads from a console dump will show many
    threads with stacktrace:
    
    com/ibm/son/mesh/CfwTCPImpl.complete
    com/ibm/ws/ssl/channel/impl/SSLReadServiceContext$SSLReadComplet
    edCallback.complete
    com/ibm/ws/tcp/channel/impl/ZAioTCPReadRequestContextImpl.readCo
    mpleted
    com/ibm/ws/tcp/channel/impl/ZAioTCPConnLink.readCompleted
    com/ibm/ws/tcp/channel/impl/ZAioTCPChannel.readCompleted
    com/ibm/ws/tcp/channel/impl/ZAioTCPChannelCPPUtilities.readCompl
    eted
    

Local fix

Problem summary

  • ****************************************************************
    * USERS AFFECTED:  All users of IBM WebSphere Application      *
    *                  Server V9.0 for z/OS                        *
    ****************************************************************
    * PROBLEM DESCRIPTION: Controller region with Intelligent      *
    *                      Management enabled hangs during         *
    *                      server stop.                            *
    ****************************************************************
    * RECOMMENDATION:                                              *
    ****************************************************************
    Controller region with Intelligent Management enabled hangs
    during server stop. The stop does not complete because two
    threads are in a deadlock. The first thread has the
    ZAioTCPConnLink read lock and wants the IM Mesh lock.
    The second thread has the IM Mesh lock and wants the
    ZAioTCPConnLink read lock.
    First thread
    com/ibm/son/mesh/CfwTCPImpl.complete
    com/ibm/ws/ssl/channel/impl/
    SSLReadServiceContext$SSLReadCompletedCallback.complete
    com/ibm/ws/tcp/channel/impl/
    ZAioTCPReadRequestContextImpl.readCompleted
    com/ibm/ws/tcp/channel/impl/ZAioTCPConnLink.readCompleted
    com/ibm/ws/tcp/channel/impl/ZAioTCPChannel.readCompleted
    com/ibm/ws/tcp/channel/impl/
    ZAioTCPChannelCPPUtilities.readCompleted
    Second thread
    com/ibm/ws/tcp/channel/impl/ZAioTCPConnLink.destroyCommon
    com/ibm/ws/tcp/channel/impl/ZAioTCPConnLink.destroy
    com/ibm/wsspi/channel/base/OutboundConnectorLink.close
    com/ibm/ws/ssl/channel/impl/SSLConnectionLink.close
    com/ibm/wsspi/channel/base/InboundApplicationLink.close
    com/ibm/son/channelfw/SonTCPLink.close
    com/ibm/son/mesh/CfwTCPImpl.closeLinks
    com/ibm/son/mesh/CfwTCPImpl.hardClose
    com/ibm/son/mesh/Neighbor.hardClose
    com/ibm/son/mesh/MemberMgr.handleIOExceptionGlobalCleanup
    com/ibm/son/mesh/MemberMgr.handleIOException
    com/ibm/son/mesh/RandNeighborMaintenanceJob.dropNbrs
    com/ibm/son/mesh/RandNeighborMaintenanceJob.dropNbrs()
    com/ibm/son/mesh/RandNeighborMaintenanceJob.run
    com/ibm/son/mesh/JobQueue.run()
    com/ibm/son/mesh/Peer.run()
    

Problem conclusion

  • Updated the code to avoid the deadlock when custom property
    com.ibm.ws390.tcpchannel.skipConLinkReadLockOnReadComplete is
    set to true in a controller region.
    
    The following shows how to set the custom property for
    server1.
    servers > Server Types > WebSphere application servers >
    server1 > Server Infrastructure > Java and Process Management >
    Process definition > Control > Java Virtual Machine >
    Custom properties
    
    Do not use this custom property without the direction of
    the WebSphere Support team.
    
    The fix for this APAR is targeted for inclusion in fix pack
    9.0.5.8. For more information, see 'Recommended Updates for
    WebSphere Application Server':
    https://www.ibm.com/support/pages/node/715553
    

Temporary fix

Comments

APAR Information

  • APAR number

    PH34576

  • Reported component name

    WEBSPHERE FOR Z

  • Reported component ID

    5655I3500

  • Reported release

    900

  • Status

    CLOSED PER

  • PE

    NoPE

  • HIPER

    NoHIPER

  • Special Attention

    NoSpecatt / Xsystem

  • Submitted date

    2021-02-17

  • Closed date

    2021-03-10

  • Last modified date

    2021-03-10

  • APAR is sysrouted FROM one or more of the following:

  • APAR is sysrouted TO one or more of the following:

Fix information

  • Fixed component name

    WEBSPHERE FOR Z

  • Fixed component ID

    5655I3500

Applicable component levels

[{"Line of Business":{"code":"LOB36","label":"IBM Automation"},"Business Unit":{"code":"BU053","label":"Cloud & Data Platform"},"Product":{"code":"SS7K4U","label":"WebSphere Application Server for z\/OS"},"Platform":[{"code":"PF025","label":"Platform Independent"}],"Version":"900"}]

Document Information

Modified date:
11 March 2021