IBM Support

IT15158: WMQ FTE AGENT HANGS BECAUSE OF A DEADLOCK AFTER IT07999 WHEN IS TRACE ENABLED

Subscribe

You can track all active APARs for this component.

 

APAR status

  • Closed as program error.

Error description

  • When trace is enabled on a WebSphere MQ File Transfer Edition
    (FTE) or Managed File Transfer (MFT) agent, a deadlock causing
    the agent to hang can occur.  The agent becomes unresponsive
    and, even though the JVM process for the agent is running, it
    does not respond to ftePingAgent requests and its status is
    reported as "Unknown" in the IBM MQ Explorer GUI and in the
    output to the fteListAgents and fteShowAgentDetails command.
    
    A Javacore of the agent JVM process shows:
    
     Thread "AgentStatusPublisher" (0x338AD800)
       is waiting for:
         sys_mon_t:0x3616E7F0 infl_mon_t: 0x3616E810:
    
    com/ibm/wmqfte/utils/FTEXMLProperties@0x90345248/0x90345254:
       which is owned by:
     Thread "CommandHandler" (0x33815B00)
       which is waiting for:
         sys_mon_t:0x33E2B2E0 infl_mon_t: 0x33E2B300:
         java/util/logging/FileHandler@0x9008ECF0/0x9008ECFC:
       which is owned by:
     Thread "AgentStatusPublisher" (0x338AD800)
    
    
    "CommandHandler" J9VMThread:0x33815B00, j9thread_t:0x333BFFF4,
    java/lang/Thread:0x903376C0, state:B, prio=5
           (java/lang/Thread getId:0x30, isDaemon:false)
           (native thread ID:0x5A5063, native priority:0x5, native
    policy:UNKNOWN)
          Java callstack:
              at
    java/util/logging/Logger.log(Logger.java:1302(Compiled Code))
              at
    java/util/logging/Logger.logp(Logger.java:1394(Compiled Code))
              at
    com/ibm/msg/client/commonservices/j2se/trace/DefaultTracer.metho
    dEntry(DefaultTracer.java:340(Compiled Code))
              at
    com/ibm/msg/client/commonservices/trace/Trace.methodEntryInterna
    l(Trace.java:682(Compiled Code))
              at
    com/ibm/msg/client/commonservices/trace/Trace.entry(Trace.java:6
    27(Compiled Code))
              at
    com/ibm/wmqfte/ras/impl/WMQTraceHandler.publish(WMQTraceHandler.
    java:237(Compiled Code))
              at
    java/util/logging/Logger.log(Logger.java:1302(Compiled Code))
              at
    com/ibm/wmqfte/ras/impl/TraceImpl.entryImplementation(TraceImpl.
    java:65(Compiled Code))
              at
    com/ibm/wmqfte/ras/Trace.entry(Trace.java:86(Compiled Code))
              at
    com/ibm/wmqfte/utils/FTEXMLProperties.put(FTEXMLProperties.java:
    42(Compiled Code))
              at
    com/ibm/wmqfte/utils/AgentStatusDetails.updateCommandhandlerTime
    Stamp(AgentStatusDetails.java:582(Compiled Code))
              at
    com/ibm/wmqfte/statestore/impl/FTEAgentStatusPublisher.updateCom
    mandhandlerTimeStamp(FTEAgentStatusPublisher.java:257(Compiled
    Code))
              at
    com/ibm/wmqfte/cmdhandler/impl/FTECommandThread.run(FTECommandTh
    read.java:235)
    
    
    "AgentStatusPublisher" J9VMThread:0x338AD800,
    j9thread_t:0x33878714, java/lang/Thread:0x90337898, state:B,
    prio=5
           (java/lang/Thread getId:0x31, isDaemon:false)
           (native thread ID:0x959081, native priority:0x5, native
    policy:UNKNOWN)
          Java callstack:
              at
    com/ibm/wmqfte/utils/AgentStatusDetails.toString(AgentStatusDeta
    ils.java:1515)
              at
    com/ibm/msg/client/commonservices/j2se/trace/TraceUtils$GenericF
    ormatAdapter.formatObject(TraceUtils.java:505(Compiled Code))
              at
    com/ibm/msg/client/commonservices/j2se/trace/TraceUtils.formatOb
    ject(TraceUtils.java:100(Compiled Code))
              at
    com/ibm/msg/client/commonservices/j2se/trace/HumanFormatter$Exit
    FormatAdapter.formatRecord(HumanFormatter.java:927(Compiled
    Code))
              at
    com/ibm/msg/client/commonservices/j2se/trace/HumanFormatter$Mess
    ageFormatAdapter.formatRecord(HumanFormatter.java:1279(Compiled
    Code))
              at
    com/ibm/msg/client/commonservices/j2se/trace/HumanFormatter.form
    at(HumanFormatter.java:288(Compiled Code))
              at
    java/util/logging/StreamHandler.publish(StreamHandler.java:315(C
    ompiled Code))
              at
    java/util/logging/FileHandler.publish(FileHandler.java(Compiled
    Code))
              at
    java/util/logging/Logger.log(Logger.java:1302(Compiled Code))
              at
    java/util/logging/Logger.logp(Logger.java:1365(Compiled Code))
              at
    com/ibm/msg/client/commonservices/j2se/trace/DefaultTracer.metho
    dExit(DefaultTracer.java:353(Compiled Code))
              at
    com/ibm/msg/client/commonservices/trace/Trace.methodExitInternal
    (Trace.java:826(Compiled Code))
              at
    com/ibm/msg/client/commonservices/trace/Trace.exit(Trace.java:74
    5(Compiled Code))
              at
    com/ibm/wmqfte/ras/impl/WMQTraceHandler.publish(WMQTraceHandler.
    java:244(Compiled Code))
              at
    java/util/logging/Logger.log(Logger.java:1302(Compiled Code))
              at
    com/ibm/wmqfte/ras/impl/TraceImpl.exitImplementation(TraceImpl.j
    ava:89(Compiled Code))
              at
    com/ibm/wmqfte/ras/Trace.exit(Trace.java:137(Compiled Code))
              at
    com/ibm/wmqfte/statestore/impl/FTEAgentStatusPublisher.getUpdate
    dAgentStatus(FTEAgentStatusPublisher.java:491)
              at
    com/ibm/wmqfte/statestore/impl/FTEAgentStatusPublisher.publishAg
    entStatus(FTEAgentStatusPublisher.java:412)
              at
    com/ibm/wmqfte/statestore/impl/FTEAgentStatusPublisher.run(FTEAg
    entStatusPublisher.java:285)
    

Local fix

  • Disable trace or, if trace is required and requested by IBM
    Support, add the following entry to the trace string
    specification:
    
      com.ibm.wmqfte.utils.FTEXMLProperties=off
    
    This disables trace in the FTEXMLProperties class which can lead
    to the deadlock scenario described by this APAR.
    
    For example, this trace specification can be configured in the
    agent.properties file with the following entry:
    
    
    trace=com.ibm.wmqfte=all:com.ibm.wmqfte.utils.FTEXMLProperties=o
    ff
    

Problem summary

  • ****************************************************************
    USERS AFFECTED:
    This issue affects users of:
    
      - WebSphere MQ File Transfer Edition V7.0.4.5
      - WebSphere MQ Managed File Transfer V7.5.0.6
      - IBM MQ Managed File Transfer V8.0.0.3 and V8.0.0.4
    
    who have enabled trace on an agent that traces the classes
    FTEAgentStatusPublisher and FTEXMLProperties, for example, by
    using the "com.ibm.wmqfte=all" trace string specification.
    
    
    Platforms affected:
    MultiPlatform
    
    ****************************************************************
    PROBLEM DESCRIPTION:
    After APAR IT07999:
    
    http://www-01.ibm.com/support/docview.wss?uid=swg1IT07999
    
    a WebSphere MQ File Transfer Edition or Managed File Transfer
    agent could hang, due to a deadlock, when trace was enabled.
    This was due to a two locks, a trace lock and a properties lock
    within the FTEXMLProperties class, being taken in different
    orders by two threads (the "CommandHandler" thread and the
    "AgentStatusPublisher" thread).
    

Problem conclusion

  • The WebSphere MQ File Transfer Edition and Managed File Transfer
    agent code has been updated to remove the need for the
    properties lock when making tracing routine calls in the
    FTEXMLProperties class.
    
    ---------------------------------------------------------------
    The fix is targeted for delivery in the following PTFs:
    
    Version    Maintenance Level
    v7.0       7.0.4.6
    v7.5       7.5.0.7
    v8.0       8.0.0.4
    v9.0 CD    9.0.1
    v9.0 LTS   9.0.0.1
    
    The latest available FTE maintenance can be obtained from
    'Fix List for WebSphere MQ File Transfer Edition 7.0'
    http://www-01.ibm.com/support/docview.wss?uid=swg27015313
    
    The latest available MQ maintenance can be obtained from
    'WebSphere MQ Recommended Fixes'
    http://www-1.ibm.com/support/docview.wss?rs=171&uid=swg27006037
    
    If the maintenance level is not yet available information on
    its planned availability can be found in 'WebSphere MQ
    Planned Maintenance Release Dates'
    http://www-1.ibm.com/support/docview.wss?rs=171&uid=swg27006309
    ---------------------------------------------------------------
    

Temporary fix

Comments

APAR Information

  • APAR number

    IT15158

  • Reported component name

    WMQ FILE TRANSF

  • Reported component ID

    5724R1000

  • Reported release

    704

  • Status

    CLOSED PER

  • PE

    NoPE

  • HIPER

    YesHIPER

  • Special Attention

    NoSpecatt / Xsystem

  • Submitted date

    2016-05-07

  • Closed date

    2016-05-27

  • Last modified date

    2017-03-10

  • APAR is sysrouted FROM one or more of the following:

  • APAR is sysrouted TO one or more of the following:

Fix information

  • Fixed component name

    WMQ FILE TRANSF

  • Fixed component ID

    5724R1000

Applicable component levels

  • R704 PSY

       UP

[{"Business Unit":{"code":"BU053","label":"Cloud & Data Platform"},"Product":{"code":"SSEP7X","label":"WebSphere MQ File Transfer Edition"},"Component":"","ARM Category":[],"Platform":[{"code":"PF025","label":"Platform Independent"}],"Version":"7.0.4","Edition":"","Line of Business":{"code":"LOB36","label":"IBM Automation"}}]

Document Information

Modified date:
10 March 2017