IBM Support

IT15138: WMQAPIFAILUREEXCEPTION: CC=2 RC=2033 OP=_GET - MQGET FFDC AND CAUSING AGENT TO END

Subscribe to this APAR

By subscribing, you receive periodic emails alerting you to the status of the APAR, along with a link to the fix after it becomes available. You can track this item individually or track all items by product.

Notify me when this APAR changes.

Notify me when an APAR for this component changes.

 

APAR status

  • Closed as program error.

Error description

  • A WebSphere MQ File Transfer Edition or Managed File Transfer
    agent generates an FFDC files with the following content:
    
      Level:      f704-FP5-20150522-1614
      Time:       06/05/2016 13:54:10:572 BST
      Thread:     68 (Thread-54)
      Class:
    com.ibm.wmqfte.statestore.impl.FTEStateStoreImpl$RetrySenderTran
    sferTimerTask
      Instance:   44ea44ea
      Method:     run
      Probe:      FFDC_003
      Cause:      com.ibm.wmqfte.wmqiface.WMQApiFailureException:
    cc=2 rc=2033 op=_get - MQGET
    
    com.ibm.wmqfte.wmqiface.WMQApiFailureException: cc=2 rc=2033
    op=_get - MQGET
      at
    com.ibm.wmqfte.wmqiface.WMQQueueImpl._get(WMQQueueImpl.java:304)
      at
    com.ibm.wmqfte.wmqiface.WMQQueueImpl.get(WMQQueueImpl.java:243)
      at
    com.ibm.wmqfte.statestore.impl.FTEStateStorePersistence$FTEMutab
    leSenderStateRef.getStateFromPersistentStore(FTEStateStorePersis
    tence.java:280)
      at
    com.ibm.wmqfte.statestore.impl.FTEStateStorePersistence$FTEMutab
    leSenderStateRef.getFTEMutableSenderState(FTEStateStorePersisten
    ce.java:249)
      at
    com.ibm.wmqfte.statestore.impl.FTEStateStorePersistence$Transfer
    RequestQueue.take(FTEStateStorePersistence.java:604)
      at
    com.ibm.wmqfte.statestore.impl.FTEStateStorePersistence$SenderTr
    ansferRequestMap.get(FTEStateStorePersistence.java:823)
      at
    com.ibm.wmqfte.statestore.impl.FTEStateStorePersistence.getSende
    rState(FTEStateStorePersistence.java:1910)
      at
    com.ibm.wmqfte.statestore.impl.FTEStateStorePersistence.getMutab
    leSenderState(FTEStateStorePersistence.java:1711)
      at
    com.ibm.wmqfte.statestore.impl.FTEStateStoreImpl$RetrySenderTran
    sferTimerTask.run(FTEStateStoreImpl.java:5065)
      at java.util.Timer$TimerImpl.run(Timer.java:296)
    
    Active Source Transfers
    ----------------------
      414d51204654453730345f514d475220d9562b5720077a29 =
    WaitingForDestinationCapacity
    
    and then the agent stops.
    
    
    The following messages are also logged in the agent's event log
    (output0.log) file:
    
      FTEStateStore  E   BFGSS0025E: An internal error has occurred.
     The exception is: cc=2 rc=2033 op=_get - MQGET
      AgentRuntime  E   BFGAG0061E: The agent ended abnormally
      AgentRuntime   I   BFGAG0139I: The agent has suspended its
    current transfers and is now stopping.
    

Local fix

Problem summary

  • ****************************************************************
    USERS AFFECTED:
    This issue affects users of:
    
      - WebSphere MQ File Transfer Edition
      - WebSphere MQ Managed File Transfer
      - IBM MQ Managed File Transfer
    
    who use scheduled transfers.
    
    
    Platforms affected:
    MultiPlatform
    
    ****************************************************************
    PROBLEM DESCRIPTION:
    This issue was caused by a timing problem between two threads
    when locking transfer state messages under syncpoint on an
    agent's SYSTEM.FTE.STATE queue.  The first thread, the
    "CommandHandler" thread, was checking scheduled transfers
    defined on the agent that were due to be started.  The second
    thread, a "RetrySenderTransferTimerTask" thread, was attempting
    to restart a transfer, which had been invoked via a scheduled
    transfer definition, that had previously entered recovery
    processing, for example in the scenario where the destination
    agent was not able to accept any new transfers at the point in
    time where the schedule transfer request was issued.
    
    When processing the list of scheduled transfers to determine if
    new transfers need to be invoked, the "CommandHandler" thread
    first checks if there is an existing transfer for that schedule
    that has yet to complete.  To determine if a transfer already
    exists for a schedule, the "CommandHandler" thread checks the
    agent's STATE queue to see if a persisted transfer state message
    exists for the last transfer requested by this schedule.  If a
    state message exists, a previous transfer for this schedule is
    still running and a new one is not requested.  Otherwise, a new
    transfer request for this schedule is submitted to the agent.
    
    In order to find any existing transfer state message, the
    "CommandHandler" thread uses an MQGET call with a syncpoint.
    Any state message that is returned is parsed and then the put
    back to the STATE queue using the original syncpoint.  This same
    syncpoint is used to process all scheduled transfer definitions
    for which sufficient time has passed for them to be serviced
    again.
    
    In the case where there are a large number of scheduled
    transfers defined on an agent, it could take a number of seconds
    to process all of the schedules.  In such a scenario, a state
    message for a currently running scheduled transfer would be
    locked an unavailable to other threads for a period of time.
    
    At the same time as the "CommandHandler" thread was processing
    scheduled transfer definitions, a "RetrySenderTransferTimerTask"
    thread attempted to restart a previous instance of a scheduled
    transfer that had entered recovery.  The thread attempted to
    retrieve the specific state message associated with the transfer
    from the agent's STATE queue.  If the state message was locked
    under syncpoint because of the "CommandHandler" thread
    processing scheduled transfer definitions, then the queue
    manager would not return the transfer state message to the
    "RetrySenderTransferTimerTask" thread and a 2033
    (MQRC_NO_MSG_AVAILABLE) reason code is returned to the agent.
    This was unexpected by the "RetrySenderTransferTimerTask"
    thread, which resulted in the generation of an FFDC followed by
    the agent being shut down.
    

Problem conclusion

  • The WebSphere MQ File Transfer Edition product and Managed File
    Transfer component code has been update such that the
    "CommandHandler" thread only browses the agent's STATE queue to
    determine if a transfer state message exists for a previous
    instance of a scheduled transfer.  This means the transfer state
    message is not locked under syncpoint and is available to be
    consumed by the "RetrySenderTransferTimerTask" thread when
    restarting a transfer after recovery.
    
    ---------------------------------------------------------------
    The fix is targeted for delivery in the following PTFs:
    
    Version    Maintenance Level
    v7.0       7.0.4.7
    v7.5       7.5.0.8
    
    The latest available FTE maintenance can be obtained from
    'Fix List for WebSphere MQ File Transfer Edition 7.0'
    http://www-01.ibm.com/support/docview.wss?uid=swg27015313
    
    The latest available MQ maintenance can be obtained from
    'WebSphere MQ Recommended Fixes'
    http://www-1.ibm.com/support/docview.wss?rs=171&uid=swg27006037
    
    If the maintenance level is not yet available information on
    its planned availability can be found in 'WebSphere MQ
    Planned Maintenance Release Dates'
    http://www-1.ibm.com/support/docview.wss?rs=171&uid=swg27006309
    ---------------------------------------------------------------
    

Temporary fix

Comments

APAR Information

  • APAR number

    IT15138

  • Reported component name

    WMQ FILE TRANSF

  • Reported component ID

    5724R1000

  • Reported release

    704

  • Status

    CLOSED PER

  • PE

    NoPE

  • HIPER

    YesHIPER

  • Special Attention

    NoSpecatt / Xsystem

  • Submitted date

    2016-05-06

  • Closed date

    2016-12-21

  • Last modified date

    2017-05-24

  • APAR is sysrouted FROM one or more of the following:

  • APAR is sysrouted TO one or more of the following:

Fix information

  • Fixed component name

    WMQ FILE TRANSF

  • Fixed component ID

    5724R1000

Applicable component levels

  • R704 PSY

       UP

[{"Business Unit":{"code":"BU053","label":"Cloud & Data Platform"},"Product":{"code":"SSEP7X","label":"WebSphere MQ File Transfer Edition"},"Component":"","ARM Category":[],"Platform":[{"code":"PF025","label":"Platform Independent"}],"Version":"7.0.4","Edition":"","Line of Business":{"code":"LOB36","label":"IBM Automation"}}]

Document Information

Modified date:
24 May 2017