IT34265: Delays when using the administrative REST API gateway to processrequests for different queue managers simultaneously

APAR status

• Closed as program error.

Error description

• The MQ administrative REST API gateway has been configured to
  administer two remote queue managers - QM1 and QM2. Using this
  setup, the following test is run:
  
  - Stop QM1.
  - Start QM2.
  - Start the mqweb server hosting the MQ administrative REST API
  gateway.
  - Submit a request to the MQ administrative REST API gateway for
  QM1.
  - A few seconds later, submit a request to the MQ administrative
  REST API gateway for QM2.
  
  After 30 seconds, the following response is received for the
  request that was submitted for QM1:
  
  HTTP 500: Internal Server Error
  {
    "error":
    [
      {
        "action": "If the problem persists, check the queue
  manager state. If the queue manager is not in the same
  installation as the mqweb server, check that the relevant
  channels are correctly configured and running. Also ensure that
  the correct user is authorized to access the relevant queues and
  perform the requested operation on the queue manager.",
        "completionCode": 2,
        "explanation": "A valid command was sent to the queue
  manager but the queue manager did not respond in a reasonable
  period.",
        "message": "MQWB0042E: Command timed out after '30'
  seconds.",
        "msgId": "MQWB0042E",
        "reasonCode": 2033,
        "type": "pcf"
      }
    ]
  }
  
  Almost immediately afterwards, a response to the request for QM2
  is returned.
  

Local fix

Problem summary

• ****************************************************************
  USERS AFFECTED:
  This issue affects users of the MQ administrative REST API
  gateway.
  
  
  Platforms affected:
  MultiPlatform
  
  ****************************************************************
  PROBLEM DESCRIPTION:
  The MQ administrative REST API gateway uses internal
  PCFRESTAgents to process requests. For each request that is
  received, a thread is started within the administrative REST API
  gateway that:
  
  - Creates a new PCFRESTAgent.
  - Uses it to process the request and send back a response.
  - And then discards the PCFRESTAgent.
  
  As part of the process of creating a PCFRESTAgent for a remote
  queue manager, the administrative REST API gateway will:
  
  - Connect to the gateway queue manager.
  - Send a PCF request to the remote queue manager, via the
  gateway queue manager, to inquire information about the queue
  manager.
  - Wait 30 seconds for a response from the remote queue manager.
  
  
  Now, the method within the administrative REST API gateway that
  created PCFRESTAgents was single threaded, so only one thread
  could create a PCFRESTAgent at any one time. This meant that if
  the administrative REST API gateway received:
  
  - Received a request for a remote queue manager (QM1) that was
  not currently running.
  - Followed by a request for a remote queue manager (QM2) that
  was running.
  
  then the following sequence of events would occur:
  
  - The thread processing the first request called into the
  internal method used to create a PCFRESTAgent for QM1.
  - It connected to the gateway queue manager, sent a PCF request
  to the remote queue manager QM1 and then waited for a response.
  - The thread processing the second request tried to call into
  the internal method to create a PCFRESTAgent for QM2, and became
  blocked as a thread was already running this method.
  - After 30 seconds, the first thread timed out waiting for a
  response from the remote queue manager QM1. As a result, it
  returned a response containing the error message:
  
  "MQWB0042E: Command timed out after '30' seconds."
  
  back to the caller.
  - The second thread now entered the internal method and
  connected to the gateway queue manager.
  - Next, the thread sent a sent a PCF request to the remote queue
  manager QM2, and waited for a response.
  - A short time later, the thread received a response from QM2
  and successfully created a PCFRESTAgent.
  

Problem conclusion

• The internal method within the administrative REST API gateway
  that is used to create PCFRESTAgents has been updated so that it
  is no longer single threaded. This allows multiple threads
  within the gateway to create PCFRESTAgents at the same time,
  which prevents the issue reported in this APAR from occurring.
  
  ---------------------------------------------------------------
  The fix is targeted for delivery in the following PTFs:
  
  Version    Maintenance Level
  v9.2 LTS   9.2.0.2
  
  The latest available maintenance can be obtained from
  'WebSphere MQ Recommended Fixes'
  http://www-1.ibm.com/support/docview.wss?rs=171&uid=swg27006037
  
  If the maintenance level is not yet available information on
  its planned availability can be found in 'WebSphere MQ
  Planned Maintenance Release Dates'
  http://www-1.ibm.com/support/docview.wss?rs=171&uid=swg27006309
  ---------------------------------------------------------------
  

Temporary fix

Comments

APAR Information

• APAR is sysrouted FROM one or more of the following:

• APAR is sysrouted TO one or more of the following:

Fix information

• Fixed component name

  MQ BASE V9.2

• Fixed component ID

  5724H7281

Applicable component levels