IT13497: WMQ FTE V7.0.4.4 AGENT REPORTS A BFGEL0007E ERROR, GENERATES AN FDC CONTAINING PROBEID JO000010 AND THEN SHUTS ITSELF DOWN

APAR status

• Closed as program error.

Error description

• A WebSphere MQ File Transfer Edition reports the following error
  in it's output0.log file:
  BFGEL0007E: The agent has received a reason code of '2195' from
  the message queue interface (MQI). The agent cannot continue
  processing and will now end.
  and then shuts itself down. At the same time as the BFGEL0007E
  error is reported, the agent generates an FDC file containing
  ProbeID JO000010 and the insert:
  Attempt to remove an invalid object from the hash table
  

Local fix

Problem summary

• ****************************************************************
  USERS AFFECTED:
  This issue affects two categories of user:
  
  Category 1:
  -------------------
  Users of:
  
  - The WebSphere MQ V7.0.1 classes for Java.
  - The WebSphere MQ V7.0.1 classes for JMS.
  - The WebSphere MQ V7.0.1 resource adapter.
  
  - The WebSphere MQ V7.1 classes for Java.
  - The WebSphere MQ V7.1 classes for JMS.
  - The WebSphere MQ V7.1 resource adapter.
  
  - The WebSphere MQ V7.5 classes for Java.
  - The WebSphere MQ V7.5 classes for JMS.
  - The WebSphere MQ V7.5 resource adapter.
  
  - The WebSphere MQ V8.0 classes for Java.
  - The WebSphere MQ V8.0 classes for JMS.
  - The WebSphere MQ V8.0 resource adapter.
  
  - The WebSphere Application Server V7 WebSphere MQ messaging
  provider.
  - The WebSphere Application Server V8 WebSphere MQ messaging
  provider.
  
  who have applications that connect to queue managers using the
  CLIENT transport and WebSphere MQ messaging provider normal
  mode.
  
  Category 2:
  -------------------
  Users of:
  
  - WebSphere MQ File Transfer Edition V7.0.4
  - The WebSphere MQ Managed File Transfer V7.5 component.
  - The WebSphere MQ Managed File Transfer V8.0 component.
  
  who have agents that connect to their agent queue manager using
  the CLIENT transport.
  
  
  Platforms affected:
  MultiPlatform
  
  ****************************************************************
  PROBLEM DESCRIPTION:
  The Java Message Queueing Interface (JMQI) is used by:
  
  - The WebSphere MQ classes for Java.
  - The WebSphere MQ classes for JMS.
  - The WebSphere MQ resource adapter.
  - The WebSphere Application Server WebSphere MQ messaging
  provider.
  - The WebSphere MQ File Transfer Edition product.
  - The WebSphere MQ Managed File Transfer component.
  
  to communicate with queue managers. If an application connects
  to a queue manager using the CLIENT transport, the JMQI will
  establish a connection handle (also known as an hconn) for the
  application to use. This connection handle will be associated
  with a WebSphere MQ channel instance, which maps to a TCP/IP
  connection.
  
  For every channel instance to a queue manager, the JMQI
  maintains a list of connection handles that are associated with
  that channel. When a new connection handle is created, it is
  added to an internal list of connection handles for the
  appropriate channel instance. When the application has finished
  with the connection handle, the JMQI removes it from the
  internal list.
  
  The following sequence of events led up to the issue reported in
  this APAR:
  
  - An application thread needed to communicate with a queue
  manager, so the JMQI found a existing channel instance and then
  allocated a new connection handle with that channel instance.
  This caused details of the connection handle to be added to the
  internal list of connection handles associated with the channel.
  - The JMQI then started flowing MQ API calls to the queue
  manager to establish that connection handle.
  - While these API calls were in progress, the TCP/IP connection
  associated with the channel instance was broken due to a network
  issue.
  - At this point, two threads noticed that the connection was no
  longer valid:
      - An internal thread associated with theTCP/IP connection.
      - The application thread.
    and attempted to clean up the connection handles that were
  using it.
  
  - The first thread checked the internal list of connection
  handles, found that the handle was in the list. At this point,
  it took some internal locks and then started the process of
  closing it.
  - Next, the second thread checked the internal list, and found
  that the connection handle was still there, so decided to close
  it too. However, the first thread was holding the internal locks
  needed to clean up the connection handle. As a result, the
  second thread was blocked.
  - The first thread removed the connection handle from the
  internal list, and then closed it.
  - When the connection handle was closed, the first thread
  released the locks.
  
  - The second thread now got the locks, and tried to remove the
  connection handle from the internal list. However, the
  connection handle was already been removed from the internal
  list by the first thread. This caused the second thread to throw
  an exception containing WebSphere MQ Reason Code 2195
  (MQRC_UNEXPECTED_ERROR) and generate an FDC.
  

Problem conclusion

• The JMQI has been updated to ensure that all of the required
  locks are obtained before checking if the connection handle is
  in the internal list associated with a channel instance.
  
  As the connection handles are only removed from the internal
  list once the locks are held, this means that it is no longer
  possible for one thread to find a connection handle in the
  internal list, and another thread to remove it from the same
  list before the first thread has done so.
  
  ---------------------------------------------------------------
  The fix is targeted for delivery in the following PTFs:
  
  Version    Maintenance Level
  v7.0       7.0.1.15
  v7.1       7.1.0.8
  v7.5       7.5.0.7
  v8.0       8.0.0.6
  
  The latest available maintenance can be obtained from
  'WebSphere MQ Recommended Fixes'
  http://www-1.ibm.com/support/docview.wss?rs=171&uid=swg27006037
  
  If the maintenance level is not yet available information on
  its planned availability can be found in 'WebSphere MQ
  Planned Maintenance Release Dates'
  http://www-1.ibm.com/support/docview.wss?rs=171&uid=swg27006309
  ---------------------------------------------------------------
  

Temporary fix

Comments

APAR Information

• APAR is sysrouted FROM one or more of the following:

• APAR is sysrouted TO one or more of the following:

Fix information

• Fixed component name

  WMQ WINDOWS V7

• Fixed component ID

  5724H7220

Applicable component levels