IBM Support

IT35658: IBM MQ AMQP service can result in a FDC being created with ProbeID XR079007 or XR071017 due to MQRC_CALL_IN_PROGRESS error.

Subscribe to this APAR

By subscribing, you receive periodic emails alerting you to the status of the APAR, along with a link to the fix after it becomes available. You can track this item individually or track all items by product.

Notify me when this APAR changes.

Notify me when an APAR for this component changes.

 

APAR status

  • Closed as program error.

Error description

  • While performing an AMQP client takeover, the IBM MQ AMQP
service can generate an FDC with either Probe ID XR079007 or
XR071017, where the FDC shows that an MQ API call fails due to
reason code 2219 (MQRC_CALL_IN_PROGRESS).

An example of the FDC is shown below:

...
| Product Long Name :- IBM MQ Advanced Message Queuing Protocol
Support classes for Java
| Program Name      :- java
| Java Version      :- 1.8.0_261
| Thread            :- name=pool-1-thread-1 priority=5
group=main ccl=sun.misc.Launcher$AppClassLoader@9572c4c2
| Source Class      :- com.ibm.mq.MQXRService.MQConnection
| Source Method     :- close
| Probe Id          :- XR079007
| Loaded from       :-
jar:file:/opt/mqm/amqp/lib/AMQPListener.jar!/com/ibm/mq/MQXRServ
ice/MQConnection.class
...
No MQM Function Stack Available


:---------------------------------------------------------------
-----------------------------------:
 : Exception cause: :

:---------------------------------------------------------------
-----------------------------------:

 com.ibm.mq.MQXRService.MQInvocationException: MQCC 2, MQRC 2219
MQRC_CALL_IN_PROGRESS
 at
com.ibm.mq.MQXRService.MQConnection.close(MQConnection.java:1313
)
 at
com.ibm.mq.messaging.TopicObjectCache.emptyAndClose(TopicObjectC
ache.java:109)
 at
com.ibm.mq.MQXRService.AMQPServerSessionV10.close(AMQPServerSess
ionV10.java:2756)
 at
com.ibm.mq.MQXRService.AMQPServerSessionV10.takeOverImpl(AMQPSer
verSessionV10.java:1163)
 at
com.ibm.mq.MQXRService.MQTTServerSession.takeOver(MQTTServerSess
ion.java:1941)
 at
com.ibm.mq.MQXRService.MQXRService.register(MQXRService.java:138
1)
 at
com.ibm.mq.MQXRService.AMQPServerSessionV10.connect(AMQPServerSe
ssionV10.java:1052)
 at
com.ibm.mq.MQXRService.AMQPServerSessionV10.access$200(AMQPServe
rSessionV10.java:106)
 at
com.ibm.mq.MQXRService.AMQPServerSessionV10$2.run(AMQPServerSess
ionV10.java:864)
 at
com.ibm.mq.MQXRService.MQXRService$4.run(MQXRService.java:1690)
 at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.ja
va:522)
 at java.util.concurrent.FutureTask.run(FutureTask.java:277)
 at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFuture
Task.access$201(ScheduledThreadPoolExecutor.java:191)
 at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFuture
Task.run(ScheduledThreadPoolExecutor.java:304)
 at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExec
utor.java:1160)
 at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExe
cutor.java:635)
 at java.lang.Thread.run(Thread.java:822)

Local fix

  • Disable the topic cache by setting the parameter
com.ibm.mq.MQXRService.topiccachesize to 0 in
/var/mqm/qmgrs//amqp/amqp_unix.properties:
com.ibm.mq.MQXRService.topiccachesize=0

Problem summary

  • ****************************************************************
USERS AFFECTED:
This issue affects users of the MQ 9.2 AMQP service.


Platforms affected:
AIX, Linux on x86-64, Windows, Linux on Power, Linux on zSeries

****************************************************************
PROBLEM DESCRIPTION:
The MQ AMQP service contains a number of internal worker
threads, each of which has its own connection to the queue
manager that the service is connected to.

When an AMQP client connects to the MQ AMQP service, it is
associated with one of the internal worker threads. If the AMQP
client performs an operation that requires the MQ AMQP service
to communicate with the queue manager, then the worker thread
will use its connection to the queue manager to issue an MQ API
call, and process the response.

If an AMQP client connects to the MQ AMQP service and requests a
client takeover, then the AMQP Service will start a new internal
thread to perform the takeover processing. As part of this
processing, the internal thread will close any open topic
handles created by the client being taken over before
registering the new client. The MQ AMQP service does this by
issuing an MQCLOSE API call on the connection to the queue
manager associated with the client that is being taken over.

Now, there was no locking around the connections that the AMQP
service used to communicate with the queue manager. This meant
that if an internal thread was issuing an MQ API call on one
connection, at the same time as another internal thread was
processing a client takeover event and was trying to use the
same connection to issue an MQCLOSE API call to close any open
topic handles associated with the client being taken over, then
the API call would fail with a reason code 2219
(MQRC_CALL_IN_PROGRESS). As a result, the MQ AMQP service would
generate FDCs containing either Probe ID XR079007 or XR071017,
and information similar to that shown below:

...
 | Product Long Name :- IBM MQ Advanced Message Queuing Protocol
Support classes for Java
 | Program Name :- java
 | Java Version :- 1.8.0_261
 | Thread :- name=pool-1-thread-1 priority=5 group=main
ccl=sun.misc.Launcher$AppClassLoader@9572c4c2
 | Source Class :- com.ibm.mq.MQXRService.MQConnection
 | Source Method :- close
 | Probe Id :- XR079007
 | Loaded from :-
jar:file:/opt/mqm/amqp/lib/AMQPListener.jar!/com/ibm/mq/MQXRServ
ice/MQConnection.class
...
 No MQM Function Stack Available


:---------------------------------------------------------------
-----------------------------------:
 : Exception cause: :

:---------------------------------------------------------------
-----------------------------------:

 com.ibm.mq.MQXRService.MQInvocationException: MQCC 2, MQRC 2219
MQRC_CALL_IN_PROGRESS
 at
com.ibm.mq.MQXRService.MQConnection.close(MQConnection.java:1313
)
 at
com.ibm.mq.messaging.TopicObjectCache.emptyAndClose(TopicObjectC
ache.java:109)
 at
com.ibm.mq.MQXRService.AMQPServerSessionV10.close(AMQPServerSess
ionV10.java:2756)
 at
com.ibm.mq.MQXRService.AMQPServerSessionV10.takeOverImpl(AMQPSer
verSessionV10.java:1163)
 at
com.ibm.mq.MQXRService.MQTTServerSession.takeOver(MQTTServerSess
ion.java:1941)
 at
com.ibm.mq.MQXRService.MQXRService.register(MQXRService.java:138
1)
 at
com.ibm.mq.MQXRService.AMQPServerSessionV10.connect(AMQPServerSe
ssionV10.java:1052)
 at
com.ibm.mq.MQXRService.AMQPServerSessionV10.access$200(AMQPServe
rSessionV10.java:106)
 at
com.ibm.mq.MQXRService.AMQPServerSessionV10$2.run(AMQPServerSess
ionV10.java:864)
 at
com.ibm.mq.MQXRService.MQXRService$4.run(MQXRService.java:1690)
 at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.ja
va:522)
 at java.util.concurrent.FutureTask.run(FutureTask.java:277)
 at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFuture
Task.access$201(ScheduledThreadPoolExecutor.java:191)
 at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFuture
Task.run(ScheduledThreadPoolExecutor.java:304)
 at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExec
utor.java:1160)
 at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExe
cutor.java:635)
 at java.lang.Thread.run(Thread.java:822)

Problem conclusion

  • To resolve this issue, the MQ AMQP service has been updated to
include some locking around its connections to the queue
manager. This ensures that only one thread can use a connection
to make an MQ API call at a time, which prevents the FDCs
reported in the APAR from occurring.

---------------------------------------------------------------
The fix is targeted for delivery in the following PTFs:

Version    Maintenance Level
v9.2 LTS   9.2.0.4
v9.x CD    9.2.4

The latest available maintenance can be obtained from
'WebSphere MQ Recommended Fixes'
http://www-1.ibm.com/support/docview.wss?rs=171&uid=swg27006037

If the maintenance level is not yet available information on
its planned availability can be found in 'WebSphere MQ
Planned Maintenance Release Dates'
http://www-1.ibm.com/support/docview.wss?rs=171&uid=swg27006309
---------------------------------------------------------------

Temporary fix

  • 



Comments


    

  • 



APAR Information




    

  • APAR number
    IT35658
    • 

  • Reported component name
    MQ BASE V9.2
    • 

  • Reported component ID
    5724H7281
    • 

  • Reported release
    920
    • 

  • Status
    CLOSED  PER
    • 

  • PE
    NoPE
    • 

  • HIPER
    NoHIPER
    • 

  • Special Attention
    NoSpecatt / Xsystem
    • 

  • Submitted date
    2021-01-22
    • 

  • Closed date
    2021-10-06
    • 

  • Last modified date
    2022-02-03
    • 





    

  • APAR is sysrouted FROM one or more of the following:
    

    • 

  • APAR is sysrouted TO one or more of the following:
    

    • 



Fix information


    

  • Fixed component name
    MQ BASE V9.2
    • 

  • Fixed component ID
    5724H7281
    • 



Applicable component levels


    








      
              
                            

              

              [{"Line of Business":{"code":"LOB45","label":"Automation"},"Business Unit":{"code":"BU059","label":"IBM Software w\/o TPS"},"Product":{"code":"SSYHRD","label":"IBM MQ"},"Platform":[{"code":"PF025","label":"Platform Independent"}],"Version":"920"}]
              

                          

          
 
        

      

    

      

        

          

            
Document Information


            

            


                        

            Modified date:
            

            04 February 2022
            

            
                      

        


        

        


      

    

  



    

  

  
    
            

          
Page Feedback