In my performance role it is surprising how often you get a bunch of similar problems. The two problems below are all to do with the amount of CPU being used by MQ applications.
Once we explored the problems it was obvious why the CPU had increased.
The first problem reported by the customer was he sees lots more CPU being used by this new application, and it is using more CPU than other production work.
We explored this and found
- Production was doing about 200 messages a second.
- This new application was doing about 50 business messages a second. Under the covers there were 10 hops. Each application did MQGET, MQPUT, MQCOMMIT - so with 10 hops this was 500 messages a second.
- Production messages were processed by CICS. From SDSF (or RMF) the only MQ cost seen was from the queue manager. From SDSF (or RMF) we could not see the CPU used by the MQ applications running under CICS; we saw about 10 CICS regions which were busy - but not running hot.
- The new application came in from a client. This means all of the costs were in the Chinit. The Chinit showed up as using the same CPU as 2 * CICS.
So the lessons learned from this were
- Make sure you are comparing similar workloads: 500 compared to 200 messages a second.
- Client and distributed queue managers will show up as CHINIT costs. In CICS and Batch etc - at a high level,the MQ costs are invisible. You can use the Usage section of the SMF 30 to see how much CPU was being used for MQ.
The second problem was we reported as "We have changed our application and now see higher costs"
The application used to be a simple
- CICS transaction does an MQPUT
- Channel moves it to distributed MQ
The enhanced application uses PubSub
- CICS transaction does a publish
- Lots of clients subscribe to the topic - and often there 10 messages output for every publish requests
There are now more messages, and more requests through the CHINIT.
In both cases once we explored the problem (for a couple of hours) the answer was obvious.