What does WebSphere MQ use storage for and does it conduct its own storage management or leave everything up to the operating system ? If MQ allocates, say, a million bytes of storage when does that storage get freed up ? Sometimes it's hard to determine just how much storage the queue manager actually needs so Supportpacs like MP16 and MP1G give direction on how much storage to allocate and what you should expect MQ to do with its storage.
It is usually true that one of MQ's principal users of storage in the queue manager address space is its buffers. While it's mandatory to have some level of buffers defined (to minimize I/O to page sets) there is a limit on how many buffers can be defined and how large you can make them. This is because virtual storage in an address space is divided between the requirements of the system and the requirements of the task using that address space. After the system's needs are met (including the base system control programs) the remaining storage is left for the user. That notwithstanding, a two-gigabyte virtual address space can often be expected to yield maybe 1.2 Gig to the user (though of course this will vary not only for each address space but for different installations of z/OS).
So private storage is chopped up into logical areas called subpools. One of the key things logically relating a set of things in a subpool is where an application expects this set to be located (eg. in private storage, CSA, or ELSQA). Many other things too, like whether an application using that storage will be expected to run in a disabled mode, or if the storage is going to be page fixed, play into the subpool that an application will request storage out of.
A look at the storage map (using VSMDATA) of a typical WebSphere MQ MSTR started task will illustrate which subpools tend to get higher usage than others :
LOCAL SUBPOOL USAGE SUMMARY
TCB/OWNER SP# KEY BELOW ABOVE TOTAL
------------------ --- --- --------- ---------- ----------
9FF988 0 7 A000 688000 692000
9E49F0 229 7 24000 3A757000 3A77B000
98B8B0 229 7 1000 0 1000
978938 229 7 0 1000 1000
974E88 229 7 0 1000 1000
9E49F0 230 7 0 163000 163000
9FE050 230 8 1000 2000 3000
9E49F0 230 8 0 1000000 1000000
9FE050 230 9 1000 2000 3000
9FF988 236 1 15000 30000 45000
9FF988 237 1 6000 2A000 30000
9FE050 249 0 1000 0 1000
9FE050 249 1 0 2000 2000
9E49F0 252 0 A000 717000 721000
How these numbers are distributed can tell a story. A virtual storage map of the MQ MSTR address space will include information about allocations in other storage keys, but those with a Key of 7 are allocated specifically under the MQ MSTR task.
Traditionally, modern products like to allocate storage above the line first (we'll hit on above the bar storage later). Storage above the line lands above the 16 Meg line and as that ATL area is used up, normally freeing of storage has to occur so somebody
else can use it. But if a runaway task causes all or most of the ATL storage to be depleted, then allocations will begin to occur below the line. If sustained long enough, these allocations will deplete all of the BTL storage as well.
Historically, WebSphere MQ never exploited 64-bit storage. At least not until 2010 when MQ's PubSub code was re-designed to take advantage of it. In addition, control blocks which were storage intensive got moved there as well. These blocks
included MQ's locks for messages in a unit-of-work or locks for queues that were open. Movement of queue indices to this storage greatly increased the limit on how many indexed messages were possible. Lastly, control blocks for security were
also relocated. MQ required that the MEMLIMIT for the queue manager be set at least to 2 Gig (either by setting the MEMLIMIT value in the startup proc for the MQ MSTR or by other means; such as customizing the limit for all jobs with the IEFUSI exit) .
Not giving the queue manager any access or enough access to this storage led to various failures. Some were easy to tie to the 64-bit limit being too small because MQ would generate explicit errors delineating this. In other cases, the queue manager
would simply fail to start, not even being able to initialize far enough in order to load its error message tables. But the point was clear that the limits should be customized before any attempts to start the queue manager took place.
The Channel Initiator, on the other hand, made no claims on storage above the bar. The CHIN's private region and common storage sufficed. At version 7.0.1 the InfoCenter documented that each channel instance would use 170 KB of private region in the
CHIN's address space (increasing in size if message sizes exceeded certain limits). It was expected that storage allocated because of greater message sizes would be freed when the current buffer size requirement dropped below its threshold
after a certain number of consecutive messages had passed (or after a heartbeat was sent or received). In addition, while storage used by the CHIN would be marked as reuseable as far as the LE environment was concerned, the z/OS VSM component would
be unaware of this. In effect, the amount of virtual storage that might be expected to be available is actually capped because of the way storage is made available within z/OS. This puts a finite limit on how many channels an MQ instance can actually sustain.
There is also additional overhead introduced if multiplexed channels are used for SVRCONN channels; ie. where SHARECNV is greater than 1. A channel with SHARECNV set to 0 will use 170 KB as mentioned above, but when setting SHARECNV to 1 or
greater that requirement increases to 351 KB making the maximum number of channels smaller.
In a perfect world these allocations would be stacked one after another in virtual storage to make the most effective use of it. In reality, virtual storage manager is like a real estate agent. Finding the perfect location to satisfy a storage request may not be the very next
lot. This is because, where storage is allocated from depends on the size of the request. If storage from 00000000 - 1A000000 was mostly allocated (with 106240 chunks, each 4K in size) this would leave a total of 1 Meg unallocated. If VSM got a request for
1 Meg of storage, it would begin a scan of its free queue elements to see if any area could fulfill the request for 1 Meg. This 1 Meg chunk might fit the bill, but only if it can be allocated as a single contiguous block. If some portion of this unallocated 1 meg falls
between some of the 4K blocks and is not located in storage contiguously then VSM will have no choice but to fail the storage request even though (in total) sufficient storage exists.
All program products are subject to this reality and so do their best to allocate and free storage most efficiently at the product level. This is a continuing endeavor. For WebSphere MQ at Version 7.0.1 the CHIN code has available a program temporary fix that
reduces storage fragmentation (which in turn can eliminate the eventual elevated CPU that fragmented storage can cause). This efficiency gain is made possible by immediately allocating the control blocks and stack storage that it's expect the CHIN will need
in order to support the channel limits that the customer has defined. Because the allocation is done all at once the storage which is carved out is naturally contiguous. This avoids forcing VSM to scan so vigorously (if at all) for free storage chunks in order to satisfy
requests that occur down the line. While fragmentation has always been there, mitigating its effects can put MQ's messaging powers to better use while lessening resource constraints for the operating system.