We have moved to the IBM Messaging developerWorks download page. Please visit the new page for latest downloads.
T.Rob Wyatt 100000R8QH Tags:  websphere client downloads download wmb messaging trials wmq 13 Comments 61,591 Views
We have moved to the IBM Messaging developerWorks download page. Please visit the new page for latest downloads.
(Republished old entry after correcting the links to images)
Author: Madhukar B V
When a queue manager is created, you get a message something like this:
Observe that 40 default objects are created when a queue manager is created. An MQ object can be a queue, a channel, a namelist, a listener (in WMQ v6.x), a process, etc. Most of these default objects act as templates for defining queue manager objects. For example, SYSTEM.DEFAULT.LOCAL.QUEUE is used as a template and its attributes are copied when a new local queue is defined using “DEFINE QLOCAL” command.
During queue manager creation, it starts some of the queue manager processes, which creates these objects, their equivalent files and directories under /var/mqm/qmgrs/<QMGR_NAME> directory, and other supporting directory structures such as queue manager LOG files under /var/mqm/log directory, and at the end the queue manager is ended.
Now, let us start a queue manager using “strmqm TECHTALK”
We will discuss about output lines of “strmqm” a little later.
Now, if you issue:
In the above output, process “amqzxma0” has a parent PID of ‘1’, whereas it is the parent for most of the other processes. This is the first process created and it is called the “Execution Controller”, simply called EC.A little bit of UNIX here. Since “strmqm” exits at the end, this process will eventually be owned by the first UNIX process “init”. (Technically, “init” is not the first UNIX process. First process is pid ‘0’, the scheduler process).
Apart from spawning other processes that are necessary for the functioning of a queue manager, the EC does few more things:
1. Establishing connection between the connecting application process and the agent process.
2. Periodically checking the health of the WMQ internal processes.
3. Cleaning up its children processes in case of queue manager termination (normal and abnormal), etc.
Hey, I mentioned about “agent” process above! It is “amqzlaa0” in the above “ps” listing. It is the servant process that receives request from applications and responds back. In other words, it acts as the agent between the application and the queue manager resources.
Few tit-bits about agent process:
1. Agent is a threaded process and can service 64-applications at a time. Each application is connected to an agent-thread.
2. New agent will be created there are more than 64-connection requests.
3. The queue manager uses a pool of agent threads and reuses them.
4. AgentClassMap and AgentClassLimit<n> are two tuning parameters to control the number of threads an agent can spawn.
I also mentioned about “internal” queue manager processes. These are the processes prefixed with “amq” which are necessary for the functioning of a queue manager. External queue manager processes are the processes like runmqsc, runmqlsr, runmqchl, runmqchi, runmqdlq, etc, and these interact with the queue manager just the way queue manager applications do.
How do these various processes (internal qmgr processes, external qmgr processes and qmgr applicaton processes) interact?
They interacting using Inter Process Communication (IPC) facility provided by the Operating System. Pipes, Named pipes, Shared Memory, Mutexes & Semaphores, Messages Queues are 5 IPC mechanisms according to System V standards. Sockets and Streams are the other two. WMQ “largely” uses Shared Memory mechanism to share the information and access to these resources are serialized / controlled using Mutexes and Semaphores.
Figure 1. Shared memory is the preferred technique of communication between two processes.
In WMQ, we mainly use two terms for shared resources – SUBPOOLS and MEMORY SETS. Subpool is a big chunk of memory, which can be further divided into n-number of memory sets. Memory sets are further divided into EXTENTS. And a queue manager will have a number of SUBPOOL for different purposes – IPCC Subpool, QMGR Subpool, APPLICATION Subpool, SYSTEM Subpool, QMPERSISTENT Subpool, etc. IPCC Subpool is the one which is used for communication between an application process and an WMQ process (mostly AGENT process).
How are these resources created and who creates them?
When a QMGR is started, it creates a set of files such as <QMGR_DIR>/@ipcc/shmem/shm.<n> where <n> denotes the kind of subpool. The i-node of these files are mixed with the “device number” to generate a “key” that is passed to “shmget( )” call to generate an unique ID for the shared memory resource created. This unique id, SHMID, can be compared to “file descriptor” when you open a file using open( ) system call.
Any application that wants to connect to the IPCC subpool will generate the key by the above technique and puts a request on the IPCC Subpool. This code will be part of the MQCONN( ) API call.
If “shared memory” is the method of communication, we say that the application is connecting to the queue manager in STANDARD bindings. Since the address space is shared between two processes here, the queue manager is prone to corruption if the application corrupts some part of the shared memory.
If you replace the mode of communication with “pipes” / “sockets”, the mode of connection is termed as “ISOLATED” binding. This can be achieved by specifying MQCNO_ISOLATED_BINDING option in MQCONNX( ) call, or by specifying DefaultBindType=ISOLATED in Connection stanza of qm.ini file. This isolates the application address space completely from that of the agent address space. But, it is less efficient compared to STANDARD binding in efficiency.
There is one more type of binding, called FASTPATH, where the application directly access the queue manager resources, bypassing the agent process. This provides the lowest level of protection and is configured only for trusted-applications.
Going back to the output of “strmqm” command:
There are two types of messages: Persistent and Non-Persistent. A non-persistent message does not survive a queue manager restart. However, queue manager can not afford losing a persistent message. But, each time a message is PUT or GET, the message will not be written or removed from the queue file on the file system. This would be very expensive in disk I/O. Queue manager keeps a buffer in memory, called queue buffer, which is flushed into the queue file periodically. To ensure that we do not lose persistent messages, each operation with a persistent message is going to be stored in what is called “log buffer” which will be flushed into the LOG files periodically. These log files are created under /var/mqm/log/<QMGR_NAME> directory based on the settings in mqs.ini file (or arguments passed to crtmqm command).
“amqzmur0” and “amqzmuc0” are the two processes that are responsible for logging the transaction information into the log files. These are called restartable (mur0) and critical (muc0) processes respectively. Discussion about the functioning of LOGGER component is beyond the scope of this article. It is sufficient to understand that each and every operation (CONN, OPEN, PUT, GET, etc) gets logged in the WMQ log, and periodically the contents of the log files that are complete are “synced” with the queue file. This sync operation is also called “checkpoint”.
Figure 3. Log records and periodic checkpoint operations.
If you imagine log file as a continuous strip of information, last checkpoint record indicates the “point-of-consistency” between the log file and the queue file. In other words, information till the log file will be safe in the queue file. If a queue manager crashes, when you restart the queue manager, the log records “from” the last checkpoint will be played-forward to bring the queue manager to a state it was when it last crashed. This is how persistent messages are recovered during a crash. Even during queue manager start operation, these log records are played-forward. This is indicated by the messages:
You must have already been familiar with the following error:
First of all, an user should belong to “mqm” group (on UNIXes) to be able to perform system administration operations such as creating, starting qmgrs, creating qmgr objects, etc.
Further, for an application to connect and perform tasks such as PUT, GET, it should have appropriate authority given. This is granted by a WMQ sys-admin using “setmqaut” command. For eg., # setmqaut -m TECHTALK -t qmgr -p guest +connect provides authority to perform MQCONN( ) operation on a queue. Where is this information stored? This information is stored in a system object “SYSTEM.AUTH.DATA.QUEUE” as a message. There is a process called Object Authority Manager (OAM), amqzfuma, which reads and authorises any connecting application process (and even WMQ internal processes from accessing queue manager objects).
“amqzfuma” is the default OAM provided by the WMQ queue manager. This is defined in the queue manager’s qm.ini file as follows:
This means, one can write their own OAM and make WMQ use these services by making appropriate entries like the one above. It is a kind of pluggable component.
Till now, we have been talking about a single queue manager and an application connecting to a queue manager directly. There are other modes of communication possible – An application connecting to a queue manager over a TCP/IP channel (CLIENT connection), and an application communicating with a remote queue manager through another queue manager.
Figure 4. Application connecting over CLIENT channel or through another queue manager.Application connecting over CLIENT channel or through another queue manager.
In both cases, the receiving queue manager will have to listen on a port (could be TCP/IP or LU6.2, etc). The common listener process is “runmqlsr”. This listens for the incoming requests and hands over the request to a process called “amqrmppa”, also called channel pooler. This is a threaded process and based on the number of requests, more number of “amqrmppa” process will be spawned.
On the remote end, “amqrmppa” works on behalf of the application and returns the service. Here is the simplified flow:
Application interacting directly with a queue manager:
APPLICATION --> AGENT --> QMGR RESOURCES
Application connecting to a queue manager through another queue manager:
APPLICATION --> SENDING MCA --> RECEIVING MCA (LISTENER/AMQRMPPA) --> AGENT --> QMGR RESOURCES
Application connecting to a queue manager over CLIENT CHANNEL:
APPLICATION --> TCP/IP --> LISTENER/AMQRMPPA --> AGENT --> QMGR RESOURCES
Lastly, in situations where more number of queue managers want to communicate among each other, CLUSTERING of queue managers helps the administration overhead by automatically defining channel connections between some of the queue managers.
As you must be aware, a cluster is recommended to have at least TWO full-repository queue managers (Refer Figure below). A fullrepository process is going to hold information about the ENTIRE cluster, whereas a partial repository queue manager INITIALLY holds information about itself and over a period of time it builds up the information which is required for its operation. These repository information are stored in the form of messages inside SYSTEM.CLUSTER.REPOSITORY.QUEUE. “amqrrmfa” is the process, called repository process. This process is responsible for retrieving information from the repository queue, updating the
Figure 5. Queue manager CLUSTER and REPOSITORY process, AMQRRMFA.
Although I have not covered each and every process, these are the basic things one can start with.
Disclaimer: Each posting on this site is the view of its author and does not necessarily represent IBM’s positions, strategies or
Collector node has been a part of WebSphere Message Broker since the 6.1 version. There are several interesting scenarios where this can be used. Here I have put-together some scenarios where I found the best use of the collector node
In summary, a collector node can help reduce lot of application development effort to store and retrieve data at various stages of a process. This just lists a small set of use cases, you can thinking of using WebSphere Message Broker and the collector node in it for several other scenarios
In the IBM MQ Java Messaging Level 3 Support Team, we regularly see customers reporting issues where WebSphere Application Server is trying to end and commit a transaction involving WebSphere MQ, and reports the following error:
There are a number of root causes as to why an XA End call might fail with the XA return code XA_RBBASE (100). The following IBM Technote describes one of the most common reasons, which is related to the Aged Timeout setting on a JNDI defined JMS Connection Factory in WSAS:
Another common cause of the exception:
javax.transaction.xa.XAException: The method 'xa_end' has failed with errorCode '100'.
is related to the use of the WebSphere MQ "Asynchronous Put" functionality. When this is turned on, any failures to put a message to a queue or topic are not reported immediately. Failures are only reported when the the unit of work that the messages were put under is ended - when the Transaction Manager component attempts to end and commit the transaction, the queue manager (which is acting as a resource manager in the transaction) reports that the transaction needs to be rolled back instead.
To illustrate this, imagine an IBM MQ classes for JMS application that performs the following operations inside an XA transaction:
Now, suppose that the queue the application was trying to put the message to was full. Because the asynchronous put functionality is being used, the call to JMSMessageProducer.send(javax.jms.Message) does not throw an exception and appears to work successfully. However, the message was not actually put to the queue, as the queue was full.
After performing these actions, the Transaction Manager issues a XATransaction.end() call to end the transaction. At this point, the Transaction Manager thinks that the message was successfully sent to the queue, and so is getting ready to commit the unit of work. The end() is flowed to the queue manager, which responds with error code 100 - the message could not be put to the queue, so the transaction should be rolled back rather than committed. This results in the Java exception noted above being written to the application server's SystemOut.log file.
Now, normally, when an application sends messages to a destination, the IBM MQ classes for JMS waits for the queue manager to confirm that it has processed the request before returning control to the application. When using asynchronous put functionality, however, the queue manager does not return the success or failure of each call and the IBM MQ class for JMS returns control to the application immediately. In the latter case, the point at which the application is made aware that the message put request failed is when the transaction under which it was performed is closed.
When not using asynchronous puts and calling the method JMSMessageProducer.send(javax.jms.Message) to send a message larger than the maximum allowed by the MQ queue, the send request would fail and the JMSException:
JMSWMQ2007: Failed to send a message to destination '<queue_name>'. JMS attempted to perform an MQPUT or MQPUT1; however WebSphere MQ reported an error. Use the linked exception to determine the cause of this error.; nested exception is:
JMSCMQ0001: WebSphere MQ call failed with compcode '2' ('MQCC_FAILED') reason '2030' ('MQRC_MSG_TOO_BIG_FOR_Q').
would be thrown to the application to subsequently handle.
This would give the application developer and IBM MQ administrator immediate feedback on the issue preventing the message put from completing. This underlying problem then becomes apparent and action can be taken to address it; i.e., in the above example case, ensuring that the queue in use has been configured with a sufficiently large maximum message size to accommodate the size of messages that the application generates.
The asynchronous put function, as well as preventing errors in messaging operations being immediately reported to applications, offers little benefit in all but a few circumstances. In the case of sending one or two messages as part of a transactional unit of work, the use of asynchronous puts would typically be discouraged.
In WSAS, asynchronous puts can be disabled on the JNDI entry for the JMS Destination. Using the WSAS Administration Console, the option can be located like so:
Resources -> JMS -> Queues -> [select entry] -> Advanced properties
We hope this has given you some insight into a very common cause the exception:
javax.transaction.xa.XAException: The method 'xa_end' has failed with errorCode '100'.
that is logged to the WSAS SystemOut.log file. If you observe this error on your WSAS system, we recommend you disable asynchronous put functionality unless you have a specific requirement for using it.
Here's a WebSphere redbook that's been made available in Draft mode. You can download the redbook in draft mode from this link: WebSphere Application Server V8.5 Concepts, Planning and Design Guide
Windows perfmon is an ubiquitous tool for System administrators monitoring various system parameters. Among the host of parameters that can be monitored, WebSphere MQ queues are included too. This post tells you more.
There are four vital WebSphere MQ queue parameters that can be monitored with perfmon.
1. Queue depth
2. Percentage of queue depth
3. Inward traffic to a queue - in terms of number of messages per second.
4. Outward traffic from a queue - in terms of number of messages per second.
To monitor any of these parameters, load the Windows Performance Monitor using the perfmon command. Right-click on the graph area and choose Add counters... to continue. Choose MQSeries Queues from the Performance objects drop down list. Choose the parameter and the queue and you wish to monitor and get started.
Alright. But why aren't all my queues shown ?
Not all queues are available for monitoring. The following set of conditions must match for a queue to be available for monitoring.
1. Queue must be a local queue.
2. The queue must have atleast one message in it OR there must be activity in the queue at the time Add Counters... was clicked.
3. Of course, the queue manager holding the queue must be in the running state.
How about Windows 64 bit ?
Windows 64 bit operating systems come with 2 versions of the Performance Monitor - one 64-bit and a 32-bit version. You will be able to monitor WebSphere MQ queues only with the 32-bit version of the Performance Monitor on a 64-bit machine. This is because the Performance Monitor interface for WebSphere MQ is a 32-bit DLL. And finally, if you'd want to figure out where to find the 32-bit version of the Windows Performance Monitor on a 64-bit machine, look into <Windows Install Dir>\syswow64
The WebSphere MQ documentation for the performance monitor interface is available here (MQ v6) and here (MQ v7)
T.Rob Wyatt 100000R8QH Tags:  events developer wtc t.rob mq wmq websphere education 10,409 Views
The 2012 WTC has been announced. Save the date, it is 15 - 18 October, 2012 in Berlin, Germany. Click here for more information or to register.
WTC is the largest European annual technical event dedicated to WebSphere products and solutions. There are over 150 unique sessions across 6 tracks!
This conference has earned the reputation for delivering deep technical content targeted at architects, developers, integrators and administrators by offering lectures and hands-on labs that focus on the best practices and practical skills required to run today’s enterprises. This year will be no exception!
Attend the WebSphere Technical Convention and expand your knowledge of SOA, CICS, Messaging WebSphere Application Servers and Infrastructure, including a focus on BPM and Cloud Computing. You can also expect to gain insight into IBM’s software strategy and learn about the latest development directions for the products in the WebSphere software platform.
The Conference Team is building a great technical agenda. Content is being organized into the following tracks:
Looking forward to meeting you in Berlin!
KiranDarbha 060000SA26 Tags:  c# .net classes xms base ibm.wmq wcf ibm.xms websphere wmq 9,538 Views
WebSphere MQ delivers the universal messaging backbone with robust connectivity that enables flexible and reliable messaging for applications, Web services, and Web 2.0. It also delivers market-leading JMS and publish and subscribe messaging. WebSphere MQ is the market-leading message-oriented middleware product that delivers a reliable, proven messaging backbone for many organizations of different sizes, spanning many industries around the world. It enables connectivity to span entire organization without limits with various choices on Skills, Endpoints, Quality of Services, and Networks. It supports wide variety of qualities of service providing a business data highway that can connect all IT platforms, applications and services across the enterprise.By implementing messaging technologies, businesses can use a consistent approach to connectivity, decoupling the business application from the complexity of handling failures, error recovery, transaction integrity, security, and scalability.
This is a cross post of the blog I posted in WebSphere India blog
Disclaimer: Each posting on this site is the view of its author and does not necessarily represent IBM’s positions, strategies or opinions. I do not guarantee correctness of the opinions or content or sample code presented here. Use it at your own risk.
SrihariKulkarni 120000KKWK Tags:  authority mq websphere oam windows setmqaut mqseries user 7,470 Views
Take a situation where you have user ids on a system who have been given access to WebSphere MQ - either as being included in the mqm group or individually through authorization commands such as setmqaut. What would happen if you delete one such user id from the system ? Will WeebSphere MQ remove the authorizations when the user id is deleted ? Or will they remain ? What happens if the same user id is created again on the same system - will the 'new' user id have the same set of authorizations to WebSphere MQ queue managers and objects as did the user id before it was deleted ? These are some of the questions this post will attempt to address.
Whenever a queue manager or a queue manager object (such as queue, topic etc.) is created, all users in the mqm group are given access to it by default. You can further fine tune access by use of the authorization commands (setmqaut, dmpmqaut, dspmqaut). On Windows, the authorization entries for each object are mapped to the security identifier (SID) of the user rather than the user name or the user id. So, when a user id is deleted from the system and WebSphere MQ is not 'informed' of thisdeletion, the entries mapping the SID to its relecant authorization will remain causing orphan entries in the authorization table. Such entries are generally harmless to the functioning of the queue manager or the security of the system. However, they can be a real headache for system administrators in figuring out who was ranted access, when and why. This is potentially a problem during auditing as well.
Another problem of having orphan entries is during the migration from WebSphere MQ v6 to v7. When the command to start a broker (strmqbrk) is run in WebSphere MQ v7, it attempts to migrate all the publish-subscribe objects and their authorizations to v7 (this is because v7 doesn't need a broker and the pubsub engine is within the queue manager). These orphan entries can cause the migration to report several errors corresponding to each 'missing' user.
How do we delete these entries later ?
Unfortunately, it is not possible to delete these entries from the authorization table. It is therefore recommended that all authorizations for a particular user be removed before deleteing the user id from the system. This article in the WebSphere MQ Infocenter has more information.
SrihariKulkarni 120000KKWK Tags:  c message .net websphere service c++ supportpac client wmq ia9h mq ia94 xms 7,072 Views
XMS C/C++ Client (Cat 3 SupportPac - IA94: IBM Message Service Client for C/C++) v2.0.1 and XMS .Net Client (Cat 3 SupportPac - IA9H: IBM Message Service Client for .Net) v2.0 are now available.
The same can be downloaded from the SupportPac site indicated below: