IBMers' Blog on Messaging A bunch of bloggers from IBM Software Labs working on WebSphere MQ,WebSphere Message Broker,WebSphere MQ Everyplace,IBM Lotus Expeditor micro broker. You can contact us at msgevangelists@googlegroups.com

WebSphere MQ v6.0.2.6 is now available for all platforms.

WebSphere MQ v6.0.2.6 is now available for all platforms. Please visit the recommended fixes page to download.
http://www-01.ibm.com/support/docview.wss?rs=171&uid=swg27006037#2

The associated MA45 and MQC6 Clients are available from the SupportPacs page - http://www-01.ibm.com/support/docview.wss?uid=swg27007198

( Mar 16 2009, 12:49:17 AM EDT ) Permalink Comments [0]

Micro broker clients

In this blog post lets see the different ways you can connect to a micro broker and some features in micro broker clients. The blog does not get into the details for every single client, but helps you understand various features of micro broker clients and some common configuration.

Here are micro broker clients currently available:

1. IA93: C implementation of WebSphere MQ Telemetry transport (Version 3) – C source code provides an API implementation for the client side of the protocol.
2. IA92: Java implementation of WebSphere MQ Telemetry transport (Version 3) - Java source code provides an API implementation for the client side of the protocol. (com.ibm.mqttclient)
3. Micro broker JMS client (In Lotus Expeditor client) – JMS API implementation for the clients to communicate with micro broker.
4. MQTT Version 5 client (In Lotus Expeditor client) – Java implementation of the MQTT Client API Version 5. (com.ibm.micro.client)

Getting started with using the clients

Some links to resources in the web which help getting started with using the client side API.

MQTT V5:

• Using the new MQTT v5 client in Expeditor 6.2
• Example usage of the API

Micro broker JMS client

• Using the micro broker JMS client
• Developing a JMS client
• Developing a JMS client (InfoCenter)

MQTT V3:

• Samples are provided in IA93. Also a small GUI wmqttsample.jar.
• Inter- and Intra-process communications

What client should one use?

Although one cannot answer this question completely (as its dependent on the various factors), in this post, I will try to help answer this question based on some scenarios which I have come across.. Micro broker JMS clients are used in scenarios when extending enterprise applications usually based on WebSphere MQ to the pervasive world using micro broker as a messaging engine. Its easier to develop as the skill set is widely available and applications can be provider independent. For example, same application can be used with minimal changes to connect to WebSphere MQ or micro broker or any other JMS provider.

If you have a new device of your own, from which you would like to send and receive messages, then MQTT would be a right choice. You can implement MQTT client side using the specification in www.mqtt.org and communicate with the micro broker that implements the server side of MQTT. You can implement your own client having a subset of MQTT client features using the specification in any language and connect to the micro broker.

A driving force for MQTT I feel lies in its simplicity to use. Also MQTT client implementation that is already available can be used in many memory constraint environment. The client takes very less memory. For example, if you like your PDA or a smart phone to communicate with the micro broker, then MQTT would be right choice. Since MQTT has a very small header, it can be used networks where data transfer costs. You can see more use cases in the red book.

Micro broker client Interoperability

The messages sent to the broker from one client and be received by a different client. For example, messages sent from a MQTT V5 client can be received by a JMS client connected to the broker. If the user properties are set in the JMS client they are retained when received by MQTT V5 client. A message sent from a MQTT V3 client will be received as a bytes messages by the MQTT V5 or a JMS client. If a message sent from a JMS client is received by a MQTT V3 client only the payload can be seen.

Security

Micro broker and its client implementations provides security features. Hence clients can connect securely to the broker using encryption, authentication and authorization features. Following links help in developing a secure micro broker V5 client:

• Quick guide to using SSL/TLS support in MQTT and the micro broker
• Developing and Deploying Applications using IBM Lotus Expeditor Micro Broker
• Configuring micro broker security

Securing a JMS client is much easier as its just a few connection factory properties that need to be set. There is another important thing to note about security is the platform support. The micro broker client can use the security features in Expeditor Device EE (jclDevice) and Expeditor Desktop EE (jclDesktop) java environment.

There are many security related configuration and features available. There have been discussed in the information center and redbook. Let’s not get into the details in this blog post.

Connecting locally

There are scenarios where the micro broker and the client runs in the same JVM. Especially when developing Lotus Expeditor applications one bundle can run the micro broker and other applications can be developed to use the same micro broker. Micro broker provides a feature where client use local binding. This feature does not use the TCP/IP layers. It makes direct function calls for communication with the broker hence making the communication faster and simpler.

Remote broker administration client

The micro broker provides a administrative API can be used to create/modify/remove broker components. This admin client is also a MQTT client that uses micro broker system topics to administer the broker. In effect it uses the publish/subscribe feature of micro broker to administer the micro broker itself. Security features can be configured to ensure that only users having permissions can administer the micro broker.

Configuration

Micro broker need no configuration for most of the scenarios. But in some cases, the micro broker configuration needs a little configuration. Let’s look at configuration from a scenario’s perspective as the information center and the javadoc explains every configurable parameter in detail.

• Getting a higher performance: Set the micro broker persistence to use memory persistence instead of disk persistence BrokerDefinition.setPersistenceDefinition() … This boosts the performance at the risk of loss in messages on shutdown. Also if the application is design to shutdown correctly every time, the shutdown persistence can be used so that the messages are written to the disk message store only when application closes normally without a crash.
• Message sequence Vs Performance: The in-flight window size can be chosen as MQTT_V3_HIGH_INTEGRITY if message sequence is really very important in all the scenarios when compared to performance. The parameter plays a very important role when there are frequent network disruptions.
• Keep alive time: This is specified using the client API. Plays a important role to keep the client connect to the micro broker. The keep alive messages are special messages that flow between the broker and client when there is no other data flow. If the client and broker are on the same machine, there keep alive can be a higher, this will reduce the flow of keep alive messages. In case of fragile networks where connection keeps dropping, the keep alive needs to be reduced to a low value.
• Last Will and Testament (LWT): Consider a scenario when running a MQTT client on a fire alarm, its important an administrator knows that client disconnected unexpectedly and requires attention. LWT is also a client side configuration that allows broker to publish a message to a topic when the client unexpectedly drops the connection. This usually happens after the keep alive message does not flow.
• MQTT persistence: If the messages should never be lost and also should never be delivered more than once, QOS 2 is a choice, but that’s not sufficient .. MqttPersistence is needs to be implemented. Consider a scenario where the client sends a message but doesn’t receive the acknowledgement and during this period the client goes down. The Mqtt persistence enables saving this message ensure once and only delivery happen even if there is a client side crash.

Buffered Clients

Now that you have seen different client features, the most interesting one is actually buffering… Simply put, its a way clients can save messages when the client is not connected to the broker. The buffering options are available with the JMS clients and can be enabled with simple connection factory properties (com.ibm.msg.client.mqtt.MQTTConstants.MQTT_BUFFERED). With this option enabled, when the client looses the connection to the broker, and an attempt to send the message is made, it gets buffered without throwing any exception to the application. The connection control policies can be set on control the behavior of this buffering.

The buffering of messages won’t happen when the client to broker connection is alive. This comes into play only when the connectivity is to the broker is broken. Also its important to configure the buffering properties like buffer size to suite the needs.

RSMB bridge to Micro broker

RSMB can also bridge to micro broker allows messages to be passed between these broker. From a micro broker perspective its just like an MQTT C client connecting to micro broker.

Micro broker broker another Micro broker

When a bridge is created from one micro broker to another, from a remote micro broker perspective its like a client connection and can’t differentiate it from an another client connect to that micro broker. The bridge can use JMS, MQTT V5 or MQTT V3 to connect to the remote micro broker.

I hope you would have an idea of various micro broker client features and some common configuration.

Blog by,
Neeraj Krishna (neeraj.krishna-AT-in.ibm-DOT-com)

( Mar 02 2009, 08:16:37 AM EST ) Permalink Comments [0]

Really Small Message Broker

A broker based on MQTT protocol which takes roughly up only 50KB of storage space and can run in only 150KB or less of memory, hence a perfect suite for embedded devices/low-powered servers.


Amazingly it has bridging capabilities to WebSphere Message Broker as well as Lotus Expeditor micro broker also !

Please have a look at RSMB


Blog by


Vijay Raj K and Neeraj Krishna

( Jan 20 2009, 12:57:27 AM EST ) Permalink Comments [0]

Redbook on Lotus Expeditor micro broker

A new red book for micro broker version 3 is out there.


Here is the link Messaging from the Edge by Using IBM Lotus Expeditor Client V6.2


This red book basically covers the new features like SSL support and MQTT version 5 client. This also has an enormous collection of design patterns that can be applied to the edge network which helps users/architects to design and deploy their solutions.

Blog by,
Vijay Raj K and Neeraj Krishna

( Jan 15 2009, 01:03:26 AM EST ) Permalink Comments [0]

WMB: Procedure to setup, configure and run SAP Connectivity Message Broker sample.

This blog describes in detail how to configure the SAP Server for running the SAP connectivity sample provided with WebSphere Message Broker (WMB). The blog further details the procedure to follow to configure SAP System for SAP Inbound Sample and configuration of SAP Inbound Adapter on Message Broker end.

1. Install WebSphere Message Broker 6.1 and upgrade to latest fixpac 03 (6.1.0.3) so that no errors are reported while importing the SAP Connectivity sample.
   
   
   
2. Users who wish to run SAP Connectivity sample need to have a login to SAP server and to do this they need to install SAPGUI to login to SAP server.
   
   
   The SAPGUI that needs to be installed is "SAP - SAPGUI 7.10+Add-ons with SAPLOGON Customizer"
   
   

3. Once SAPGUI is installed open the connection and add the login details as follows:

   
   Click on SAP LOGON select Systems tab and click on ‘new item’ then press ‘Next’ to go to the following screen and fill in the SAP server details
   

   
   Save this setting and now we are ready to logon to the SAP Server.
   

4. 
   For successful deployment of message flows with SAP nodes, we must configure the message broker runtime with the prerequisite
   SAP Java Connector libraries (SAP JCO) and JAR file must be made available to the Adapter Connection Project.

   The JCo libraries and JAR files is available at the following location

   Users need to have valid user id and password to access the above files.

   Unzip the files under C:\SAP_JCO

   .

5. Configuring SAP system (for SAP Inbound sample)

   We need to configure the SAP Server to identify the broker as target destination.

   Create the RFC connection from the SAP system to the broker
   

   
   
   1. Log in to your SAP system.

   2. Go to the SM59 transaction, Configuration of RFC connections.

   3. Click TCP/IP connections and click Create.

   4. Check that the connection type is set to T

   5. Set RFC Destination name to an appropriate value, for example SAMPRFC1.

   6. Select Registered Server Program in the Technical Settings tab

   7. Set the Program ID to match the Program ID that is specified in the broker adapter, for example, SAMPRFC1.

   8. Click Save to save the settings.

   9. Enter a description in the Description 1 field, for example, RFC for Sample.

   10. Enter a description in the Description 2 field, for example, your name.

   11. Click MDMP & Unicode.

   12. Select Unicode.

   13. Click the green tick on the pop-up window.

   14. Click Save
       

Read the full blog here (Please download and rename the SAP_CONNECTIVITY_DOC.zip.jpg to SAP_CONNECTIVITY_DOC.zip and then unzip)

( Dec 19 2008, 12:45:49 AM EST ) Permalink Comments [0]

Redbook : WebSphere MQ V7.0 Features and Enhancements

( Nov 18 2008, 04:21:46 AM EST ) Permalink Comments [0]

WMB: Parsing messages with Message domain

This first section of blog explains how to parse messages which contains Message Domain values set in the RFH2 header and the second section explains how to parse the length encoded data basically created by the TCP/IP clients or server using WMB.

 

Section 1

We may come across with processing some messages created by an adaptor with having MRM domain set in the RFH2 header also set.

The constraint is that we can’t do any changes in the message created by the adaptor. Still we are required to process the message through the message broker using MQInput Node.

 

Here the message contains wrong Msg Set name as “4” in the RFH2 header and we are not suppose to change the message set name and still process it successfully through the MRM parser used message broker flow.

 

How to resolve this problem?

Create the following flow using MQInput Node àResetContentDescriptor NodeàCompute Node à MQOutput Node.

Set the following properties in the MQInput Node and ResetContentDescriptor Nodes respectively

MQInput Node doesn’t override or reset the Message Domain properties as mentioned in the above property editor. Since the message set name and message format are not correct in the input message coming from an adaptor, we need to reset them using ResetContentDescriptor node as mentioned below. Here the Message set BIB_ICBS_Messages and CwXML message format is created in the WMB Tookit to parse the Adaptor messages.

The compute Node “ProcessMessage” can be used to manipulate the incoming message. Its optional depends on the business requirements.

 

 

Section 2

Parsing length encoded data:

 

In a banking domain, sometimes the message comes from TCPIP client or TCPIP server, with having data length and data embedded in the message itself.

For example:

Msg1:

00550800822000000000000004000000000000000729110323203140001

 

Msg2:

0069020022200000080200003000000502114530023105050211023105012DANPACIN2008

 

In the above message the first four characters shows the length of the message and after that everything is data.

 

How to split the above messages into length and data using a MRM parser?

 

You can select data element separation as “Use data pattern” here.

 

 

For “length” field type: Set Data Pattern as “.{4}”

 

Similarly,

For “data” field type: Set Data Pattern as “[A-Za-z0-9]*” and Specify the Length Reference as “length”.

 

Blog By,

Abhinav Priyadarshi (pabhinav-AT-in.ibm-DOT-com)

( Nov 02 2008, 01:09:53 AM EST ) Permalink Comments [0]

WebSphere MQ v6.0.2.5 is now available for iSeries

WebSphere MQ v6.0.2.5 is now available for iSeries.

Please see the Recommended Fixes for Websphere MQ website -> http://www-1.ibm.com/support/docview.wss?rs=171&uid=swg27006037#1  for more information.

 

( Oct 30 2008, 11:27:17 PM EDT ) Permalink Comments [0]

Installation and uninstallation of WebSphere MQ on Linux

This article outlines the sequence in which the packages have to be installed / uninstalled on Linux. The dependencies of the installation packages are also explained.

 

Installation

Following is the sequence in which various WebSphere MQ v6.0 for Linux packages have to be installed.

 

1. Gskit
2. MQSeriesRuntime
3. MQSeriesJava
4. MQSeriesSDK
5. MQSeriesKeyMan
6. MQSeriesServer
7. MQSeriesClient
8. MQSeriesSamples
9. MQSeriesTXClient
10. MQSeriesIES30
11. MQSeriesConfig
12. MQSeriesFTA
13. MQSeriesMsg_en

 

Uninstallation

The following is the sequence in which packages must be uninstalled. This sequence is essentially the reverse of the sequence of installation.

 

1. MQSeriesMsg_en
2. MQSeriesFTA
3. MQSeriesConfig
4. MQSeriesIES30
5. MQSeriesTXClient
6. MQSeriesSamples
7. MQSeriesClient
8. MQSeriesServer
9. MQSeriesKeyMan
10. MQSeriesSDK
11. MQSeriesJava
12. MQSeriesRuntime

            

Dependencies 

The following table explains the dependencies that each package has on other packages.

 

 

Package Name	Install dependencies
MQSeriesRuntime	No dependency
MQSeriesJava	MQSeriesRuntime
MQSeriesSDK	MQSeriesRuntime
MQSeriesKeyMan	Gskit
MQSeriesServer	MQSeriesRuntime.
MQSeriesClient	MQSeriesRuntime.
MQSeriesSamples	No dependency
MQSeriesTXClient	MQSeriesClient MQSeriesRuntime MQSeriesJava
MQSeriesIES30	No Dependency
MQSeriesConfig	MQSeriesRuntime MQSeriesIES30 MQSeriesJava
MQSeriesFTA	MQSeriesIES30
MQSeriesMsg_en	MQSeriesRuntime

Follow the above sequence and use '-ihv' for installation and '-e' flag for uninstallation. 

Note : For abnormalities --nodeps, --noscripts and --replacefiles are used along with rpm. The gskit can be updated specifying -U option.

      

Lakshmi B Gangigatte and Abhinav Priyadarshi
India Software Labs
Bangalore

( Oct 22 2008, 10:30:26 AM EDT ) Permalink Comments [0]

Tools for diagnosing Java related issues

As some parts of the messaging software use java, we feel its important to give a summary of java tools that help in diagnosing some runtime issues. Micro broker is 100% java, WebSphere MQ client API is developed in java and there are parts of Message broker that is developed in java. There are situations when the messaging client or the server runs out of memory or start running very slow or gets into a deadlock. The problem might lay in the software that is developed on top of the messaging layer.

The first step is install the IBM Support Assistant. The support assistant tool is developed on top of expeditor platform and provides the user the ability to download and install the required tools. Its got the ability to update the tools also easily, hence enables you to worry less to managing these tools.

To get all the java related tools. Just go to Updates -> Find New -> Tools Add-on. You'll now be presented with a list of diagnostic tools for java under "JVM based tools". Its also good to download the "Tech Preview" tools also.

After installing them you can launch them using "Launch Activity" -> Analyze Problem -> Tools tab and choose a tool.

Here is a summary of tools:

1. IBM Thread and Monitor Dump Analyzer - For analyzing dead locks, this is best tool. Its displays all the thread details from the java core. The tool clearly identifies the deadlocks. The ThreadAnalyzer can also be used for this.
2. IBM Thread and Monitor Dump Analyzer - If your application is taking too much memory, you can generate a heap dump and analyze it with this tool. It even tells you the probable leak suspects.
3. IBM Monitoring and Diagnostic Tools for Java™ - Garbage Collection and Memory Visualizer and IBM Pattern Modeling and Analysis Tool for Java Garbage Collector (PMAT) - Basically for the performance issues.You can also see the memory usage over time. It gives good recommendations so that you can change some JVM level garbage collection policies to suite your needs.

Health Center: The above tools are for post analysis. Here is a tool that would help in monitoring a running java application.

Blog by,

Neeraj Krishna and Vijay Raj K ( Oct 13 2008, 09:48:27 AM EDT ) Permalink Comments [0]

WebSphere MQ v6.0.2.5 is now available for Unix and Windows.

WebSphere MQ v6.0.2.5 is available from the web site http://www-1.ibm.com/support/docview.wss?rs=171&uid=swg27006037#1

In addition, the availability of the SupportPacs MQC6 (the WMQ v6 client) and MA0V (the Async SOAP SupportPac), refreshed to WMQ v6.0.2.5 level have been timed with the availability of the Fixpack. The latest refreshes of the SupportPacs are now available from the following URLs.

MQC6 -> http://www-01.ibm.com/support/docview.wss?rs=171&uid=swg24009961&loc=en_US&cs=utf-8

MA0V -> http://www-01.ibm.com/support/docview.wss?rs=171&uid=swg24009880&loc=en_US&cs=utf-8

( Oct 07 2008, 12:29:12 AM EDT ) Permalink Comments [0]

WebSphere MQ V7.0 is now available

WebSphere MQ V7.0 is now available !!!!!!

The announcement letter is here. One can download using Passport advantage from here

Also read more about MQ V7 in these links:

• IBM WebSphere MQ V7.0 delivers the universal messaging backbone for SOA and Web 2.0 with enhanced ease of use, performance, and resilience
• WebSphere MQ information center

( Jun 30 2008, 09:35:49 AM EDT ) Permalink Comments [1]

WMQ v6.0.2.4 is now available

WebSphere MQ v6.0.2.4 is now available for Unix, Windows and iSeries.

Please see the Recommended Fixes for Websphere MQ website -> http://www-1.ibm.com/support/docview.wss?rs=171&uid=swg27006037#1 for more information.

( Jun 26 2008, 11:47:56 PM EDT ) Permalink Comments [0]

HL7 application integration into IBM Websphere Message Broker V6.0

Healthcare Industry

Healthcare organizations use a large number of discretely-developed IT applications. The exchange of healthcare related data among these systems is required both for administrative and for medical reasons. In an attempt to resolve challenges faced during system integration of healthcare applications, there has been an evolution of messaging standards such as HL7. Many healthcare industries are looking to build solutions by re-using their existing infrastructure. The existing applications need to communicate with each other and share data through a centralized processing unit. HL7 standard is widely used by different healthcare applications to process healthcare data in a standard format.

Message-oriented middleware (message broker or communication server) is being used to deal with some of the integration issues, for example, application integration, translation of code, mapping of events and message routing.

This article aims to describe how to integrate HL7 applications with Websphere Message Broker.

IBM Websphere Message Broker

IBM Websphere Message Broker (WMB) is a business integration product. Business integration is the coordination and co-existence of all the business processes and applications. It involves bringing together the data and process intelligence in the enterprise, and harnessing them so that users can achieve their business goals through their applications.

The use of Message Broker offers a scalable platform that has access to data sent by various applications in the organization. It improves the dependability and ease of access of data and concentrates all available message flows to a single point. Creating a central point for data improves the quality of healthcare service, the integration of data, and the central management of the links.

HL7 application integration into WMB V6.0 - Sample

WMB is used to integrate healthcare applications built on HL7 standard by performing message transformation, enrichment and routing. Even if the existing healthcare applications are based on other languages like .NET languages, C or Java, they can be integrated using WMB.

WMB provides HL7 message set to parse HL7 healthcare messages coming from different healthcare segments. It integrates healthcare applications built on various HL7 segments. One of the segments is Patient information (PID) segment.

Description of the Sample

The HL7 sample demonstrates how users can use the MRM parser to model the PID segment of an HL7 message in XML and TDS formats. It will demonstrate how the PID segment of HL7 message can be translated from TDS to XML message format and vice-versa.

The HL7 sample contains two message flows. The first message flow converts HL7 messages to XML messages (refer Figure 1), and the second message flow converts XML messages to HL7 messages (refer Figure 2). The message set has been created to parse the HL7 messages containing "MSH (message header) and PID (patient information) segment of HL7 Standard" present in TDS format and in XML format (refer Figure 3.).

Figure 1: Message flow which converts HL7 message to XML message

Figure 2: Message flow which converts XML message to HL7 message

Figure 3: Message set to parse HL7 messages (PID segment messages)

HL7 sample artifacts can be downloaded from:
http://www-1.ibm.com/support/docview.wss?uid=swg24014296

Download the HL7.zip file. Please follow the installation instructions as per ia9n.pdf. The HL7 message flow and message set details can be found under the samples gallery.

Benefits of using WMB as middleware

• Integration of healthcare applications helps to reduce the cost of application upgrades and modifications.
• Any healthcare application developed using ERP, SAP, Siebel, etc can be integrated using WMB.
• Cost and problems associated with point-to-point connection can be reduced as it is easy to maintain the connections, migration of systems is easy and also cost involved in creating new connection is less.

By, Sowmya H Dayananda and Abhinav Priyadarshi ( May 02 2008, 12:12:41 AM EDT ) Permalink Comments [1]

An Introduction to WMQ Queue Manager Processes

An Introduction to WMQ Queue Manager Processes

When a queue manager is created, you get a message something like this:

Observe that 40 default objects are created when a queue manager is created. An MQ object can be a queue, a channel, a namelist, a listener (in WMQ v6.x), a process, etc. Most of these default objects act as templatesfor defining queue manager objects. For example, SYSTEM.DEFAULT.LOCAL.QUEUE is used as a template and its attributes are copied when a new local queue is defined using “DEFINE QLOCAL” command.

During queue manager creation, it starts some of the queue manager processes, which creates these objects, their equivalent files and directories under /var/mqm/qmgrs/<QMGR_NAME> directory, and other supporting directory structures such as queue manager LOG files under /var/mqm/log directory, and at the end the queue manager is ended.

Now, let us start a queue manager using “strmqm TECHTALK”


We will discuss about output lines of “strmqm” a little later.

Now, if you issue:


In the above output, process “amqzxma0” has a parent PID of ‘1’, whereas it is the parent for most of the other processes. This is the first process created and it is called the “Execution Controller”, simply called EC.A little bit of UNIX here. Since “strmqm” exits at the end, this process will eventually be owned by the first UNIX process “init”. (Technically, “init” is not the first UNIX process. First process is pid ‘0’, the scheduler process).

Apart from spawning other processes that are necessary for the functioning of a queue manager, the EC does few more things:

1. Establishing connection between the connecting application process and the agent process.
2. Periodically checking the health of the WMQ internal processes.
3. Cleaning up its children processes in case of queue manager termination (normal and abnormal), etc.

Hey, I mentioned about “agent” process above! It is “amqzlaa0” in the above “ps” listing. It is the servant process that receives request from applications and responds back. In other words, it acts as the agent between the application and the queue manager resources.

Few tit-bits about agent process:

1. Agent is a threaded process and can service 64-applications at a time. Each application is connected to an agent-thread.
2. New agent will be created there are more than 64-connection requests.
3. The queue manager uses a pool of agent threads and reuses them.
4. AgentClassMapand AgentClassLimit<n> are two tuning parameters to control the number of threads an agent can spawn.

I also mentioned about “internal” queue manager processes. These are the processes prefixed with “amq” which are necessary for the functioning of a queue manager. External queue manager processes are the processes like runmqsc, runmqlsr, runmqchl, runmqchi, runmqdlq, etc, and these interact with the queue manager just the way queue manager applications do.

How do these various processes (internal qmgr processes, external qmgr processes and qmgr applicaton processes) interact?

They interacting using Inter Process Communication (IPC) facility provided by the Operating System. Pipes, Named pipes, Shared Memory, Mutexes & Semaphores, Messages Queues are 5 IPC mechanisms according to System V standards. Sockets and Streams are the other two. WMQ “largely” uses Shared Memory mechanism to share the information and access to these resources are serialised / controlled using Mutexes and Semaphores.

Figure 1. Shared memory is the preferred technique of communication between two processes.

In WMQ, we mainly use two terms for shared resources – SUBPOOLS and MEMORY SETS. Subpool is a big chunk of memory, which can be further divided into n-number of memory sets. Memory sets are further divided into EXTENTS. And a queue manager will have a number of SUBPOOL for different purposes – IPCC Subpool, QMGR Subpool, APPLICATION Subpool, SYSTEM Subpool, QMPERSISTENT Subpool, etc. IPCC Subpool is the one which is used for communication between an application process and an WMQ process (mostly AGENT process).

How are these resources created and who creates them?

When a QMGR is started, it creates a set of files such as <QMGR_DIR>/@ipcc/shmem/shm.<n> where <n> denotes the kind of subpool. The i-node of these files are mixed with the “device number” to generate a “key” that is passed to “shmget( )” call to generate an unique ID for the shared memory resource created. This unique id, SHMID, can be compared to “file descriptor” when you open a file using open( ) system call.


Any application that wants to connect to the IPCC subpool will generate the key by the above technique and puts a request on the IPCC Subpool. This code will be part of the MQCONN( ) API call. 

If “shared memory” is the method of communication, we say that the application is connecting to the queue manager in STANDARD bindings. Since the address space is shared between two processes here, the queue manager is prone to corruption if the application corrupts some part of the shared memory.

If you replace the mode of communication with “pipes” / “sockets”, the mode of connection is termed as “ISOLATED” binding. This can be achieved by specifying MQCNO_ISOLATED_BINDING option in MQCONNX( ) call, or by specifying DefaultBindType=ISOLATED in Connection stanza of qm.ini file. This isolates the application address space completely from that of the agent address space. But, it is less efficient compared to STANDARD binding in efficiency.

There is one more type of binding, called FASTPATH, where the application directly access the queue manager resources, bypassing the agent process. This provides the lowest level of protection and is configured only for trusted-applications.

Going back to the output of “strmqm” command:


There are two types of messages: Persistent and Non-Persistent. A non-persistent message does not survive a queue manager restart. However, queue manager can not afford losing a persistent message. But, each time a message is PUT or GET, the message will not be written or removed from the queue file on the file system. This would be very expensive in disk I/O. Queue manager keeps a buffer in memory, called queue buffer, which is flushed into the queue file periodically. To ensure that we do not lose persistent messages, each operation with a persistent message is going to be stored in what is called “log buffer” which will be flushed into the LOG files periodically. These log files are created under /var/mqm/log/<QMGR_NAME> directory based on the settings in mqs.ini file (or arguments passed to crtmqm command).

“amqzmur0” and “amqzmuc0” are the two processes that are responsible for logging the transaction information into the log files. These are called restartable (mur0) and critical (muc0) processes respectively. Discussion about the functioning of LOGGER component is beyond the scope of this article. It is sufficient to understand that each and every operation (CONN, OPEN, PUT, GET, etc) gets logged in the WMQ log, and periodically the contents of the log files that are complete are “synced” with the queue file. This sync operation is also called “checkpoint”.

Figure 2. Log records and periodic checkpoint operations.

If you imagine log file as a continuous strip of information, last checkpoint record indicates the “point-of-consistency” between the log file and the queue file. In other words, information till the log file will be safe in the queue file. If a queue manager crashes, when you restart the queue manager, the log records “from” the last checkpoint will be played-forward to bring the queue manager to a state it was when it last crashed. This is how persistent messages are recovered during a crash. Even during queue manager start operation, these log records are played-forward. This is indicated by the messages:


You must have already been familiar with the following error:

First of all, an user should belong to “mqm” group (on UNIXes) to be able to perform system administration operations such as creating, starting qmgrs, creating qmgr objects, etc.

Further, for an application to connect and perform tasks such as PUT, GET, it should have appropriate authority given. This is granted by a WMQ sys-admin using “setmqaut” command. For eg., # setmqaut -m TECHTALK -t qmgr -p guest +connectprovides authority to perform MQCONN( ) operation on a queue. Where is this information stored? This information is stored in a system object “SYSTEM.AUTH.DATA.QUEUE” as a message. There is a process called Object Authority Manager (OAM), amqzfuma, which reads and authorises any connecting application process (and even WMQ internal processes from accessing queue manager objects).

“amqzfuma” is the default OAM provided by the WMQ queue manager. This is defined in the queue manager’s qm.ini file as follows:



This means, one can write their own OAM and make WMQ use these services by making appropriate entries like the one above. It is a kind of pluggable components.

Till now, we have been talking about a single queue manager and an application connecting to a queue manager directly. There are other modes of communication possible – An application connecting to a queue manager over a TCP/IP channel (CLIENT connection), and an application communicating with a remote queue manager through another queue manager.


Figure 3. Application connecting over CLIENT channel or through another queue manager.Application connecting over CLIENT channel or through another queue manager.

In both cases, the receiving queue manager will have to listen on a port (could be TCP/IP or LU6.2, etc). The common listener process is “runmqlsr”. This listens for the incoming requests and hands over the request to a process called “amqrmppa”, also called channel pooler. This is a threaded process and based on the number of requests, more number of “amqrmppa” process will be spawned.

On the remote end, “amqrmppa” works on behalf of the application and returns the service. Here is the simplified flow:

Application interacting directly with a queue manager:
APPLICATION ---> AGENT ---> QMGR RESOURCES

Application connecting to a queue manager through another queue manager:
APPLICATION ---> SENDING MCA ---> RECEIVING MCA (LISTENER/AMQRMPPA) ---> AGENT ---> QMGR RESOURCES

Application connecting to a queue manager over CLIENT CHANNEL:
APPLICATION ---> TCP/IP ---> LISTENER/AMQRMPPA ---> AGENT ---> QMGR RESOURCES

Lastly, in situations where more number of queue managers want to communicate among each other, CLUSTERING of queue managers helps the administration overhead by automatically defining channel connections between some of the queue managers.

As you must be aware, a cluster is recommended to have at least TWO full-repository queue managers (Refer Figure  below). A full-repository process is going to hold information about the ENTIRE cluster, whereas a partial repository queue manager INITIALLY holds information about itself and over a period of time it builds up the information which is required for its operation. These repository information are stored in the form of messages inside SYSTEM.CLUSTER.REPOSITORY.QUEUE. "amqrrmfa" is the process, called repository process. This process is responsible for retrieving information from the repository queue, updating the information, etc.

Figure 4. Queue manager CLUSTER and REPOSITORY process, AMQRRMFA.

Although I have not covered each and every process, these are the basic things one can start with.

( Apr 27 2008, 01:14:02 AM EDT ) Permalink Comments [5]