Meet the experts: Mark Taylor on WebSphere MQ

This question and answer article features Mark Taylor, technical strategist for WebSphere® MQ, who answers top questions from users about developing applications for, or running systems with, WebSphere MQ (formerly MQSeries) on the distributed (non-z/OS) platforms.

Mark Taylor (marke_taylor@uk.ibm.com), WebSphere MQ Technical Strategist, IBM

Author photoMark Taylor has worked for IBM for nearly 20 years, and has worked in a variety of development and services roles. He wrote code for the early versions of MQSeries, porting it to numerous Unix operating systems. Mark spent the last five years in the Technical Strategy department where he is now responsible for defining the functions included in of new releases of WebSphere MQ. Mark is also a frequent speaker at technical conferences.



22 January 2007 (First published 03 February 2003)

Also available in Russian

This month we asked Mark Taylor to answer your about developing applications for, or running systems with, WebSphere MQ (formerly MQSeries) on the distributed (non-z/OS) platforms. WebSphere MQ messaging products enable applications to exchange information among more than 35 IBM and non-IBM platforms, including Linux and Windows® 2000. Mark is a technical strategist responsible for defining functions that are added to new versions of WebSphere MQ. For more information, see the developerWorks WebSphere Business Integration zone.

We appreciate and thank all the WebSphere developers who submitted questions.

Question: How do we achieve tranparent Failover for MQSeries Java™ Client in a MQseries cluster set up? For example, if a queue manager goes down in middle of a put or get operation, the client should be able to switch to the next available queue manager and continue processing in a transparent manner to the application. (submitted by SR)

Answer: The word "cluster" has different meanings. A WebSphere MQ (WMQ) cluster is designed to assist with routing of messages between queue managers. An application still has to connect to a single, known, queue manager within the cluster before the messages can be transmitted. There is no concept of connecting from a client to any member of a WMQ cluster. An alternative meaning of cluster is its use as a failover capability, implemented using a technology such as HACMP. These permit queue managers that can move transparently between machines. There isn't the space here to go into all the possibilities, but both types of cluster can work together very successfully.

Transactions, which are required to meet once-only delivery guarantees, are managed per-connection. If an application or queue manager abends, then any active transaction is backed out. This means that any client application needs to be able to handle a CONNECTION_BROKEN return code and decide how to proceed. You cannot expect to lose a connection during a PUT or GET and continue. Instead, you have to explicitly reconnect and then decide which work you want to rerun, knowing that the previous operations will be rolled back. This transactional integrity is vital to reliable delivery, and is why you cannot redirect requests to another queue manager immediately.

So the question is then, where do you reconnect?

If you are using a failover cluster, then the failed queue manager becomes available on a new machine, but with the same TCP/IP address. Repeating your original connection request attaches to the restarted queue manager and you can repeat any work that would have been backed out from the previous connection.

If you really want to connect to a different queue manager, then client applications written in C can use a wildcard during MQCONN, along with the external client channel file (AMQCLCHL.TAB). This selects the first available queue manager from the table that meets your selection criterion. However the current version of the Java and JMS classes cannot access that file, so they have to code a list of alternative connections. You can do this either explicitly (for example, setting the TCP/IP address of each queue manager in your program until you successfully connect), or by iterating through entries in the ConnectionFactory details accessed through JNDI.

Question: MQ is running on Unix box. I have an NT application which uses the MQ's services for messaging. I want to develop a simple MQ Monitoring tool, which would give me Q depths and the message contents in the Queue. This monitoring tool needs to be integrated in the NT application, so that the user can view the Qdepths and messages in Q. The application uses ORACLE and is in C programming language and Java. What options are available? We cannot afford to go for 3rd party tools. (submitted by SR of Deutsche Bank)

Answer: There are a number of APIs and techniques for discovering the depth of a queue. Depending on your application environment, you might want to use MQINQ directly or send PCF messages to the command server. Reading the contents of messages non-destructively requires you to use MQGET with the MQGMO_BROWSE flags. Use of these interfaces is demonstrated both in sample application code shipped with the product and in SupportPacs. In particular, take a look at MS02 and MS0B as they help with the construction of PCF messages. See the full list of SupportPacs. Several standalone monitoring programs are also available there.

Question: Can you explain the naming path from a Websphere V5 JNDI application to a Websphere MQ JMS queue? Is there a good plan for using local provider queues on desktops during development and rolling out to an enterprise name space for deployment? (submitted by RW)

Answer: Every JMS provider seems to have a different way of specifying how to connect from an application program, and different ways to define the attributes of the connection and the queue being used. Because one of the goals of JMS is to be vendor-independent, the standard requires common interfaces that the application program can use regardless of the provider. Hence, the ability to store information in a directory which is accessed at runtime, and automatically used by the JMS provider. For more information, see Implementing vendor-independent JMS solutions.

With the WMQ JMS implementation, the Queue Connection Factory defines how the application connects to a queue manager. This is frequently in Client mode across a TCP/IP network. Once connected, you can use the Queue object to open a queue on that queue manager. The queue name administered in the JNDI space is the same as the queue name seen by WMQ. Creating a queue in WMQ through the runmqsc command does not automatically make it available for JMS applications. You need to put a Queue object of the same name in the JNDI through the JMSAdmin program.

WebSphere Application Server V5.0 includes some limited administrative capability, which combines both the runmqsc and JMSAdmin functions for a single default queue manager, known as the Embedded Messaging provider. The queues defined are available only in that default queue manager. If you want the full flexibility of the JMSAdmin tool with the ability to connect to different queue managers, you have to install a fully-licensed copy of WMQ V5.3 on top of the Embedded version.

The path from JMS to JNDI gives one level of isolation and indirection, making it convenient to change the "real" queue manager dynamically, perhaps from a development to production system. However, I'd also recommend that all object names, including the link to the JNDI service, be configured outside of the application (perhaps as properties which is referenced at runtime). Remember that the JNDI link could easily refer to a local file on a workstation or to an enterprise directory server. Changing externally-configured properties makes it trivial to move between environments. If your organization has already defined naming conventions for other applications using WMQ, then you can carry these forward into the JMS world.

Question: How can I tell what version of WMQ I've got installed on each platform?

Answer: You can use the operating system specific command to list which versions of products are installed. For example, lslpp on AIX, swlist on HP-UX, pkginfo on Solaris. There is also a mqver command, currently undocumented, which is available across the V5.2 and V5.3 products. This shows the base level, such as 530, of the product. The CMVC level contains the date on which the product was compiled. This is useful for IBM service to know as they can relate it back to a CSD number and custom efixes which had been generated. We do intend to make some enhancements to this command, which will show the CSD number more clearly.

Question: When will we be able to use shared queues on Distributed platforms?

Answer: I'm reluctant to answer this with the word "never", but I think this is a function that is very unlikely to appear on platforms other than z/OS. A shared queue, which is accessible by multiple queue managers simultaneously, relies on the services provided by a fairly specialized piece of equipment, the Coupling Facility. This is a combination of hardware and software that gives function beyond a simple sharing of memory between machines. For example, there are dedicated operations that can handle updates to list structures. Such systems are not currently available for other platforms. Our investigations into techniques that would allow shared updates to a queue without a dedicated Coupling Facility tend to fail because of performance and recovery concerns. Distributed two-phase transactions are just not efficient enough to match what you can achieve with dedicated queue managers. For now, if you need the parallel scalability and recovery that shared queues provide, then z/OS is the place to be.

Question: My application seems to be losing messages. Why?

Answer: WMQ is very careful not to throw away messages unless it has been told that it is allowed to. The most common reason for an apparent message loss is that the message has been defined to be non-persistent. Many people seem to think that you can make queues persistent, but the DEFPSIST attribute simply sets a default value. Persistence is really a message attribute and not a queue attribute. An application program can choose to either set an explicit value or allow the default to be picked from the queue attribute. Non-persistent messages are discarded when a queue manager is restarted, and may also be discarded when a channel fails. When you think you've lost a message, always check whether it could have been sent as non-persistent.

A second reason that messages might be "lost" is that the queue manager has not put the message to the expected queue. The message is not permanently lost, simply misplaced! Take a look at the Dead Letter Queue (DLQ) defined on each queue manager through which the message might have been routed. If there was a problem sending the message, then it has probably gone to the DLQ.

The final common reason for the appearance of a lost message is when a client application abends during an MQGET(WAIT) operation. On restart, an expected message has been removed from the queue. To avoid this problem, ensure that you are using a transaction (MQGMO_SYNCPOINT) for the MQGET call. The message is rolled back to its queue when the SVRCONN channel attempts and fails to send it to the client program.

Question: I cannot purge messages from a queue because it is in use. How can I tell who is using it? And how can I stop them using it?

Answer: Look at the DISPLAY QSTATUS command that was introduced in WMQ V5.3. This shows all of the processes that have a queue open, both application programs and channels. The channel programs keep a cache of recently used queue handles, which might keep a queue in use. Although the channel eventually releases that handle, the DISPLAY QSTATUS information shows the channel name so you can force it to end immediately if necessary.

Question: An existing application is putting messages to a queue. I now need to be able to send copies of those messages to a second queue, but cannot change the application. Is there a way to do this?

Answer: If you are using WMQ V5.3 on any of the Distributed platforms, then take a look at the API Exit interface. This allows you to intercept every MQI call made by an application. Whenever an application does an MQPUT, you have an exit make a corresponding MQPUT of the same data to a second queue. An alternative is to change the queue used by the application. This is easily done if it is using an ALIAS definition, or the application reads its output queue from an external configuration file (to be a temporary holding queue). Then write a new application that retrieves the message and puts one copy to the original output queue, and a second copy elsewhere. You might also, of course, be able to use one of the WMQ message broker products as a fully-flexible intermediary.

Question: How do I control the userid that is used for authorization from a JMS application?

Answer: If the application is connecting to the queue manager in local bindings mode, then the userid is associated with the running process. If the application is connecting to the queue manager as a client across a network, then the MCAUSER attribute of the SVRCONN definition controls which userid owns the connection. If the MCAUSER is blank (the default setting), then most systems cause the JMS application to be running with mqm authority, unless the client program has provided its own userid.

For very old C clients, such as on Windows 95, the provision of a client's userid was done by setting an environment variable.

For current C clients, the userid is automatically picked from the running program and sent to the server.

For Java and JMS clients, the userid has to be supplied from within the program. This can be done either by setting the values in the Environment class, or by using the version of the JMS createQueueConnection method that takes userid and password parameters. If one of these is not done, then no userid flows across the channel and the default from the MCAUSER is taken.

All three of these mechanisms have an equivalent security strength, which is very little, as the client is simply asserting a userid in all cases. There are no guarantees about the ability of a user on the client machine to set an environment variable, create a new user, or modify executing code.

For assurance of the identity that is coming from the client, then you must either use channel security exits, or use SSL authentication. If the password is used in the createQueueConnection method, then that is made available to the security exits. For an example pair of security exits written in Java for the client and C for the server, Look at SupportPac IC72.

Question: Can I use WMQ across the Internet?

Answer: Most connections across the Internet ought go through one or more firewalls configured to isolate machines and provide security. While you can use WMQ directly through a firewall by opening up the port number to which a channel is connecting (1414 by default), there are a several additional facilities that administrators might like to use.

If you want to restrict the sending side's local port number (this is not normally needed, but is sometimes requested), use the LOCLADDR attribute of a channel. This allows the initiating channel, such as a SENDER or CLNTCONN, to say that the outbound socket comes from a specific TCP/IP address, and within a known port range.

WMQ channels do not, by default, have any security controls applied to them. Once you start connecting across the Internet, look at using the SSL facilities available in V5.3 for authentication, encryption, and data integrity.

For an even more flexible approach, look at SupportPac MS81, the WMQ internet pass-thru. This is a fully supported package that can act as a proxy in the DMZ between firewalls, relaying WMQ channel protocol flows. It can also encapsulate these protocols using HTTP, which makes it even easier to traverse firewalls.

Comments

developerWorks: Sign in

Required fields are indicated with an asterisk (*).


Need an IBM ID?
Forgot your IBM ID?


Forgot your password?
Change your password

By clicking Submit, you agree to the developerWorks terms of use.

 


The first time you sign into developerWorks, a profile is created for you. Information in your profile (your name, country/region, and company name) is displayed to the public and will accompany any content you post, unless you opt to hide your company name. You may update your IBM account at any time.

All information submitted is secure.

Choose your display name



The first time you sign in to developerWorks, a profile is created for you, so you need to choose a display name. Your display name accompanies the content you post on developerWorks.

Please choose a display name between 3-31 characters. Your display name must be unique in the developerWorks community and should not be your email address for privacy reasons.

Required fields are indicated with an asterisk (*).

(Must be between 3 – 31 characters.)

By clicking Submit, you agree to the developerWorks terms of use.

 


All information submitted is secure.

Dig deeper into WebSphere on developerWorks


static.content.url=http://www.ibm.com/developerworks/js/artrating/
SITE_ID=1
Zone=WebSphere
ArticleID=14056
ArticleTitle=Meet the experts: Mark Taylor on WebSphere MQ
publish-date=01222007